How to use Containers, OpenShift and Kubernetes with Red Hat

[Music] all righty uh welcome everybody uh welcome to our our event today the cockroach hour um today we're going to talk about uh we have we have a special guest from rad hat joanna so we're going to talk about openshift we're going to talk about containers we're going to talk about kubernetes all this goodness and i'm going to presume we're going to end up talking a little bit about open source 2 after meeting our our presenter here so um but let me just give a little quick guidance before we get started here uh you know we do have a qa panel there is a chat panel as well um please engage however you like uh you know we enjoy questions on on the cockroach hour um you know tim vale is joining me again if you're a repeat customer and i know tim manages a lot of the chat going on and members of our team are actually in the chat so we've had a couple sessions with some really really lively uh chat going on the background uh we'll be monitoring looking for questions along the way uh so not a typical webinar where we present a bunch of slides and come back uh we'll be trying to do these things um all the time so uh and then finally and actually there's the first question is will a recording be available absolutely so uh we will make the recording available uh typically dan on the team gets it up there within an hour to uh under the cockroach labs uh youtube channel uh but we'll send it out to everybody as well so um so with that thank you all for joining uh and uh today's session is uh you know we've been asked before like can you tell if these things are basic intermediate or advanced you know i think of this session as more of a basic uh conversation we're not going to get into some any command line we're not going to get too deep into distributed transactions like we've had in the past um this is a little bit more kind of higher level stuff workloads kubernetes definitions of these things uh where people are getting tripped up on these things uh and and and talking through those sort of things right and so um but please do ask questions we will send mugs out to people i have a mug around here somewhere i model it but uh so with that let's uh let's bring the cameras on so uh tim and scott if you guys want to join me there you are tim and then scott see so everybody in the crowd it's uh it's it's it's it's it's guys rain here and uh flannel shirts today that's final shirt wednesday our diversity is really low and i'm sorry everybody i will endeavor to get better this one in particular sorry um but i i promise that the conversation will be lively uh and and uh you know we'll keep it to some really interesting topics here but um you know first and foremost uh thank you scott for joining us um scott mccarty is a principal product manager uh in charge of containers at red hat so um scott what do you work on at red hat you know i mean that's i give you a title is like what four words uh what what are you what are you working on at red hat but it's all about the syllables like if you're a principal product manager that's like what seven eight syllables and you throw containers in there's another couple there yeah another three syllables you're good to go yeah i was i used to have a title that was 16 syllables i thought that was way cooler but i used to make fun of myself much more um so in this role i uh i live in what's called so so there's two main platforms at red hat if you will like you know there's open shift and there's rel and i live in sort of this nexus in between the two where where my team our team builds um technology like uh cryo pod man build a scopio we work on run the c we work on like all these low-level bits um i also manage the road map for red hat universal base image so sort of all these primitives is what i call them they're like the basic flower sugar eggs and water of containers that we basically build for red hat i should have worn my rocket t-shirt today um knowing knowing that what you work on um so anyway we'll we'll come back to that though if anybody did yeah we consider we consider podman the sort of spiritual successor of rockets it is it is it is but i know but the little rocket logo was so great and everything um scott how long were you at uh how long have you been at red hat i have been at red hat nine years yeah that's awesome you know it's amazing you know you there's the tenure is long i've met i've met people you know dozen well over a dozen years what is i guess red hat's been around for how long 20 years now it's we had on our company call today uh mike evans passed 25 years so we've been around since what not you know over 25 years i don't remember the start date 93 i want to say 93 was the start date so i guess i can finally say like there's some people who have been there a couple dozen years so yeah but you know to me you know one of the most important companies of my generation and all of our generations in terms of you know i think all three of us here are um open source advocates and zealots and all others i you know i don't have a neckbeard but i definitely love open source so um so so thanks thank you scott for joining us and then tim vale uh you want to introduce yourself well hey everybody uh tim vale here head of solutions engineering at cockroach labs been here about two and a half years and as you mentioned spent a lot of time in an open source prior to that so glad to be here glad to be talking about all this fun stuff and and tim brings in an angle of you know engagement with customers every day you know both large and small you know smaller companies using cockroach cloud uh larger companies trying to deploy uh you know cockroachdb on kubernetes or you know in these kind of distributed environments um you know in a single cloud multi-cloud lots and everything we're going to talk about hopefully all those things uh today um my name is jim walker i am uh i i'm in product marketing here at cockroach subs i am just a week short of two years at this company so man scott you're just way beyond me in any sort of tenure um but that's okay i you know i was at coreos before this so i kind of almost could have made it to red hat but that's okay so um so but welcome everybody um like we usually start good morning good afternoon and good evening uh please do use chat and um and uh you know we'll come back to this uh but but let's just let's start the comp not thank you we're not we're not done yet so i'm gonna stop sharing so that it's that it's us on here so um so scott your your title is principal product manager of containers so when you have to describe what a container is to somebody you know and let's just say it's uh you know the the the the most highest level what is it yeah so it depends on who i'm talking to for this audience i'd say it's probably more technical on it so i describe it as even the simple technical definition i'd say is is it's two things there's there's a runtime component and then there's an at rest component so i jokingly say that like containers are fancy files and they're fancy processes at rest they're just fancy files on disk you know they're tar balls that have some metadata wrapped around them that you know are defined by the open containers initiative and then at run time um you know they're just fancy processes that have extra controls in place constraining them um and so i i often say like you know i i wrote uh uh uh before before red hat acquired core os you know uh i i wrote an article that took a spin on their ceo's article about you know like what like can you run databases in and you know and he asked all these different questions that he gets that are kind of i don't want to say he said they're dumb but they're we get all these questions that are remedial they're like can i run a database in in a in a container and i'm like replace the word container with process and then ask that question again right and i run a database in a process and you're like well i don't know any other way that you can run a database so like these these if you use the right mindset you know these questions they're pretty easy to solve themselves you know answer themselves but uh so i joke they're just fancy files and fancy processes and then there's one other component there's a registry server it's just a fancy file server and other than that that's it fancy files fancy processes fancy file servers yeah i mean nothing nothing really new i mean i guess it's just i mean name spaces have been around for a long time here y'all so like you know this is just something if anybody understands you know linux it's a pretty easy kind of transition to do this so you know but scott if i think about containers and your your title right is principle product manager for containers and i know there's container platform and kubernetes engine and there's openshift is there anything to innovate in containers or are they just kind of are we at are we at steady state and basically you know uh you know the oci has defined this and you know docker did a great job of pushing everything you know are we at steady state with containers well so for me no i mean it's funny because like my day in and day out i'll admit i'm down in the engine room i jokingly say you know most of the time these are things that like a cio won't understand what we're doing they have to just kind of trust that we're constantly adding new features and making it better but they don't really necessarily understand what that is um then nor should they need to that's kind of why you buy something like open chip but that said no my my days are fairly intense um you know we're always i mean there's a there's a ton of features that we're still adding in cryo and pod man in particular like you know i'll give you an example like what we're working on right now is image mounts so something docker never thought about is and nor should they have needed to i mean it was just it was new but you're like oh i want to fire up one container and then i want to modify another container with that so can i bind mount in this container and then like scan it analyze it or maybe even modify it and so we're we have these two features that we're kind of working on in podman called overlay mounts and then also image mounts and so you can imagine this is the flour and sugar and eggs and water of bake you know the cake that is containers you if you like if you really look at the way a container works it's just a bunch of overlay file system layers and you know the all of them are read only except for the top one and the top one you write to and then when you save the container you just add one more layer it's actually not nearly as complex as people think um and i have diagrams and graphs i go into these container internal labs if you have google container internals labs you'll find all kinds of deep dive information i give but in a nutshell once you learn that that's kind of how containers work you realize you can kind of do it inside of a container as well so like we're looking at like how can we bind mount a container image into a running image you know into a running container and then like maybe you do a read you know a read write layer that's like a temp ffs and then when this container goes away this one goes away too but we can scan it and add things and you know there's all these like basic use cases where you want two containers to be able to talk to each other in a like a really deep way through like a posix file system yeah you know essentially that's like that's like one of the things we're doing right now um we need this for like for example anchor and and twist lock and aqua sec and all these security scanners they want a way to like bind mount in an image analyze it read only so you can't muck it up but but you know but know what's going on and then save all that data back to a database somewhere that's like the perfect use case for that yeah these kinds of features we do all the time and i think it's just kind of one of those things i mean as we become much more mature for with distributed systems i mean look at really who was this new and distributed systems five years ago okay like really a handful maybe of companies and and if some of them probably weren't doing it very well uh you know you walk around cubecon today and wow there's a lot of people interested in this stuff and actually pushing the bounds of these things and you know i think pulvey and and what the what the team did at coreos to actually help drive some of these core initiatives of you know like you know they had rocket which was their container run time and i know you know there's been you know competing versions of this but like to think of what an operating system looks needs to look like in a container i think you know that to me that's what's the to me that's the that's the interesting piece right like you know what what you know you like yeah these things are going to run on an operating system people don't realize like in that container is a an os right let's shrink the size of your container right and so i presume like just that that like linux level of expertise is kind of where you're living right scott yeah it is and it's weird because there's like sort of two competing narratives one narrative is the os doesn't matter anymore but then when you dig into the covers you're like wait a minute actually like it matters more in a lot of ways than it ever has you know and so that's a strange part you go how did coreos get sold for 250 million dollars if the os doesn't matter you know like and how did red hat get sold for 34 billion if the os doesn't matter you know like there's there's definitely two competing narratives depending on where what your interests are and goals and you know desires are in the universe yeah i mean the os always matters i don't care what anybody says even with functions as a service i don't see how it does yeah right i mean like it's gonna matter yeah like really serverless it's still servers and there's still an os running on those things and i tell you what there's a whole lot of optimizations that happen it has to happen at that layer to actually make serverless a reality in my opinion i think we are only scratching the surface of what that world is going to be and you know what is a truly serverless environment to me it ends up being something kubernetes like if you was if you will almost got where it's you know you are you know communicating at that layer um for all those things to happen right so yeah absolutely and then yes there's sort of two sides in my world there's the container images side and the container host side there's kind of the two main things that work on the container images side we're always looking at things like serverless and uh distro lists and things like that and like both of these words bother me like serverless and distributors they're neither neither of them are actually without servers or without distros like just because you don't put a package manager in a container image it does not mean that there's not a cadre of human beings going out that are subject matter experts in all these different pieces of software and packaging them in a way that is consumable like that's still the same business requirement even if you don't use rpms or maybe you don't use a you know manager but somebody has to still go do that work for you and you need to be able to consume it easily is serverless is just somebody else's servers and then that's what the cloud is right it's just somebody else's servers right like ultimately so uh and there was a lot and i joked realist it's just somebody else's distro you'd even if you look at like the google serverless stuff a lot of the different build languages rely on debian packages yeah they don't include the package manager and you can't rebuild them but they're still pulling from that dependency tree right so it's like dependency trees still matter even in 2020 you know yeah so so let's one other thing that's related to this and and you know i i think some people are newer to containers or aren't familiar with kind of the complexity of you know you and i talked a little bit before this about the supply chain and and how do we get applications to production you know there's also things like ansible or you know like you know core asked we had quay right um you know can you just just describe to me what what those tools play like uh in this whole kind of supply chain because i think they're actually pretty important for people to understand right because the point is not easy yeah absolutely so this is probably a good segue into describing what openshift four is versus three okay so so you know historically like if you look at openshift three we used ants we basically had a kubernetes distribution openshift we had ansible deploying that kubernetes distribution on rel you know red hat enterprise linux which is a linux distro and so you kind of you can kind of understand all the primitives there right you're like oh it's a configuration management system a kubernetes distro linux distro and you kind of wire it all together you do upgrades you know like that's pretty easy to understand that's probably how the whole world did things you know you go back to 2005 i was using a homegrown cf engine that we had written at american greetings doing e-cards you know and i left there in 2005 uh but but you know i called that web 1.0 that was the tail end of web 1.0 um e-cards are still making more money than you think even in 2005. but uh but um you know we did do distributed systems with cf engine right like we were doing with a cf engine equivalent you know that we were basically pushing out to a thousand linux web servers um the business problem there is the same right it's running 75 different services across a thousand different nodes and we if you look at openshift 3 we're kind of doing that the same as we'd always done except that we were deploying the kubernetes distro so like you had one workload that you needed to deploy to all these with the config management system and then all the other workloads where the application workloads were deployed with kubernetes yaml you know you could think you'd hit that cube api and deploy all of your stuff there but then when you go to opengf4 we actually kind of extended the api down a layer to the cluster itself so i don't we i'll admit i don't think we even red hat have done a good job of explaining what openshift 4 is but it is a profoundly different way of thinking about kubernetes it's you treat the cluster itself as you treat every object in the cluster including the nodes as objects in the kubernetes cluster so like it's essentially extending that that you know wait let me back up and say config management works on the concept of like you know defined outcome right like i want this thing to have user added or whatever like right make it you know i'm you know item potent but do it don't do it 20 times but add it make sure it's there you know kubernetes works on a very different paradigm where it's like defined state actual state and it's constantly it's got a timer going you know a controller it's basically constantly looking for looking at the defined state looking at the actual state comparing the two and trying to like make them look the same right so that's a very different paradigm because you know config management might run on a deploy and then maybe maybe if you were super brave you'd run it once a day or once an hour or something like that kubernetes is doing it all the time like all the time at any given you know you know times per second you know or minute you know it's looking at the state managing the cluster itself as part of that defined state is what openshift 4 does so you manage the nodes you manage the container engine you manage everything so if you look there's this thing called the master uh or the mco is what we call it and it basically is an operator and an operator is basically just an extension of kubernetes that takes that defined state actual state and applies it to other things right and so we're basically taking this mco and applying it to the node and the container engine that's running on that node which are the kind of two main things you need to configure on a on a you know historically we did that with ansible but now we're moving it into the cluster so now there's a single api you literally hit the kubernetes api uh this rest api and you can manage everything from the node to the engine to the containers that are running on it to the state of config files on the cluster you're treating the cluster as a single computer now through a single api endpoint and so that changes those two paradigms are very very different so then that changes not to be super complex but you're to answer your question what roles do these things play they're different in each of those worlds that's right that's right and i think it all it all collectively kind of comes out as open shift and the value in openshift is this greater simplification this abstraction up away from kubernetes basically and and you know these kind of crazy configurations and you know i don't like to write yaml i think i find it to be you know look at i avoided cobalt early on in my life i don't like to count spaces um you know so like i think it's kind of one of these things that i you know why deal with any of that i don't you know i want command line but i don't you know maybe i'd maybe a bit of a ui to manage and and to to do all these different things but you know for me you know what you're all doing with openshift is is truly tremendous and it's all in open right i mean it's all open source right and so there's kind of this boundary though between kubernetes and what red hat is doing in openshift right it's a very tightly integrated kind of you know platform what's happening here how do you guys choose what happens in say the kubernetes community and you know with the stuff that clayton and that whole team is driving and coding away on and then what the openshift is seeing and you're a product manager you kind of sit between the two sides right and so how does that work internally with you all so this is this is actually touches on i i don't know if i shared i shared the link i don't know if people that that that join can see it in the chat or not but uh this touches on this concept of open source as a supply chain so yeah i think there are fundamentally two ways that people look at open source you know there's sort of the so let's go three there's three way there's companies that look at technology as a closed core you know and i'm borrowing uh you know acquaintance of minds of terminology very specifically there uh you know joseph jax calls it closed core you know it's a it's the idea that we're gonna build everything proprietary and we're gonna deliver all value to the customer through this proprietary means right then there's debate about what does open core mean i'll fully admit there's an open debate about what does that mean right red hat is at the other end of that spectrum where essentially the vast vast vast vast you know 99.9 of our supply chain yeah is open source i i leave room for you know a long tail there of there's little pieces of glue code and things that exist in our environment like our build systems and you know there's things behind the scenes that aren't necessarily public but it's not because we don't you know it's not it's not you don't need them to run openshift at runtime or anything it's just you need some glue code here and there to make your specific environment work um you know so so i'd say this becomes a supply chain choice right like like if you look at if you look at openshift it's made up of basically there's sort of two major pieces of the supply chain well actually i'll say there's kubernetes is a huge chunk of the supply chain for it i would say that's like the motor right like in a car let's let's make a car comparison the motor is kubernetes like you need an electric motor or a gasoline or a diesel motor you have to decide this if you're a product manager like you have to decide which motor meets the needs of my customer right you go my customer wants to pull trailers so maybe we only have diesel and electric as an option you know maybe gas isn't good enough um these are the kinds of decisions that the pm makes right and so then you look at openshift i'd say kubernetes is the motor and then you look at all the things that run on top of kubernetes um you know well first let's start with what's below it red hat core os below it that is you know you look at the supply chain it's fedora then rel and rel core os is basically just a different snapshotted version of the exact same bits are in rel so the supply chain is linux you know is the motor for that and there's a whole bunch of thousands of other projects around it you know for the door handles and everything else and then you come down into a distribution like fedora then down into rel and rel corels so you could you could start to imagine in your mind a map of like this crazy deep web of you know all these different projects that go into openshift so the supply chain for openshift is basically kubernetes linux upstream you know major motors and then downstream you go to okd okd is sort of a again an open source freely usable you don't have to pay a subscription to use it you know sort of interim project think of it as almost like a distro kind of like fedora where it brings it all together and then and then there's openshift so how do we decide what goes where you know like that's the biggest question right yeah basically if the kubernetes kubernetes you know i this is what i try to talk about in my article about like what is open source product management like your upstream supplier should do something different than what you do like if your upstream supplier sells fuel injectors and you sell fuel injectors you're gonna have a problem like like you guys are never gonna be able to differentiate each other and you're never gonna make money if the upstream supplier sells a fuel injector and you sell a car it's a lot easier to differentiate so yeah you have to look at the use case right like the use case for openshift is it's a i always joke it's a it's a 200 you know it's a it's a dump truck that can go 200 miles an hour and it carries 10 tons of dirt that's not something that everybody needs but if you need a dump truck that goes 200 miles an hour and handles pretty well and carries 10 tons of dirt that's an interesting use case right like it's a it's an advanced use case that basically people that don't want to have to manage the entire supply chain themselves handle and so we decide based on what the need of the user is essentially yeah yeah and so let's let's abstract that exact point up scott and so you know yes there's the internal workings of the car and like fuel injectors and the engine and the drivetrain and the you know the chassis and all these things and and and i think those things are important and i think what openshift is doing is pulling all that together so you get a car yeah i'm sorry in this case a pickup truck right and so yeah you know what what you know if i was to just you know i'm new to these things like you know you see you say it's a pickup truck you know how do i work this into my you know my my architecture today right as a as an enterprise organizat what am i using it for like what's the value i'm getting out of openshift and kubernetes it you know for for at that layer right like without getting too deep into the weeds like you know what are the workloads people are using we'll come to that as the next thing but like what what is it what's why do i want to buy this thing ultimately yeah exactly well and so to be fair it's not that the workloads are necessarily different although they can be um you know the user of object wants something that just works out of the box and will upgrade for you know 10 years you know whatever seven years you know in a row like you're not gonna get that value out of the upstream right like if you're upgrading kubernetes every six months you're gonna have a lot of work on your hands the bottom line is that's just a lot of work for you to do and that costs money that costs engineers and engineers are hard to hire and so it's pretty natural where the the fit is right like we want to make the experience a life cycle fips compliance security you know so government organizations big you know large enough enterprises that they they would try to tackle this themselves and realize that they don't want to do it because it it costs a lot of money and it's these are very hard engineers to get a hold of um you know i'd say it's more based on like a business understanding of like do i really want to tackle all this myself right do i want to buy a bucket of parts and build a dump truck or do i just want to buy a dump truck take it to you know ford and have them service right and i think and i think that's one of the things that people struggle with with kubernetes right there's a lot of pieces it's complex i have to deal with storage and networking and security and all these different things and i think what what openshift does is simplify a lot of that for the end of the pre-built solution and you know exactly and like having a package solution around those things we used to do this with hadoop i mean could somebody roll their own hadoop distribution eight years ago whatever that yeah sure you bet like why would you do that right and then you need indemnification all these things and i think exactly intelligent companies like like a red hat does it you know i think one of the other areas that that you know you you just touched on very clearly and people are asking you know like why why is this important you know back at core s this whole concept of tectonic was how do we do rolling upgrades and how do we automate upgrades of software right and so there you go you know upgrades is probably one of the biggest challenges of kubernetes that's right yeah i mean because each pod has to be updated and there's different layers of that have to be updated in each pod right and then the control plane itself needs to be updated right and i think that's one of those core benefits and you know talking about the operational complexity of kubernetes well just understanding the damn thing took me a while to figure out right like running it in production is it's difficult and i think that's where the value for you know i i that you know that to me scott that's you know then again i'm not the product manager i see on product marketing see this is how we work together buddy well and so the funny part is like you would never you would never find a business in the fortune 2000 the russell 3000 or 2000 that would build their own linux distro right i mean maybe in if they're if their business is like you know embedded systems where they're doing some very very specific thing but you're not gonna find anybody doing you know building their own linux distro to run sap on it right like that just doesn't even make any sense whatsoever and i think it's a little bit easier to see with a technology that's a little bit older and more mature because you you start to realize you're like oh yeah again like the perfect car for a family would be like mom and dad build the car specifically for the family with the exact size seats for every family member's butt and you know like the exact safety request nobody does that that's irrational and like and not that's not efficient right and so the market provides sort of an approximate solution that's pretty good that's worth paying for essentially like mom and dad would rather buy a minivan than go build a minivan that's perfect for their family like this is so obvious with cars it's so obvious with linux but it's still not completely obvious with kubernetes because it's new and people look at it and they go they go oh this is strategic to our business so we want to build it and add all these things to it and then maintain over time and like one to three years into that they go yeah this was not necessarily what we actually wanted to do like we would actually be better if we were building stuff on top of kubernetes that actually met our business needs as opposed to building kubernetes right right exactly and i think that's where people are struggling right now they get too deep in the rates right i mean tim you see people out there using kubernetes all the time right and so you know what are the workloads you're seeing kind of people shift into kubernetes is it all of them or is it you know and what's that journey look like for them yeah you know it's it's funny i mean i i think i think kubernetes still for the folks that we talk to is is a bit aspirational um yeah you know it's on everybody's roadmap and certainly those who are are are doing it are doing it at the app layer so you know customers are certainly talking about well you know i'm moving my application workloads you know my microservices to to kubernetes uh type architectures but obviously here cockroach labs what are we selling you know we're selling the database and and we want to um want to and can run cockroachdb on kubernetes and so you know we're starting to have the conversation of people not only not only using kubernetes to deploy application servers but to start to deploy you know a storage layer and so that that kind of really at times makes people's head kind of explode what do you mean i can run a database you know as we were talking about in containers well sure of course yeah i can run database and kubernetes absolutely uh for all the same reasons that you'd want to run it in your application in kubernetes you know yeah and so but yeah we're still not everybody's there yet but they're getting there they're definitely getting there and i think it i think it's it's one of these things we're going through a paradigm shift like literally it's going to take all of us time and and you know scott you're in it right like you're like when you're deep in it you're you're just there whereas like there's a mass majority of people just don't understand like like what does what what is the the the result of distributed system for you and and what we're trying to do and the complications i let me let me shift a little bit into the data and and you know what that actually means in these systems you know the first time i ever saw cockroachdb and here i am now with this company a few years later uh pulvey from coreos was up on brian i forget his last name the guy who runs openstack i forget i forget the guy's name he was like the president of the openstack consortium or whatever forget what it was got but we're at openstack summit yeah they were at openstack summit and they wanted to show off like kubernetes and like how you couldn't kill this thing and what's what was the distributed application well it was the database like holy cow like i could have this distributed database right and so i think we're seeing people think about kubernetes you know when it first came out people were like oh it's for stateless workloads what is that i don't even know what that means you know what i mean are you still hearing a lot of those same kind of concerns from customers scott like where am i using this thing how am i going to use it like that's great i i trust you red hat and like this openshift thing's awesome i'm sold our customers struggling with like the workloads and and how to how to instrument and how to make it all happen i i presume that's one of the reasons why openshift is here in the marketplace and all that right so it's absolutely so so one of the one of the most popular talks i've given over the last five six years has been my migration talks like like if you look every generation we've went from like bare metal to virtual machines i mean we can go back as far as you want mainframes to mini computers to unix boxes to linux to virtual machines but but there's always been three options right there's lift and shift like can i just take this thing i have and run it on this new thing and does it give you some benefit right and then there's like augment you're like well we'll move part of it and then we'll add some things to it to like maybe make it work better and then there's a let's just write all this crap from scratch and obviously it gets more expensive you know you go from lift and shift to the cheapest to like rewrite everything the cio does not want to hear that you're rewriting everything so this is constantly like there's always going to be a business challenge here right and so yeah absolutely my not one of my number one conversations is what can we put in openshift or what will work well right exactly what do we run on rel and what will work well there and what should we just leave alone because it's not worth moving in and these are like yeah absolutely and they're they're really complex questions depending on the environment that require you know a lot of times it results in consulting going in and analyzing you know a whole a whole portfolio of applications and you're like these 32 run here these 72 run there you know and it it's not an easy question to answer but at a minimum i mean the thing i try to guide people is if you could separate the code the configuration and the data like whatever that means like with mysql you can separate the code configuration and data but the problem is the data is a is one thing unless you're using galera or using something like cockroach you know like if you have a single piece of data that's like one chunk that's you know uh uh a uh you know i don't you can't it's not it's not distributed you know set up to be distributed essentially have multiple replicas and things like that if it's not cloud native which we tried to teach people with openstack but i think we just ran through that wall and just people kept going like i could run it like a virtual machine now with containers i think it's finally we're at a hard place for you if you really want to run this in kubernetes you need to build a you need to be able to imagine a couple things one this thing could run anywhere because kubernetes can decide where it can run it whatever it wants at any given moment and then two it might restart it a thousand times so like you've got to be able to kind of handle two things right like can move anywhere at any time and restart a thousand times like apache can handle being restarted a thousand times an oracle database not so much like you know like you restart an oracle database a thousand times on thousand different nodes that's probably going to create a problem good luck so something that like something like cockroaches designed for that is going to handle that a lot better than something that's not designed to do that oh no and i think it comes back to like the core principles of distributed systems right like build it as a small single atomic unit so it can be deployed the same way everywhere right um they're they're they're built for scale so they can easily scale them so they communicate in that way um they will survive at all times right and i think those are those core principles come back you mentioned something scott i found kind of interesting you know it's like look at i can lift and shift i'll take the thing vm and all dump it into a container and run it right i can make some small changes to it augment augment and kind of like okay maybe it's kind of distributed at that point you know whatever that means yeah right maybe we run part of it outside the cluster part of it in the cluster right exactly and it's like or or do you just re-architect from the ground up and you know here at cockroach labs we chose to re-architect from the ground up and you just kind of set something off in my mind and and i kind of realized why i joined this company is because if you're going to run a database on kubernetes you have to re-envision it it's it's not as simple as just running it you know mounting a persistent volume using the storage class and figuring that out you if you try to augment or you try to lift and shift it's not built for that world right and so you know we realized that with the database is this some of these things like if i'm a consumer of openshift or i'm going down the path of kubernetes like maybe my application is just not built for this new world is is that a question that organizations should be asking themselves oh yeah absolutely absolutely because if you force a round peg in a square hole it's not going to work well and you're going to end up with more nightmares you know like i said cram an oracle database on on on kubernetes and see what happens you know like right it'll work great for a few days months years you know until one day kubernetes like runs into some problem where it can't schedule it and it restarts it 200 times and then the database gets corrupted and you know you just end up with these crazy problems right because it wasn't designed to run in a distributed way and then i kind of somebody was asking a question was like what are the challenges of moving the database into kubernetes and i think that's exactly it scott right like if you're going to run a legacy database like you want to run mysql or postgres and in in a pod okay great but you need to understand that pod is ephemeral and it may die and so even with stateful sets which is a kind of a key concept in kubernetes if if somebody doesn't know what it is i would check out stateful sets a very interesting way of actually maintaining some sort of state when a pod dies right the the simplest explanation right um without that you know it was it was a complete lost cause but even with that it's still a problem right because like it still won't handle 100 restarts that's the problem that's the other problem i tried to explain it's not just about where it lands it's also about how many times it lands there right now there's there's there's time and space like i always try to explain to people there's time and space of kubernetes and you have to decide like can this thing handle being moved anywhere in the universe a thousand times per second you know that's a different problem set you know has to be designed for that now there's things like apache can handle that pretty well you know you're right like if you're you know maybe you have a very small website a blog you know like it can probably handle that with just a mysql database have it but you do that at scale that's probably gonna fall down you know that's where it starts falling down yeah and it really it's choosing the right workloads to run in these environments and where it makes sense where are you going to get the efficiencies and why do you need that efficiency i think is the biggest question right and i think that's where i've always kind of had these conversations about workloads and kubernetes and openshift and all these things it really comes back to that um going back though and just talking about applications moving them over and this sort of thing there's this concept of an operator right and a kubernetes you know rob zimski a friend of mine i love rob rob's another brilliant product manager at red hat and you know and just a really solid human um totally you know and i remember brandon phillips rob and i having a conversation about operators actually i think brandon wanted to call him controller operators and rob was like controllers i'm like no man it's just operators like you know d should be people be thinking of that like is that a is that the conduit to make your app work in in in kubernetes you know or is that just basically something that you know the software engineering teams that are building things that are going to be deployed they should be thinking about right like you know what i mean so like is is this something that's like consumer grade or is it more like you know the the industrial grade supply chain you know vendors have to think about that i think it's both i think both groups of people have to think about it i mean that's that's probably again one of the biggest conversations i've had over the last few years is like explaining to people what operators are the way i've broken it down and i've bumped this off rob to see if he was cool with it but like you know if you think about traditional operating operating environments you know you know operational excellence what did it take to achieve operational excellence you deployed an app on a server and then you had a human being also called a sysadmin that would make sure those things two things work together and that's how you achieved operational excellence right in this kubernetes world you really need a robot sysadmin called an operator that now you deploy the app on the platform you know and now it's a cluster treated as a again kind of an openshift for paradigm is that you could treat the cluster as a single computer basically but you're spanning across a bunch of nodes um so it's the it's the platform the app and then a robot sysadmin that you deploy side by side with the app that takes care of the app so like you know the operator can do things like back it up restore it you know check on it if it's broken you know i talk about like if it restarts 200 times and the tables get corrupt the operator will know what to do right like the operator is what helps handle you know cod all these sort of i call them cloud immigrants they might not be cloud native but they've immigrated into the cloud and so that's right you know you know kind of coddling that that's what the robot system operator does and so the knowledge now moves from we used to have that in a run book and a wiki and instead of in a wiki we now moved it into the operator right like yeah and like for a while there has to be able to do that work now that's right and for a while there we moved it into the sre's head right and so the sre was basically going off and building scripts and like you know sre being kind of one of these artifacts of google and kind of their whole approach you know again it's like is this giphy which is google infrastructure for everybody right like yeah kind of you know and i think the operator was codifying i think so too some of the things that the sre was doing because you know doing operating these things at scale are incredibly difficult like how do you actually manage certificates in a distributed environment across every pod and make sure that everything's gonna be sorted out like okay vault's gonna probably solve one problem for you but like uh you still need to distribute and make it these things are complex it's about velocity in my mind so like if you looked at the velocity of the standard application you could fix it once a day once a month once a year you know i had i had a server that we booted up you know put rel i forget what it was six or seven on it and it ran you know probably five or six sorry and you know it ran for 10 000 days or whatever it ran and then we shut it off like it was it was ridiculous it booted once and it shut down once we never had to reboot it even there was never been a remotely exploitable kernel bug so we never had to reboot it like this is what people don't understand that kind of velocity is very different than the velocity of i might need to do this multiple times per second or minute you know like and so when you look at when you looked at the way config management works in a traditional environment if you if you're if your for example your monitoring system notices a web server goes down your config manager could go out you literally could have nagios talk to ansible to go out and restart the web server right like this would happen once a month once a week that's fine like that feedback loop is slow enough and and you know it's there's enough slack it's kind of like a 1960s car it's like you know like the gear you find the gear and you can feel it grab and you're like but that's what makes it fun that's fun right like it is fun but when you're talking about inside the kubernetes cluster this defined state actual state methodology is it's an ooda loop that's basically happening all the time and so the idea is you're deploying the operator in this tight ooda loop you're now in the reactor right like it needs to be able to happen and react at the state of the kubernetes database so at the speed of the kubernetes database state change right and so that's what that operator does you deploy the automation inside the cluster and it has access to the state data unlike you you know so historically you had a monitoring system a fault monitoring system a config management system this and then the underlying platform and all of these different disparate systems had to talk to each other and they didn't share a database but if you deploy all this in kubernetes it's all saved in the xcd database and i think in my mind that velocity that that can achieve you know of restarting a container within seconds of it failing you know is a different animal than than doing it you know oh well it took us two three four minutes to get this thing back up okay no problem yeah at scale with capacity that doesn't work yeah and it's it's interesting you talk about state and how you use that in the context of the operator is funny when we you know when i first joined cockroach labs you know i was we got to build an operator and i remember you know guys on our team that understand kubernetes like what are you talking about we don't need an operator and actually because we were designed as a distributed system that can actually natively you know survive a failure of a pod or region or whatever like what it because you can do that you don't need a lot of those kind of core operator things that that some applications may need because you got to think about scale right or resilience right scale is how do you deploy and actually get all these things to work right and and or rolling upgrades like these sort of things are not always easy to deal with you know how do you apply patches to a huge one yeah you know how do you how do you how do you apply patches and i think you know those are the core patterns that i see in operators and and you know what we have on operator i know it's it's uh we're container certified and then our operator is now certified by red hat as well probably you know pushing out some news before cubecon but what the heck the people on the thing will know right um but yeah we did it because well there's certain things there are certain deployment patterns that actually are difficult to do and it goes beyond just being a distributed system and i think that's the kind of stuff that uh that i think people you know upgrades yeah i'm sorry it's a perfect example yeah oh no that's gonna say you you you really struck me when you say upgrades upgrades is probably the hardest thing again go back traditional environment and imagine upgrading a mysql database from like i don't know a major version you know three to four or whatever you know and you go you go that's a lot a lot of interaction like you've got to like shut the database down back it up run some kind of up you know install the new version of the software before you do that you know how do you do that like which order do you do these things in like like you have to like shut it down export or you know export the database then shut it down then remove the software add the new version or maybe install them side by side then you've got to like run some upgrade script that changes the schema then you've got to like stop fire the new one up oh it didn't work oh now how do we get the schema back an old version to restart that's not easy god that's that's one instance that's a stovepipe database yeah that's one instance start talking about you know i have 50 containers all right that's why you had maintenance windows that were well eight hours long what i'm gonna ssh into each one of the servers and start figuring this out and do it like you know so like the upgrade process applying patches which apache you know patches are so incredibly important today you know this means security problems are just this is massively important so um like you can imagine like historically assisted men would go and fiddle fart with all that figured out then we got to automation where we had like an ansible or a shaft for a cf engine we could kind of codify those rules right but it still took i mean do an ansible upgrade with something it'd still take hours you know sometimes to upgrade things like you don't have hours right like you've got to go figure all that out at the factory and then deploy it in production like i joke i jokingly say containers are about building at the factory not at the dock right like like historically you go back to 1800s we take all the lamps and pianos and barrels and boxes and crates and like literally on carts you know with horses and we take them down to the dock and load them on and it took freaking three four five days it would take months to load a ship people don't understand this like a month and then we ended up getting the containers and you're like we loaded the same amount of stuff that's just as complex because we did it at the factory where it was air sealed and cleaned that's right and we and we did it in salt water with open air like just put it on there turn some twist ties lock the thing down that's the idea of an operator like the operator is able to like handle just these final last mile operations with access to the state data of what's going on in the entire system at the same time but since you have that unified state data it's like a robot system that has access to the it's like if i hooked my brain into like you know a distributed system and i'm like oh i can feel the state of all the things you know it's the matrix right and you didn't have that kind of power with a regular configuration management system but you do with an operator because right and it can check all these things in real time and figure out what's going on right and honestly it was a you know i if i think back a couple years ago it wasn't that kubernetes couldn't do this it's just that kubernetes has its own layer and it has to do what it does which is basically let me manage state ultimately kubernetes is one thing it it it listens to fcd and says i need to have this is the state that i need like that's it like literally if you're gonna break it down in the most simple thing fcd is just all it does is the database it says here's my state and as you said scott i need to make sure that the state looks like this it means five instances of this six instances of that 120 instances of this piece and and that's all it does it's really all it does ultimately in the end right like there's a lot of complexity and there's networking and storage and all these things that have to happen right it handles defining the state really well right now managing the state changing the state right exactly and when there's things you gotta do in between the state change that's right like that's right and i think you know if people think about operators and like where they fit that's how they fit there's the thing that's basically controlling and making it it is the the uber robot and then you got a robot controlling the robot right i like i like it it's robot sre is what this is so that's what it is it's a robot sra and then openshift4 is basically about having robot sres for both the applications which is what regular operators do and then we've also written system operators that manage the host and the engine and then all the other software that runs in openshift so like it's actually quite simple once you really let me do my job and make my my robot overlords happy as well um let's talk about the red hat marketplace and um and what you guys are doing there and i think you know people are asking a lot of questions about like complexity of applications and are we just basically building things because engineers have free cycles or you know like what what is the value of the marketplace and you know we're we are definitely in the marketplace and we're seeing you know adoption through there like what what is the vision and and what's going on with openshift from that point of view so in my opinion the marketplace is like the final culmination of what we've wanted for 20 years like if you go back when i first started doing this like 98 you know i i did there wasn't really a concept of cloud but automation was already like even in 2000 2001 we knew it automated unix sysadmins knew what automation was right right but the problem was is everybody built automation for their specific environment everywhere right like and so if you really think about if you had a thousand different companies building automation there was a thousand different automations they were building and so it was a thousand times the amount of work that you need to do yeah and then i would say by like 2005 to 10 even not you know 11 12 we had open stack and we started to see the writing on the wall they're like wait a minute i can interact with the api and then i've got to build the automation once and i can deploy the automation at the thousand sites but write it once and you know we could see it but but it didn't quite go to the point where you could like select the automation that you want and deploy there was still a baker in the you know there's a chef in the kitchen that was still choosing the ingredients and figuring out how to how to make the food but there wasn't a menu right like there was no menu you couldn't just order it was like cooking at your house and i saw a lot of projects fail because i was actually a solutions architect at the time for red hat and i remember adding one customer in particular they had this wildly ambitious plan they used servicenow and like all this crazy homegrown automation with like ansible and chef and puppet or whatever they were using and uh you know they had like all the moving parts of what's in the marketplace today but there was no standard way to do the automation like it was still an open system that didn't have constraints placed on it and i'd argue now for the first time since we've standardized on kubernetes we've kind of standardized on this operator framework which really is a framework of like how you do the automation you now have the ability to standardize a marketplace like this is like options trading like options are standardized you know there's a set price and a set date and so you can trade them in chicago like you can trade nobody knew that in like 1776 nobody's like oh we'll just trade these options i'll i'll buy something at a future time like we realize that if you set the standards you can actually trade these things on the market that's what we've finally done we finally figured out how to get software into a market and trade it essentially for dollars you know uh because all the automation is standard and so now you know if you go there and you look at cockroach gb you click on a button it'll install and you know it will install because it was built on specific versions of openshift and so you know that will work like you know that's that's we're really close to the dream you know if you will the dream state we're finally to the electricity where you just buy the electricity it actually works that's right that's right and i think it's just taken time you know i've i've you know i've contended over the past five to ten years what we've done is we've basically abstracted out all the components of the software delivery supply chain from from the fingertips of the of the developer all the way through to the the mouse click of the user and everything that has to happen in between we finally have codified that into its constituent parts and it's all api based so that you know that entire supply chain just as we've done in the past with with a physical supply chain like a wool jacket that you're talking about right like you're you're you know like the time it takes and all the components from the raw materials uh to to to to packaging to to getting it in the store putting a price and a skew and all we basically created a a very well greased automated supply chain for software and i think kubernetes is just basically now expanding that to to even deeper depths of global uh you know deployments and i think doing that has has made huge changes and it wouldn't be done without you know the likes of the companies like the chefs and the then and puppet and like those were huge massive stats everybody that you know like the cloudbees team and what they're doing in ci cd is a huge piece of this right like i think about launch darkly and what they're doing with future flagging like how cool is that that we have that in the supply chain i feel like openshift scott is is the culmination of a lot of those things it it's it's it's ford versus you know the one-off car right like i mean it's kind of the you know that that's where we're headed right it's like the the full like end-to-end production facility correct yeah i i liken it to like swatch people don't know but in 2000 like 10 ish they created the first automatic watch that was touchless from from the plastic to the time it rolled off the assembly line and you literally put it put it in a box and everything and you just went in the truck and it went to the store there was historically for like 500 years we've been making watches or whatever it is there always had to be a watchmaker the closest they'd gotten it like into the 2000s was having a watchmaker fine-tune it at the very end and they'd right but they figured out a laser way to do it now that it's like completely touchless manufacturing what we've done is that touchless manufacturing but like we've instead of just for one company for one product line we've done it for an ecosystem of that's right you know it's like now cockroachdb and even your competitors and your friends and allies and all these different like pieces of software they can all build on the same the same essentially assembly line that basically allows them to have this touchless manufacturing and the trick is you know it's all going to end up in different stores and in different countries and all these different things so let me i'll ask you a bit of a directive question i'm a i'm a believer in this thing but you know do we live in a multi-cloud world is that basically the future i think there's some people who say like no way it's never going to happen like it's too complex like i'm a believer in it personally what what's your opinion i think we already have it here's what nobody realizes just like the cloud do you remember the narrative seven years ago when we were like you're already using amazon and you don't know it yeah right well you're already using azure and amazon and you don't know it like the cio doesn't know but he has developers in this group doing this thing and developers another group and i've seen i've seen cios do stuff as crazy as not a lot like literally working with a credit card company to limit where people spend money because you know they can't get a handle on it they can't control where everybody's going i would say i don't know 80 of the fortune 500s probably already have multi-cloud and they don't know like and so now it's going to be it's going to be like well you know five years from now when we finally realize and wake up oh we actually are using wait we're using how many seven different clouds what are we let's figure out how to make all this stuff work together right like like you know like we definitely need to be able to get a handle on this and then control the costs and blah blah blah i asked the right question you just almost threw your microphone right off the table dude i mean like scott i'm passionate about this one i i'm super passionate about this one in fact you know we created a multi-cloud conference last year called escape and i know you know the red hat team was extremely supportive of it because i think they share in the vision you know i personally you know my you look at i've been an open source person for a while and you know jim whitehurst and paul cormier you know the leaders of red hat have done such a phenomenal job actually understanding that the world is not homogeneous right the world is heterogeneity and it's always going to be that way and by the way the world is also open and and i think that's exactly and i think that's that's a lot of the trick and you know i you know i commend you guys all y'all at red hat like i think the the delivery of open shift and what i'm seeing in openshift for and you know i'm sorry but i'm i'm an old core os person so i got you know i bleed right like the tectonic thing was very near and dear so i mean there's so much of it it really is and and it's just really really proud to see this really coming to light and and you know to see the marketplace and it's in its reality and you know looking back to you know to my conversations with brandon phillips a long time ago like man it would be great because we would just push button and deploy you know and and by the way not even worry about upgrades because that's just all automated in the background like to me that the it's the app store of of of b2b apps that's right as opposed to just a b2c app on your phone right that's right and and you know kudos to the team um and and on and everything that's been going on but um we're at the top of the hour um scott thank you so much um that was a great conversation uh i had a hard time getting a word in edgewise but that's good it was really good sorry tim you know like hey i can just as easily be quiet on these things i like it too it was great it was awesome yeah so thank you yeah super insightful so um but seriously thank you uh thanks for joining us scott it was really really a lot of fun say hey to rob for me over there um i will and thank you for having me i appreciate it alrighty um and everybody yeah recording this will be available um there are no slides for you to send um but i hope it was useful for everybody um if you did i know uh i think i believe we we still send out a survey so please do fill out the survey let us know how we did um we're always trying to improve these things as always it's it's constant constant evolvement um so for on behalf of three guys with flannel shirts um thank you everybody for joining us today and one guy didn't wear a hat he didn't get the memo it's cool though so thank you everybody for joining us and and i hope you have a enjoyable rest of the day bye now

Channel: CockroachDB

Views: 455

Keywords: Getting started with containers, Getting started with Kubernetes, Getting started with Openshift, Getting started with Kubernetes on openshift, How to run software on Kubernetes, what does multi cloud mean, How to get my workload on kubernetes, How to use containers Openshift and Kubernetes with Red Hat, How to use Kubernetes with Red Hat, How to use Containers with OpenShift, Kubernetes Openshift CockroachDB, What workloads can I run on OpenShift, How to use containers

Id: a4CLC6JOdr4

Length: 57min 31sec (3451 seconds)

Published: Wed Oct 21 2020