GitOps Happy Hour (Ep 5): Operators in a GitOps world

[Music] [Music] good morning good afternoon good evening and welcome to another episode of get ops happy hour i am queer short principal technical marketing manager here at red hat i'm joined by my fellow principal technical marketing manager or are you senior principal i forget what you are now dude my bad joined by my fellow uh co-worker uh christian hernandez the get ops extraordinaire uh welcome to open shift tv today we're going to be talking about operators in a git ops world so christian let's let's tell everybody what that means yeah yeah let's see everyone yeah yeah so by the way thank you uh everyone for last time we had a um a good session get ups happy hours a special kubecon edition um uh we got a lot of good feedback right that's nice to get all the the talking heads the uh the um uh the the the thinking minds behind what what get out is gonna be uh red hat so uh that was really cool so if you haven't caught that uh go ahead and go to um the youtube channel right you can see all previous streams there and um yeah so today we're kind of talking about operators um this kind of uh topic came up in my head here um because anytime i talk to anyone about just cloud native architecture and just in general there's just so many um so many automation tools right there are tools they're like especially when you talk about operators and openshift because it's like well like what do i need operators i have helm and it's like well what why do i need um operators if i'm using get ops to trigger the automation it's just there's there's this whole um i don't say there's like like venn diagrams right like it's like a it's a multi it's a venn diagram that looks like a star yeah yeah yeah exactly yeah it's a venn diagram that it's like you can use like a little bit of everything how does that all fit in right so um oh and by the way uh uh chris i know chris you're you're you're drinking an ipa i'm i'm having wine because um being from california yeah ah good point yeah i'm actually uh drinking cigar city brewing from my favorite brewery in florida it's finally up here in michigan so i was stoked once yeah it's always nice when you can just get it like locally instead of right drive to florida yeah [Laughter] i ain't driving to florida no more yeah long drive now yeah that's a long drive yeah there were some breweries in milwaukee that i really really liked um and i was like you know back when we can travel um that i'm like man i wish i wish they had some of this stuff here in la but anyway um back to the topic at hand right i just kind of want to say we're the happy hour here um is um we talk a lot about get ops and how uh you know get ups in terms of like um no not like miller no not miller i'm a faux hipster so you know only microbreweries um the um you know get ups uh by pull request and um you know all that how argo and you know a lot of your sync tools acm flux does a lot of the automation for you right so but then we have operators and if you've been around the red hat ecosystem have been been around openshift a while all we do is talk about operators right so i think yeah um i think uh it would be a good time to talk about how operators fits in in the overall get ops architecture right so i do have presentations so let me share my screen here i'm not gonna um uh i'm not gonna kill you guys oops let me not slide us to death uh yeah i don't want to slide you to death i think in the um uh is it desktop three or desktop two what's weird is like i have like one two three but like on linux is it's one three two like it's really weird but anyway oh so it's funny story like you might notice if you haven't watched the show recently uh there's no flags behind me so when i moved my desk up here yeah like i wanted to rearrange some things right i wanted to put the tall monitor on the right hand side i wanted the laptop on the left hand side and yeah it completely freaked out this monitor was turned like a 90 degree view this monitor was normal so it was like i had to like scroll and everything was out of order so like just getting it back to normal took like like nine or ten tries so it was ridiculous it's like this shouldn't be this hard yeah well i like i just given up like i just let linux do whatever it's saying i think my monitor one is two monitor two is three and i don't know it's yeah i just keep one monitor on my linux box there's a reason for that multiple desktops but one monitor one monitor yeah maybe i should do that here um so you got you guys you see the presentation right these yes am i presenting the right thing all right cool um so i'm gonna kind of talk about operators from an overview right so i know you guys hopped on to talk about get ups but first i think we need to talk about operators right and um what operators are and what what they provide right so um you know if you've ever been to any of these streams or talked to anyone at red hat this shouldn't be new to you because all we do is talk about operators right so um the idea behind the operators right it's like you want a cloud-like experience everywhere right so what's what is um what does that actually mean right so um operator and the operator framework is um uh a way to codify operational knowledge right we call it an operator because the idea is operational knowledge we want to codify that right and and what is that you know what does that actually mean and i think andrew right by the way if any of you um haven't caught andrew's shift for administrators office hours i'll um highly recommend that especially if you're an admin that's a great show as well you need help great show right so um but i think i think andrew kind of described it he described it on the show as well very perfectly right as an admin right or as as a devops person right let's take a simple use case if you spin up a mysql database right if you're if you're doing that over and over and over and over again right you essentially do the same steps all the time specific for your environment right like it's not the same steps for everyone but it's just for specific for your environment you perform the same steps over and over and over and over again right so um and a lot of the ways people have already codified that right so we have ansible right i think ansible has a very large ecosystem that massive she built a company he actually built a company around that before red hat acquired them um and so you know a lot of that stuff's already codified so um we want to kind of bring that into kubernetes native way right so we we've already codified i think people say operators codified operational knowledge i think people have already done that and i think companies have already done that i mean for a while now right like even before ansible existed i did things with uh puppet and chef um to codify a lot of this operational knowledge right um and even before then people would just fire off uh i remember doing uh solaris administration right it's just nothing but a bunch of shell scripts running in ssh right so like we've already kind of done that right so we kind of took that idea and um put it in a kubernetes native way right and it really is to um deliver applications uh automate workflows and uh to really um provide a sas like experience for end users right so um so really it's you've um you take the knowledge of you know operators and operators in that like people right so like an actual you know either an sre either you know devops engineer admin guy they know how to set something up right you embed that knowledge you take that knowledge and you put it in what we call an operator and an operator and i'll go over it a little bit is a um kubernetes native way right we took the principles of uh custom resource definitions and uh put a controller behind it to actually do something right so we embed the knowledge and operator and operator all it does it manages deploys and um automatically things like deployments staple sets um secrets config maps anything you want anything you want to be able to codify you can it's not an operator right um and so really what um what are the capabilities right for in in the sdk right so we have an sdk in order you know um in order to easily build operators um right so like uh and this is a cncf project now by the way yes by the way the the operator framework is actually in the sandbox i believe they just got into sandbox they got into sandbox so they it's a cncf project now right um so like everything red hat does we like to continuously donate things um to the open source community um so we have uh helm right so like so if you're already having helm charts put that in an operator you're good to go um ansible right so for you um uh for you admins right so you admins hold a special place in my heart because i was the former admin um nsre so it's you know i understand the pain um your ansible scripts you know if you're already using ansible you're gonna fit right in right so i've written a ton a ton of ansible uh operators right so if i can do it it's not hard right i don't so don't think it's like don't think it's hard right so like if i can yeah if i you know i just took out my ansible scripts i put them in operator and you know minor tweaking and they just work right so um so it's um very very extensible and then for anything else you just have go laying right so that's kind of like the catch-all i would say i think um if you can do it and go you pretty much do it um anywhere right so so like i said before we did uh we did a whole thing on this once didn't we like ants will offer yeah um i feel like i think yeah yeah you probably have a streak if we haven't done a stream about it we probably need need to get the ansible team on it's time for operator from scratch yeah i got it right here yes yes actually yeah me and uh jason did it right yeah we did a uh ansible operator from scratch so and then i did a workshop several workshops now on uh well you were on the ansible team so if anyone would know that yeah no uh that was like that was part of the draw right they knew they were working on answer operators they knew i knew a lot about kubernetes so it was like we need to hire chris [Laughter] yeah yeah exactly yeah so we need that guy yeah like the venn diagram of ansible and kubernetes like world was like only a handful of people at the time a couple yeah yeah a couple years ago was like three people and you were one of the three it was like literally something like that um so operators essentially like i said before is nothing special right so if you are in the kubernetes world cloud native world um there's something called a custom resource definition right so very high level right so you know i don't want i can actually you can actually talk the whole hour about just crds right but um custom resource definitions essentially extends the kubernetes api right so when you do something like cube ctl get pods that pod is an actual object you can actually extend that api so you can do cube ctl get uh um barf right yes we're talking about spaceballs earlier on the chat um why not the um you know the cube ctl get barf and um and you can have the kubernetes api understand that right or you can say you know same same thing as a pod so that's what a custom resource is um so you can do things like in this example get fcd cluster right um what an operator does is um you know you can have a custom resource and a custom resource definition but kubernetes doesn't actually do anything with it unless you have a controller right and that's what kind of like what the operators is is that an actual controller i would say a controller on steroids right um it watches the uh events it does a reconciliation it does you know xyz everything it needs to do whatever you codified in it right and then the operator then um modifies um and manages uh the kubernetes native resources right so again deployments staple sets auto scalers um there's nothing uh there's nothing nothing special about them other than what you've written in the actual operator or what the isv um writes in there there's you know we're trying to build an entire ecosystem around um operators right and trying to get our isvs and you know third-party vendors uh to be able to um write operators so that way you can use it in your openshift cluster right so um so with all that in mind right um you need a way so like after a while you have you're going to have a ton of operators you're going to have operators a couple of operators and couple of operators meta operators all kinds of operators yeah yeah you're going to have you're going to have operators depending on other operators so um we need so this so we have the idea of the operator lifecycle manager right which is from a high level an operator that manages operators right so it's kind of like a a layer a layer on top so just from a a quick overview right um the the operator lifecycle manager manages the lifecycle of your operator right so you can say hey for my cluster i want my operator to be pinned for this specific cluster i want it to be pinned to version uh 1.3 right so you can have it and then 1.3 the operator does whatever it does um if there's an upgrade to that operator you can choose to either upgrade it or stay on that version um and it's kind of like a cascading effect right so if you upgrade your operator to here in this example from 1.3 to 2.0 then your operator may upgrade to other components right so the operator may upgrade the software or you know the version of my sql whatever it's managing it'll upgrade it right so um so yeah so you have um so you have operators that do some sort of automation you have the operator lifecycle manager that automates the automation um and so we also um you know have the operator hub right so the operator hub you know just kind of quick commercial here is you know um away you know for you know it was launched by red hat for way for like you know um you know red hat aws google whatever uh to be able to publish their operators so that way you can consume them uh the idea is in any kubernetes cluster right so it's not necessarily an openshift specific thing but you know any kubernetes uh um cluster can use an operator um and so the operator hub is integrated directly into the openshift console yeah um if you were on the channel earlier you actually uh witnessed or waited on me to install ocs earlier which happened in a matter of minutes and that is a metaphor yeah yeah and actually uh i actually um glad you brought up the meta operator because like my example is gonna gonna be that meta operator sort of thing so um [Music] so um so yeah so now we have the operator hub so then you um actually you know you the royal you if you have an operator um you can actually publish a community operator and have openshift users consume that right so yeah and maybe tweak it and make it better yeah yeah maybe tweak it yeah so i i think um um not not not to our own horn here but you know this is a red hat channel um if you you know i think like the quickest startup the quickest startup idea could be like basically oh hey um we're gonna publish our application on the operator hub and now you're exposed the entire red hat customer base right yeah like a certified operator in operator hub will get you access to every openshift customer yeah every openshift customer ever so you know kind of just if you ever think of an idea it's probably a good for an operator it's probably a good idea to put in the operator hub so um this next slide i think is my absolute favorite slide ever that um i and this is a slide that one of our uh oh yeah i know it's far cry i know yeah you know what it's like first of all first of all it makes you laugh when you first see it right and then it makes you think because you're like oh that doesn't make sense right because that's what you're doing with an operator um so kind of just in summary right um first of all i just want to enjoy this this yeah the gift is amazing here this is this is my this is my favorite slide i don't know it's just it'll always be my favorite slide so far um so operators right so like you're codifying operational software life cycle knowledge right so you're essentially doing what this gif is doing right this is like you're putting all your knowledge into um it's kind of the evolution of i'm going to replace you with a small shell script right right i mean now i'm going to replace i'll replace you now with an operator right right and you can actually write operators there's i think it's called flank it's a way to write operators with bash so you can do that and and as someone mentioned in chat you can write them with quercus and you know yeah the sky's the limit there's quite a bit of like cop is out there kopf uh you can write them with python so yeah like yeah you sky is kind of the limit as far as what you want to operate i'm gonna check out that bash one because that yeah yeah i can do it in bash yeah game over right like i can ride operators all day at that point yeah yeah i mean but ansible same thing right like yeah with ansible same thing i've been doing it with ansible just because that's just what i don't i didn't have to learn anything new um so it just fit perfectly right so uh so operators you codify operational knowledge built on kubernetes primitives right so cr days um sdk golang ansible helm as chris mentioned sky's the limit currently right pretty much you know there's a sdk for anything now um it's meant for automating applications right so it's meant for automating deploying application stacks uh yeah but openshift kind of just took that and ran yeah right see and the entire openshift four platform if you haven't been paying attention is built on operators so even the platform itself installs itself and manages itself with operators right so there's something called the uh cluster operators the ceo and the cvo right the cluster version operator that manages the actual cluster itself is built on top of our operator so we went all in so we don't just tell we we're not we're just dog fooding we didn't just tell people to do it we're doing it ourselves right yeah we we done did it we we don't did it right like we're not just telling you we're preaching right we're preaching we're practicing what we're preaching um and then to manage operators you manage it with all olm right so uh you don't have to right you can obviously install operators you can totally have independent operators right the idea behind olm is to manage your operators for you kind of deal yeah yeah so like you can either manage in manually or you can manage them via olm um so you can control versions and upgrade so you can have actually a little bit more control over your operators that way right and it's and it gives you like that catalog sas experience right um you know i'll you know give me click you know like the previous example let me go back a couple slides uh you know i click a button and it gives me an xcd cluster right so like that's kind of like the sas um version of that right so um so cool so um if you want to find out more about operators uh where people go operator hub.io operator.io somebody is a great place to start we have a book on how to write operators i posted that that's just jason dobby's book yeah so if you've seen jason on the channel he helped write that book with josh wood who's also been on the channel before um that's right yeah yeah like i have it on my bookshelf right here to give you any indication of how important it is yeah i've act i've actually referenced that book several times and and just trying to figure out how should i do thing x yeah yeah yeah because sometimes you're like well what's the best practice here it's kind of good to reference something right [Music] you know make sure you're on the right path because i've always people always want to see if they're on the right path at least right they're not yeah like people want to make sure they're not spinning their wheels and wasting their time right yeah yeah exactly well people want to innovate but they don't want to be too crazy right it's like they want wanted to be maintainable yes yeah yeah exactly yeah um so um so um get ups and operators right so it's like i like to say i always like to make now it's just like peanut butter and chocolate right so like two things that you don't think should go together but actually work together pretty well um and so uh really like i said before there's just like so many automation tools right so where um you know automation's like everywhere automation's everywhere there's if you on the cncf diagram if you see that poster of the landscape i had it earlier yeah no no no no i had it i had i printed it out for my interview with ashesh [Laughter] there you go yeah if you've seen this before you know what we're talking about yeah yeah yeah yeah it seems like everyone's trying to sell you some sort of automation right so there's automation in the face everywhere um so um i i when i was first kind of dealing with get ups um exploring this this this this buzzword right um i kind of just broke it down it's like what are you know like where does it all fit like what are operators really right so like let's just break it down right so an operator um is nothing but a crd right so the custom resource is a saved in yaml right you submit a yaml to um kubernetes cluster right the automation is triggered by the ammo right so by the cr right i can either so i have an operator that the pods are managed by the operator right so even if you scale the deployment my operator will go no no um you know i control the scale right so like so all the automation is triggered by the ammo um and like i said right changes are triggered by changing the ammo um other changes are reconciled right so like if you make a change outside the operator the operator will reconcile that say no you need to change everything um with me so like that's kind of like some of the issues i had with openshift four coming from three to four it's like well i modified the deployment why did i do this it's like no everything's an operator don't right yeah don't don't don't modify the employment modified the deployment is actually an artifact of the operator right like that's how you yeah exactly yeah it's yeah it's a hierarchically it's it's it's you know you're going you have to go above right um but one of the cool things i didn't mention in the other slides but um with operators you can easily replicate you know if you have the same cr once you install the operator you can easily replicate that to many clusters um so you can kind of like rubber stamp these things um clusters right so um so then you know how does that fit together so what's get up so like manifests are saved and get in the yamaha correct right so like we do everything yaml um or json but you know i'm a mo guy um yeah so the automation and the sig yeah the automation the sync tools read the manifest which is the ammo uh changes are made by changing the ammo and pr in the ammo right so um as as you see like there's a lot of similarities between how you manage an operator and how you would manage uh get ops right so um so so you interface with um the operator framework or just like the operators in general uh via yaml right so um it's just yaml at the end of the day right so yeah it's it's all yaml all the way down as they say yeah yeah it's yeah it's yam all the way yeah exactly so if um if you're operators it's just a bunch of yaml right i think that fits perfectly in with get ops because you're just storing that gamma right so um so so the question is it's not if it operators um fit into the gitops workflow because it does because it's just yaml right but where is um where's the point of demarcation right so that that's that's the thing right so should you save all the manifests that um your operator creates like should you save the deployment should you save the secret should you save or should you let the operator handle that right and um i think that's the most important thing is like not if operators and git ops works together because they do is like where is your point of demarcation right so um operators are animal driven and the yama for the operator uh should live and get right and i think that's where the um the point of demarcation is is this yaml right so you have a yaml for the operator that it's going to live and get and you pass that over to the operator and then you let the operator do whatever it needs to do um with that yaml whether you know the um you know there's a there's a lot of um there's a lot of ideas around okay like when you deploy a kubernetes cluster the first thing you do is you export all your yaml um you put it in git and that's how you manage your cluster um if you're using openshift or an operator driven uh cluster that changes right because you're not gonna export every single um every single yaml file right every single configuration you're not going to um like you said chris the deployment is now an artifact of the operator so you're not going to save the deployment you're going to save the artifact you're going to save the the operator configuration um and that's where the higher level thing right so it's it's the point of demarcation so i had a long long time ago right at um hashtag last job right um at last job um that was yeah yeah exactly yeah that's what i'm saying so i'm reaching back right yeah uh where's your way back um we had kind of a workflow kind of like a similar automation where um and i think my ex boss described it pretty well and i think it fits here is like you have a pipe right you have the main line and you have where the water comes out of the faucet right so you can either have this turned on and this turned off or this turned off in this like you need to um um you control the flow at a different point right so if you're going to turn on the water you have to leave this open if you're going to turn on the water where you are you have to leave this open or if you're going to turn on the water here you need to leave this open so it's um depends where you want to control the water right the water flow so um i hope that analogy came out um correctly if my experience it didn't come out in a way that made me cringe so that worked okay the answer is watching i apologize if i butchered that but it's um it's really where the point of demarcation is the point i want to get across is where you want to you know have that demarcation right so so frank asked a question i think it's sure yeah good time to stop and ask it to you uh must a developer specify each used crd version like group version kind of the created objects whoa oh let's just move down to decide on which kubernetes version the operator can run on or how is the supported cluster version managed so that's a very good question oh gbk matters right like yeah yeah it does yeah yeah it i mean for the cluster definitely it matters especially if they're doing that um that alpha thing because right like if you're if you're using like what is it 1 15 you're you've got an alpha crd if you're using 116 version of kubernetes you have the the the ga version of the crds so or crs so you can you can yeah like that matters right like if you're on an older version of kubernetes you kind of have to be careful with that right so that's where like i like to use a little bit of ansible to figure that part out right like hey tell me what version i'm on okay now deploy the the operator kind of deal yeah what and actually this is um you know this is kind of it you brought up an interesting use case chris and by the way thank you for asking that question because it kind of um percolates right kind of makes you think right where you can have with a combination of um you know get ops practices obviously but to customize right so where you can have like the template and customize you have the ansible um operator you can actually have like a generic version of your template deployed in many versions of your clusters now um i think we only support n minus one and just like in general like um just like in the community kubernetes is n minus one right n n minus or n minus two um uh i think it's n minus two i forget who i think is n minus two you're right now that i think about it yeah because it's the current version anyways yeah yeah if you're running 1.15 you should probably upgrade so that's all i'm saying um but um but yeah you could have you could have multiple versions of uh of kubernetes um and you do need to look out for that uh crd version right like uh the the well yeah what they call the the gsv right uh or the um the group version kind yeah group version yeah yeah zvk there we go um so yeah so you could have using templating using um uh you know like customize and using ansible you can kind of like you said figure out okay what class cluster version am i on i'm going to start patching some of these so that way like ingress just graduated to v1 right right yeah right so if you're using beta ingress you're going to get a lot of warnings right now i don't know if you guys realize that you get a lot of warnings once you deploy you're like oh this graduated to v1 so good question yeah no great question thank you frank for asking it cool thank you um so one more this is the last slide right and then we'll go into kind of just a little little hacking on the command line here right um other things to consider right so since openshift is built on operators there's just like other things now you need to consider right um so like like i was saying before cluster operators in the cvo right so you have to kind of think about where um you know where where is that sword and where's that demarcation line right so i some of that's not clear to me yet just yet right so by the way a lot of the stuff on this list is not clear to me yet right so we're not going to talk about this this will be the later show once again oh get a chance to yeah so this is kind of like a teaser right um machine config operators right so if you've worked with openshift you know that we have a machine config and machine config operators right so um and machine sets right so like we have a red hat core os machine set windows containers is coming by the way um if you guys don't know not only am i the get ops guy on our team i am also the windows containers guy so maybe i'll do like a a joint show about windows containers and get ups and that's going to be kind of cool that g8 here yeah now that'll yeah it'll ga next week we're targeting this week or the week after i think it's the week after um i forget but jp dade your thing is coming and it's coming way sooner than you think yeah it's coming soon machine configs right so machine configs for those who don't know is um when openshift boots up a node all this configuration is actually stored in ncd and openshift applies those via machine configs as you know uh machine configs you can have like a hundreds of them so we have something called a machine config pool which is a aggregate of all those machine configs like where are those stored right do you get up to those um there's just so much to consider in an operator world where that demarcation is it kind of gets a little hairy it kind of gets little there's a venn diagram multi hyper dimensional ven venn diagram right is that what i would like to say um so yeah so uh like i promised that was my last slide um it's uh any any questions in the chat that we can get to before i start hacking so i kind of answered this but you know frank also asks so the operator framework cannot check if a special operator version can run on the kubernetes cluster we have to keep the operator always up to date to run on old as well as new clusters so this is an interesting question again frank thank you for it yeah what i do is i you know what if i'm running an operator i want to know what version of kubernetes i'm on so i know which kind of crd or cr to call which version of it so i always have a little script that just says or a little playbook essentially that asks that question and registers that for me so i know to deploy to version a b or c right like because i know there's only a handful of api versions i'm going to be able to deploy to at any given point in my environment right now it's one so if i'm writing that operator i'm going to write some logic to do like a quick oc or a quick you know call in where the ansible to grab that version of kubernetes so that i know which api to use and then i can have those variables in my uh operator later on what's what's really cool about the operator sdk is that the downward api is available to you so you can just start querying the um the kubernetes api really really easily um i do it uh probably not in depth as you can do it but there's like this whole you got the once you get the the sdk downloaded and loaded into your cluster you have access to the kubernetes api so you can do a lot of fancy things um and i think a lot of it depends on your ci cd process right um one of the things i've been threatening to do is talk about the ci cd practices on this um because get ops you can't have get ups without ci cd right and kind of been kind of you kind of kind of been pushing that topic off a little bit a little understandably because so understand that yeah it's a lot of nuance to it right like you can't just there's a lot a lot of nuances you can't just go out there and have i think one episode it's gonna be like an episode and a half maybe two right like before you go to gonna be like i think it's gonna be like a series right okay fair enough so uh uh jp dade has said um talking about sent four so yes i'm wearing the scent force one t-shirt right because i am uh we should have coordinated our t-shirts i know right you should have told me what you were wearing yeah yeah um because as an admin right like i'm a c linux guy what's really one of my favorite things about openshift that um i don't think we talk about enough we kind of blanket security over it but really um yeah that wash right now yes absolutely love it um as yes because we have the sea legs one of my favorite things about openshift is that i don't have to think about sc linux right that is abstracted away from me it does all the sc linux stuff for me um whereas in the past as an admin actually had to know a lot of se linux i don't know if any of you took the rhce tests version five version six there's a lot of messy learning stuff on there and it's like yeah yeah that's the exact face i made um yeah exactly just because it's not that it's hard it's just like something else you need to think about and just another nuance right where that now that has abstracted um abstracted from you know openshift and i think that's like one of my favorite things about openshift that we don't talk about enough i don't think is that openshift abstracts the need for you to know sc linux so um your developer doesn't need to even worry about it you don't even need to worry about it sc linux when you're running openshift so um it's kind of abstracted for me so that's yeah that's kind of um that's pretty cool so um yeah stop stop disabling us linux.com absolutely um we do not gonna make dan walsh cry and no one likes to hear him cry no one likes to hear down walsh yeah cries in uh boston right i don't know yeah i mean if you hear something from the northeast direction stan walsh crying about seven four zero yeah so uh make this a little bigger uh so i have a cluster here uh cube ctl and this is a fresh cluster uh get nodes um is it still up all right cool uh yeah actually one of my favorite things is a sort buy i do sort by workers but i don't know why it becomes last in the list but anyway so i have three masters three workers i just installed this this morning um so what i'm gonna do is let's first install argo so by the way i haven't tested this on um yeah yeah so like this might not work so i apologize it's been one of those weeks where it's just been crazy back-to-back meetings chris i don't know i know you i know you know so i don't have to complain to you right it's just back-to-back stuff i haven't been able to test this on 4.6 if it works on 4.5 so if it doesn't work i apologize so we'll debug on the fly here so let's install argo first um one of the things that um the canadians right the the get ups canadians at red hive say that um our wonderful red hat canadian friends yes yeah yeah there's uh a lot of the new get ops language is shell because you just basically do um well let's think about this for a second do an until right you do an until apply and it won't return until it's successful i mean right but remember a lot of kubernetes the management of it the installation of it all that fun stuff was written in bash at the beginning right so it's like that's right redoing that cycle so you know the get ops maturity will start showing here as soon as you know more languages start getting involved yeah yeah so let's see here come on go go go go go all right um until unchanged configured unchanged no matches for okay so i was waiting there there we go all right yeah yeah that which is why what the until is um so let's do this watch oc get uh you pods spell i believe in you all right so that'll fire running that's running cool um so that's and i'm going to deploy this repo it takes a while for this repo to deploy so i'll deploy it first and then i'll talk about what's going on here um go baby go baby no whammies great a great yeah right cool this is doing a hormone good job buddy all right swing all right so let's uh do octotree by the way whoever someone on the channel finally finally by the way you know what it's been a lifesaver to be honest with you uh yes so thank you i don't know who yes it was me and narendev [Laughter] there we go yeah miranda was like christian you need octotree and i was like christian go to octotree.io sign up done [Laughter] so uh so yeah so i have to keep clicking around right so i can see here so i have um so as you know argos td uses a concept of applications and um so if i could go over here if i look at um i'm going to be installing the efk stack right so the fk stack is um log aggregator and it's actually yeah elastic search elasticsearch fluency cabana yeah efk right and as like like like you said it's a meta operator so um so it's one of those like you need like three operators right like you gotta install all three of those programs connect them together that's what a meta operator is for so i have this efk application that basically says um it's this is argo cd application it says uh go to the manifest efk install so let's go here manifest efk install using the f out of that octo tree yeah i love this option um can i go directly to the directory that can i click i think if you click the directory name i haven't quite figured that out apparently now if i could go to the directory but yeah um but you click something inside and then go up a level which is what you did so yeah yeah yeah that's that i always do it that way so by the way if you guys know please put it in chat um so i have here um the the cluster logging operator right i have this instance um which is basically it says i'm gonna use a storage cast gpt2 so um it says to use base right so in this base directory i have all the operator configurations that i need right so in order to get an operator up and running you need an operator group a subscription and the actual cr for the operator right so um and there's this implicit namespace right so like you need it in an ancient space so um i didn't mention it because you need a namespace for everything nowadays right so i have um um you know openshift logging right which is the namespace i want to create uh i need another namespace for the um to the actual operator for the actual operator to live there um but the actual magic happens is first in the operator group right so i need an elasticsearch operator group so that's that here right so i say all right so i'm going to create operator group um uh for elasticsearch and i need an operator group for there's another one for the cluster logging um right so it basically says for this operator i want to target this namespace um and then once i have that then i need to subscribe to the operator right so then i have a subscription right so subscription just says i want to subscribe to this particular version of the operator right because your operator may have different versions so you need to which goes back to the gvka thing right like version 3. one can handle older clusters 3.2 can handle current clusters 3.3 can be ready for the next release of cluster kind of thing yeah exactly right so then you have so then that could be so your channel that could be something you either do it ansible or if you do it customize you can do it you know wherever right but that's basically where you say i want to subscribe to this particular version right um and uh oh yeah i will share my repo link yeah i was trying to look it up but yeah thank you for seeing that yeah yeah um and so uh so this gives me so basically as you can see here um i don't have anything specific like uh well let me go back to base i don't have anything i wish i can click on the directory anyways um i'll treat people i wish i can click on directory um so we'll see if they're watching yeah see if they're watching okay so we don't know right um so yeah so i have um i don't have anything in here specific to um the actual deployments so if notice here everything is either an operator group a subscription um or the namespace right there's there's nothing defining like oh hey uh i need i'm saving my deployments and my deploy or my deployment configs or my stateful sets or my storage or yada yada yada right i'm all i'm saving is the operator manifest and then i'm gonna let the operator handle do its thing right so nice um this is a good thing so yeah yeah yeah so so my point of demarcation is the operator right and operator hub i'm going to just submit the manifest and then the operator just does its magic so all right hopefully we'll see so i just found a cool feature in octotree not to hype it up anymore but it actually has dark it has dark mode for all of github as a feature yeah yeah that's why yeah i was actually looking at that i was looking at because you know i like it so much that i was just thinking about just like just buying their one of their tiers yeah they're um i have fifty because remaining and this is definitely probably gonna be an expense that i do yeah yeah i mean and it's like it's like cheap it's like it's like 30 bucks a year or something like that um anytime i like something i always buy or definitely you're like you gotta support crypt i yeah let's encrypt i always donate to let's encrypt just because i just use a crap out of that yeah i donate to eff every month so yeah the uh so yeah yeah um how do i yeah okay there we go yeah now i want to register then now i'm gonna go into argo so if this works this is the first time if it works is the first time i see it work so um [Laughter] that's scary i don't know what that is well i haven't had time to test it but it should it should work uh you know it should yes yeah so i got the efk well it's out of sync let me try to sync it um let's just prune it let's just manually sync it another operation is in progress oh i probably have to cancel it um let's go back jp data is not so much an expense account it's just that if there's tools that make our job easier we're encouraged to uh not give money to red hat we're encouraged to get that money back from right yeah yeah [Laughter] let's go back here so i might have a error here i have it's missing okay [Music] yeah this is uh so by the way um and this is something um i've been trying to play with if you saw here this is something i might talk about in in for their episodes i have um a sink waves right because there's like certain things that need to happen in certain orders um in order for something to work that's just life right it's just right just life in i.t in general right you need to do abc before you do d um and that's what sync waves are and triggers and things like that and i've been playing around with that um but looks like let me just delete this resource here i'm joking uh remember back in the old days yeah i'm just going to delete this server people would freak out no you don't know what that's connected to [Laughter] yeah i have i have uh um yeah there's there's like funny stories out there where it's like someone unplugged an xbox and the webster website went down i was like what right yeah you're like you don't even know [Music] yeah i've got some pretty funny stories from those three days people accidentally unplugging things or wiring things incorrectly potentially you know wow it actually lets you pick fonts and everything well that's cool sorry i'm still talking about up oh yeah it's just playing with octo yeah octatree here yeah yeah this is seriously messed up here this is why i probably should have tested it yeah i'm trying to sync the individual um it might be because you have the right storage um oh boy are you just spinning or ssd which which is your cluster seeing them well i'm on aws so oh uh storage class gpt2 okay that's that works that should be that should work yeah sync can i sync this guy here where do i see the logs here let's do this let's just just debug this live here was my commit so let's see here let's see what it looks like uh oh i know what i think i did hold on manifest efk okay that's fine gpt2 that is silo instance no i do have it okay so i have this skip dry run on missing resource ah yeah not eve naughty me oh wait how do i go back now i'm stuck uh oh here we go here we go again there we go by the way i just bought a license to octo tree yeah i mean you should support the things we like to use right yeah i agree sync them apps uh which one uh yeah that's a problem that is a problem um which is it which component it's the actual instance of login right so like i said it's the actual cr right for uh the login component that matters that doesn't matter right everything else is good doesn't look like so i would think it stops syncing let's terminate that braided silver this is just as bad as the time his github get ignored yes [Laughter] remember folks this channel is designed for all of us to learn together right like we embrace failure on this channel because if we don't if we don't fail and learn from it like nothing's perfect in life right so we want to make sure we give you a very real experience and this is going to show you how to troubleshoot problems later um we don't we do it live these aren't polished demos for a very good reason so let's sync that okay let's see what happens here well it's funny everything else is sunk sink synced sync you've been drinking a lot of wine um yeah it's just this guy partial synchronization not recorded in history yeah well i know that stop syncing how about that you know what's funny it doesn't let me like because i have auto sync turned on when i stop it it'll like it like it kicks back on um oh yeah that's right oh wait i can do this okay yeah see look it just like it kicked off again like i don't want to i want to debug this damn it um let me debug you well what's what's funny um uh braided silver uh yeah i i i know um i was i was talking to her about like because i've been doing a lot of goal line lately and i was like you know that's okay we're doing golang well doing golang but like i came from like the web front is doing i going from php i went to golang and she's and she's like that's like going from being a fish to being a bird i'm like get that pretty much yeah like i don't think anyone has ever put it quite so eloquently as as that's very well said yes yeah so let me uh let me actually i'm gonna cheat a little bit guys sorry uh because i need to see braided silver ass can you write in a weight somewhere even if it does turn it back on it's like a sleeper or something yeah so that's what this these sync waves are so like if i have um oops where's shift be yeah i have a sink wave you can do like you know it starts at negative 20 and goes to plus 20 right so it's kind of like a um uh it's kind of like the like a golf course right oh okay yeah yeah yeah so um so yeah so like zero is actually the highest number i put on because i have it like negative five negative four so it should just work so let's see here uh i'm gonna cheat a little bit here tmp just to see if i can get this going i need to figure this out why this is a problem on 4.6 okay so that's the problem something no matches are probably they're logging what are they what are they changing yeah that's that's that's exactly that's exactly my question so i guess um i don't know who asked the question but like this is now that that's that same issue just now biting me in the butt oh so as eric jacob says when in doubt um read the docs yeah yeah aren't you the one with the fancy gcp shortcut didn't you tell us about that yeah um i do ocp space uh logging oh sorry no ocp not gcp gosh well yeah yeah no thanks thanks for reminding me um no no problem i use it all the time thank you for telling me so cluster logging logging openshift i o v1 that should be fine is it hang on let me see i've got a 46 clustering i wonder if i could do this i have no idea what that area is uh let's go back to this guy here i need the whole http guy yeah so even customizes uh open shift monitoring projects hmm it doesn't even like my [Music] looks like this doesn't work on 4.6 i should probably looking at the 4.6 stuff right now well i wonder if i oh well the name space is open shift monitoring i'm not sure if that changed from four five to four six though i don't think it did so scrutiny still and start over here oh cluster monitor yeah exactly so this is yeah i need to go back and see why this isn't working oh i've got that trick shape you date hang on buddy i got you so you add a search engine in google settings um it's it's actually really cool yeah it is so i just have to type ocp ocp and then i just type whatever i want yeah so you if you put in um if you go to your browser's setting for search engines and you say manage them what you basically do is you know you create you add a search engine and um let me edit this so you give it a short code of whatever you want you know open shift or ocp is what we use and this is the string that i actually use in that field so that way every version that gets updated you just change the number and you got the latest docs that you're searching anytime you type ocp so add it as a special search engine it will save you tons of time but make sure that the version number is the version number the cluster you're using right now yes yeah um and that's that last little bit right there 4.6 so i always yeah like so every release i just change it um i think i got a race condition here yeah i need to figure this out here does anybody have any get ops questions for us that are yeah not related to what's on screen or any openshift question for that matter anyhow we always have a question that i'm at that's right um yeah i think i think i really messed this up let's see here so oopsie get applications let's see delete applications so at least give me a long time but i think i'm gonna be running into yeah well i think i'm getting into finalizer issues oh that's right yeah so this is well this is something that um also i think i may switch to one of the one of the red hat canadians um uses a job to actually trigger the efk deployment instead of actually giving the cr um that makes sense yeah um well mikey a job yeah he uses the job in order to make sure the cr crd is in place before he deported operations thing yeah yeah yeah but he uses a job which is which is nothing more than a shell script which is why we exactly which is um which is why we yeah we always joke that bash is the new a git ops language um that basically just loops and makes sure once the cr is available then he then he continues on with the um deployment here but this isn't deleting i wonder why i probably have storage issues is what i'm thinking because if i do oc get pv there's no persistent volumes we'll see you get pvc there's no persistent volume claims either um and it needs uh storage oh boy family's not my answer no worries uh there is a way there's like a weight equals no uh oh there's alexa something went wrong but everything worked just fine that's weird jp jpd says that string isn't working for him that's weird so you have to add it as a new search engine by the way and i apologize for the dog yeah i got some uh some dangling issues here i think i may also have our back issues let's go over here let's sync this guy here yeah the reason i think i have our back issues is because when i go here and i delete like this guy here says i don't i cannot delete resource as the user that i am so you may have r back issues as well yeah so will this even delete we don't know fun yeah it's fun oc get events argo cd let's go here uh carlos santana from our colleague on the ibm side we have been trying to create a tecton pipeline with argo cd for operator and olm bundles catalogs interesting i hit a blocker running opm yeah so like opm so for those who don't know is opm is a way to bundle your operators right because you have an operator so what an operator when you're building an olm into olm you need to build like all the metadata right so like when someone subscribes to version one of my operator it goes out and downloads or gets the this particular tag of this particular image right so um and you can bundle different versions right so you could have one one bundle with different versions depending on what you subscribe to you get a different version of the image right so um so that's what that's what the question is he's talking about right now about the opm bundle um in tecton so i guess what you're doing is so we need to see any folks push code okay so you're so i guess mr santana here big fan of your music by the way um i bet he never hears that joke um never it tries to auto tries to automate so when when you when you push the code the code like i guess compiles you create the operator and then you're just trying to automate adding that bundle into the new bundle into the operator i guess is that what uh what you're doing okay so it's like you need a whole versioning system to then plug into the catalog yeah carlos i need to hook you up with some of my friends send me an email buddy yeah i can i can get some people that are real smart on that for you sso you want uh dash dash insecure you want this guy i'm trying to use the argo for my last ditch effort i'm using the argo cli okay to see i have a hard stop in 20 minutes that's all oh no worries i know we're even over time now that's fine people like your show so that's okay yeah i'm like langdon show i'm just kidding wow i'm fired shots fired yeah yeah the sun doesn't even say yeah it's the wine um uh app list okay so braided silver asked is this a fresh cluster if not did you stop any other running operators to make sure nothing was conflicting oh yeah it was a fresh cluster um i could do by the way you can do oc get co and you get the cluster operators um and it tells you the uh the status so i i've always this is such a programmer or engine not even a programmer engineer mindset um you see false but false means good in this case nice super helpful thank y'all yeah in my mind that this seems bad but the degraded status is false so um maybe they should put it like i mean regular people non-engineers would want you to see no there instead of false but well yeah no degraded no right so like that would make more sense to me but um but anyways none of these operators are degraded we're not here to make sense of things yeah i know we're just here uh progressing goes false that means there's no operators progressing and you know available equals true so um so yeah so let's do uh argos cds ergo cd uh app get and this is here tells me yeah so it tells me this so there's there's some issue with uh order order of operation looks like um it doesn't want to delete it for whatever reason yeah for whatever reason it doesn't want to delete it and i believe we'll see uh project uh open shift oops let's look at the actual openshift blogging oc get even also get pods and see here nothing okay good oh so you get uh events nothing well that that sucks well at least you're honest i wanted to see something uh defaults let's do oc delete uh product i'm just gonna drop the hammer for my area let's see what happens i'll see projects unlocking not there uh okay come on finalizers okay so that's missing let's delete the operator group probably don't want to do that i'll really mess up my cluster so subscriptions let's go see get subscription subscription what is your github username christian h eight christian h814 yeah okay see here alpha oh i just realized my my error error in my ways so um yeah so i subscribe to the 4.5 channel instead of the 4.6 channel ah um that's why i may be having trouble so what so okay so let me back up and i'll i'll explain what's happening right um which we probably won't see this work within the next you know couple minutes that we have left here yeah um but what end up happening so when you subscribe to a channel and it's an older channel um and and you and there's a newer version and so like all these things like i had like this perfect storm that just happened and you have your um uh and actually andrew pitt he he warned me about this too so i'm just gonna kind of like lang didn't warn me about the github thing uh he warned me about this uh error coming up here so let's um let's take a look here um where is it uh it's in one of these let me see if i can find it real quick um so like you can have your your operator automatically upgrade right so you can have um your um there we go install um approval equals automatic right so what ends up happening is that um what ends up happening is that i install 4.5 i have upgrades set to automatic so then it um once it fully installs then it upgrades itself to the latest version right so right um what what's happening here and i messed this up so bad that it's not gonna reconcile right like i have to like delete everything and start over um is what would end up happening is that um um i uh i installed 4.5 it saw okay i'll solve 4.5 oh you want by the way there's 4.6 available so i'm going to upgrade you to 4.6 um so it was waiting that time that upgrade time right so it and it went ahead and installed the entire operator right and then tried to upgrade itself um and so i ended up with the race condition so anyway beautiful so that sucked so but at least you know why there you go well at least i know why right yeah yeah sometimes we have to tweet out what broke after the fact that that's happened before actually now we don't have to it's in the recording that's in all right so um cool yeah you know maybe next time you just do a quick run through with it working yeah at the beginning we touch the next yeah exactly so cool awesome so yeah so yeah no great show uh there's an sop for this i'm almost certain uh yeah carlos santana we we all need to talk i feel like with what you're trying to do um so i nudged uh scott mccarty and dan in the github issue you opened so we'll get that figured out anything else christian other than thank you audience we appreciate you tuning in yeah yeah thank you um appreciate you tuning in um i guess you can uh you can tweet at me you tweet at christianh814 or find me on github it's the same name i use the same name across everything so um so yeah so yeah carlos santana wants to know if you have any experience with the argo roll allowance so yeah argo rollouts is something i want to i want to cover in future episodes because i think it's um it'll be helpful um yeah totally too yeah so awesome cool well good talking to you uh i hope you all stay safe out there and we will catch you next time yep definitely all right bye everyone you

Channel: OpenShift

Views: 639

Rating: 5 out of 5

Keywords:

Id: IeN4EQcGLvY

Length: 81min 0sec (4860 seconds)

Published: Fri Dec 04 2020