GitOps Guide to the Galaxy (Ep 23): Directory structure battles

[Music] [Applause] [Music] [Music] [Music] foreign [Music] so [Music] [Music] good morning good afternoon good evening welcome to another edition of get ops guide to the galaxy i am chris short your host with the most for this expedition that we are going on in the next hour or so christian hernandez is here with me uh christian the get ups guide to the stars i will say yes to the stars i'll be your captain today in journey get off the sky to the galaxy um i don't know about you chris it's been a crazy week for uh i mean it's been like non-stop right like it's like it's like you wake up it starts and then you go to sleep that's when it ends and it's like yeah can we not do that all day long can i get like a yeah like a show in there on tv yeah yeah exactly well then it's like your day-to-day stuff right just like oh wait now that now that the meetings are done now i get to go to my yes no it was funny last week was like a lighter weight week for live streaming and yeah you i use todoist and they have like a karma and you know they track how many things you complete per week and i actually sent a screenshot of it last week because it was like you completed 34 oh look at that i wasn't on the air so much not as many meetings so it was like oh that's amazing this week it's like 10. yeah yeah exactly yeah so um well you get it where where you can right i think it also has like with with kubecon and uh and for us getupscon there's a lot to do and it's coming up quick so right um again like for everyone who's watching this we got um october 12th right is the day zero for um get ops con and then uh kubecon cloudnativecon is the rest of that week so for those of you who are um uh flying out to los angeles say hi to me i'll be there uh for those of you who are virtual um you know you say hi to him anyway right yeah say hi to me i may not say hi back because well i'll say a generic high right to the camera because we'll be um it'll be a hybrid event as as most of you probably already know so um so yeah so today's uh as i was preparing for today's topic today's topic directory structures right it's been it's kind of an interesting a topic because yeah there's it's a hotly debated topic and it's also um you know you're you're you're either on one or the other side right you're either very opinionated or you're like hey do whatever you want right and it's it's there's a lot of information a lot of ways of people doing things and it could kind of get overwhelming right like if if you're starting off in your git ops journey if you're you know if you're just starting and then right away you're like well what's the best way to you know like if i'm putting everything in get like how do i configure everything right so um i went over briefly long long time ago so for those of you who haven't uh been with us the whole time um you know you can catch past episodes it's literally like the first episode i kind of touch on directory structures a little bit um but i've only like i really like grazed the uh uh i guess the top i like the top of the tippy not even the tip of the iceberg but like the part that you can see like the little snowflake on top of it um that's how much we got into it but i we did talk about a little bit um and so i figured i'll i'll do a a show a stream about um best practices right so um not necessarily like hey this is what you need to do but more like hey there's a suggestion here are some suggestions and some of the things to look out for because um again just like everything else it's an answer you absolutely hate it really depends on what you're doing everyone always hates that answer it's like well it depends like well it depends on what well i mean what are you doing i don't know tell me it's like you know what a blind code damn it yeah yeah exactly i'm just [Laughter] you know i just want to deploy my code um so uh i kind of set up a kind of best practices and hopefully it'll it'll trigger some some conversations here so um i know i try to do a lot of keyboard stuff today is mostly presentation and talk right um you know i mean it's a directory structured conversation yeah yeah we're gonna talk about multiple ways to skin this cat yeah yeah yeah so let's uh let me share my screen here and it's always the fun dance of oh yeah you know what there's i actually found a setting in zoom that maximizes the screen when you hit the share button ah oh also like you can have like a pre the predestin that'd be cool like it was like always share this screen it would be nice it would be nice i don't know if that feature's in there but i need to tinker with it for sure yeah yeah so for those if you if you anyone out there knows let us know um so i have this presentation here i gave the presentation to you chris he's gonna going to upload it right now he's gonna share it out you know whenever right so um as we're as we're talking here so it was processing and now all i have to do is actually hit this right hit share stuff so um yeah so i'll i'll be jumping in and out of the this presentation that's why i'm not in full screen mode so let me know if you guys can't see anything i'll i'll try to zoom in but um and you can follow along there's just yeah just drop the link in chat oh there we go the link's right there so so right so um so you've decided to get ops right so like this i figured um i started off like absolutely yeah yeah you say yeah so you've decided to get ups here's a pamphlet to help you out right so um so for those um for those of you who have been following the get ops working group um i'm gonna put the uh the link in the chat we have the get ops principles um you know the beta release came out we have a release candidate also out right just follow that link open getups.dev all the information is there um and the reason i'm dropping you that in is because the one of the getups principles is you have a single source of truth right right so um so you know much like everyone else when i started with get ops i basically was like all right cool so like i'm gonna dump all my ammo in the directory and like and then i'll sync that with my repository and yeah and then and then you know that works great right profit right and it actually does work and it actually is pretty simple um but much like everything else is as easy to get started uh difficult to master sort of thing and now you know you probably ran into things like i did where you're running into issues is like okay well that's probably not a good idea right so like how do i deploy this you know i have this this don't break this yeah how do i not break this right like how do i have this dump of yaml in a directory and how do i apply it to multiple clusters and how do i apply this to multiple environments i don't apply this to multiple clusters in those environments right like you start getting into this like okay well the verse of things yeah metaverse of things it's like okay well having a directory with all your yaml uh is probably not a good idea and you started running to these things like okay well how do i um you know how do i organize my directory structure right it's like all right you did a poc right hey this works pretty well it's like the first question is like okay well then how do i that's literally the first like first question is what is get ops and second all right what git ups controllers are there and third is like okay this works well how do i how do i configure my direct structure it's like one of the like early questions right and um a lot of the tools built around get ups you know they've already um they already faced these similar questions right and and um they're like okay like anyone starting with get ups automatically that's like the first thing right and so people building tools around it um have kind of have an opinionated approach right so um i'm gonna go over some of the tools it's kind of like hey if you know if you don't really have a strong opinion and you want to just say hey just give me like the same default there's some tools out there that'll do it i like how your like tool tool tool gerald yeah yeah so gerald if you're if you're watching that yeah you'll you'll see this here pretty soon here so there's um i'll drop this um i'll drop this blog here um so i have um there's uh argo city um there's uh argo autopilot right and argo autopilot is in the argo cd project right sorry in the argo project right the argo project has argo cd uh argo workflows and this one here uh argo autopilot um and um and it's basically kind of like a as the graphic suggests it's a hands-free argo um so if i go back to the presentation here um here's a sample uh repository layout they have here and this is basically um this is a test someone did with with argo autopilot and it and it kind of kind of breaks it down into um things like my application how do i bootstrap right and i see that william you know ask how do i um you know deploy existing resources and things like that right and you know that's kind of like the how do i deploy argo you know get ops friend only way that's kind of like their idea of bootstrapping and they have an idea of projects um not to confuse you even further right so we have openshift projects and there's argo cd projects and then there's argo autopilot projects which is something completely different um um projects can be seen as environments right so like they're that's how they use it so anyway um are going to say environments then yeah it's one of those things where like they thought about it later they're like oh yeah we probably shouldn't say but you know tech debt is real um yeah no kidding so um so argo autopilot creates those directory structures for you so you don't have to like you just it creates it for you and you just need to know where to put your yaml um that's that's one thing um red hat has our own we have um something we spoke about before and i swear one day i'll get bose to come on and talk about cam uh cam's another one i've been on the talk about that yet we briefly talked about it okay um so cam has a concept of yes right of uh bootstrapping right so it's essentially the same idea except it's a day two thing instead of a day you know one sort of or date even day zero thing and can you name it name space that won't confuse anyone yeah yeah exactly um and basically um see with with cam they do have the the concept of environments and configurations and stuff like that so this is kind of what you get the directory structure um and then flux cd flux doesn't actually have a directory structure but they actually have a pretty good article um uh flux has like guidelines like what to do and i actually read this it's actually really good i actually do a lot of this and i think as we talk about later most people start around this kind of path so that's an article you can read and then there's general gerald noon right um you don't know gerald is a uh as an architect here at red hat and you know he's he started with the get ops thing right around the same time i did um and he actually has he actually went and documented the crap out of this so he has um uh directory structure and he has you know you know what level it is you know the name of the folder and like what what it's for and he actually goes down into the weeds here so there's um so good news is that um people face this problem before and um you know there's recommendations out there right so if you're you know if you're looking and you're like well like what have other people started doing i recommend looking at um something like what gerald or the the the the community of flux has done um that they actually just kind of say like you know here's an explanation if you're kind of like hey i don't really care just give me a directory structure that i can work with something like argo autopilot or um or cam could be something that um that you can use so but if you're like me you're strongly opinionated right and you want to do your own thing um you know the the the answer is really there's no really no magic bullet right because there's no really really right no one's really right in this regard right right everyone really does what works for you yeah everybody knows what works best everyone does what makes sense for them right and like i said before the answers that everyone hates it really depends right but um but it's true yeah it really depends but there are some best practices right so we're you know i'm gonna kind of go over best practices and not necessarily directory structures um there is like i put in the chat and like we have in the presentation there's loads of information out there if you really really want someone to tell you what to do there's there's plenty of opinions there's there's lots of opinions out there 100 opinions so grab a pencil right i imagine this is you right looking at it at the computer screen grab a pencil and um uh and take some notes right because i'm going to go over my best practices and some of the best practices that i've read out there um so the first thing is dry right just like with coding infrastructure as code get ups it's all the same thing we're all talking about code and we're all talking about don't repeat yourself right and you know um basically is avoid duplicating your yaml so um you know like how do i essentially the question is like well how do i deploy all this yaml across multiple clusters without copying and pasting yaml everywhere right so it's it's like a um it's a trap right it's like a true cycle yeah yeah [Laughter] so it's like so again with my example right i dumped everything in one one yaml right it's like well if i want to deploy this deployment to another cluster and i just want to change the the scale right instead of replicas one replicas equals two do i just copy that directory and just change one line right it seems like a big waste right it seems like i'm just repeating the same animal over and over and over again right so um so income's customized to the rescue right customized yeah it's customized right so um customize has been around for a while they do it's even built into the cli like you know cube ctl or oc since like 1.14 it's been yeah it's been there for a little bit it's been there for a while right and what customize is is a patching framework right so you um it's it's essentially it's like okay well i want to take this yaml and i want to patch it right so um so you can do environment specific changes by using the same um uh same yaml right so it's not a templating per se right although you can use it like templates um it essentially it produces raw yaml so the idea is that it just gives you raw yaml um and you can directly apply it so um and the basic concepts of customize is that you have a series of bases and overlays and the base is essentially what's common amongst all clusters right so the so if if you if you think about it right you're you know what's coming across deployment the differences between the deployments from one environment to another can be it's actually very minimal right it could be like the secrets you use the image you use and maybe the scale but like the rest of it like in in terms of like a skeleton is essentially the same right so you don't um you know you don't want to be copying all that just to change a few things right so this is the concept of an overlay essentially i want to overlay my changes on top of the base configurations so um and the thing is is the base has no knowledge of the overlay it's just raw yaml yeah it's just yaml so the thing about customize is that you just you just use raw yaml for everything so um and then the overlay is essentially just a series of patches against that so how that looks like is um here's an example here so you have you know your application and you have the base so you have things like service route deployment common things yeah basic kubernetes objects yeah basic kubernetes objects and then there's a special uh file here called customization um which basically says hey when customize run read these resources right and so basically it's kind of saying read these resources anytime someone calls customize and just displays blanky i'm like and then in overlays um essentially you're like okay this is the diff this is the difference right so this customizes saying all right so as my base use you know go up to directories and use the space um and in the dev environment i'm going to change my route and then i'll probably in a different name space and this is what uh the patches are here so like you basically patch the resource and the patches could be like a json patch uh json patch json was it six something i know uh what's that what's that so um oh oh oh yeah uh json 6902 i was about 69 something yeah yeah 6902 so 69. it's just um if you don't know what what that is is um um i'll find it's just yeah common uh kubernetes patching right like if you're if you're doing cube ctl patching you'll you'll fit right in that it's not it's there's nothing crazy right um and then so why do we use customize right so i use customize for various reasons right first again it goes to the dry principle right don't don't duplicate it removes the need for duplication of just yaml constantly i don't have to copy yaml from one folder one one repo to another one to another right i just i have one set of repos um and um it's um you can create a hierarchy right and so that gives you kind of uh some flexibility um it could introduce a big complexity with that flexibility it's kind of like one of those things is like it's really really flexible but like you can really it's like a like a ball of twine right like you're trying to follow things um but it's kind of it's one of those things like to an outsider like if you don't know customizers you're like how do you know what that is but someone who's been using customize a lot you're like hey yeah oh i know that goes to here that goes to there so it's one of those things like you really have to do it a few times um it's easy but it's it's also easy to mess yourself up a little bit um what i really like is overlays can reference remote repositories right so here um you know going back to the using customize i'm like referencing like the base i have to go up two directories i have to copy that directory there um but you can actually reference remote repositories so here's an example here i have um this is uh red hat community of practice another gerald noon um special where he had the what we call the get ops um catalog basically here what you can do is you can say my bases oh hey my base actually exists on that git repo over there so like you don't really have to copy the yaml right you just have to have one file and basically load in what someone else has already saved um and even that so like even if you use this what's funny here is if i go to base and go to customization that even reference yet another game another git repo so you can actually start chaining start chaining things together um so you don't you don't have to copy yaml everywhere um so you don't even have to do um you don't even have to do this right like copy the bases you can reference a base from another gate repo so that's pretty right so yeah that's pretty handy right right yeah security team wants to do some things to the route or maybe you have some kind of certificate thing that's different in prague compared to devon state yeah it could be different stage yeah or like me as a if i want to consume uh like in a catalog form right like i want to consume you know something that my administrator is providing for me i don't want to mess around with how to install it but like that's that's their problem or like that's for them i just need to rename it and yeah yeah exactly i just needed a different ink space and so you don't have to copy the entire thing right you can just reference other things so uh what's pretty cool is that it validates yaml right so you know it which is hard and it does a good job of it yeah it does a great good job of it and sometimes it's annoying it's like just apply it no there's an error it's like but i know what i'm doing um [Music] [Laughter] well i'm like fine fine i'll let it validate right um and what's really cool it's agnostic right just customizes just right since it's raw yaml it works with argo cd with flux with acm right with whatever whatever tool you want right and so uh you know you keeping your repos agnostic gives you ultimate flexibility so um that's what i really like about customize you can use it with pretty much any tool that interfaces with kubernetes so i can't talk about templating and patching without talking about helm right helms another helms another way of doing things um if you're used to using helm repos um or helm charts you can bring those over um as well right to kind of solve this problem of duplicating yaml um you know for this kind of a um a uh what do you call it a high level view right for those who don't know how the helm works but kind of like review kind of you have the idea of a chart right which is a template and your values and then it generates your yaml based on those right so essentially it's a um a templating engine essentially where customized patching framework is a templating framework um and so if you know kind of have like the ideas like you have you know your template and then you pass uh parameters to it and then it just produces a yaml file right so um who is responsible for other yaml decorations like dns policy cluster first ink in case like those those system layer things are going to be handled by the actual admins right like yeah but the actual admins yeah yeah um who's ever in charge of your uh clusters yeah yeah yeah yeah yeah words words are hard right yeah so you have um you know again like i said um you know you just private parameters and it'll can do that as well some things to look out for right uh when using either helm or customize right so um i have a blog there's two ways to use helm um so i'll put that blog in the chat yeah um i wrote a blog about helm and argo cd so there's two ways to use argo cd and um both of them are right and but just one of them isn't get ops friendly so if you if you're if you're doing if you're doing get ops and you want to do things to get ops away there's a specific way to use that in inargo city so it's just kind of like things to look out for right um because remember get ups is declarative so um doing things imperatively then use the then you um then you have a different source of truth right so then you think you break the the principle of get off so you have a single source of truth so the thing you have two instead of one so um so remember if you're going down the path of helm which is fine i use helm a lot i love helm i have a shirt i wear it sometimes i love them um um but just just remember there's there's there's a there's a way to use it that he that keeps it uh get off friendly so um and to keep it as a single source of truth right with customize there's gonna be a lot of customization.yaml files so i worked with a with a customer of ours and they're like man there's you know the customization.yml files like yeah it saves you from writing have to copying elmo's back and forth but you have like the customization like yamaflas are like rat droppings they're just like everywhere they're just like they're like you'll have a folder what with the christmas tree red droppings i don't know i'm just quoting the customer oh okay fair enough they're like they're like red droppings right so like you'll have a situation where you have a folder that just has one customization.yaml file and it references like eight things and um you know it like i said before you know you can mess yourself up a little bit um so um just look out for there could be a lot of customization.yaml files um also there's not a um either or right you don't have to use customized or helm you can use both of them in conjunction right and um most tools are flexible with that i think all tools that i've used like um like argo city acm flux they let you use both so it's it's so it's it's not like an either or like you can mix and match what makes sense to you for instance i'm deploying an application that i use a database backend i just use the bitnami um mariah db um helm chart to deploy my db because it's just easy right i don't have to worry about that um then helm take care of that so yeah you don't need to write your artisanal yaml for things that are yes yeah yeah exactly yeah and like with the knowledge of like an actual dba yeah yeah yeah potentially your limited knowledge set right like you have like a known good thing yeah yeah exactly yeah if i have a known good values.yaml file i just use the home chart so let that um and last point right the very very at the bottom um using customize will dictate your get ops repo structure so if you remember um uh right here right so like if this is customized if you're using customize to not repeat yourself that's gonna dictate a lot of what you write into your um into your git ops repo so um so that's just kind of like one one thing uh to kind of look out for right and maybe if we have time i'll show you kind of like this little project i'm working on to kind of show you the directory structure that i've been coming up with heavily customized um uh influence as you'll see but um [Music] but yeah so that's kind of like the point there if you're using customize if you start the path down customize you're like you're like halfway there i think right so you're okay in your directory structure let's say okay well at least i know how to you know not duplicate yaml and like create environments for myself and you know all that's really um um i just saw that comment that's that's all really good artisanal yamo goes well with avocado toast yes it does by the way in l.a i'm in l.a right we're like the whole you're like in the avocado yeah for those of you who are coming to kubecon i will show you some nice avocado places multiple places it's like monday you can go here tuesday you can go here again if you're on the east side of the city this is the place i have yes exactly exactly now if you're on the north so um so cool so um you know that's kind of like one of the best practices here um okay so now you kind of figured out the repost structure but so like now what right so like what's what's where i go from there right now that i'm using customize you know i kind of have an idea of how to you know customize makes me use a particular structure um so i'm already kind of halfway there where do i go now right so um it's not all unicorns and rainbows right once you start using customize as as some of you have might known here uh gerald if you if you're watching this i'm sure as some gerald and i have discovered about uh uh structures right so um so there's different ways all right so now that i have um a structure like what do i do with my repos right do i have like a mod what we call mono repo do i have a single repository with everything in there right like like my whole just dump everything in a single repository do i have separate repos for environments to have a repos specific for clusters like like now the question becomes do i do monorevo do i do poly repo do i do a hybrid of the two um so just kind of some of the things to keep in mind um originally argo cd um i mean i guess even still today they realized this has scale issues with mono repo so if you're going to use argo cd big argo cd deployments buddy yeah let me tell you yeah yeah um it has scale issues right so meaning that if you're gonna use monorepo you just kind of have to um you know give your argo cd deployment a little bit more memory maybe scale it out a little bit it it um it has just issues with with monorepo so yeah i mean i've i've seen where people are like argo cd controller and then it's just like scroll scroll scroll scroll scroll so many things there and i feel like that's an anti-pattern right yeah so well what's funny is uh william who's actually on the chat right now one once um uh once showed me he goes here take a look at this video and he and he showed me the repo and like the first five minutes was him scrolling and like i'm like and i went back to chat i'm like hey the first five minutes is just you scrolling the eagles actually the whole video is just me scrolling this is how long it is how big the repo is um um so yeah so to do so this is a good question from from william here to decide between monorepo versus polyrepo should it start with the operations team what the operation team looks like is it distributed is it one team these other questions so like these are some of the questions that um and that's kind of like one of one of the bullet points is here the way your organization is laid out is going to come into play and will kind of dictate a lot of this for you right so you kind of have to ask these questions right you have to say okay you know what does my operations team look like where are those demarcations right what is operations team sre teams development team um what are those teams look like and where are those responsibilities uh um marked in so gotta ask those questions to kind of come to a conclusion of whether you're doing a monorepo or separate repo polyrepo um and then um so again multiple repositories you kind of get into the the idea of having a um i wouldn't do it this way but i think this illustrates this point is that you have a repo like per environment for for instance or a repo like if you're using um uh micro services right maybe you have a repo per service not necessarily one repo with all services in it right maybe you have a repo with all that um that's kind of uh one way to do it that's kind of the way i do it i actually see things the way the question that i ask is that is the cluster i'm on is it multi-tenant right or is it uh not meaning like okay how many teams yeah how many teams right like so like if i'm if i'm an admin maybe i'll control the cluster in my repo but then i'll have a multi-tenant system where it's like okay you know um these developers have their own instance of argo cd that then they you know um deploy their application using a specific namespace yeah exactly 100 so this is kind of like what the multiple repositories come into play and how your organization is laid out um you can also go full devops is what i like to call full full devops right and have a single repository um and um and like single repository meaning like all right like i have a one-to-one relationship between a repository and a cluster and so i have my whole definition of a cluster in a git repo and that just might be devops style right just one big repository for a cluster and changes and all that go into one get ops work uh get workflow for your get ups environment so um so yeah so you have this single repository um which is fine right and remember i um i'll put this in the in the in the chat if you're using argo cd just remember there is scaling considerations if you're gonna do a mono repo right so um if you guys again if you haven't seen past episodes the intuit guys came in the guys who actually wrote argo city and they've actually said like hey yeah like we know this issues here and um you know we're working on it so they they they already know right so um but just something to keep in mind see here um i only have a few slides left here let's talk about um questions to ask yourself right and i didn't realize this i thought this was a big number one yeah well like i was i was trying to think like all right things to think of question mark question mark could have been yeah i don't know well i was trying to decide between maybe the face emoji thank you face emoji i'm a big fan of that there's probably i don't know how you made that that big and didn't realize it i don't know yeah well this is the pdf maybe it doesn't show on the google slide so um so i have a few tips and things to ask yourself right and i think uh william put in the chat very i was very um uh some of the questions they need to ask i like yeah so i like some of those um uh the question mark uh thinking face right whatever um oh another another good comment i'll i'll get to that comment in a bit here so some of the questions here is about um so some of the tips so utilize customize and utilize to refer to other git repos right so if you're not doing a mono repo if you're doing poly repo um you can aggregate a lot of those repositories um in a single repository so you can do something like hey for this deployment i'm taking these get re get ops repos and you know making one and i'm deploying that to a cluster you can do something like that make sure it's easily repeatable i have a cool workflow it's like hey i have another cluster i just copy my overlay to another name change one or two things and then off i go so um so make sure you're you're it's easily repeatable right um and then um i didn't say this before it's one of the first things i always say when people starting off in this journey keep your application repo separate keep your code repo separate from your deployment rebound like yeah different types over here infrastructure yellow's over there you got to keep those separate keep those separate because they have different life cycles so um you know you don't want a scaling you know application code hasn't changed and i want to scale it you don't want like a build of that application to kick off because you changed the replicas from one to two right so or vice versa you upgrade or vice versa version and now yeah we got to rebuild the application or something like that yeah yeah so there's there's a um they're just different life cycles it's just not you know that's just it just makes your life easier um so things to ask yourself right i think um one of the questions one of the things william says do you want job security is one of the questions do you want job security get get into get ops um so i thought that was funny um so uh you know do you have an environment specific configuration or do you have a cluster specific configuration you have a mixture of both um me and gerald we were butting heads for a long time i for the longest for the longest time have the opinion that you have cluster specific consideration uh configurations or you have environment but you never have both because one is a subset of another right so you know if you have um two clusters in a dev environment and one of them slightly different welding you actually have a cluster specific configuration and not a environment specific consider configuration um but there is so after speaking to a lot of customers and after you know going over a lot of use cases um going on this get ops journey right we're going to the galaxy right um there are some some instances where you're going to mix both where it's um and my i guess my last tip or last thing i want to kind of want to go over in the slide dump here um yeah so so william very good point in in when when gerald noon um watches this later he's gonna he's gonna chat me he goes i would argue it's is you have three you have cluster environment and team right and um gerald actually so if you if you look at gerald's repo he actually has a concept of tenants right and that's what a team is right so like a team within a so you actually do have like three um so there is uh um so i actually acquiesce i actually thought okay hey yeah gerald you're right um i jokingly say gerald opps i i succumb to gerald opps where yeah you do actually do have three um i mean i like geraldo that's fun i like gerald ups i i've been using it for a while yeah seems to be working okay for me so um another way to approach this and this is something that um i kind of did with one of the customers i was working with is that you kind of have a fan out or a cascada uh approach right so for you engineers fan out right or case cascade same same idea right um the um you have a there is such a thing as especially if you're working with your security teams which you should be working with security teams a lot especially with literally everything please everything right devsecops is is a thing right yeah it's the way now it is the way you've seen the headlines lately yeah sorry yeah exactly yeah naveen for tenants do you mean name spaces so yeah so this is kind of what what um um what uh the idea i agree with gerald but the naming i kind of go differently yeah and he and he kind of says if you look at his repo he kind of mentions it as well um when i think of tenants i think of name spaces so i actually say teams right instead of cluster environment team so kind of like what william put in the chat that's kind of what you mean um when he says tenant he actually means teams so um so yes so so that's that is probably a poor choice of language i'm sure it doesn't translate well you know yeah yeah exactly yeah so um yeah so uh yeah a team can have multiple clusters it's a function right exactly yeah yeah so you can have um so yeah so i think i think team is uh is is a good uh um a good analogy i i like the name tenant because like a team can be a tenant on a cluster right but like you said it's probably a bad it's a bad choice of words because it confuses with name space yeah um so um yeah which i agree with so i mean like like we said at the beginning there's more than one way to get off this cat yeah yeah the only one way to get ups is cat so um so you have a so like if your security team or operations team you're gonna have a base configuration right so no matter especially in an organization no matter what cluster i install it's gonna have to have these basic sets of cluster configuration right and um like i mentioned before in the other you can reference other repos from customize then you can branch out to say okay and now i have a dev stage of production right within that dev i have yet another repo of cluster one cluster two cluster three ad nauseam and then that eventually makes it onto a cluster right and the same for stage and prod right where it fans out um in this particular way i um so you're talking about you know three four five repo's deep um so if you have a change that needs to go to this one particular cluster you just change this repo and you know it makes it in there if you are if there's a security patch that needs to happen you can do it in this repo right the the base configuration repo and then it just basically cascades out to all the other ones uh same for the environment you you kind of get the idea here so um so here uh the question good question here so uh basically is it better to have different repos on an application level or an environment level like pod and or like prod and dev so um you have different yeah great question right so this is another one of those um one of those things where it's um again it depends right so i think the application level um would be like in the environment layer i would i would say um i agree with that that's where i think i think most of that um where that stuff would go so and then someone said uh again the follow-up question was what's the meaning of cluster repo so cluster repo is um the repository where your cluster-specific configurations go so for things like um like security policies would go in a cluster repo or things like um you know connecting to an identity provider right your login information would go into that repo so um i like the the com i'm holding for dev third ops right so it's uh wait we're gonna have dev something ops is that's like the new thing right so yeah um dev get ops they have given up there's a there's ai envelopes there's this yeah oh my god don't get me started coming to a theater near you yeah coming to it come into a creator near you so um so yeah so that's that's pretty much it of my of my tips um uh these are good tips man like yeah yeah so there's it's kind of just some of the things you have to ask for questions um i again just to kind of uh wrap it up kind of little bow is the uh get familiar with customize get that power under you um because that's going to dictate a lot uh ask yourself questions like you know you know what does my environment look like what does my organization look like uh who's controlling what that will then be on customize then we'll tell you what what kind of repos are you going to be using so um so one of the to be solved issues is one of the things that coming out from william here one of the to be solved issues we have in telco if i update the base on a on a cascade or layered setup how to prevent immediate remediation to make sure remediation happens on authorized maintenance windows but that's well that's all but also that's an old versus new mentality correct right so um um you you never commit so the the so there's there's kind of multiple layers here you never commit to something that you don't want live right away so right just you know like if it you know think think again remember stage that thing yeah and wait yeah converge it until your maintenance window and then just hit merge yeah correct and then just then emerging during uh or you can use some other tool to kick it off during your maintenance window or you can use the crc or something like that yeah anything there's there's also in argo cd there's called sync windows i think it's called yes let me see if i can find it yeah sync waves sync windows uh argo cr figure what's called it's called uh sync windows here we go okay yeah so sync oops uh sync windows here um allow you to sync things at a certain time nice so it's like crown job format we all yeah cronjob format it's like hey um only sync this at this time or during this windows right like sundays at 8pm or whatever so you can do that as as as well right so you have a sync window um to set up on argo cd so that's another way of doing it but again uh i kind of i kind of agree with with william here it's kind of like the old hat versus new hat mentality it's just like well you know if if you kind of have to change switch the mentalities like if if i don't want it in production right now then don't merge it because that branch you're tracking is your source of truth right think of it as a database of the live or what's what's going on right now um so so yeah so um so yeah so william yeah so if you weren't aware of the sync windows there you go 100 there you go you use those you can say hey i only want you to do old school with the new school man yeah yeah now we're doing both right so like if you're doing um uh where's that oh wait it was in my my slide right here so that's on the application um level right so you can have it pretty much on any level of where you want it right so on the right on the cluster first so essentially it's like hey for this application you know sync it saturday at midnight or whatever um oh my gosh yeah all right let's see virtual case clusters inside nam namespaces i'm assuming this is for a test case not an actual yeah so you can um there is there's a project that actually lets you do that uh but it's a project that chris doesn't like the name so maybe i can just like put it what what project it's uh i'll just type um oh no don't put that on my channel please don't so bad yeah so so the problem with um uh kubernetes in kubernetes um is that you can't really google for it it's it's right um it's like a circuit it's like a infinite loop kind of thing yeah yeah um in uh there's um anyways well we'll find well we'll i'll tweet it out later i think no i think we retweeted it and we talked about how the names should be changed but we just we talked extensively about how the name should be changed yeah yeah none of our none of our tweets were answered yeah yeah i want to file an issue but like i don't want to file an issue at the same time it's like you should know better than this yeah yeah they call something in kubernetes land like it just violates the code of conduct yeah yeah exactly um so it's uh but yeah so for uh uh who who asked this question here uh oh uh naveen yeah so you can you can run kubernetes inside of a sign up kubernetes namespace using kind um so i'll just let you google for that um it's a brand new project but it's possible so um uh see here uh yeah kind kinds the the name of kubernetes in docker and then which now you can use podman for you can use podman for yes correct i haven't gotten that working yet but i haven't got that working yeah i haven't read the documentation yet but i haven't dived that deep into it but i do have podman machine here on the apartment machine you can't do it right here yeah yeah you can do uh ocp over ocp vert you can also probably do uh snow which is single node openshift yeah as a virtual machine yeah so you can you know put that in your cicd pipeline um to test some of these things right now yeah i highly recommend kind in your pipelines for sure yeah yeah for sure yeah yeah so for like nine times out of ten if it works so if it works in kind it'll work in open shift that's easy if it works in openshift it may or no wait yeah it's one way that right i think it needs to be like it needs to be hardened more to work yeah yeah no if it works in openshift it'll work in kind yeah yeah um sometimes if you don't build your containers right it won't work in openshift because that's um uh if if you're running like as rude or something openshift will kick that back so yeah um maybe one day i'll do a um a stream about pipelining with get ups because i think that's a very um you know how to how to test something yeah right bring it from that test to qa to fraud process yeah especially for the clusters right because that's kind of what we're talking about right now exactly right because we're talking about this cluster even work yes or not yeah exactly if i make this change how does that going to look like in my cluster yeah you know you have to fire off you have to spin up a cluster and test that change so i'll i'll do some um i think i may be booked up for the rest of this year so maybe it's something early next year yeah yeah so something we don't want to get ben around for the actual yeah developer that'd be cool yeah that'd be cool oh man having him on that'd be great it has so much so much praise for that guy he single-handedly changed changed how we how we test right he just happened he made he made everything easier just kubernetes contributor workshops the whole nine yards just got a whole nine yards by one project yes this guy's one idea kind of just for for everyone right even even uh here at red hat a lot of us is kind um so cool so um last four minutes um i don't know if you guys have any questions or any any other comments um hopefully you enjoyed this kind of like uh the overview of kind of the kind of the best practices yeah this this has been insightful and eye-opening to i think our audience and myself right like i always think it's great when i learned something on our shows right so yeah so we got oh jay that would be a good show so i will definitely put on the list we have a backlog of you know so i can put that on the list and it'll um we'll eventually get uh get that so that's that's a lot of cool things so um i think that's it do we have another show before kubecon uh oh we do we do on the seventh yeah on the seventh and i think that's when oh okay cool so next show so uh um october 7th right so it's the show right before q con um we'll have michael foster on so michael foster is the uh tmm for acs so for those of you who are using uh who are using stack rocks or now that you guys are using um you know red hat customers using ac acs security advanced cluster security so we'll be talking about the um um uh the application delivery um pipeline right so it's kind of like that the the idea of building security into your delivery system um so that's kind of i like talking about some of these things that are kind of like acm the cluster management where like git ops is kind of like the core of something bigger acs is kind of like the same same idea um he'll be on talk about that so um that'd be a good show yeah so how do i stop sharing um gotta find the uh the bar gotta find the the oh there's a big old red button yeah uh so yeah so uh that's it let's see if there's any anything other in the chat here no i just dropped the link to the calendar for folks i haven't updated it for next episode yet but now that this one's done i can so you know it's a timing thing you know as always it's always about timing timing is always it's always hard all right so uh yeah folks this is this is actually our last stream for the week uh which is awesome um well i get to crunch numbers tomorrow so yeah that'll be fun uh yep yeah stephanie was pinging me we're trying to crunch some numbers back and forth here so yeah yeah you know math is involved and lord knows i'm not good at it so that's that's always fun yeah that's why i have a computer do the math for me nice shirts thank you yes yes i have cool shirts yes and uh so yeah stay safe out there folks and uh everyone we'll see you next week yep cheers you

Channel: OpenShift

Views: 312

Rating: 5 out of 5

Keywords: Red Hat, GitOps, Red Hat OpenShift, OpenShift Container Platform, OpenShift, Kubernetes, K8s, OpenShift GitOps, GitOps Operations, DevOps, Continuous Delivery, CD, Repo Structures, GitOps directory structure

Id: HDg5vh97zmI

Length: 57min 25sec (3445 seconds)

Published: Thu Sep 23 2021