Why GitLab CI/CD? - 5 hacks to improve your CI/CD set-up!

welcome everyone uh thanks for joining today's webinar we are very excited to have you on my name is timothy tran and i'm a field marketer here in apac at gitlab so in today's webinar we've got jonathan joining us from singapore who would tell us more about why gitlab cicd with five hacks to improve your current ci cd setup as well as special guest rob williams from my previous webinar joining us to answer all your questions during the live q a at the end it's just a little housekeeping before we get started this webinar is being recorded so the presentation and slides will be emailed to you after the webinar is finished if you've got any questions during the presentation please just type them in the chat box in your zoom control panel below i'll compile them during the presentation and we'll give some time for rob and jonathan to answer all your questions at the end we'll also have a couple of polling questions to kick off the session it will just pop up on your screen and all you need to do is pick an answer that is most applicable to you so let's get the show started shall we i'd like to introduce to your jonathan liam he is a senior technical account manager at git lab prior to joining git lab jonathan worked in the big data analytics and application monitoring space having seen the the cost to rectify production issues jonathan wants to help organizations like yourselves adopt devops best practices to fix problems earlier and quicker we've also got rob our panelist for the q a um he's an experienced devops engineering consultant who in the past has worked with large enterprise organizations to build and deploy web applications as well as roll out digital workflow transformations before joining us here at get lab where he's been helping organizing organizations bring more modern devops best practices to more companies uh jonathan please take it away all right hi everyone um i'm just monitoring the chat it looks like some people are having uh issues with um voice or something can you hear me all right tim it's okay uh yeah i can hear you okay anyone else having problems ah okay i think just just uh one person okay right thanks thanks for feedback yeah okay okay cool um yeah so um i'll just move forward um sorry about that i was monitoring the chat because i think some people were having issues there so first and foremost uh thanks everyone for coming so my name is jonathan and uh i initially and tim has actually uh introduced me so just to get an idea about like uh the you know get a few around the room like how everyone is comfortable with the topic on like ci cd and devops i have a few polling questions um it'll be great um to get your feedback on that so i can you know tailor my presentation in a way that helps that you know in if there are certain words that are more technical so i might give a bit more of an explanation so let's just get on with it so with the first question is um how familiar are you with continuous software development so yeah this again is anonymous um it'd be we just want to get an idea on like what people are having all right so we have about 70 80 of the people responded also let's get just give it a few more seconds right um so so these are the poll results i'm not sure if you can see the poll results um but we we basically have some level of familiarity i think it's a little bit somewhat familiar familiar and not familiar uh so they're they're i guess a lot of people in this uh call are quite familiar or at least know what devops is about uh just two more questions about that so the second question would be are you currently using gitlab ci cd okay eighty percent of people answering we are pretty much next to next yes and no so uh that's that's good to hear actually um so um it'd be great um so for maybe the folks that are using gitlab cicd you can help us as well to uh answer some of the questions if they appear on the chat as well so we're getting uh for the poor results we are getting pretty much neck to neck on about 50 yes 50 no uh on on salt and last but not least the last question would be how far are you along your devops journey from one just started to five very much work all your teams are using it um and um you know basically you're here to fact check whether i'm i'm talking about the right things uh okay so yeah i um some most people as i can see are within the earlier stages uh from one to three which is great i guess and so i hope this session can be a little bit more informational about how we see it here at gitlab and let's just move straight into the presentation um right so in 2011 mark and anderson who was a former founder of netscape mentioned this cycle time compression may be the most underestimated faults in determining winners and losers in tech so what does this actually mean is that the quicker that you get feedback from your users from your tests the faster then you know you're off track and therefore that results in less time less money and less effort that is wasted for you to get back on track right so in the past as we we know as software development lifecycle we had the waterfall methodology of how we develop products uh there is of course the design stage where you get the designs from the end user from the customer and then you go on and then uh six months later you say hey customer six months later i'll come back to you with a prototype or so and we'll discuss it six months on when you reach the when you have some prototype ready uh what happens is the customer might have moved into a different space their business might have evolved their requirements might have evolved and they come back and tell you hey you know this is not what we want these are the new requirements that we need but what have in that scenario you have basically wasted a lot of time because you are already moving in the wrong direction that gave rise to the idea of like uh agile methodologies um in which you get to see your feedback faster you don't have such long um uh iterations where you where where iterations can be three months or six months long but what you see is that um you um in agile methodology uh you have a smaller number of features that you would develop on you would then push them into production and you ask the customer for feedback quicker devops came around around the same time as after agile methodologies came about in which you you then have you give power to the developers to allow them to be empowered to know how the feedback is like to to not wait for uh to have its siloed teams where developers only develop and in your operations teams only just monitor the end product so organizations then move to the next stage right so organizations who try to adopt devops try to bunch of buy a bunch of tools to help the organization and they said hey you know i just bought a bunch of tools developers here you go please use it automate it you know automation is always a very big word and then they want the developers to actually adopt devops sensor too but as we know uh or also as uh the polls a lot of people do know about devops devops isn't just about two this sounds a bit hypocritical coming from product companies such as gitlab but i still strongly believe that it tools are just in eight in in helping you to adopt a devops methodology at your workplace but it really boils down to really the people processes and the tools as really the last thing we can't really help you with the people and processes we can have a chat later on and then as a technical account manager that's my job as to guide customers along that journey as well but today we're just going to be talking about tools and how we can simplify these tools for you uh just as a side note there's a very interesting book called the phoenix project that you can consider reading about devops it's not very heavy it talks a little bit about devops in general and gives you high level overviews and it's a little more engaging if you want to learn a bit about devops but that's not what i'm here to talk about today i myself consider i consider myself as a millennial i mean i am a millennial and i only had my first phone when i was like 14 or 15. uh back then it was a nokia phone uh it didn't have any uh cool features i only could play snake on it and at that point in time we had dedicated features and functions for each different tool so we had really good cameras from nikon and canon we had like mp3 players we had a walkman which played your your cds and so on and so forth we had dedicated computers pages that did what they did really well right so they were very good in their own right but what we ended up with was that we had so many devices that came right with the next component so i think uh the first one that we that i think a lot of people might think about is the first iphone the first iphone was really where the uh the way where you actually see all of these different tools come together in a single uh in a single uh device so you had on an iphone you could have your phone your calendar your cameras your your your you could see whether you could hear music you could browse your web and then you basically replace your alarm clocks uh a lot of the time right uh in the first iteration of the iphone of course the the camera wasn't the greatest but as you know now we are i don't even know where we are at now with the iphone x or uh 2x or something um the the features and functions have really uh improved over time so i think that's kind of where we see ourselves at gitlab to be in the same direction as um we want to have an all-inclusive platform right a complete devops platform that that runs end to end right all the way from manage to plan create verify package secure release configure monitor and defend all in a single uh all in a single platform only in a single pane of glass where you can actually see everything all with using the same permission model so you don't have to integrate all your ldaps with your many different tools so and so forth all in a single data store and that kind of simplifies it and more often not we also preach about a lot about it in within the company about collaboration between the tools so if everyone is using the same tool it doesn't leave a lot for you know um how do you say conflicts between teams which is like hey you know my tool says this but your two says a different thing so having one single two is what we then um would uh prefer or what we what we uh that's our vision for the devops life cycle unfortunately today uh as we are only going to be doing this for about half an hour or so um we um we will only focus on these three three stages again um with me we have rob who actually talked about a little bit about a different topic we had summer which was the previous speaker as well which talked about devsecops another topic in in general uh but what today we're going to talk about is these three stages which is verify package and secure or what most of us known as cicd um uh we will this is an ongoing uh webinar kind of session and then we'll be talking about new like other stages as well so stay tuned if those are the things that you're more interested in so let's define what ci cd is so ci cd in a very very short definition also so it's embodies a cultural set of operating principles and collection of practice that enable application teams to deliver code changes frequently and reliably i think one thing to note here is that you don't see the word tools right so again i want to emphasize this even come coming from a company that sells a tool is that tools are just a way to help you to enable that culture their operating principles and um hopefully true today i will share with you some tips or some tips that our larger customers and gitlab are using today that help them to actually adopt these principles a bit better by the way yeah i think by the way i think rob and tim and the rest of them are monitoring the uh chat so feel free to ask any questions if you um you don't have to answer questions at the end of the presentation you can ask it throughout but we will address them at the end of the presentation as well so why ci cd so why do people use ci cd because obviously it detects errors faster it reduces integration problems it allows teams to develop faster and most most uh and first and foremost is it helps you to develop to deliver value as quickly as possible um gitlab has its own way of cic or the way that we actually have a methodology around at ci cd and this is uh to understand it first is to understand the gitlab flow so right in the middle here you do see the master branch so the master branch is where how a lot of people see as the source of true when you were to actually make a code change you were to actually develop something new in terms of your a new feature for your application you can create an issue first and then that would as well as a merge request you commit your changes your ci pipeline runs uh you then have your security scans uh and all your other tests that you have in in your pipeline you review the applications you approve the changes based on your team lead or how your approval changes are and then you merge it back into master right so how we see this works is that this is the git labs flow and this is the cornerstone to how gitlab continually integrates and deploy right why is this important because you know a lot of the times when developers they develop on their own laptop or my macbook machine also you know it works fine the code works completely fine on my macbook machine because it's a lab environment right so there are no external factors there are no you know uh networking factors there's no like uh uh other factors that involve like even for your for your your web applications or how you integrate with other tools so having the the following this gitlab flow to test it even before you merge it into master is very important in our case site tracking a little bit um that has the whole idea of that desect as summer uh our our senior architect uh that's based out in melbourne uh talked about in our previous webinar um that's also really important um so as you can see here we have the gitlab ci so you actually integrate continuously the other portion of it is the security testing in the pipeline so following back on my previous slide which is about the um the gitlab flow so you test all your or use tests for your security vulnerabilities you test for your um your license compliance you set your test for your code management so and so forth and then you give the feedback directly to the developer even before it gets released to the security team right so get this this empowers or has the idea of shifting left to empower the developers to actually get a feedback on whether the the new features and functions that they are actually developing is in compliance has no security issues uh so and so forth right so again i can't really uh go too deep into this topic but if you would like to have a chat about it let us know and then we can have a separate conversation about it um expanding a little bit about the gitlab ci cd workflow so within the continuous integration component you can have it as i mentioned earlier your code quality test your performance test you can integrate with your third party uh tests as well you if you have like junit testing um you have unit testings available uh in your own flavors you can definitely put it as part of your ci pipeline and then you can use the the the features that are available from gitlab which is like container scanning dependency scanning fast testing which is something that we've added recently as well to our security stack and moving on to the next component of it so like your package registry or how you manage your packages for maven and mpm and even to your release stages where you integrate with technology such as kubernetes for your canary deployments and all right so today is going to be a very high level review i have i am not able to go through every single feature of them but if that's something that's interesting to you and you want to have a discussion we can always have that later on um so in general how we see our ci cd pipelines is this firstly there are the main stages you have the stage for build test review and deploy these are just some example stages that you can have as part of your ci cd pipeline uh it doesn't mean that it's limited to only these four you can have custom stages as well if you want you can have build you can have you can split up these stages as you need to and you have can have many multiple number of jobs in these stages that can run in parallel or you know only after or sequentially after one stage of after one job has finished right so that kind of gives gives an overall or like a high level overview in terms of setting the ground in what gitlab believes in ci cd and how we actually see the devops landscape to be in our own words uh the next thing that we want to talk about which is basically the meat of the presentation is the five hacks all right so um what led me to even come up with this topic in the first part is actually observing customers and talking to a lot of the people such as rob and summer and a lot of my colleagues as well what are the customers what are customers asking for today or what are customers facing problems and challenges in their organizations today that has led us to this topic which is five hacks so first let's go on to it so first things first is the problem where customers always come back to us is that uh first things it it as the poll suggests so some of you you guys some of um your teams are having issues with or you find it very imposing very difficult to switch to devops one of the reasons why is that a lot of them you don't have the proper resources you don't have the proper people with the correct toolsets to be able to adopt these devops practices or this or to adopt ci cd involves a very steep learning curve and i agree to a very to a large extent but how do we solve this problem all right so i'm just going to go through the problems first and the next few slides will actually talk about the issues the next one is how do we ensure consistency of our application builds over multiple cloud environments as a lot of companies are moving to the cloud native environments also ensuring consistency over across azure aws across gcp is getting more and more difficult and how do we help that number three how do we handle secrets and environment environmental variables as per company guidelines or even by audit guidelines per se uh number four how do we effectively run tests monitor applications that involves micro services and multiple technologies especially when we work in silos in a development unit right so these are some of the problems that we see our customers come back to us and ask us how can get that help in this scenario and these are some of the solutions that we think or some customers that adopt these solutions and we think that might really help in this case um right so first solution simplify your ci cd uh the steep learning curve uh you don't have the proper two sets and and and all that developers find it very stressful to actually learn deployment codes such as your deploy your jenkins jenkins file or even writing in gitlab ci so before i start um maybe just to have a bit of does anyone know what this tool is that i'm showing on the screen here mixer i see mixer any other guesses also ah okay all right i i see a lot of people know what they're talking about okay filter yeah yeah okay so i think that yeah there are many people who actually know what this does um so yeah this is actually a thermomix and if if you like squint uh oh we even have the actual model which is uh tm5 right so so if you screen hard enough you can actually see the brand on it this is not a product placement nor are we sponsored in any way but what this tool actually does is that it's a food processor and cooker per se so it's an all-in-one blender cooker bread maker and it basically is is able to uh cook anything it has it has basically over 600 recipes that you can actually cook and all you need to do is just put in the uh the ingredients and you'll actually cook the dish for you right so that's pretty amazing right and i i i think s had it to forget it okay right i i see there are some uh fan uh fanboys or fangirls out there that really like this too so in in the same vein of threat uh i i think where we are coming from is that we also believe that you know we don't want these things to we want to automate some of these things we want to make it easy so what how would how how nice would it be if you were to be able to actually you know uh you you can sense what the ingredients are so and then automatically help you to deliver the end product as we go i guess that's where we come in with um the idea of auto devops so with auto devops is the idea that all you need to do is commit code push code and gitlab will automate the rest for you right so what gitlab does is that if you push a code in a proper format and of course there are some configurations that to be done i'm not going to give you the the sales pitch is like oh just put it in everything's fine but there are some uh very basic uh configurations that you need to do um and it will automatically help you to create these stages so from create merging and building to verifying code quality testing security in different ways also for packaging for releasing configuring and monitoring if some of these things are set up properly and that's what we think is the best way to bring these best practices to develop to delivering software uh this is an example when i basically use a template of a spring boot application that's written in java so i i didn't write any ci file i didn't do anything i just start with a template and automatically it would help me to and i run a pipeline it would help me to actually set up these stages for me depending on what kind of different technologies you are using we would then have different pipelines as well that will help help you to set it up accordingly right so that's my first uh thing auto devops auto devops again it can be used in conjunction with your with with your customized ci pipelines as well it's not just the case where you just have to use autodevops and you know it's a fire and forget uh you can use it in conjunction and but it what i'm trying to drive at is that it's a good start point right so for teams who do not know what to do at the start and are trying to adopt devops practices that's a good start point um for you to start and to get that feedback quick enough all right so that's one down there's a four other small to go so point two effective deployment of cloud native um applications so how do we ensure standardizations of deployment across multiple cloud providers and deploying on native uh applications on cloud native right so um this is typically how a lot of developers or if like a lot of our customers that make use of gitlab and and in this problem is that maybe they do have a gitlab instance they have and they they have many projects right and they have many projects and they span across multiple um products as a cloud vendor so they might be deploying to um amazon so aws or google gcp as well as microsoft azure and they have different technologies they might be trying to use gitlab to as git ops to actually deploy infrastructure as code or even application as code but it's very very difficult to actually ensure that this standardization happens so why is that difficult say for example if uh your application now support nginx 9.0 also but uh nginet com nginet comes up with a new version of it how do we ensure that that is standardized across all your three different cloud vendors how do we ensure that all our application adheres to all these security best practices within organization as well so um we at gitlab i think we leveraged on a few different methodologies or tools as part of our devops process and first things first is the idea of build packs right so the answer to that is buildpack so we started out with a heroku open source based project that's called herokuish and now we are moving actually to a new standard for um cloud native uh build packs right so in in a nutshell what uh herokuish or heroku in cloud native build packs do is that when you deploy code uh what it does is it analyzes the code it selects the build packs or if you have certain assigned build packs to your your your code retrieves the assets help you to compile the code and then create an app slug for you um to actually deploy into your environments in conjunction leveraging with a helm charts and auto devops um it can then help you to deploy straight into production or your uat or testing environments in this case right so that's part one of how we can actually enable that that that standardization why is this important because build packs firstly they help you to maintain a control between the apps and the developers to ensure that standardization also allows for compliance there's very little effort and intervention when you're using build packs and uh as well as why why are we i think that one of the questions that we we got is that why are we moving away from herokuish to actually uh cloud native build packs um so first things first herokuish is a open source project that's maintained by the community we are not sure when support will stop or when people are actually not going to contribute to this project anymore at the same time we also it wasn't heroku-ish wasn't born out of a cloud-native background um so the docker images are a little bit larger and they're the long end in our team we actually uh monitored and the build times are a little bit longer so moving on towards uh uh something that the cloud native computing foundation is working on is something that is more cloud native it's a bit lightweight and the build times would then improve uh accordingly and that would also help you to speed up your efficiency and um automation uh process with autodevops okay so that's part one a of it so part uh part two a of it part two b of it is actually our kubernetes integration right so uh we believe very strongly and we we invest heavily in integrating with kubernetes to actually have some of this dashboard to get that feedback early on right so having the deploy boards canary deployments um including autodevops kubernetes monitoring and uh integrations with web terminals that's where we are have we are at with kubernetes integrations and what you get to see out of it is like um how how fast or how having a little bit more control and a little bit more view into your deployment models so part of your cd component where you can actually see how many instances or how many ports are up and running in your kubernetes deployment you can actually also see in terms of your canary deployments on production are they working fine are they up and running you can also monitor it directly in for your node metrics such as your average memory utilizations and cpu utilizations and all of these ones are all available in the gitlab platform so you don't have to switch to a separate two sets like um using uh maybe a kibana or grafana dashboard to actually view all of these statistics and metrics you can see them all directly in the gitlab platform so again going back to the idea of having a single devops platform where you can see end to end in terms of like your feedback and and all that right so that's point number two okay so first one we talked about auto devops um to to actually go um uh to actually climb uh to go past that that that mountain of like you know adoption uh the second one we talk about how to maintain that standardization and number three handling multiple pipeline projects so over here in the screen if you can see here this is the basic gitlab application architecture you have multiple components such as you know your in your nginx your workhorse your um unicorn that's running on ruby on rails you have redis so and so forth so this is a architecture that might be very familiar to you uh even in your organizations where you have multiple services or micro services they are talking to each other and teams in general wouldn't be working on every single part of this uh architecture so you have a team that maybe will be working on unicorn and another team that's working on your workhorse right but how do we test this effectively so at gitlab we do a lot of um we do uh we make use of we make use of our own products and we are able to actually test this in in in parallel right so able to test on our own application and how it interacts with uh different applications right so that might be the same at your organization or that's what a lot of our customers are talking about where application teams might want to see how my new code that i've been uh that i'm pushing in how it interacts with a different component a different uh yeah a different component when when something new gets pushed up that gives rise to the whole idea of multi-project pipeline so you specify a branch you can pass variable variables from uh to downstream pipelines and if downstream pipeline fails it will not fill the upstream pipelines do we do it so in your ci pipelines you can actually specify a new stage called bridging and in that new stage called bridging bridge sorry you can then specify the project branches and your strategy on how you actually want this to run what it would look like in your cia cd pipeline would be this so firstly you have your initial branch that may be written in as i mentioned it could be running on ruby on rails so it goes through the entire build process it actually deploys all the way to your downstream and then once everything passes it would then actually trigger the downtrend project which is another project in gitlab to actually run their own tests and their own ci pipeline yeah so that's how we deal with multiple projects and a lot of these days like advanced projects that you know have micro services that are talking to each other so that's point number three right so we we we've also covered the the issue on like how do we test effectively for multiple projects uh and fourth right so how do we optimize resources using gitlab rules so so prior to rules the gitlab prior to actually 12.3 uh there were only conditionals that we could allow so for own we had conditionals that allow you to specify only or accept uh into your in in into uh your conditionals uh and after 12.3 we actually change the rules a little bit better so it's a lot of improvement after 12.3 so we provide the users a bit more control um when on how ci pipeline should run we understand that compute resources are very expensive and if you maximize your resources um you can save a lot of money right so you don't maybe certain stages you don't want it to run for uh you you can have you can specify tax for certain ci pipelines uh where for example if it will be part of this conditional it doesn't run and it therefore saves a lot of computation power so i had a previous colleague at my uh previous uh company that was telling me that a certain large emanci in korea was spending approximately about 800 million uh dollars on aws spending uh in korea so i think that's one of the biggest problems that a lot of large organizations these days face which is trying to reduce their compute spend and how to actually have more control on on on these things it's very important right so even in gitlab uh prior to 12.3 we had a lot of these kind of errors so you would commit a certain code and that certain code might because it was because of the text that you placed in and because it was a merge request as well it could potentially run two parallel ci drops at the same time running exactly the same thing uh that number one is a waste of resources you don't want to spawn ci jobs both at the same time because they are basically running the same things and at the same time you don't want to make your runners busy in this case so how do we actually put rules into our ci pipelines right so this is another hack that we we use or or that we suggest moving on from uh on just using only or at or accept is to use rules in your blocks so specifying a rules in your block and specifying your your conditional so for example in this case if your ci pipeline source equals to web then it would then run the script as it says there are a lot of clauses so just now we mentioned in our previous example there is the if clause that you can use there are also other clauses such as changes or exists and there are other operators that you can use um to actually specify your gitlab uh ci uh to have that additional control also so some of the ones that uh that are very uh popular is say for example if you uh if the environment is you are deploying on is to uh production then there are only certain type of jobs that would kick in if you are not if you're only deploying to a test environment then you can only spend you you'll be only running a certain set of jobs uh again this is one of the examples so in this example you can see that um in this job for example if the ci pipeline source is a merge request event or if it's a scheduled uh project so you can then have the control to actually run the script um hello hello world or echo hello rules right in this case so there are a lot more examples but we i don't have the time to actually go through all the different examples also so just do a quick google and you probably can see what some of these examples are in terms of your ci pipelines okay so the last last but not least i think it's uh environmental variables and secret management um and i think this is really important today so over here on my screen you can see uh i hope no one here is working this is not a call out to the company so if you're working for like facebook t-mobile or like uh robin hood uh this isn't this isn't like trash talking but i think the idea behind it is uh what we want to talk about is uh that a lot of people uh we we still today uh face the problem of having to store a lot of our passwords in clear text and we don't do them in a proper manner um not only just passwords but also environmental variables that might be crucial in terms of connecting to your environments in the cloud or even your on-prem or certain other features and functions right so um get that so during so that this is a very important component and i think this component also ties in greatly into how we actually manage these secrets as well as these environmental variables well enough so i'll talk about two areas that you can actually control these variables so first one is under your ci cd uh settings as part of your variables you can actually specify your variables here uh and if you can see the ver the values here actually uh you can't actually see them you can review these values and the and you can con you can control like who can see these variables um via our back sources so a row based access uh sources so maybe only the admins can see it but if you were a guest user or you're a developer you can't see these variables um and then you can also control it in a way where certain variables are only activated for certain branches right so you can have the idea of protected branches versus unprotected branches where these variables will only be accessible or you can actually call these variables if you're only on a protected branch so that's one way of actually managing these secrets or these environmental variables so say for example if you are not if some of these variables are not so confidential or not so secure you don't need it to be secure you can definitely put it directly into your gitlab ci file as well as a variable and you you can then reference it uh from that point onwards but what we see here is that um there is another component which is uh highly requested by a lot of our customers as well as a lot uh a lot of our customers and a lot of people are starting to use this in our latest version as well is integrations with external secrets in ci so uh as of our last uh our last release also we've actually had achieved um vlog integrations with hashicorp uh so as you can see from the diagram here this is basically how you can use an external secret application uh to actually do that whole authentication process and not store it within gitlab if that's how your organization sees it to be um i've also had some conversations with other um customers and there were there are interest in other secrets management uh toolsets such as maybe cyber vlog as well and so and so forth right so i think we've only started on this journey relatively new so um i think uh as an open source company also i would suggest if you would want to see a certain feature from us please create an issue on our uh through through gitlab uh and then you know suggest why why this is important and then i think our developers our product managers will then um you know they'll they'll pick all of these up and then they will start to focus their development efforts on actually supporting these technologies as part of our secrets management right so i guess that's pretty much it from my point so that's point number five um and these are just five hacks uh or five tips that uh that some organizations that are using gitlab are using um that they are doing it today and i suggest that you give it a try or if you are using if you are already a gitlab ci user cicd user um or if you are not you know there's always a free trial that you can take up on and you can always try in your organization um yeah so i guess that's it from me and i think we can move on to the next part which uh i will let rob take over awesome um thanks for that jonathan very insightful presentation um you know it's great to hear all those tips and tricks and i hope everyone has a lot to take away from this um so if you guys got some time you know stick around for the q a first off we'll start off with a question from lewis um is auto devops available on the self-hosted version yeah definitely so so i actually saw this one uh get answered in the chat as well so some people are already on top of it right um auto devops is is totally available in the self-hosted version it's um the the auto devops can basically be thought of as a set of templates that are that all work together to create one pipeline but then uh in addition to that pipeline being working both in the the sas version and on-prem you can then componentize the jobs and create custom pipelines from it great um any what's the difference between gitlab um and spin maker like from a ci cd perspective yeah well like from the cice perspective um there's there's some things like the auto devops pipelines right like um things like uh integrating tightly with kubernetes and some of our operations dashboards so a lot a lot of the the biggest benefits that you're going to see with good lab versus a tool like spinnaker do this is sort of around the edges of the ci cd uh space right because we integrate with the fully with the the create tools fully with the monitoring tools and that sort of thing um and that lets us do things like the feature flags and the secret management that jonathan was talking about earlier definitely um is it possible to have multiple pipelines in the same respiratory um in the case we have multiple apps in the same replay that's a really interesting question and it's a use case that can happen a lot especially in the old like um sort of mono repo structure that uh a lot of repositories still are in um and and the best answer that question is that uh while there can't be two explicitly defined pipelines per se what you can do is dynamically change how uh the the jobs run so that uh they only run when certain conditions are met and what that has uh ends up being is a a very dynamically generated uh pipeline that can run different jobs based on different factors so that when you push up a job it might have an environment variable set and then that can trigger the jobs that that you need to run to run and the ones that you don't want to run can be turned off perfect um so what do you recommend for handling secrets in gitlab pipelines and on developers local machines having secrets stored in pipeline cio settings doesn't allow for easy updating or changing i mean that doesn't help for secret management on developer desktops does gitlab have a solution or should we look into tools like how should you call fault i'll look like even if you if you go and get uh hashicorp vault right if the developers are still putting the the passwords on their desktops then hashicorp's not going to be able to help you too much um what what we talked about right the um the the secret management in the uh the project variables what jonathan was been talking about earlier is the the recommended way of working within um within gitlab uh but the the hashicorp vault integration as well is is very uh deep and like it actually it works very well and very seamlessly to to get your secrets uh in and out of uh bolt into your environments and pipelines cool that makes a lot of sense um so how do you put put an approval method in pipeline for a particular job interesting so so uh the job will have certain success or failure criterias right so a job is considered to be successful when it reaches the end of its of the pipeline script so for each pipeline job there's a set of instructions that that the gitlab runner machine will run and then uh if it successfully reaches the end of that then it's a successful job if there's some sort of error that gets raised by your script before that happens or there's some sort of something gets raised and some sort of exit code gets given then that's a failure condition for that job okay this one's a good one we get this quite a lot is gitlab ci faster than github action um how do you compare well really it really depends on like what you're doing right so uh in in terms of like uh the only thing that i can sort of point to to to compare us is the analyst reports right um and and gitlab ci is the the number one continuous integration tool on the market at the moment so that's going up against the likes of jenkins the likes of uh github actions so that's uh yeah let the credentials speak for themselves i suppose yeah definitely definitely um so how do you how how do you block the gitlab ci yml to be edited while merging from another branch well when the the the the way you do that is is by enabling merge approvals on on the merge requests right it's impossible to stop any individual file from being changed in the git repository right that's the whole point of the repository is that you you get you get like the version control you see all the different versions um in order to stop the the pipeline from sort of changing and this sort of thing and especially in terms of like um long living branches that get merged in after a while when there could have been like lots of changes based on um other other branches um there's this concept of merge trains within gitlab ci which allows you to set specific merge requests as uh prerequisite for other merge requests so that means that if there is a change then uh it has to go through that merge train first and make sure that all of the um all of the mergers have been put in i see i see in the chat could also try file locking um yeah great um does gitlab have a solution to auto deploy jira custom plugins automatically to running jira instance uh at the moment i think that that still has to be configured you have to configure the the um the integration there but but once you do have it integrated there is a a full range of features including closing get lab uh jira tickets from the gitlab uh interface when you push commits up i've got another commenting question from louis in my case i have 10.net apis in the same repo and dev stage production branch should i use conditions on the pipeline file to trigger the build for a specific folder from a specific branch origin yeah definitely so that's that's one of the most common um uh usages of the gitlab rules is having specific branches run specific jobs so we can see this in the the auto demos pipeline one of the one of the cool things it does is it has a a separate pipeline for master versus uh for every other branch and on the master branch uh in addition to all of the the regular jobs your build your code quality all about security scanning um including the the auto review app for the dynamic application security testing it'll then deploy to a staging environment and then have the option to to have either incremental rollout timed incremental rollouts to print to a production environment manual deployment to a production environment or even canary production to a deployment to a production environment and then after that deployment it even has uh performance testing built in so that those are some of the things that are just specific on the master branch there so uh you can definitely like pick up individual jobs for specific branches especially in the the dev stage prod branch branching model right so period parent child pipelines seem to track the entire results compared to multiple projects with triggers do you have any road maps of providing consistent view on them uh so so as you said there's no like uh upper level view you can you simply have to like navigate up and down to the parent child pipelines because they they are separate projects and they they have a separate context so i think that at the moment there's like they're just trying to maintain that uh that sort of context there so i'm not sure about a timeline for providing a more global view on them have you heard anything uh yeah i think i agree on that point i think they um they they want to see that separation by the same time you know if it's something that you think is useful you know submit a issue on our project and our product leads will probably take a look at them i mean it's a good point to add as well so yeah right um does gitlab support open road by actin uh i'm sorry i haven't haven't heard of that um uh depending on what it does potentially um yeah i haven't heard of this one as well so um okay we can come back to it later um so does gitlab support versioning uh so so in terms of versioning so so for git lab is a version control piece of software uh one of one of our well one of the functions of gitlab is a version control software right um that that sits in the the create uh devops stage so so we've broken it down to the 10 devops lifecycle stages and jonathan here has really been focusing on the the verify package and release stages uh which lets you uh enact pipelines on individual commits and branches within the the version control software so based on that you can definitely have lots and lots of different versions up and running just being managed by gitlab yeah definitely i'm just wrapping up with the last few questions now um so this is a question from bhaskaran i have a i have the same question on we have been requested for function id from gitlab how to create the function id in gitlab um option id i know i haven't run into any requests like that um jonathan have you seen anything happening in terms of functionality i haven't seen any yeah i haven't seen that as well yeah we probably need a bit more context from that it's a bit hard to uh you know because i think it's quite specific yeah feel free to shoot us through an email um with more details and then yeah we can get definitely get back to you on that um next question how effective is metrics performed by gitlab ci can you list the types of metrics that gitlab cid is performing so specifically for the gitlab ci itself it performs uh some a lot lots of security and code quality metrics so it provides a code climate report a vulnerability list so that's a vulnerability list both for vulnerabilities found in your code as well as any libraries that your code's using found in from static application security testing and dynamic application security testing on top of that it provides metrics about what licenses are being used by your uh project so it shows you all the different licenses that are being used by the various uh libraries that you're using within there and brings all of that into the the merge requests so that you can sort of see all those metrics uh provided to the developer at uh at build time um on top of that there's uh some uh analytics given about the gitlab ci itself right um there's a ci insights panel which has uh like over over a period of time you can see the number of successful pipelines how many minutes ci pipeline minutes you're using that sort of thing so you can get an idea of um if your pipeline's getting more or less successful or if you're using more or how that's trending there's a fair bit of analytics there yeah there's quite a quite a few metrics there um just to wrap it up one final question at what step is it recommended to run container security tests so so git lab recommends that we run that in the verify stage so uh if we think back to jonathan's uh devsecops workflow the the new workflows is the the shift left in security so moving it out of that uh uh an end stage in release and moving it more into the verify stage that happens in every commute definitely yeah so look just to wrap up um this brings us to the end of the webinar keep an eye out for the next topic um we'll be talking about closing the feedback loop for a better stage planning which covers things like better portfolio management through roadmap planning and burn down charts granular planning through multi-level epics and how to use how to make use of value stream analytics so look thanks everyone for tuning in um i hope you all got a lot out of it as much as we enjoyed providing this information to you um look in saying that we'd like to hear your feedback about these webinars so far um we'll drop a link in the chat um below with a survey link if you all could fill that out for us um just to let us know you know what what we're doing right what we're doing wrong what you'd like to see um in these sort of webinars um yeah that'll be great so again just a reminder that the webinar is being recorded um and we'll send you through a copy of this after the webinar has finished all right thanks everyone see you later thank you you

Channel: GitLab

Views: 7,585

Keywords: Product, git, server, version control system, CI, Continuous integration, pipelines, git repository, Conversational Development, collaborate, integrate, integration, software development, tutorial, open source, on-premises git solutions, developers, programmers, code, VCS

Id: 0JCKe5lt1p4

Length: 59min 6sec (3546 seconds)

Published: Wed Oct 21 2020