Multi-Stage CI/CD Pipelines as Code with YAML For Azure DevOps

hello everyone welcome to our video multistage sea OECD pipelines as good with Yemen for a sure DevOps so we will see a complete tutorial on how to configure and sit as your PI applying as a code using UML configuration language and also how to build multistage pipeline which mean that we have multistage in the same y ml file we can like build and deployment and even more stages if needed so first I will give you a quick overview about the demo work a flow this will give you a graphical representation about all the steps in this demo and also it will give me the chance to explain and teach you many important information about the configuration then we will go for the demo and at the end of the demo I will give you some tips in order to able to troubleshooting Azure pipeline especially for the hosted Asian because we don't have access as a physical machine you know because Microsoft hosted Asian provided as a software as a service so we don't have access for that the tips will just give you an example he'll to troubleshooting the hosted Asian but first let me give you a quick introduction about myself my name is Mohammed Radwan I'm a developer technologies MVP and principal DevOps consultant I have been doing software development for more than 15 years now working on several projects for different enterprise customers across different region and countries i heavily participated in the community developed several frameworks presented mini sessions holding various certificates for agile and devops i started back in 2002 as a classic sv developer relocated multiple times in multiple countries working with several technologies and for different role and positions and for the last eight years I moved to consultancy based role where I focused more on helping different companies some of them from the fortune 500 to improve the software development and delivery using DevOps and automation I have developed and authored several frameworks command line tools and guides you can find them on github and agile devops marketplace I like to share my experience this is Hawaii I have a YouTube channel and blog where they have hundreds of videos and posts that share different topics from my real experience I have presented mini sessions in several user group conferences events and customer site as well in different countries so I got the chance and the opportunity to work in different culture project size and company size as well which helped me to have different experience here's some links for my activities okay so first this demo will not cover any infrastructure as good there is other videos that I created which cover how to create an infrastructure as good as a part of the pipeline using ansible terraform and arm as well but this video will not cover this part so the first step is to start by creating the infrastructure manually through log into the azure portal and creating my resource group and then using a template which will create a web app and sequel server on Azure so the first is by creating an azure service plan it is just one template that will create all the component so this includes an azure service plan a web app as your sequel and also configure the azure firewall rule so the web app can access the second server once I complete that then I will open the azure DevOps and then navigating to the pipeline the first step is by creating a ml which is the build pipeline as a code and choose the location where I want to store my ml file which is usually is the repository of my project and I will use a template with asp.net template which is ready-made the ML file for some tasks and since my application is an asp.net cool this template will has the default tasks needed which is restore all the nougat package build the application run the unit tests and so on and then I will run a continuous integration build so we can see that the build is running and the unit testing and all of that then I will open the ml again and start editing the ml filed by adding a step to start creating the package of the web app and storing that package on the artifacts and then running a build again so this task will package the web application and de storing that on the artifacts once this complete I will start editing dialog in and start adding a new stitch so the build and the deployment stage and in the deployment stage I will start adding some tasks to pick the package and deploy that package to the whip air and then start running the second stage which is the deployment and then the first step in the task is to bake the package download the package on the pipeline agent and then deploy that package to the web page after that I will change the connection string of the web app using the connection string from the web configuration in my application to make sure that it will be translated correctly and then browse the application from the azure web app so let's see the demo so let's look at the sample project that I will use in this demo which is Porsche Unlimited with application a website for car parts and this is the source on github this is a key feature SB dotnet 5 support Linux and Manu using visuals to 2017 and many others ok first I will navigate to the demo generator where we can generate a project template so here I will select agile devops organization and choose the required template here assertive obsessions and here I will choose a template navigate to parse Unlimited and then select this template here I will just type the project name I will type the name for unlimited project then create project so this will create the project including different component as we can see teams for the columns swimlanes this includes the source code the pipeline you can watch this you for more information at our docker control ization and we ran how docker can fit with devops & cie CD by appliance as we can see now the project is created so let's navigate back to Azure DevOps and refresh the page here so we can see this is the project let's open the project and as we can see here there is meaning work items created even some of them is in the dona state and there is many poor requests committed and so on so this project can be used for demo or understanding or training purpose okay let's navigate to the repos so we can see the project as we can see here we have parse unlimited with application which is created using asp.net 4.5 so if I navigate here - this is the documentation of the project let's open that this is for how we deploy the project and let's another gate to the web application directory or folder so as we can see this is an MVC application and also it includes this - project so we have here unit test selenium test this is for UI testing if I want to also to try the automation of UI testing as a part of the build it's out of scope but I'm just showing you what is included in this project so and this is a unit test so let's see navigate now to the pipeline so we have a pipeline created part of the generated project I will pose these pipelines select it and both this pipeline then return it back here and then let's navigate to Azure portal and I will create new resources so I will create the back end of my project which it will be the web app and the sequel on Azure because this demo will not include creating the infrastructure as a part of the pipeline in other labs and videos part of them is creating the for a structure from the code under the source control but this is out of the scope of this demo so here we are going just to create the infrastructure manually using the azure portal so we have a template here it's called web sequel so if we navigate here to this template then click create so from the name of the template we expect to have a web app and sequel on agile so here I will name my web app name so I will just call it with f-secure at one my name and then here I will select create a new database and here I will name the database with Apps equal DB then for configuring the settings I will create new sequel server on Azure and here is the name of the second server and then just put the username and password for the sacred I can put in username and password this will be used in the connection string in the web app later we will see that and here I will change the location let's choose the UK so and as we can see here this option allow other service to access server this will allow the firewall on the sequel server so the web app can access the database if I didn't check that then I need to configure this rule manually then select and then create so this will create the web app and the sequel server with this resource group name web app sequel radwan so as we can see now it's complete so let's navigate to resource groups and as we can see we have new resource group created with a sequel rot 1 if I click that we can see that this is the resource created this is the database this is a web app and this is or sequel server on Azure so let's navigate back let's navigate to the web app and click on configuration so we can see the connection string created as a part of the template so here this is a connection string as we can see here the name of the database that I could hit the part of the template the username and the password that I use to create the sequel so let's close that we have also some configuration for the app settings let's navigate outside from here and get back to the azure DevOps so now I'm going to create a new pipeline and in this time I will create a pipeline as code using UML not just the classic or the visual one if I want to use the visual one then I just go for the classic editor here and since I'm going to use the pipeline as a code then I will choose this and here the first is to ask me where I want to store my pie applying code so I will choose here this ripple and which is a person limited here as we can see we have some reddit tml template which includes some template for tasks and here I will choose the asp.net template so as we can see once I create that this will generate the ml file with some predefined tasks for building a speed net application so the first is to change the agent image since I know that my application required to build the asp.net with running the unit tests so I need the visual studio 2017 image so I will change that and as we can see here I just will use this image and here is the name of the value that I can use in Yemen so here I will put the name and then just save and run or I can save and run later so let's save first and then just edit again and run as we can see the trigger here is not defined and this means that this will be continuous integration so every time you save this by applying it will automatically trigger a new bill so let's navigate to the pipeline I prefer to open to instance so we can see different views so here let's open this pipeline so we can see the running of the build from the outside let's open another instance so we can see that from inside so you can see how we can have different views so this is in details and this is like an abstract view for the build so as we can see this is the tasks in the belt just to check out the project get the project from the source control start restoring all the packages using the new gate package and then building the application using the via spelled and then running the unit test this is then you get command let's see that from outside as we can see you can watch this video for more information about Cooper natus cluster healthy deploy docker image to Azure Cooper needed service and how to configure CI c Depot appliance for an end-to-end development and deployment scenario now it's building the application by speeding up all of that but I leave that to have the full experience so as we can see here running the unit test now the test is complete and as we can see I can navigate to the test result from here if I click here this is a test and we can see that I have total 16 tests 14 paused and 2 unknown but all of them there is no problem or I can just get out of this running job and navigate to the tester from here it is the same view okay so let's get back to the pipeline as good and click Edit so we can add another stage and this is very important because now I'm going to use a multi-stage build and deployment in one EML file as a code so in order to do that I just need to go under the trigger and start adding a stage so I can name that this file has no stage when I running that so this is like a single or one stage so I don't need to define stages but if I need to have more than one stage then I need to define the stages so first I will just have a new line and enter this be very careful about the species and indent because this is very restricted so as we can see now I have the stage built then just remove the extra space that not need it because this is very important and as you can see here we have this correction which means that this file has an error and I must fix all the error before I can see if and even if I could see if and it has any syntax error about the spacing or the indention this will cause the pilled and deployment to - has an error in the running and it will show me that so be very careful about the species and the intention in this file I will show you how to do that exactly so now I remove this species another space here as well so now it seems fine so when I collapse the stage as we can see the pool and all of that is not in the stage but I need all of the content inside the stage in order to do that I just expand the pool and the stage as well and I will select all the part that is not under the stage and make two tabs for indention again this is very important so all this part is under the current stage which is the belt so as we can see now it seems there is no error and if I collapse the stage we can see that all the stage now collapse correctly and there is no element outside this stage which indicator that this is correct so now I'm going to copy that stage be very careful about the spacing to copy the space as well and then create another stage this will be for the deployment as you can see I get back to the beginning of the line to make sure that I will paste in the right place then paste here so now I'm going to change the name from build to be deployed this is the second stage here deploy and I don't need any variable here so I will delete that with all species and I will delete all the steps because I will create new steps so as you can see now I added the second stage and I need to create the steps in this stage so I will use the graphical UI to add the steps to zml so if I navigate here and choose the app service deploy because I want to deploy my application to the web app on a sure so I select tasks so the first is to choose the subscription my subscription and authorized that to the Azure subscription so I can have a permission to deploy that and here then I can see all the web app and I have the web app that I created earlier in this demo which is the web app sequel at one and here I will delete that and just put that with the drop folder so so this task will start looking for the web package on the drop location on the azure DevOps because I will copy my web app in the drop location this is another task I will show you that later now and then let's add that so as we can see now I have this task added to the deployment stage in the ml file but as we can see it still has error so I need to invent that part under the deployment stage with two tab while I'm selecting it and as we can see the settings here this appears automatically which also indicator for a correction I have that correct and it seems working fine so now the deployment is ready to take the web package from the drop location and deploy that to the web app on the azure but in the build stage I don't have any web package that put that on the drop location so I just need to add another task in the build stage so I can create the web package and put it with package on the drop location so I will go here for the build stage and search here for publish package so here the publish artifact and then here this will be the drop location and the build artifact staging directory then click add to tabs so this task will publish the web app to the droplet as we can see here we have the drop here this is the folder and also in the deployment I just need before deploying the package to the web app I need to download the package on the agent first so I will add another step to download the package from the agent so here download build artifacts and here just write the artifact name which is the drop and then click Add to tabs for end engine so now I have complete the two stages so let's double check if I collapsed that and that it seems very good so let's expand this as well and now just save this will automatically trigger because it's continuous integration so let's navigate to the pipeline open to instance this sometimes happen because I need to confirm the authorization for the pipeline so I will just need to click here authorize resources and then run new so as we can see now the build is running let's open another instance so we can see that from inside click here so this is the build and deployment from outside and this is from inside so it will go all the process that we explained before and of course has a new task which is publish build artifact which is here so run the build the same tasks just show you here this is from outside how it looks like and here from the unit test publish the artifact on the drop location so we can see here unlike the previous build this build publish the web package to the artifacts because we added the publish package tasks to the build stage so this published here so it can be picked up by the deployment pipeline so it can pick the OP this package and deploying that to our infrastructure as we can see here I can access the artifacts here this is the drop location because I publish the artifact and here this is a test and here this is the way package published on the artifacts so let's see that from outside here now the deployment will start this is from outside and this is from inside let's expand that so as we can see it will check out download the artifacts from the drop location and then start deployment you can watch this video for more information about gate with animation you will see different animated command like French Mirage rebase cherry pick and many others now it's complete it listen navigate here to see that from how it looks like from outside as we can see it's completed successfully with some warning but no problem it just completed successfully here and here as well so if I navigate to the repos we can see that so we can see the connection string so if we go for the web.config we can see the connection string names different connection strings so i will copy it and navigate to my web app this is a resource group and the web app configuration and update the connection string name too so this is the correct connection string name so the live application can translate the connection see if that and then navigate back to the web app and prowess so as we can see this will load the web app from our web app and as your sequel server with all the data this is the web app as we can see it has some products for car parts as a tips in order to troubleshooting the problem about the package was not exist I needed to troubleshooting and in this I used a command which is dire P s this will display all the existing files and directories or folders in this location and I can run them on the build even so I can display all the existing file and just ctrl C so I can exit from that so I will use the same command with the command task on the build pipeline so I can understand the hosted agent Microsoft hosted agent which mean that I don't have access its the hosted agent is provided as a services so I don't have a physical access to the buildin I don't have a remote access so I can navigate to that so what I needed to troubleshooting is to display what is the file exists in the pipeline agent so in order to do that I will come here in the deployment stage and start searching for command this is just a tip it's not needed and here I will just type a message tell me all the folders and files in this path and here I will put the same command so this will tell me exactly what is inside the building and of course to tap for indention save this will run the build automatically let's navigate here this is the normal file I don't need it I just sure won't show you here so this is the command line in the desk so let me get back as you can see it's start displaying all the files and folders in this and of course it starts with telling me all the folders and the file and dispose and listing all of them I can also if because this is a Windows machine if it is a Linux machine I can use bash instead of command line with this command there is also a very good documentation about using bash how to using bash of variables and displaying that has great content also for more information about bash command at the end I would like to thank you for watching the video please if you have any question or comment don't hesitate to reach out you can find more information on my blog which will appear at the end of the video along with some other related video thank you [Music]

Channel: Mohamed Radwan - DevOps

Views: 32,506

Rating: 4.9093199 out of 5

Keywords: Multi-Stage, Azure Pipelines, Azure, YAML, Multi Stage CICD, Pipelines, Continuous Integration, Continuous Deployment, CICD, CI/CD, CICD Process, DevOps, Build Automation

Id: i77vEEVAfB8

Length: 29min 38sec (1778 seconds)

Published: Sun Dec 29 2019