Enterprise Process Automation Using DevOps

hey everyone good evening and welcome you all to our another meetup session of azure developer community and for today's session we have monajeet from microsoft who will be guiding and leading us on the topic of enterprise process automation using devops so before we get started with today's session uh i'll just brief you all about what is uh this azure developer community as it is under this meetup session comes under that so this community initiative was started on 26 january this year with and motive to help each and everyone in the country to learn network and upskill so by being in this community you can attend regular developer focus events here and learn on various technologies from key experts and community contributors here and also you can grow your network by sharing your knowledge and ideas with like-minded people here and also you can up-skill by giving back to the community by sharing your experiences and knowledge with other people by upscaling and empowering them and we are spread across in 102 cities of india and these communities are speed headed by our amazing organizers who help us to drive this uh sessions and help others also to learn in this journey and also we have a speaker's panel here in the community where speakers on various technologies expertise come and hear give talks so similarly we have uh monajit today who will be giving an amazing session so without any further delay uh i would like to bring moonjit on stage yeah so hi manuji hey hi shefali hi everyone yeah how are you i'm fine sophia thank you so much yeah uh so before uh starting with the session um won't you please can you give us a brief introduction about yourself and what session you have today in for us yeah over to you absolutely thank you so much chef thanks everyone for joining so myself manojit i um i work in microsoft as a cloud solution architect uh primarily focusing on cloud native technologies and that means all containers and serverless app services devops mobile applications so these are my area which i love to work most although my current role need me to go a breadth of everything but that this is the area which i want to focus i would love to focus more on that today we will be talking about uh the process automation through devops how you do that in enterprises we will talk about some of the the the features that uh the devops tools on azure brings in for us right we will be unbiased on the actual tool selection we will not talk about it although obviously the tool uh references will come in uh but our focus will be to understand how devops can be utilized to automate your end-to-end process not just uh you know the code checking and the code checkouts and and the deployments but just beyond that okay so that is the that is a short introduction of what we are going to do today in this uh one hour time frame and since it's a virtual session please feel free to uh post your questions as i think sheikh ali and it will guide you so accordingly we will take up the questions at the end because it's a short time frame i will keep 10 minutes at least for or questions thank you introduction so uh hey everyone if any of you have any queries in between the session or when the session is going on you can comment down in the chat section here and uh surely once you will at the end of the session he will look into it so i would i will bring the presentation uh here and you can start off right sure chef let me switch my screen let me know uh just help me uh because i will not be able to see that screen is is it is it fully visible yes yes it is great and my voice is also clear i hope right right great okay uh great everyone so uh without much delay we will start discussing about uh the devops solutions and as i said right the process automation and all so basically um the main reason for uh all the all these discussions around devops that started off 10 years or 10 12 years back was primarily to uh you know cater to the needs of the the modern application design right the engineering is now getting more and more evolved it's not the old only client server technology or doing some monolithic applications the engineering is doing a lot of stuff and as engineering is evolving uh the new type of applications new softwares are getting built right so that's where the need for these type of uh devops solutions are various devops solutions and related tools not only devops the devops and its related components and services those are now imperative that we know how these things work and we also know the best practices around them right now if you look at uh the the the current uh application uh landscape right all over the world i mean across all the cloud providers and everyone you see that uh you know cloud native is one of the way that we build our applications cloud native means containers serverless app services then intelligent apps that means you bring bring your own custom models or you use ready-made models to deploy models means i'm saying artificial intelligence and all the cognitive models right you deploy them the models are big in size sometimes they do they come with their own processes each model comes like an application right they load data they have its uh you know other processes inbuilt into it so they need more scaling they need you know better performance as well as they need to uh you know be deployed on multiple different uh disparate systems right so that is these are the two primary things then we also uh have the uh customer-centric applications like more like bots or i mean which are using these intelligent applications or cloud native application those are mostly front-end applications they are uh they might be help desk applications or everything is more right somebody said some uh you know experts commented that all the mobile applications or desktop applications one day will only be bots so it will be a bot application only that's what we are thinking of now if that is the way or that is the kind of improvement that has happened over the last 10 12 years in terms of software development then that means the deployment of that also needs to change right there are numerous instances where we have seen that how applications basically uh got into trouble big applications from great enterprises have failed because just they don't didn't have a proper devops mindset right so the first and foremost thing is that to understand what is devops because earlier it was only ci cd people used to talk about continuous integration and continuous development right now it's beyond that right it's it's it's about the entire process if we can uh automate or control through some tools so that when that comes into play uh i will uh you know breeze pass through these things because we have already discussed but uh then it is now uh the challenge uh that we need to address so what are the challenges that we have uh very quickly i will address these challenges and move on to some examples basically uh now the first thing is that the business needs a faster uh deployment cycle and continuously innovate so if the application is now being deployed onto cluster onto an kubernetes cluster or or on azure as an app service or as a function or serverless app it is working fine that's good what about the next version is the next version going to break the earlier version is the next version going to be still uh you know working on top of it are we only deploying the delta or we have to deploy the entire stuff so those are the questions or challenges that people have been dealing with for not only for last few years it's long uh time you know discussion the moment we started the client server technologies and all these were the things that were always happening right it's not about one desktop application running for 10 years now it is more flexible and flamboyant so so in in that case that means we need a solution that will bring in more frequent and stable releases this is the first and foremost thing every enterprise whatever be your marketing catch line of devops or sales catch line of devops the ultimate technology benefit is this and this is the requirement from every customer i hope since you guys are also facing lot of customers you would know that this is the basic question everybody is asking this frequent release stable release both are needed right i will be doing some companies some big companies the startups to the major uh customer facing companies they are doing almost uh you know 10 plus releases on every day for one product only for one product so you can imagine how frequent that would be and each of the release has to be stable because that might be 10 releases not for one region one product getting deployed on to multiple regions and all has to work right so those type of things the failure rate has to get you know reduced a lot now that means the it's a business need and technology has to respond that's where devops come into play uh now when it comes to play that means there are a lot of challenges because the first and first challenge is the dependency of the software so as you guys are also you know doing or probably traversing between the developer world and the operations world you know that the biggest challenge for any software on any technology stack whether it is node.js or dotnet or java or even mobile applications like xamarin or ios or android the biggest challenge is the dependency of the software a software will not break if there is no dependency so if i have a if i have a code base like this let me know if the switch of the screen works or not is my visual studio code window is visible anyone if yes or no is fine yes it's visible right here i just wanted to check the switching works uh here as well okay thanks uh so uh if my software is is a very standard software like this where i have a function application don't look at the other folders don't get you know driven by the other folders and all this just remove it from your mind this is just one application validate ocr application it's a function app so if i extend this it is nothing but a function and which is a dot net function if that is the case then i'm fine i mean that is the ideal thing that i'm i'm dealing with only one function and then i'm deploying it anywhere i can do all these stuff right but the moment i say that no this function also has the dependencies and serious dependencies i mean if i change the version of my durable task this is just a library i don't even get into that but just you know using it as a reference so if i change the version of this it might break my application so that is the that's the dependencies are the biggest thing which bogs us down when we actually think of doing frequent releases right and stable releases frequently is still fine the stable release frequent stable release so that means i need to have a solution that can help me in ah handling this stuff right so that is one thing and then the other thing is uh is the configuration right so uh even in this application you could see now this is a simple name application that so i'm using it just to explain these might be my configuration thing right now these configuration parameters are easy it's an ocr api key application ocr api key that my function uses to do some ocr processing uh some url which does the uh which does the which actually calls the ocr api in the back end and some other parameters right now these are secret parameters i am still using it hard coded because ah okay anyway i will talk about it that how we handle this situation but the point here is these parameters might change my instrumentation key for app insights might have expired or deleted or something has happened right so these configuration changes and the drifting of this configuration so i need more parameters and my code needs even more control of the parameters so those are some of the challenges i'm just saying uh which basically uh you know forces us to find a solution like devops uh which brings all these all these under one umbrella so that we can manage it better right and the developer challenges i mean i talked about developer and iit challenges both basically so it's a integrated interdependent infrastructure is one challenge for i.t configuration is one challenge a collaboration between div and op team this is what i always talk about in any session whether it is devops or whether i'm talking on containers or or any other you know related technologies this is the biggest challenge the the third point on the left the the dave and the ops guys are always at loggerheads always at the fight right uh i mean devi is saying it's not my problem ops is saying how do i how do we how do you say that it's not your problem it's definitely your problem so that that tussle goes on right so uh these are iit challenges on the developer side also i talked about like the dependency on the libraries the release cycles one bug comes up at the qa team which when i was also into the developer hat i still remember right when the qa team tells me that there is a bug at five o'clock in the evening that's the last thing that i want right i'm about to go back to my home and then i got to know there is a bug and i know that if i fix the bug even in next 10 minutes it can create another three more bugs right so i have to be very careful about it so all these dependencies and even if the my my architecture is fully object oriented fully following all the design patterns these are the things that are still going to be there always now to avoid that ah and then definitely industry so i t developer all these things gone industry also is asking that hey forget about your challenges i need more monitoring i need more observability i need to see the containers uh in details i need um the support uh you know beyond uh your your given technologies i need to be more open source so a lot of things it's just a tip of the iceberg which they are asking right so uh then uh what happens and and then finally also the need for the compliance so this is also we will see today i mean what i am telling you now all these challenges we will see how do we tackle these challenges in devops and then also we go and uh do that uh in the process automation into it right now this is another thing which is uh probably not from the business not from industry directly but from the security team your procurement your security your licensing they come back and say that my applica your application that you developed is not compliant with the policies that i have so this governance teams requirements are even even trickier and even a critical before you release it right i have seen apply i have been part of an application where when devops was not a buzzword in the industry i'm talking about 10 98 10 18 years back and then we i saw the customer pulling down an entire application from apple app store just because they had small data lag in the in the monitoring server right at that point of time it was google analytics was very popular only a solution or some flurries and all so one of the solution basically was giving a data which is slightly deviating but the customer was a customer facing media and entertainment giant so they could not uh you know have an application with that amount of deviation because that was giving a completely wrong picture and when we saw actually the issue uh from the from just coming out of my developer hat or architect head i could understand oh my that's the problem that means this is where the entire graph is changing it is giving a completely wrong impression because more and more users are using and clicking those functional buttons right and the for every click if there is an error so just an example that if we are not compliant if we are not uh doing the proper monitoring or analytics of the application then there is no way uh to go into production if it is there it has to be rolled back so that also brings the capability of resiliency in our software design our software has to be resilient so that we can roll back we could roll back because it was only a mobile application but what about an api right roll backing an api means you have to take care of lot of things right so with uh keeping all these things in mind uh basically uh this is what a devops taxonomy says right you need to have a continuous planning right you need to have continuous planning that means you would have a project plan behind the scene with the sprints and i think you guys know about these things i will not go into that continuous planning of that which is the culture that provides then it would have a continuous integration and the delivery that means you would whenever you change into your code it will be you know getting prepared for deployment that is the integration piece that is the build will happen the binaries will be created then you would go into the delivery mode which is the deployment continuous delivery and then the operate the post delivery work is even more important that's where your architecture and the resilience and all these things will come in right post deployment or at the time of deployment how you are basically maintaining that your software product quality is great you are not uh you know leaving any stones unturned you are actually checking all your all your [Music] the problem areas that's the key for any devops and azure devops or github these kind of solutions that you get on azure on azure ecosystem they address these things particularly right we will maybe we will come to that slightly later but just in in short uh these are all the four uh or like i would say all the periods that we should always uh be worried about if we do not go into each one of these that's fine it's not that everything is relevant for your organization or for your customer but majority of them is right the quality the security uh the collaboration the collaboration is often not looked at very uh you know hugely but when you are on at ground zero you would see that without collaboration how do you do that so we have lot of good integration points we'll talk about it and the improvement of the application so every time you do a change and then you bring in new features that should also be stable so that is something is ah these are the capabilities and the practices what azure devops brings in we have azure boards for exactly what you saw in the previous slide the planning piece of it then we have azure pipelines which you saw in the second pillar which is the integration and the delivery we have azure repos uh or you can bring in other repositories also like github and all uh we are talking about azure repos we can we similarly get a report as i said i am not going to talk about any particular solution develop solution and azure both are great and uh it's you know we will be unbiased for today's session on which one to use and all purpose of the session is not that it's general devops right but since we are talking on azure yes so on azure depose and github repos both bring you the capability uh to manage your uh the software uh lifecycle better right and then the test plans uh test plans we all know that the test plans basically allow us to create our testing strategy then do all the functional non-functional testing and then you know then allow the uh application to go on production only if everything is successful and artifact is something that we will talk about it for now if you if you were really excited to know what the artifact is artifact is basically the packages so you maintain nuget packages you can be doing with maven or you can be doing some node.js packaging the package.json all these are from the systems they are they're all mutually exclusive right uh and they're maintained they are uh their life cycle is maintained uh in a completely different way but all have one thing common that they all have a server where they store these things so we have we are leveraging that and we are saying that okay azure artifacts azure devops as a server will give you that benefit you can manage all your artifacts into one location rather than going to new gate and do this you mention it here and all the updates from the uh from the respective repository azure artifact will maintain and what you will do instead of managing it going to separate separate libraries like newgate or package.json or maven they would all go to azure artifact and get the latest version from there so it's just another you know the repository of uh repository of the packages quickly i will touch upon how microsoft within microsoft we do uh you know some of the best practices not everything but some of the best practices which will give you an idea and then we will move on to some of the examples in between i would just quickly check if there is any uh questions or anything at this point of time okay no me is my voice not is it better now sorry somebody says voice is not great so let me know if it is clear now okay sorry for that so so uh within microsoft what we do basically um is that there are few four five things we always follow one other thing is to um is to have a clarity and and also listen to our customers always listen to our customer what is the problem and then we are always transparent so whenever the the product is about to be released or getting released and this is one of the very good practice that every software you know vendor should follow right instead of you you know deploy creating and deploying applications without looking at the customers feedback is one of the biggest challenge because uh then there will be always a gap so knowing your customers challenges and then what they're expecting so you might have built something which is working great but not meeting the customers expectation so there's in your azure devops uh you would have always so if i go back to my window again and if i [Music] open my one of my project this is my azure devops organization i'm going to this project here right so you can see that azure device already provides lot of uh you know various ways that you can interact with within your team and even outside your team also right seeing a bit of sluggishness my uh so you can actually create your wiki pages from here you can code you can you can actually use this wiki to communicate with your um with your customer as well as within your team members right we also have integration capability uh with teams now i'm just trying to uh you know highlight that what are the things that you can do to make sure that you collaborate within your team as well as outside your team to get feedback and then improve your software so it's just a standard wiki that you can always use to uh you know have your documents your diagrams everything uploaded here right and then the other important thing let me switch back to my i think under wrong one so let me open uh another application maybe so uh the the other uh option that you should always use is a dashboard uh now this dashboard is just uh you know created as and when you you can basically build this dashboard just by clicking few buttons right uh if you are using it you already know that right whatever you are doing on your project you can basically create a widget and then put it over there but what is good part about it is that all your project team members so if i go to my project settings and if i have my teams right and i can add multiple members into the team there are different groups so based on that your team members who are part of this project they would have a view a unified view of this of this uh dashboard right so dashboard gives you an option to uh com you know give a quick view of the entire project now different people will be interested to know different sections of the project somebody might be only thinking about the user stories and all these stuff somebody would be more on the bugs so if i click on the bugs i will be getting to know okay there is a there is an issue already happening which is unassigned which i can assign it to myself and do work or the stakeholders who are responsible for uh you know talking about the quality of the product so they would be checking these and getting to know this quite easily right i will talk about one more important thing which let me come to that later but uh yeah so maybe i think you can see the wiki here also the good part of the week is that it is good for everyone in the team as well as outside so outside in the sense if somebody is joined as a guest user from your customer end or um your the other vendors in so they can be part of it and the the good part is you can share these information right like this is just a standard information you can you can have your documents and diagrams everything uploaded anybody joining the team that is another challenge for developers right anybody joining the team how do they know that what is going on in the project so this week he can be your one of the place to do that now the [Music] the other important thing in terms of in terms of uh the the clarity part of it right we always say that you try to measure not everything whatever is needed for you to measure talk to the customer and get to know what they are expecting these are the standard right you try to measure what the performance the uh the efficiency of that how the entire project flow is working uh the the whether the customer is satisfied or not those type of things you should measure don't try to measure something which is not related to the operations right which are very very specific to uh project management you might be tracking it into the board section so you can see azure devops has these two segments right one segment on top we call this extended devops or i personally call it extended devops and whereas the section from repost in pipelines and test plan these are my real devops i mean the devops that we do day to day but the external devops piece is the project management which earlier looked at uh separately from devops now it's part of the solution so that you don't have to go to some other disparate system do the management over there and then come back so you can all have an integrated view here right but anything that is that should be measured as a management piece of project management piece of it do it separately but only focus on the measurement of the stuff so the parameters which i was showcasing here these are something which are important but if you look at the complicated awards the line of code the capacity of the team these are something that is not exactly a devops stuff that you need to ma you need to measure but you need to do that these are not something that are not irrelevant to the project it's relevant for the project management not for the actual software so you should basically segregate these two things in such a way that you can uh manage both of them right so that's the reason i said that when you so here whatever software i have built in right all my pipeline analytics that how much uh of the pipeline has run properly so if i go to the overview section what was that yeah so this is my entire project stat on the stat statistics on the right hand side right now out of this my pull request my comments all these problems and the bugs and all these are my something which i am related directly right but if something which is as i told you right the stuff which are not so relevant they should be or which are specific to project management you should maintain it here like we are doing it here in the dashboard section and then you can go down a bit right you can do you can you can have a look at the boards see what are the features or which one of these are you know unassigned but you know still lying in the backlog so those type of stuff is more of a project management which you should handle over there right okay now uh i'm just uh you know passing through some of the slides because i wanted to take you to more of a sample kind of a thing these are something when i share you can always go through it later the important thing is to also uh have a zero distance approach so talk to customer i think we we talked about it so more you use the forums the team's integration with the azure developer so now we can actually have the integration with azure device directly from teams as well as you probably if you are using it you know that otherwise let me show you quickly you can go to your you know go to the the service connections and from there also you can actually you know connect to your uh connect to the teams and then create um create a complete channel so basically what will happen so whenever there is a bug or anything created you come to you you get notified in the team the teams get notified into the team's software and then they can collaborate and then update it again from there right so uh now from there onwards the other important stuff which we always talk about in devops is the breaking of the team right how you divide your team into multiple um multiple uh groups of people who are relevant to your software development right like uh for let me show you an example also here as well so um so if i go to the pipelines for any of the projects right i will come to this pipeline in detail because this is one of the thing that we have implemented for a big customer and you know a couple of customers in this way and then we saw a great benefit in terms of software life cycle and all this stuff so we'll come to that but if you look at the pipeline any pipeline that i open here right i can bring in uh i can i can manage the security of that pipeline now this pipeline is responsible for deploying or building my applications and all here as you can see i have already inbuilt groups these groups are standard from azure devops or github they know these type of groups should exist at least right so you can always use these existing groups and bring your teams under any one of these heads or you can bring in groups from other services so i can actually have i have created an uh build service group and then put users underneath i can also create groups in azure ad right so if i can do that i don't know whether that tenant would work in a separate tenant let it open in the meantime let me talk on this so uh we can actually bring in uh the themes from my azure ad into this entire flow so if i have a team structure like this we we always manage it in here just because uh there is no confusion so what happens that all authentication the rollback access control and everything is taken care of um from the from the azure ready perspective all in one place right so let me get into that in the meantime let me check if there are any question okay yeah so if i go to my as i'm just opening a separate tenant because my microsoft internal tenant doesn't allow me to be an admin or owner of this admin of this azure already so i have created a separate tenant which is which is free which doesn't have any subscription in it but i use it for all my authentication needs so if i go to the users or groups rather sorry if i go to the groups so you can see i have multiple groups here but the first three what you see a cluster admin or a cluster architect or cluster developer because i have created these groups for an aks cluster i created this type of grouping right you can bring in the groups specific to the projects across your organization so you can have a stakeholder group a completely build administrator group who are shared across multiple teams so it doesn't matter that how you handle them you just need to bring those guys under under any one of these heads or bring the group here as a separate group it will be you know displayed here if you come in and then you decide which one to allow like build administrators these are all standard you can always change them but the point here is that is the way you should control access to your project and bring in more security into the team so all different teams working but having a very controlled environment working on this so this is important now the other thing is that like we have a sorry yesterday so like we have the program management and the development of the testing team uh then we would have more uh focus on engineering and the operations guys right so those type of different teams will also come in and engineering guys are only for the product team they are building the product so you need to bring in those type of uh thing and then the feature team is most important we will see that in an example that feature tv is is most important who are maybe only working on specific features then the new features that is coming in that will be integrated uh into the main product by the engineering later on right so they they should be a separate team also with with a varying degree of access and this is also we always suggest that instead of fully horizontal the ui team working on the ui api team working on the api data team working on the data they all have great knowledges but they can always collaborate and that collaboration is the vertical piece of it right so where the teams know each other teams can the the success of a project apart from using the proper devops tool is also to leverage the tool the best of its capability right so azure device github they both have these type of capabilities you can actually have a team across different uh tech stack right as you have seen you can create maybe a build administrator team then from the coding also you can group them in such a way that the the cross technology people can walk and collaborate better right and if and more and more they collaborate better the productivity of the team increases otherwise this is not a very productive team right all working in silos and these silos bring in lot of challenges which you will see later but this is what so this is the uh change in the entire team structure right we have uh the [Music] cross discipline teams they are self-managing then you know the the teams are all small 10 12 members each having a varying degree of responsibility but as i said they also can intersect there is an intersection between these two so or all these teams so that they can collaborate and then share the code and that basically improves the quality of the software ultimately right because if ui and api are completely segregated which used to be even seven eight years back is always a problem always a big gap so if they work hand in hand which are now basically with the containers in place um more and more containerized applications rolling out it's always a joining hand of uh disparate teams into one right uh okay we will talk about these things later or maybe i can just skip them later also because it's all standard straightforward stuff i will take you to something which is very very important to understand we'll come to that now what we what i wanted to show you in in the meantime just i can maybe i can pause for it yeah i think i will answer that question rs i will take that up good question so now uh the other thing is that um so what what we have done for one of the customers i just wanted to highlight that first right this is purely in terms of the devops process the project management piece is not here in my sample because this is my subscription they have done extensive good work in terms of project management uh but i'm not going into the details of that because that's quite standard we you all know what needs to be done and our tool is providing a lot of features the important thing that we had done on top of the existing thing that azure devops provides was to basically manage the pipelines in an extremely granular fashion so if you look at the pipelines here and i hope you are all familiar with the cicd kind of a thing right so ci is this uh like whenever i make some changes in my code uh this is the uh build pipeline that you know gets executed the navy first of all the naming convention you can see that right which environment right which environment and is it it's a ci pipeline the purpose everything is named accordingly and then after that let me click on that and after that what we do here is if i try to edit the pipeline see the first and most important thing is this one right now it it might be a bit of debatable statement but this is what i'm sharing what i have faced uh in my work with various customers we can use hosted machines right azure pipeline machines as you can see we can select that but i am using a self-hosted machine why because majority of the applications that you deploy on azure or any cloud they need security they need isolation so in in my case it was maybe the private cluster of aks this cluster if i try to access from the portal i will not be able to do that because it is a private cluster it doesn't allow me to access any one of you you can see it is not allowing me because it's a private cluster so i can't even see the name spaces on the workloads and all i have to get into a machine which can do that for me i don't know if that is the session is disconnected let me connect the session in the meantime but but the point here is that until and unless i make sure that my jump server i i have a jump server you know created and then access from there but that is for my developers access how do i automate that process so that means i need to have a a machine like this where okay just give me a second i think my connection is for a different machine let me connect it separately so the point here is the the moment you have a this is my devops machine okay this is my devops machine i have a self-hosted agent all running in a it it is running on azure but it is peered with my cluster so entire communication that is happening with my cluster is over a private endpoint or a private network rather i mean what i mean to say is all sitting either on a peered network or on the same same video so that means whatever i am doing if i am doing some kind of if i want to see the status and all it's all happening and the code downloads and the execution of the code everything is happening within that within the machine itself it's not happening anywhere outside okay now that is one part of security so that means you need to make sure that you have a machine which is um where was that sorry yeah so you need to have your uh yeah so you need to have your self-hosted agent mapped so that you can make it secure second point in the release pipeline if i go to this folder view you will see the how we have actually uh make sure that all our uh the entire process is sequential so if you can see we are doing the zeroth step that means if any new member joining in they know that this is this is something that i have to do at the basic step right and this is the setup this is a setup of an aks cluster or it can be setup of an infrastructure also right this is my update infrastructure anything that i want to update all my pipelines will be part of this group with this and and then all other stuff like cluster the rollback access control the network policies the monitoring everything we have grouped separately all separate pipelines and with varying degree of access so that means the stakeholder can access monitoring not the developers developers will not have access so here in this case i don't have that entire team structure but i'm showing you how we have do i have done that so if you go to the security and from security itself you can decide who of the team can access to this one so this is one of the important thing security at the pipeline level right as well as at the project level with that if i open which one i was talking about yeah maybe this project this is a simple sample application sorry a function application you would see two important things here you can see that the same approach what i saw what what you saw in the aks cluster concept here also the same thing i am running uh the same project i have two different pipelines one for entire infrastructure automation so this project this pipeline is going to deploy is going to deploy my entire setup it's just a publishing artifact here but um it's its purpose is to copy all the files that are needed for setting up the infrastructure and the corresponding release pipeline the corresponding release pipeline if i try to edit that you will see this is doing the main job and some of the important things we have mentioned here it is a setting up of a script this is my arm template uh sorry this is the powershell script that is running which internally deploys the entire infrastructure for me right this pipeline will not be accessed by a developer only for the architects and in that we are also adding lot of other structure if you can look at uh the the different steps in between like we have the uh manual intervention step so at every step whenever the infrastructure architect is trying to deploy an application right they would have to be very careful so once they run the script and they're trying to hand over this to the development team you can have a manual intervention step like this to show them that so if i can show you how the release pipelines actually got run so let me quickly open i'm just running out of time so quickly touch up on this thing so you can see the setup job has finished with setup script then manual intervention topped me with this message that have you done these things now these jobs are something outside your devops how you are managing your key vault whether you are adding your keys in the key vault or not but that is a warning to the infrastructure guys to complete this before they hand it over to the development team so always a check that you need to put it into this thing right and then finally when you hand over these to the development team they are running this is my development team's release so it is named as setup that is by infrastructure team and this is for development team city they would they will basically have a single job to deploy the function that's true but this is where the things are quite interesting and important so you can add your gates you can add your gates over here i think i added a gate in a different project sorry let me just go back to that quickly so in my release pipeline i can actually add and this is very important uh team to do that so i can actually add you know gates saying that if these conditions are not satisfied the release will not go so that means at every step it's not about only ci cd and deployment it's about how you make your uh getting and the checks you know streamline so if i go a pre-deployment approval i will add maybe uh you know the the email address of myself saying i am part of the team i should always get an email only when i approve i am the approver after the approver it will go through the gates and what the gates are doing gates are waiting for a query item see this query item is very important until and unless this query item is satisfied i will not allow the release to happen and how the query item is created that's where i said that you should always fall back on the extended devops right so that's my board my board the project management board there i have queries created so these queries okay all queries so in these queries you would see i have queries in this folder which says okay so if my query is run you can't see the query in a different project okay let me open this one and quickly show you the queries here yeah so i i i have multiple queries you can see the critical bugs is one of the query so if this is not done if this is not in a resolved state it is now unassigned it has to be assigned and resolved i will not allow the release to happen so that means where different teams across multiple go is working this is an extremely important step okay before i stop for this thing let me quickly take you to some of the architectural stuff so that you there is lot of content first of all into this so you can always go through it when you have i mean when you get this take it's not easy to cover in the short span of time but what i want to highlight here is some of the architectures sorry for the uh quick uh pass on to this one just a minute yeah so the two things i would always uh try to highlight is how we suggest our like apart from cicd the other thing that you need to think about is the branching strategies right so this is one of the thing that you need to keep in mind you have multiple options of doing that but this is the best practice you can see on the left hand side top left corner we are talking about a simple branding strategy but features bug fixes all separate so your main branch is going in a linear fashion but your feature is separate they are getting merged into main your box fixes going in separate parallel line merged into men this is okay i think you all know that on the right hand side is even more interesting and important stuff we are not making the release okay i you should look at the bottom diagram first we are not making the release from the main branch this is an important concept people always make this mistake don't release from the main branch we are taking a separate release branch release 20 which is for my release so main branch will never be used for release main branch is still main branch it still flows linearly right the release happens any bug fixes in the release happens then and ultimately release will stay as is or if there is anything that needs to be merged to main branch we will do it after a lot of checks and pull requests and then finally again your other development life cycle will continue so at any point of time you would always see that your release branch main branch and feature branches all go in parallel and that's where you create the segregation of the responsibilities some simple architectures quickly before i stop for q a um the code the visual studio code or visual studio whatever is your ide you push the code uh this is an example of deploying azure web apps right uh the pipelines it goes to the web app you can we have the definite task for doing that and then uh the important thing is the line number six and the seven so the application the moment you deploy you should have application insights in place your azure monitor should be integrated so if that the if the project if the application is crashing you know an alert from there and you can easily switch back uh your deployment through azure devops right similarly in case of containers the life is even easier because if you go on from point number six to azure container service you all know that definitely because the container inside is running within the cluster right so it can always throw you errors and give you an um understanding whether your pods are running or not so you can always have a quick view you can always have a quick view of your system like i am looking at my cluster right now i know that there is there is a primary namespace where what is going to be my view of each of these projects right how much memory and cpu is taking the moment i see that there is a crash something happening not working i should go back to my build stage i should definitely open my project which is ratings api it's a it's a container based application i know okay i have done it so what i can do basically i can if if i have the multiple runs here right so i can actually use this so let me go to the release pipeline and if my pipelines are all segregated with the reference i can redeploy any one of the previous which was working so i know release two had a problem so i will go to release one i will select this and i will just deploy this one again without making any changes right so that's a direct deployment so basically you should have your architecture and this devops thing created in such a way that you can do all these things easily right and this is a complete view of how it should work so you can see the inner loop is important inner loop is your developer loop they all talk to each other deploying applications updating deploying and then ultimately when it goes to the channel it goes through pipeline it takes the container images from a registry deploying onto the production cluster now i'm not showing that okay i think i should have shown you where is that diagram yeah this is the one okay so uh this is important so you can see this is a more of a production scenario i have uh the deployment going to be happening on aks cluster but before that look at if i can zoom it a bit look at line number six the six is from github actions or azure pipelines it coming to your dev test registry the deployment happened your development and qa aks clusters both are running separately with their own loop right they have connection to cosmos db or whatever doesn't matter so on when your q a is ready what you're doing initially is nothing but from eight to ten you are running the pipeline for your production with the same uh image that you have actually deployed on your registry so you now move the uh look at the line on top where dev test container registry through azure pipeline passing the container image to the production registry that's it your deployment will happen from there because you have a notification center that whenever a container registry gets an image you can start your build or you can start your production bill manually also doesn't matter but you are safe that it is tried tested has gone through all the loops and then coming into the play right so there are some six seven examples i've given all good architectures you can go through it and uh i believe i don't know uh i'm running out of time i think so already three minutes past my time uh but just wanted to highlight these processes at high level short time for such a deep discussion but um yeah i would wait for the questions um i i will stop here and we'll take the questions if you have anything please let me know i will take some of the questions which i can see here shivali is it is it good to take the questions now or yes we can we can take there are two questions i think yeah so common problem faced is that team members are not cross-functional and team members are pure they are pure op so teams must learn both stuff without discrimination um yeah so basically uh there are two things uh as i said if they are completely disparate teams then also azure devops is there and you need to bring in people who are the device architects right ops architects uh i'm not saying infra architects or operations guys they can act as a bridge between them they would be doing all the deployments they would do the pull request validations checks and everything right that is the layer you have to bring in but if the teams are cross-functional then your life is easy that means then people can actually uh change the hats and then then do it but it's a challenge i agree with you but then the solution is also there if hope that answers your question rsa to govern all this should there be a horizontal yeah i think that's what i just told you right yes exactly so there should be a devops team or devops architect uh and people underneath so that who can manage or bridge between these two cross-functional team yeah right yeah you will get the tppt assist yes how to deploy sql scripts in azure develop santosh so we have the backpack files so we have a task uh if you guys can still see my screen um your pipeline are you familiar with backpack deployment then i can tell you quickly that there is a task available in azure devops and even in github as well which allow you to uh bring in your dac pack file as your sql database deployment you can use this right and that is sql server deployment on prem or on azure server but this is the pass you can bring in that task and then you can select your files the backpack file and it will be deployed hope that answers your questions and push interesting to understand how database changes are rolled back in devops well i think again uh that's a great question i would probably would request you to come back to me over email my email is by the way is here uh or it can be shared as well by uh the team um where is my email yes sure okay yeah this is my email you guys can write it down so you can reach out to me over email who asked this question naveen so uh basically yes um the the rolling back of the changes if you use backpack file and the one that i just shown you it's easy but then the problem is always your application might break right so that's the trickiest part of it so where you need to basically have a plan uh properly and if see the the basic thing is your deployments your your pipeline should be different right if your pipelines are different and they they are maintained in that way so you can always do that and reduce the amount of error that can come in into this we can talk about it if you reach out to us uh over email how lower environment can integrate with azure uh sagar can you please uh be a bit more uh can they ask are muting themselves so they can only ask over the chat lower environment the queries can be answered they can ask here only in the chat okay how lower environment can integrate with azure if you can tell me a bit more detail sagar over email uh so basically what happens here is uh depending on the software that you are deploying whether it is a container uh it's it's fine i mean it's an aks clustered and all your ah like for example you can you could see uh my window right you could see my window that um sorry for that so you can see my window i have different name spaces where are my name spaces yeah you can see my different name spaces dave and the primaries and all so by the name spaces i can basically manage all my clusters different environments lower environments and then deploy onto the production whatever is needed similar thing is needed for all other services also like for example app service we have a slot option so you can deploy on to dev slot and keyword start and finally swap the slot with the production options are there if you need to understand more you can mail me in details yeah does azure provides ways to monitor the customer user transaction or yes absolutely so that's a good question thanks so basically it is um that's what i had shown in the diagram also right the application insights uh and the integration with application inside so ideally your whether it is a container based application or non-container application you should have this line number eight on or here line number six on uh so that you can actually uh track and maintain uh this type of the the functional stuff that how the users are interacting and all that is in built-in azure obviously and what you can do if that gives some error you can always that's what i was showing you you can always switch back your pipeline deployment again back to some other earlier version which was working and then deploy it again right so that is very much doable out of the box yes to govern all these i think i answered those questions all any further question team uh please do let me know yeah i think uh we are good uh only and thank you for this uh amazing session i think all the questions got answered and if there are more any questions we are running out of time so if people come back and pose their questions so we'll be sharing that live url um here in the with you so you may you maybe you can answer them offline is it fine sure i think i can see rss question also but yeah i know it is well beyond the time so you can actually send me an email with this question i will respond back this is very much possible and we can do it and yeah if there is any session plan we will definitely run for it no issues yeah thank you for this wonderful session monarchy thank you so much for watching yeah so before we conclude today's session few things to mention that as we all know that uh me this meetup session is part of azure developer community so if you haven't joined this um this community you can visit the site azdev.community and you can go through and about the upcoming sessions and all other information so you can join nearest community and we are part of this amazing community also uh apart from this we are also running in very exciting activity it's called cloud quizzing challenge it's very fun and interesting life quizzing challenge where you can come and participate in live quizzes on of on azure technologies so our next quiz is tomorrow that is 10 december so if you and haven't registered yet you can go and register yourself the site is cc.config.com it's reflecting on your screen here and also if you talk the leaderboard in the quizzes you can you you can get exciting prices and goodies and more so this is it so thank you all for joining for today's session and yeah okay thank you manoji thank you so far thanks so much everyone thank you everyone

Channel: KonfHub Tech

Views: 42

Keywords:

Id: gCldOrWR0to

Length: 71min 42sec (4302 seconds)

Published: Thu Dec 09 2021