🔥DevOps Tools Full Course 2023 | DevOps Tools Explained | DevOps Tools Tutorial | Simplilearn

foreign welcome to this devops tools full course by simply learn in this complete course we learn everything that we need regarding several devops tools we have Matthews and chidanand to guide us through this journey so let's look at what's in store for us we shall begin with having a basic understanding of devops with a short animated video after that we'll dive deeper and understand the basics of devops like its relevance in the current scenario and several devops phases once we know devops we shall begin with learning different devops Tools in detail you shall learn about get followed by GitHub and in the process we shall learn about various git commands second we shall see what Jenkins is Jenkins Pipeline and how we install Jenkins after that we'll move on to Docker and check out what Docker is and Docker container is all about we'll also learn about the different processes and how to install docker moving further we'll understand the basics of kubernetes the difference between kubernetes and Docker and kubernetes on AWS the following tool in the list will be puppet where we shall know what puppet is and why we use puppet and the process to install it moving forward we learn about ansible and its installation process and what are the benefits of using ansible the last tool on our list is selenium where we shall learn about selenium and all about its sweet tools followed by its installation processes while we're learning all these tools we shall understand them with the Hands-On demo making the understanding much more effective and more accessible finally when we have a piece of deep knowledge regarding devops and several devops tools we'll look at some of the most prominent devops interview questions that will help you excel in your forthcoming devops interviews but before we begin if you haven't subscribed to our Channel already make sure you hit the Subscribe button and the bell icon to never miss an update so without further Ado let's begin right from the start software development comprise two different departments the development team that develops the plan designs and builds the system from scratch and the operation team for testing and implementation of whatever is developed the operations team gave the development team feedback on any bugs that needed fixing and any rework required invariably the development team would be idle awaiting feedback from the operations team this undoubtedly extended timelines and delayed the entire software development cycle there would be instances where the development team moves on to the next project while the operations team continues to provide feedback for the previous code this meant weeks or even months for the project to be closed and final code to be developed now what if the two departments came together and worked in collaboration with each other what if the wall of confusion was broken and this is called the devops approach the devops symbol resembles an infinity sign suggesting that it is a continuous process of improving efficiency and constant activity the devops approach makes companies adapt faster to updates and development changes the teams can now deliver quickly and the deployments are more consistent and smooth though there may be communication challenges devops manages a streamlined flow between the teams and makes the software development process successful the devops culture is implemented in several phases with the help of several tools let's have a look at these phases the first phase is the planning phase where the development team puts down a plan keeping in mind the application objectives that are to be delivered to the customer once the plan is made the coding Begins the development team works on the same code and different versions of the code are stored into a repository with the help of tools like git and merge when required this process is called Version Control the code is then made executable with tools like Maven and Gradle in the build stage code is successfully built it is then tested for any bugs or errors the most popular tool for automation testing is selenium once the code is passed several manual and automated tests we can say that it is ready for deployment and is sent to the operations team the operations team now deploys the code to the working environment the most prominent tools used to automate these phases are ansible Docker and kubernetes after the deployment the product is continuously monitored and nagios is one of the top tools used to automate this phase the feedback received after this phase is sent back to the planning phase and this is what forms the core of the devops lifecycle that is the integration phase Jenkins is the tool that sends the code for building and testing if the code passes the test it is sent for deployment and this is referred to as continuous integration there are many Tech Giants and organizations that have opted for the devops approach for example Amazon Netflix Walmart Facebook and Adobe Netflix introduced its online streaming service in 2007. in 2014 it was estimated that a downtime for about an hour would cost Netflix two hundred thousand dollars however now Netflix can cope with such issues they opted for devops in the most fantastic way Netflix developed a tool called the Simeon Army that continuously created bugs in the environment without affecting the users this chaos motivated the developers to build a system that does not fall apart when any such thing happens so on this note here is a quiz for you match the devops tool with the phase it is used in a B C d none of the above today more and more companies lean towards automation with the aim of reducing its delivery time and the gap between its development and operations teams to attain all of these there's just one Gateway devops and if you're also looking forward to doing the same and Excel in devops check out simplylearn's postgraduate program in devops design in collaboration with Caltech ctme the program can help you master several in-demand devops skills and tools like continuous integration and delivering devops on cloud kubernetes Docker and much more hello this is Matthew with simply learn and today we're going to go through and introduce you to devops wouldn't go through a number of key elements today the first two will be reviewing models that you're already probably using for delivering Solutions into your company and the most popular one is waterfall followed by agile then we'll look at devops and how devops differs from the two models and how it also borrow and leverages the best of those models we'll go through each of the phases that are used in typical devops delivery and then the tools used within those phases to really improve the efficiencies within devops finally we'll summarize the advantages that devops brings to you and your teams so let's go through waterfall so waterfall is a traditional delivery model that's been used for many decades for delivering Solutions not just IT solutions and digital Solutions but either way before that it has its history it goes back to World War II so waterfall is a model that is used to capture requirements and then Cascade each key deliverable through a series of different stage Gates that is used for building out the solution so let's take you through each of those stage Gates the first that you may have done is requirements analysis and this is where you sit down with the actual client and you understand specifically what they actually do and what they're looking for in the software that you're going to build and then from that requirements analysis you'll build out a project planning so you have an understanding of what the level of work is needed to be able to be successful in delivering the solution after that you've got your plan then you start doing the development and that means that the programmers start coding out their solution they build out their applications to build out the websites and this can take weeks or even months to actually do all the work when you've done your coding and development then you send it to another group that does testing and they'll do full regression testing of your application against the systems and databases that integrate with your application you'll test it against the actual code you'll do manual testing you do UI testing and then after you've delivered the solution you go into maintenance mode which is just kind of making sure that the application keeps working there's any security risks that you address the those security risks now the problem you have though is that there are some challenges however that you have with the waterfall model the cascading deliveries and those complete and separated stage Gates means that it's very difficult for any new requirements from the client to be integrated into the project so if a client comes back and it's the project has been running for six months and they've gone hey we need to change something that means that we have to almost restart the whole project it's very expensive and it's very time consuming also if you spend weeks and months away from your clients and you deliver a solution that they are only just going to see after you spend a lot of time working on it they could be pointing out things that are in the actual final application that they don't want or are not implemented correctly or lead to just general unhappiness the chance you then have is if you want to add back in the client's feedback to restart the a whole waterfall cycle again so the client will come back to you with a list of changes and then you go back and you have to start your programming and you have to then start your testing process again and just you're really adding in lots of additional time into the project so using the waterfall model companies have soon come to realize that you know the clients just aren't able to get their feedback and quickly effectively it's very expensive to make changes once the teams have started working and the requirement in today's digital world is that solution simply must be delivered faster and this has led for a specific change in agile and we start implementing the agile model so the agile model allows programmers to create prototypes and get those prototypes to the client with the requirements faster and the client is able to then send their requirements back to the programmer with feedback this allows us to create what we call a feedback loop where we're able to get information to the clients and client can get back to the development team much faster typically when we're actually going through this process we're looking at the engagement cycle being about two weeks and so it's much faster than the traditional waterfall approach and so we can look at each feedback loop as comprising of four key elements we have the planning where we actually sit down with the client and understand what they're looking for we then have coding and testing that is building out the code and the solution that is needed for the client and then we review with the clients the changes that have happened but we do all this in a much tighter cycle that we call a Sprint and that typically a Sprint will last for about two weeks some companies run sprints every week some run every four weeks it's up to you as a team to decide how long you want to actually run a Sprint but typically it's two weeks and so every two weeks the client is able to provide feedback into that Loop and so you were able to move quick quickly through iterations and so if we get to the end of Sprint 2 and the client says hey you know what we need to make a change you can make those changes quickly and effectively for Sprint three what we have here is a breakdown of the ceremonies and the approach that you bring to Agile so typically what will happen is that a product leader will build out a backlog of products and what we call a product backlog and this will be just a whole bunch of different features and they may be small features or bug fixes all the way up to large features that may actually span over multiple Sprints but when you go through the Sprint planning you want to actually break out the work that you're doing so the team has a mixture of small medium and large solutions that they can actually Implement successfully into their Sprint plan and then once you actually start running your Sprint again it's a two-week activity you meet every single day the two with the actual Sprint team to ensure that everybody is staying on track Rank and if there's any blockers that those blockers are being addressed effectively and immediately the goal at the end of the two weeks is to have a deliverable product that you can put in front of the customer and the customer can then do a review the key advantages you have are running a Sprint with agile is that the client requirements are better understood because the client is really integrated into the scrum team they're there all the time and the product is delivered much faster than with a traditional waterfall model you're delivering features at the end of each Sprint versus waiting weeks months or in some cases years for a waterfall project to be completed however there are also some distinct disadvantages the product itself really doesn't get tested in a production environment it's only been tested on the developer computers and it's really hard when you're actually running agile for the Sprint team to actually build out a solution easily and effectively on their computer computers to mimic the production environment and the developers and the operations team are running in separate silos so you have your development team running their Sprint and actually working to build out the features but then when they're done at the end of their Sprint and they want to do a release they kind of fling it over the wall at the operations team and then it's the operations team job to actually install the software and make sure that the environment is running in a stable fashion that is really difficult to do when you have the two teams really not working together so here we have is a breakdown of that process with the developers submitting their work to the operations team for deployment and then the operations team may submit their work to the production servers but what if there is an error what if there was a setup configuration error with the developers test environment that doesn't match the production environment there may be a dependency that isn't there there may be a link to an API that doesn't exist in production and so you have these charges that the operations team are constantly faced with and their challenge is that they don't know how the code works so this is where devops really comes in and let's dig into how devops which is developers and operators working together is the key for successful continuous delivery so devops is as an evolution of the agile model the agile model really is great for Gathering requirements and for developing and testing out your Solutions and what we want to be able to do is kind of address that challenge and that gap between the Ops Team and the dev team and so with devops what we're doing is bringing together the operations team and the development team into a single team and they are able to then work more seamlessly together because they are integrated to be able to build out solutions that are being tested in a production-like environment so that when we actually deploy we know that the code itself will work the operations team is then able to focus on what they're really good at which is analyzing the production environment and being able to provide feedback to the developers on what is being successful so we're able to make adjustments in our code that is based on data so let's step through the different phases of a devops team so typically you'll see that the devops team will actually have eight phases now this is somewhat similar to Agile and what I'd like to point out at the time is that again agile and devops are very closely related that agile and devops are closely related delivery models that you can use with devops it's really just extending that model with the key phases that we have here so let's step through each of these key phases so the first phase is planning and this is where we actually sit down with a business team and we go through and understand what their goals are the second stage is as you can imagine and this is where it's all very similar to Agile is that the coders actually start coding but they typically they'll start using tools such as git which is a distributed Version Control software it makes it easier for developers to all be working on the same code base rather than bits of the code that is rather than them working on bits of the code that they are responsible for so the goal with using tools such as git is that each developer always has the current and latest version of the code you then use tools such as Maven and Gradle as a way to consistently build out your environment and then we also use tools to actually automate our testing now what's interesting is when we use tools like selenium and junit is that we're moving into a world where our testing is scripted the same as our build environment and the same as using our get environment we can start scripting out these environments and so we actually have scripted production and environments that we're moving towards Jenkins is the integration phase that we use for our tools and another Point here is that the tools that we're listing here these are all open source tools these are tools that any team can start using we want to have tools that control and manage the deployment of code into the production environments and then finally tools such as ansible and Chef will actually operate and manage those production environments so that when code comes to them that that code is compliant with the production environment so that when the code is then deployed to the many different production servers that the expected results of those servers which is you want them to continue running is received and then finally you monitor the entire environment so you can choose zero in on spikes and issues that are relevant to either the code or changing consumer habits on the site so let's step through some of those tools that we have in the devops environment so here we have is a breakdown of the devops tools that we have and again one of the things I want to point out is that these tools are open source tools there are also many other tools this is just really a selection of some of the more popular tools that are being used but it's quite likely that you're already using some of these tools today you may already be using Jenkins you may already be using git but some of the other tools really help you create a fully scriptable environment so that you can actually start scripting out your entire devops tool set this really helps when it comes to speeding up your delivery because the more you can actually script out of the work that you're doing the more effective you can be at running automation against those scripts and the more effective you can be at having a consistent experience so let's step through this devops process so we go through and we have our continuous delivery which is our plan code build and test best environment so what happens if you want to make a release well the first thing you want to do is send out your files to the build environment and you want to be able to test the code that you've been created because we're scripting everything in our code from the actual unit testing being done to the all the way through to the production environment because we're testing all of that we can very quickly identify whether or not there are any defects within the code if there are defects we can send that code right back to the developer with a message saying what the defect is and the developer can then fix that with information that is real on the either the code or the production environment if however your code passes the the scripting tests it can then be deployed and once it's out to deployment you can then start monitoring that environment what this provides you is the opportunity to speed up your delivery so you go from the waterfall model which is weeks months or even years between releases to Agile which is two weeks or four weeks depending on your Sprint Cadence to where you are today with devops where you can actually be doing multiple releases every single day so there are some significant advantages and there are companies out there that are really zeroing in on those advantages if we take any one of these companies such as Google Google any given day will actually process 50 to 100 new releases on their website through their devops teams in fact they have some great videos on YouTube that you can find out on how their devops teams work Netflix is also a similar environment now what's interesting with Netflix is that Netflix have really fully embraced devops within their development team and so they have a devops team and Netflix is a completely digital company so they have software on phones on Smart TVs on computers and on websites interestingly though the devops team for Netflix is only 70 people and when you consider that a third of all internet traffic on any given day is from Netflix it's really a reflection on how effective devops can be when you can actually manage that entire business with just 70 people so there are some key advantages that devops has it's the actual time to create and deliver a software is dramatically reduced particularly compared to Waterfall complexity of maintenance is also reduced because you're automating and scripting out your entire environment now you're improving the communication between all your teams so teams don't feel like they're in separate silos but that are actually working cohesively together and that there is continuous integration and continuous delivery so that your consumer your customer is constantly being delighted a very warm welcome to all our viewers I'm Anjali from Simply learn and today I'll be showing you the installation for git on windows so we'll start by downloading and installing it on our system we'll then have a look at the git bash interface we'll type in some basic git commands next we'll create a local repository that is we'll create a repository on a local machine we'll then connect to a remote repository and finally we'll push the file onto GitHub first things first we need to download and install git so download git for Windows and I'll select the second link so 2.19.1 which is the most latest version of git that's the one we want for Windows system choose your version so mine is a 64-bit system and it's downloading so this will take a while so git is on a system click here run so go to next we don't have to change this path uh just so just click on in quick launch and on desktop next next again next nothing to change here either and install so now git is getting installed on our system so we don't need to view the release notes we just want to launch the git bash so let's just stick that and then click on finish and your git bash interface appears here so we are on the master Branch the first thing we do is we'll check the version for our git so the command is git dash dash version and as you can see version 2.19.1 on our Windows system which is exactly what we just downloaded we'll now explore the help command so let's just type get help config so config is another command and as I hit enter the manual page for the second command opened up which is config so what help command does is that it provides the manual or the help page for the command just following it so in case you have any doubts regarding how a command is used what a command is used for all the various syntax of the command you can always use the help command now this another Syntax for using the help command itself which is git config dash dash help enter this does the exact same thing as you can see now that we looked at the help command let's begin by creating a local directory so mkdir test now test is my new directory I'll move into this directory so CD test great so now that we are inside our test directory let's initialize this directory so get init is the command for initializing the directory and as you can see as you can see the path here this is the local path where a directory is created so I'll just show you the directory test and it's completely empty what we do now is we'll create a text file within this new directory that we created so new text document and I'll just name this demo I'll open this and just put in some dummy content so hello simply learn save this file and go back to your bash interface let's just check the status now so get status and as you can see our file has appeared it's visible but nothing is committed yet so this means that we have not made any change to a file through the git tool itself so the next thing that we are going to do is we'll be adding demo to our current directory the next command that we will be applying is the commit command and when you add certain files to the current directory the comment command is applied on all the above directories so git commit minus M and a message that will appear once the file is committed so as you can see one file is changed and one insertion I'll just clear the screen next thing we need to do is we need to link our git to our GitHub account so the command for doing that is git config Global user dot username and this will be followed by our username so let me just show you my GitHub account so this is my GitHub profile and my username is simply learn Dash GitHub so guys before you begin this procedure just make a GitHub account type in my username here simply learn Dash GitHub and enter and there you go a git is successfully linked with GitHub next thing we do is we'll just open our GitHub and we'll create a new repository give a repository name so I'll get name it test underscore demo and create Repository great so our repository is created this is our remote repository what we do next is just copy the link and then go back to your bash interface now we need to link our remote and a local Repository so git remote origin and then paste the HTTP link and now that our local repository and a remote repository are linked we can push a local file onto a remote repository so the command for doing that is git push origin Master as we are on the master branch and that's done so now let's move back to GitHub I'll just click on test demo and as you can see here our local file has been pushed to our remote repository with that we have successfully completed our demo thanks Anjali now Matthew and shadanan will take you through the remaining part of the course hello this is Matthew from Simply learn and today we're going to go through and explain how you can use git for your software development so what are we going to cover today so we're going to introduce the concept of Version Control that you will use within your devops environment then we'll talk about the different tools are available in a distributed Version Control System we'll highlight a product called git which is typically used for Version Control today and you'll also go through what are the differences between git and GitHub you may have used GitHub in the past or other products like gitlab and we'll explain what are the differences between git and git and services such as GitHub and gitlab will break out out the architecture of what a get process looks like how do you go through and create forks and clones how do you have collaborators being added into your projects how do you go through the process of branching merging and rebasing your project and what are the list of commands that are available to you in get finally I'll take you through a demo on how you can actually run git yourself and in this instance use the software of git against a public service such as GitHub all right let's talk a little bit about Version Control Systems so you may have already been using a version control system within your environment today you may have used tools such as Microsoft's team Foundation services but essentially the use of a Version Control System allows people to be able to have files that are all stored in a single repository so if you're working on developing a new program such as a website or an application you would store all of your Control software in a single repository now what happens is that if somebody wants to make changes to the code they would check out all of the code in the repository to make the changes and then there would be an addendum added to that so um there will be the the version one changes that you had then the person would then later on check out that code and then be a version two um added to that code and so you keep adding on versions of that code the bottom line is that eventually you'll have people being able to use your code and that your code will be stored in a centralized location however the challenge you're running is that it's very difficult for large groups to work simultaneously within a project the benefits of a VCS system a Version Control System should demonstrates that you're able to store multiple versions of a solution in a single repository now let's take a step at some of the challenges that you have with traditional Version Control Systems and see how they can be addressed with distributed Version Control so in a distributed Version Control environment what we're looking at is being able to have the code shared across a team of developers so if there are two or more people working on a software package they need to be able to effectively share that code amongst themselves so that they constantly are working on the latest piece of code so a key part of a distributed Version Control System that's different to just a traditional version control system is that all developers have the entire code on their local systems and they try and keep it updated all the time it is the role of the a distributed VCS server to ensure that each client and we have a developer here and developer here and developer here and each of those our clients have the latest version of the software and then that each person can then share the software and appear to pay here like approach so that as changes are being made into the server of changes to the code then those changes are then being redistributed to all of the development team the tool to be able to do an effective distributed VCS environment is get now you may remember that we actually covered get in a previous video and we'll reference that video for you so we start off with our remote get repository and people are making updates to the copy of their code into a local environment that local environment can be updated manually and then periodically pushed out to the get repository so you're always pushing out the latest code that you've code changes you've made into the repository and then from the repository you're able to pull back the latest updates and so your git repository becomes the kind of the center of the universe for you and then updates are able to be pushed up and pulled back from there what they this allows you to be able to accomplish is that each person will always have the latest version of the code so what is get get is a distributed Version Control tool used for source code management so GitHub is the remote server for that source code management and your development team can connect their get clients to that certain remote Hub server now git is used to track the changes of the source code and allows large teams to work simultaneously with each other it supports a non-linear development because of thousands of parallel branches and has the ability to handle large projects efficiently so let's talk a little bit about git versus GitHub so get is a software tool whereas GitHub is a service and I'll show you how those two look in the moment you install the software tool forget locally on your system them whereas GitHub because it is a service it's actually hosted on a website git is actually the software that used to manage different versions of source code whereas GitHub is used to have a copy of the local repository stored on the service on the website itself git provides command line tools that allow you to interact with your files whereas GitHub has a graphical interface that allows you to check in and check out files so let me just show you the two tools here so here I am at the get website and this is the website you would go to to download the latest version of git and again git is a software package that you install on your computer that allows you to be able to do Version Control in a peer-to-peer environment for that peer-to-peer environment to be successful however you need to be able to store your files in a server somewhere and typically I a lot of companies will use a service such as git Hub as a way to be able to store your files so git can communicate effectively with GitHub there are actually many different companies that provide similar service to GitHub gitlab is another popular service but you also find that development tools such as Microsoft Visual Studio are also incorporating git commands into their tools so the latest version of Visual Studio team Services also provides this same ability but GitHub it has to be remembered is a place where we actually store our files and can very easily create public and shareable is a place where we can store our files and create public shareable projects you can come to GitHub and you can do a search on projects you can see at the moment I'm doing a lot of work on blockchain but you can actually search on the many hundreds of projects here in fact I think there's something like over a hundred thousand projects being managed on GitHub at the moment that number's probably actually much larger than that and so if you are working on a project I would certainly encourage you to start at GitHub to see if somebody's already maybe done a prototype that they're sharing or they have an open source project that they want to share that's already available um in GitHub certainly if you're doing anything with um Azure you'll find that there are thousands 45 000 Azure projects currently being worked on interestingly enough GitHub was recently acquired by Microsoft and Microsoft is fully embracing open source Technologies so that's essentially the difference between get and GitHub one is a piece of software and that's get and one is a service that supports the ability of using the software and that's GitHub so let's dig deeper into the actual get architecture itself so the working directory is the folder where you are currently working on your get project and we'll do a demo later on where you can actually see how we can actually simulate each of these steps so you start off with your working directory where you store your files and then you add your files to a staging area where you are getting ready to commit your files back to the main branch on your git project you want to push out all your changes to a local repository after you've made your changes and these will commit those files and get them ready for synchronization with the service and we'll then push your services out to the remote repository an example of a remote repository would be GitHub later when you want to update your code before you write any more code you would pull the latest changes from the remote repository so that your copy of your local software is always the latest version of the software that the rest of the team is working on one of the things that you can do is as you're working on new two features within your project you can create branches you can merge your branches with the mainline code you can do lots of really creative things that ensure the that Aid the code remains at very high quality and B that you're able to seamlessly add in new features without breaking the call code so let's step through some of the concepts that we have available and get so let's talk about forking and cloning and kit so both of these terms are quite old terms when it comes to development but forking is certainly a term that goes way way back long before we had distributed CVS systems such as the ones that we're using with get to Fork a piece of software is a particularly open source project you would take the project and create a copy of that project and but then you would then associate a new team and new people around that project so it becomes a separate project in entirety a clone and this is important when it comes to working with get a clone is identical with the same teams and same structuring as the main project itself so when you download the code you're downloading exact copy of that code with all the same security and access rights as the main code and then you can then check that code back in and potentially your code because it is identical could potentially become the mainline code in the future now that typically doesn't happen your changes are the ones that merge into the main branch but also but you do have that potential where your code could become the main code with Git You can also add collaborators that can work on the project which is essential for projects where particularly where you have large teams and this works really well when you have product teams where the teams themselves are self-empowered you can do a concept what's called branching in git and so say for instance you are working on a new feature that new feature and the main main version of the project have to still work simultaneously so what you can do is you can create a branch of your code so you can actually work on the new feature whereas the rest of the team continue to work on the main branch of the the project itself and then later you can merge the two together pull from remote is the concept of being able to pull in Services software the team's working on from a remote server and git rebase is the concept of being able to take a project and re-establish a new start from the project so you may be working a project where there have been many branches and the team has been working for quite some time on different areas and maybe you kind of losing control of what the true main branch is you may choose to rebase your project and what that means though is that anybody that's working on a separate Branch will not be able to Branch their code back into the mainline Branch so going through the process of a get rebase essentially allows you to create a new start for where you're working on your project so let's go through forks and clones so you want to go through the process so you want to go ahead and Fork the code that you're working on so this is this scenario that one of your team wants to go ahead and add a new change to the project the team member may say yeah go ahead and you know create a separate Fork of the actual project so what does that look like so when you actually go ahead and create a fork of the repository you actually go and you can take the version of the mainline Branch but then you take it completely offline into a local repository for you to be able to work from and you can take the mainline code and you can then work on a local version of the code separate from the mainline Branch it's now a separate Fork collaborators is the ability to have team members working on a project together so if you know if someone is working on a piece of code and they see some errors in the code that you've created none of us are perfect at writing code I know I've certainly made errors in my code it's great to have other team members that have your bag and can come in and check and see what they can do to improve the code so to do that you have to then add them as a collaborator now you do that in GitHub you can give them permission within GitHub itself and it's really easy to do supervisual interface that allows you to do the work quickly and easily and depending on the type of permissions you want to give them sometimes it could be very limited permissions it may be just to be able to read the files sometimes it's being able to go in and make all the changes you can go through all the different permission settings on GitHub to actually see what you can do but you'll be able to make changes so that people can actually have access to your repository and then you as a team can then start working together on the same code let's step through branching and get so suppose you're working on an application but you want to add in a new feature and this is very typical within a devops environment so to do that you can create a new branch and build a new feature on that Branch so here you have your main application on what's known as the master branch and then you can then create a sub branch that runs in parallel which has your feature you can then develop your feature and then merge it back into the master Branch at a later point in time now the benefit you have here is that by default we're all working on the master Branch so we always have the latest code the circles that we have here on the screen show various different commits that have been made so we can keep track of the master branch and then the branches that have come off which have the new features and there can be many branches in git so git keeps you the new features you're working on in separate branches until you're ready to merge them back in with the main branch so let's talk a little bit about that merge process so you're starting with the master branch which is the blue line here and then here we have a separate parallel branch which has the new features so if we're to look at this process the base commit of feature B is the branch f is what's going to merge back into the master branch and it has to be said there can be so many Divergent branches but eventually you want to have everything merge back into the master Branch let's step through git rebase so again we have a similar situation where we have a branch that's being worked in parallel to the master branch and we want to do a get rebase so we're at stage C and what we've decided is that we want to reset the project so that everything from here on out with along the master branch is the standard product however this means that any work that's been done in parallel as a separate Branch will be adding in new features along this new re-based environment now the benefit you have by going through the rebase process is that you're reducing the amount of storage space that's required for when you have so many branches it's a great way to just reduce your total footprint for your entire project so git rebase is the process of combining a sequence of commits to form a new base commit and the prime reason for rebasing is to maintain a linear project history when you rebase it you unplug a branch and re-plug it in on the tip of another branch and usually you do that on the master branch and that will then become the new Master Branch the goal of rebasing is to take all the commits from a feature branch and put it together in a single Master Branch it makes it the project itself much easier to manage let's talk a little bit about pull from remote so Suppose there are two developers working together on application the concept of having a remote repository allows the the code to the two developers will be actually then checking in their code into a remote repository that becomes a centralized location for them to be able to store their code it enables them to stay updated on the recent changes to the repository because they'll be able to pull the latest changes from that remote repository so that they are ensuring that as developers they're always working on the latest code so you could pull any changes that you have made to your thought remote repository to your local repository the command to be able to do that is written here and we'll go through a demo of how to actually do that command in a little bit good news is if there are no changes you'll get a notification saying that you're already up to date and if there is a change it will merge those changes to your local repository and you get a list of the changes that have been made remotely so let's step through some of the commands that we have in git so git and it initializes a local git repository on your hard drive get ads ads one or more files to your staging area git commit Dash M commit message is a commit changes the git command commits changes to head up to the git command commits changes to your local staging area get status checks the status of your current repository and lists the files you have changed get lock provides a list of all the commits made on your current Branch get diff the user changes that you've made to the file so you can actually have files next to each other you can actually see the differences between the two files get push origin Branch name so the name of your branch command will push the branch to the remote repository so that others can use it and this is what you would do at the end of your project git config Dash Global username or tell get Who You Are by configuring the author name we'll go through that in a moment git config Global user email will tell get the author of by the email ID get clone creates a git repository copy from a remote Source get remote ad origin server connects the local repository to the remote server and adds the server to be able to push to it git branch and then the branch name will create a new Branch for you to create a new feature that you may be working on get checkout and then the branch name will allow you to switch from one branch to another Branch git merge Branch name or merge a branch into the active Branch so if you're working on a new feature you're going to merge that into the main branch a get rebase will reapply commits on top of another base tip and get rebase will reapply commits on top of another base tip and these are just some of the popular git commands there are some more but you can certainly dig into those as you're working through using get so let's go ahead and run a demo using get so now we are going to do a demo using get on our local machine and GitHub as the remote repository for this to work I'm going to be using a couple of tools first I'll have the deck open as we've been using up to this point the second is I'm going to have my terminal window also available and let me bring that over so you can actually see this and the terminal window is actually running git bash as the software in the background which you'll need to download and install you can also run get batch locally on your Windows computer as well and in addition I'll also have the GitHub repository that we're using for simply learn and already set up and ready to go all right so let's get started so the first thing we want to do is create a local repository so let's go ahead and do exactly that so the local repository is going to reside in my development folder that I have on my local computer and for me to be able to do that I need to create a drive in that folder so I'm going to go ahead and change the directory so I'm actually going to be in that folder before I actually create make the new folder so I'm going to go ahead and change directory and now I'm in the development directory I'm going to go ahead create a new folder and let's go ahead and created a new folder called hello world I'm going to move my cursor so that I'm actually in the hello world folder and now that I'm in the hello world folder I can now initialize this folder as a git Repository so I'm going to use the get command init to initialize and let's go ahead and initialize that folder so let's see what's happened so here I have my Hello wall folder that I've created and you'll now see that we have a hidden folder in there which is called dot get and we expand that we can actually see all of the different subfolders that git repository will create so let's just move that over a little bit so we can see the rest of the work and now if we check on our folder here we actually see this is users Matthew development hello world dot get and that matches up with hidden folder here so we're going to go ahead and create a file called readme.txt in our folder so here is our hello world folder and I'm going to go ahead and using my text editor which happens to be Sublime I'm going to create a file and it's going to have in the text hello world and I'm going to call this one readme.txt if I go to my Hello World folder you'll see that we have the readme.txt file actually in the folder what's interesting is if I select the get status command what it'll actually show me is that this file has not yet been added to the commits yet for this project so even though the file is actually in the folder it doesn't mean that it's actually part of the project for us to do that we actually have to go and select for us to actually commit the file we have to go into our terminal window and we can use the get status to actually read the files that we have there so let's go ahead and usually get status command and it's going to tell us that this file has not been committed you can use this with any folder to see which files and subfolders haven't been committed and what we can now do is we can go and actually add the readme file so let's go ahead I'm just going to say at git add so the git command is ADD [Music] readme.txt so that then adds that file into our main project and we want to then commit those files into the main repositories history and so it's that do that we'll have the the get command commit and we'll do a mess message in that commit and this one would be first commit and it has committed that project what's interesting is we can now go back into readme file and I can change this so we can go hello get is a very popular Version Control solution and we'll we'll save that now what we can do is we can actually go and see if we have made differences to the readme text so to do that we'll use the disk command forget so we do get Dave and it gives us two releases the first is what the original text was which is hello world and then what we have afterwards is what is now the new text in green which has replaced the original text so what we're going to do now is uh you want to go ahead and create an account on GitHub we already have one so what we're going to do is we're going to match the account from GitHub with our local account so to do that we're going to go ahead and say get config and we're going to do Dash and it's going to be a globaluser.name and we'll put in our username that we use for GitHub and this instance we're using the simply learn Dash GitHub account name and under the GitHub account you can go ahead and create a new repository name in this instance we call the repository hello dash world and what we want to do is connect the local GitHub account with the remote hello world.get account and we do that by using this command from get which is our remote connection and so let's go ahead and type that in open this up so we can see the whole thing so we're going to type in git remote add origin https backslash by sash GitHub .com slash simply learn Dash GitHub and you have to get this typed in correctly when you're typing in the location hello dash world dot gate that creates the connection to your hello world account and now we want to do is we want to push the files to the remote location using the git push command commit git push origin master so we're going to go ahead and connect to our local remote GitHub so I'm just going to bring up my terminal window again and so let's select get remote add origin and we'll connect to the remote location github.com slash simply learn Dash GitHub slash hello dash world dot get oh we actually have already connected so we're connected to that successfully and now we're going to push the master Gish so get push origin master and everything is connected and successful and if we go out to GitHub now we can actually see that our file was updated just a few minutes ago so what we can actually do now is we can go and Fork a project from GitHub and clone it locally so we're going to use the fork tool that's actually available on GitHub let me show you where that is located and here is our branching tool it's actually changed more recently with a new UI interface and once complete we'll be able to then pull a copy of that to our account using the forks new HTTP URL address so let's go ahead and do that so we're going to go ahead and create a fork of our project now to do that you would normally go in when you go into your project you'll see that there are Fork options in the top right hand corner of the screen now right now I'm actually logged in with the default primary count for this project so I can't actually Fork a project as I'm working on the main branch however if I come in with a separate ID and here I am I have a different ID and so I'm actually pretending I'm somebody else I can actually come in and select the fork option and create a fork of this project and this will take just a few seconds to actually create the fork and there we are we have gone ahead and created the fork so you want to say clone or download with this and so this is the I select they'll actually give me the web address I can actually show you what that looks like I'll open up my text editor that's the correct I guess that is correct so I'm going to copy that and I can Fork the project locally and clone it locally I can change the directory so I can create a new directory that I'm going to put my files in and then post in that content into that file so I can now actually have multiple versions of the same code running on my computer I can then go into default content and use the patchwork command 20 so I can create a copy of that code that we've just created in the college that's a clone and we can create a new folder that we're actually putting the work in and we could for whatever reason we wanted to we could call this where folder Patchwork and that would be maybe a new feature and then we can then paste in the URL of the new directory that has the forked work in it and now at this point we've now pulled in and created a clone of the original content and so this allows us to go ahead and Fork out all of the work for our project onto our computer so we can then develop our work separately so now what we can actually do is we can actually create a branch of the fork that we've actually pulled in onto our computer so we can actually then create our own code that runs in that separate branch and so we want to check out um the the branch and then push the origin Branch down to our computer this will give us the opportunity to then add our collaborators so we can actually then go over to GitHub and we can actually come in and add in our collaborators and we will do that under settings and select collaborators and here we can actually see we have different collaborators that have been added into the project and you can actually then request people to be added via their GitHub name or by email address or by their full name one of the things that you want to be able to do is ensure that you're always keeping the code that you're working on fully up to date by pulling in all the changes from your collaborators you can create a new branch and then make changes and merge it into the master Branch now to do that you would create a folder and then that folder in this instance would be called test we would then move our cursor into the folder called task and then initialize that folder so let's go ahead and do that so let's call create a new folder and we're going to first of all change our root folder and we're going to go to development I'm going to create a new folder call it test and we're going to move into the test folder and we will initialize that folder and we're going to move some files into that test folder close one test one and then we're going to do file save as and this one's going to be test two and now we're going to commit those files Okay add and then we'll use the dot to pull in all files and then git commit Dash m what else did make sure I'm in the right folder here I don't think I was and now that I'm in the correct folder let's go ahead and and get commit and it's gone ahead and added those files and so we can see the two files that were created have been added into the master and we can now go ahead and create a new Branch you call this one get branch test underscore branch and let's go ahead and create a third file to go into that folder this is file three into file save as we'll call this one test three dot text and we'll go ahead and add that file and to get add test three Dot txt and we're going to move from the master Branch to the test round branch kit check out test underscore branch I switched to the test branch and we'll be able to list out all of the files that are in the tech in that Branch now and we want to go through and merge the files into one area so let's go ahead and we'll do git merge test underscore branch and it's well we've already updated everything so that's good so otherwise it would tell us what we would be merging and now all the files are merged successfully into the master branch there we go all merged together fantastic and so what we're going to do now is move from Master Branch to test branch so get check out test underscore branch and we can modify the files the test 3 file that we took out and pull that file up and we can now modify and we can then commit that file back in we've actually been able to then commit the file with one changes and and we've seen as the text free change that was made now we can now go through a process of checking the file back in switching back to the master branch and ensuring that everything is in sync correctly we may at one point want to rebase all of the workers kind of a hard thing you want to do but it will allow you to allow for managing for changes in the future so let's switch to it back to our test branch which I think we're actually on we're going to create two more files let's go to our folder here and let's go copy those and that's created we'll rename those tests four and five and so we now have additional files and we're going to add those into our branch that we're working on so we're going to go and let's select get add Dash a and we're going to commit those files get commit Dash a dash m adding two new files and it's added in the two new files so we have all of our files now we can actually list them out and we have all the files that are in the branch and we'll switch them to our Master Branch we want to rebase the master so we do git rebase master and that will then give us the command that everything is now completely up to date and we can go get check out Master to switch to the master account this will allow us to then continue through and rebase the test branch and then list all the files that are all in the same area so let's go get rebase test and just go branch and now we can list and there we have all of our files listed in correctly hello and welcome to git commands my name is chidanand and I am with the simply learn team today I'm going to cover a few essential git commands and walk you through a demonstration for most of it I begin by installing a git bash on the git client and configuring it for its first time usage once I'm done with configuring the git client I'm going to spin up few repositories locally and work on these repositories using git commands once I'm happy with my local changes I'm going to commit few of them and then push these changes to a git server of my choice for today's demonstration I plan to use github.com which is a cloud hosted free service and it provides free account registrations for anyone eventually I will cover few topics regarding git branching and merging because these two in my opinion are the killer features of the git distributed Version Control tool kit is one of the most popular distributed Version Control tools of recent times and like any other distributed Version Control tool it allows us to perform various Version Control tasks without really needing a network connection before I jump into my demonstration let me spend some time explaining to you the git client server working model what does it take for a couple of users to collaborate and start working with Git the bare minimal thing that any user would need in order to start working with Git is something called as git bash or the git trained this comes as an installer for all popular operating systems like Windows all flavors of Windows Linux Mac OS and other operating systems so once you install git bash you get a command line utility using which you can fire up your git commands and ensure that you can bring up repositories you can work with these repositories by adding files to it committing changes to it and all those git commands would work perfectly well so you can bring up a small repository and you can work with your repository using this git bash but what does it take for you to share this repository with another user who has also got get bash installed on his system this is where you would need something called as a git server so for user one or one user to share his repository with another user he would need to collaborate using something called as a git server so in the present Market there are a bunch of git servers which are popularly available some of them are free some of them come with a cost because they are licensed a bunch of these servers that I can think of are GitHub which is one of the most popular kids servers that is around in the market for a long time now it comes with two variants one is the cloud hosted one which is the github.com and then the other one is an Enterprise server which comes as a black box that can be installed into your data centers so typically organizations who don't want their source code to be put up on the cloud but go for this GitHub Enterprise servers wherein they buy this servers and these servers are hosted on their data centers other popular variant of the git server is big bucket this is from the famous atlasian products and it integrates very well with all other atlassian products like jira in recent times there's one other variant of the git server called gitlab which is getting very very popular these days because gitlab not just provides a git server it also provides something called as a runner this is a part of the continuous integration strategy where as soon as you have a source code check-in that is going in you have this Runner that kicks up and builds your particular project so all these are bundled together in a git lab and gitlab also provides you with a Community Edition which is almost free of cost having said that there are a bunch of these tools available for my demonstration what I'm going to do is I've already registered myself on the github.com by providing my username and my password and I'm going to use a free account so whatever reposit is that I'm going to put up on this server is all public by default so I will register Myself by using my username or my email ID and my password so this set of authentication is also referred to as https authentication so in order for me to share my repository with say user 2. the first thing that I got to do is have an access to a git server where both user 1 and user 2 are set up so once I have that I can push my repository onto this server and provide right kind of access so that user 2 or the other developer whoever wishes to collaborate with me on this repository can use his credentials as long as is registered properly on the git server he can use his credentials and pull down this repository and work with this repository so this is the https way of authenticating and sharing repositories another popular way of sharing repositories or working with each other is called as SSH authentication as many of you would know SSH means nothing but creation of a private and a public key a bunch of keys public and private are created on the client machine while I create a keys I will be asked for a passphrase I need to type in a passphrase and remember my passphrase and then I take this public key and kind of post it onto my account within my git server so that whenever I'm going to connect next time using my command prompt or any of the tools it's going to challenge me for my passphrase as long as I remember my passwords correctly I can authenticate myself and get into the server so both https and SSH are popular ways of communicating with the git server having said this let me just get started by installing the git bash on my local system and then fire up some of these git commands and build up some repositories make some changes in my repository commit these changes and later you use my GitHub credential to connect to my GitHub server and push my repositories out there let me now go ahead and find my git bash for my windows so that I can go ahead and install it so I just type in git client for windows open up a browser and type in for this the first one that I find is a GUI tool I don't need this GUI client I need this one so this is the git client or the git bash that I was referring to it's about 40 Megs and let me just download this pause this video for some time so that the download happens and come back to you once I have the download the download seems to be complete when I'm recording this video the version that I have the latest version that is there is 2.19.0 and this is a 64-bit version for Windows so I've got that downloaded let me go ahead and find it and install this EXE all right I go ahead choosing all the default options I want it to be installed in SQL and program files under the git folder everything looks good for me I'm not going to make any changes here get Bash from Windows yes I may want to use git from Windows command prompt that sounds good for me open SSH that's fine next card looks good no I don't want any of these new features so what I'm doing is installing the git bash or the git client so that I can start using my git commands and the motion that I'm using is 2.19.0 which as I record this video this happens to be the latest version all right so looks good I don't want to see the release notes I'll say again launch git Bash so what I get is this window so this is the git command prompt so let me check if everything is looking good it's a Git Version it sounds good it's a git help all right a lot of git help commands everything looks good so I ended up installing the git bash I can go ahead with all other commands now that I have git bash installed on my system let me open up a git bash prompt and start using it but before I do anything I just need to configure It For the First Time stating what would be the username and the email ID with which I want the git bash to be configured with the command to do that is get config hyphen iPhone Global which means that this setting that I'm going to set would be a global setting user dot name and this the name of the user would need to be specified within a double code in my case I am using a simply learn account which is the official account and this is the particular username of this account the other setting is nothing but git config hyphen iPhone Global this is also Global setting and this would be the email ID the email ID need not contain a double quote so in my case this is the email ID that I would be setting up it to work with all right let me check if the configurations were set correctly or not all right if you see this the username and the email ID has been picked up the way I wanted it to be picked up looks good so far all right so what do we do next by default whenever you open up the git prompt you're placed into your home directory let me create a folder here which would be a repository folder from where I'm going to create all my multiple git repositories so I will create a folder called let's say git underscore demo all right there's already a folder called git demo so let me just delete that in the first place and let me create this git demo again all right let me navigate to this particular folder so this is the folder let me open up that folder using a command prompt this is my git demo so this would be a folder so I would use this folder as a base folder for creating all other repositories as a part of this demo so let me create my first repository a repository is nothing but a folder so let me create my first folder called first repo and let me get into this repo all right as of now this is just an empty folder if at all I navigate to this you could see that this folder is empty there's nothing much in this folder at all at this moment okay so let me create a repository out of this how do I create a repository of this is pretty simple I just run a git init command when I say kit in it it means it's going to initialize a repository as of now the folder contained nothing in it was an empty repository so it initially is an empty repository in this particular folder for all I look at my content you would see that there's a hidden folder this would be a hidden folder since I've configured my Explorer to view all hidden folders I see it otherwise you wouldn't see it so a folder by name dot get is created within this first repo and this was created because we ran the command kit init if I get into this folder I see a bunch of directories and other configurations if you see there is something called as a hooks directory a couple of configuration files that are here in this directory all these relate to one or the other type of a client hook that can be enabled and this info this objects this references a bunch of stuff that is there so what is this folder this is a magical git folder and this is created whenever a repository is initialized so by default if at all you get into any git folder or any git repository you would see this dot get folder a word of caution do not get into this folder and try to modify anything because if you fiddle around with this and corrupt this particular folder your whole repository will go for a toss so let me come out of this and let me show you another thing now if you notice here when I was in the folder the repository did not have anything after this now when I say git in it there is something called as a master that shows up so typically what happens is whenever a git repository is created for the first time it creates a default Branch the name of the branch is called master and that is why you would see Master within braces this doesn't mean anything else other than we have a repository out in this particular folder and by default there is a branch called master and we are currently on that Branch if at all I create multiple branches in this folder whenever I navigate to different branches the branch name within this brackets would change all right so far so good so we created a folder we made it a git repository by initializing that with the git in it and then I'm not put anything yet in the repository so let me try to do that let me create a few files within this particular repository so I use a touch command touch is a Linux command it kind of creates this particular folder if not you're not comfortable with the file creation by this way you can always use a file explorer and create some files in this so I create a file called touch master.txt I'll open up in the notepad okay so this is my first file that's what I'm going to write inside this I save this and I come out of this all right now let me do a git status command so what does this tell you there's no commit set you're in the master Branch there is an untracked file and this shows up in red it also gives you all the command that is supposed to be run so if at all you want to add this file you would say git add and the name of the file and then you're going to commit it so what happened is that when the repository was initialized it was an empty repository now that a git notices that there is a new file in this repository called master.txt if at all you want you want us to let git know that you want to track this file we need to add this to the repository so the command to add that is called git and I can specify either the name of the file or I can give a wild character saying git add dot now if I run the git status command again it says there's no commits yet added a file and the file which was earlier in red now shows up in green and it says it's a new file master.txt all right it also gives you something called git remove command if at all you want to unstage or undo your changes so what has happened is the file which was earlier untracked has been added into the git index or it is ready for staging or it's in a staged State now I can go ahead and commit this file if at all that's what I want to do so let me do that I'll do a kit commit minus M and I give a message that I want I would say this is my first get commit all right I run the git status command again and see it says on the branch Master nothing to commit it's all nice and clean so typically what happened was I created a empty directory made it into a git folder by doing a git in it I put a file into it I made some changes to the file I added that file into the repository and I committed to the repository if it were I run the git log command it says this was the commit ID or the commit number or the commit hashtag and this is the author if you notice this is the email ID and the username that we set earlier and this was done on this date and this is the commit message that is there all of it looks so good let me do one more commit into this repository so I will touch Master One Dot text I'll put in another file I'll open up this particular file write something into this all right I save this change let me also open up the older file make some more change into this all right so what I did I modified the older file that existed I also added a new file to my repository now if I run the git status command it says modified this file which is showing in red and it says you have one more file which is an untracked file all right now let me add both these files because these are the changes that I want to incorporate I would say git add dot I run the git status command again and it says now this is a modified file this is a new file looks good to me let me go ahead and Commit This so git commit with the message that says this is my second commit all right if I do git log now it shows both the commits the first commit is the one that that shows up at the bottom and the topmost one is the latest commit that is there so far so good if at all you notice I'm doing all these git commands without really connecting it to any git server I don't need any network connection for doing any of these things I can keep on doing more and more git commands and run git commands the way I want to without really having any connections to the kit server now that I have my changes the way I wanted and I've checked in all the code changes into my local repository I may want to share my repository with somebody else and work in a collaborative way so the way to do that is as I mentioned earlier to host this repository or to push this repository onto a git server I already have a GitHub account wherein I've registered myself using my username and password so I log into my GitHub account using my username and credentials and let me go ahead and create a new Repository my repository is already there on the local system so what I essentially need is just a placeholder for my repository so that I can link them together and push the content from my local repository to the server if you notice the name of my repository is his first repo so let me copy this I would create a folder structure or a repository structure on the server with the same name so that when I pushed my repository from my local box it will go ahead and get created in the server so the repository name should be the same as what I created on my local box description I would say this is my first get Hub Repository on the server this is just an optional description that will show up and if you notice I can't have any private repositories so whatever repositories that I'm going to create here would be a public repository so I choose public and I don't want to initialize this with a readme file or anything of that because I already have a repository with some content with some code checked in all that I need is a placeholder for my repository on the server so with this option with just a name and with an optional description and the repository being public I go ahead and click on the create repository button so if you notice it gives me a bunch of commands that I would need to run in order to create a repository and push it onto the server and also important point it shows me two URLs one is the https URL and the other one is the SSH URL as of now we do not have any SSH keys that is set up so let me use this https URL to connect my local repository with the repository on this server so I've done all these things I already have the repositories set up all that I need is a placeholder for me to connect my local repository with the server URL so this would be the command that I need to copy but essentially links or ads and origin with the URL of the git server I just copy this command and run it here get remote add the name of the remote is called origin and it points to an https URL which is nothing but the placeholder that I've created on the server all right so let me check if it's added correctly okay the URL looks good I'm all set I've got two commits in my repository I'm done with all the changes so let me try to push my repository onto the server the command for doing it for the first time is git push hyphen U this is for linking the Upstream origin master so git push hyphen U Upstream linking origin master link the master branch of my local repository with the master branch on the server repository and where is it going to push it is going to push by default to An Origin and the origin points to this URL all right let me try doing this so it essentially opens up an URL asking me the credentials with which I need to log into my GitHub server so let me just copy and paste the username and the password with which I have registered I would say login okay success so I'm able to create a repository on the server and the content seems to be pushed and it created a new branch called Master it is linked the master Branch from the local repository with the master branch on the server so let me go back to my server and refresh this page and see if all my commits have come in master master one dot text for all I see the commits they seem to be two commits this is my first comment this is my second comment and if at all you notice each of this commit comes with a hash key or a hash ID which contains the details of my commits in this commit the files were added this was a line that was added if you look at the second commit I added these lines I made this you know one line chain that was there I also added this new file which was not existing these are the two commits that existed and if you see the timestamp of the Commit This is the commit which was committed for 15 minutes ago so what does this mean is that the commit is the timestamp when we actually commented the code in the client the pushing is just after we have committed the change we have pushed it so there's no time stamp of the push that happened this completes the first part of the tutorial wherein I create a repository on the client and then I pushed it to the server after creating a repository Skeleton on the server now let me demonstrate to you the SSH configuration that is required for creating a pair of SSH keys and ensuring that I connect to the GitHub server using the SSH mechanism so for doing that I need to First create a pair of private and public key the command to do that is SSH Keygen hyphen T I will use the RSA mechanism for creating my keys Capital C and I need to specify the email ID using which I'm going to connect to all right I need to create the keys for all right so generating a private and public key pair it says let me know what folder should I create the keys in so by default the keys are created in the user home directory under a DOT SSH folder so this is pretty good for me so let me go ahead and say that that's fine enter a passphrase so let me enter some passphrase keep in mind that I would need to remember my passphrase because when I'm going to authenticate myself using SSH mechanism it will challenge me for this particular passphrase I need to enter the right passphrase so that I get authenticated so the key is generated and by default it is created in this particular folder so let me go over to that particular folder and this is the folder where the keys is generated you would see a public key and a private key that is created let me open up the public key with a notepad and then copy the contents of the whole file and now I would need to stick it in on the server so that I can get myself authenticated using SSH keys so I have logged in to GitHub using my credentials and go to my settings and you would see an SSH and gpg keys let me click on that as of now there are no SSH key or gpg keys so I would click on new SSH key and let me paste my public key that I've copied and I would say add SSH key it prompts me for my password just to make sure that I've got the right kind of authentication to be adding a new key so this one looks good that means the keys were added successfully let me check if my SSH keys are working well so I do SSH hyphen capital T I would say get at github.com this is just to double check if my SSH keys are authenticated correctly are you sure you want the connection yes you want to continue yes all right now a check for my passphrase let me enter my passphrase I hope I remember it correctly all right pretty good I've been successfully authenticated so that means now my SSH keys are good and just by using my passphrase I can connect to my GitHub server using SSH all right so for the Second Use case what I would do is create a repository on the server with some content in it and then clone that repository cloning is nothing but making a copy of that repository linking the origin so that I create an exact replica of the repository which is there on the server on my local drive so let me first create a repository on the server I'll create a new Repository let me give a name for my repository and let me kind of initialize that with the readme file I would call this my second repo give some description this is my second repo that is created straight on the git server on the GitHub server to be precise it's a public repository this time I am going to choose this option the reason being that I don't want to create just a skeleton as of now I want to create a repository with some readme file in it a random readme file with no content in it but then I want a repository to be created on the server all right I say create a repository So Random readme file with some content in it gets created and I can see there are two URLs to this repository https and SSH so this time let me copy the SSH URL so that I will use this URL to clone this repository on the server rather from the server to my local box all right so let me navigate to my root folder the folder that I created for making all my repositories git repositories if you see my first deposit is already here so let me clone my repository the second repo that I created on the server the command to do that is git clone the URL of the repository that I want to clone oops looks like I didn't copy the URL correctly let me copy that again copy this URL all right so if you notice there is no https to it I'm just cloning the URL that is SSH URL of the Repository it asks for my passphrase again okay so I just had first repository earlier now I have something called a second repo this is an exact replica of whatever was the repository content that was created on the server so if I get into the repository there's just a simple readme file that is there let me try to add a new file out here I would say Notepad second dot text some text file I'm going to put in here yes definitely you will not find that file there this is my second file that I will push onto the server I'm going to put in some content out there and save this and if I do get status it says this this is a new untracked file you want to add this file onto the repository do you want to track this file yes I definitely want to track this file let me also commit my change to the file with a message this is my first commit for the second repo I'm giving some particular message and this time around since I cloned my repository from a server the origin is already set for me I don't really need to add the origin and all that stuff because what happened in the first case was that I was creating a repository from my local box and then pushing another server which is why I had to set up the Upstream branches and stuff like that in this case I have cloned an existing repository from the server so when I clone I get the replica along with all references to my repository so I don't need to do anything more I'll then push the content I'll just say get push to be in the safer side I would say origin and master ask me for the passphrase okay it posts the content onto the server let me see if the contents have come in here okay wonderful I had only one commit now I see my second comment so what was added as a part of the second comment it says a new file was added all this line that never existed got added this is the second file the content exactly what I pushed onto the server now this completes the tutorial when I create a repository on the server cloned it made some changes to the repository and pushed it to the server using SSH a quick recap of all the git commands that we ran till now get init is used to initialize the repository so if at all you are creating a local repository you can get into any of the folder which is not a git repository and run this command whenever you run this command a magical folder by name dot kits gets created in the folder and from that time onwards git will start tracking any changes that happen to that particular folder it also creates a new repository and a branch by default the branch that is created is called the master Branch git add dot is a wildcard character for adding any number of new files into your git repository if you have a bunch of files one file two file 10 files a folder containing multiple files in it all of them could be added into the repository using this command so once you add any number of files into the repository you can keep on adding more and more files into the repository once you're done with that you want to commit the changes that has happened in the repository you use the git commit command git commit hyphen M with a meaningful message this will commit all the work items that you have done in terms of the files that got changed the more files that got added all this with one particular message get status will give you the status of all the files and folders any file that got added any file that got created any file that was deleted all the status of the files will be obtained using the git status command git log will show you all the commit history with the latest history or the latest commit showing up on top git add remote origin this command is used whenever you want to link any local repository to a server and this is when you want to really push a repository from the local to the server for the first time git push hyphen U origin and master this is the command that you would use whenever you want to push contents from the local repository to the server over git clone and the URL of the server this is the command that you would run whenever you have an existing repository on the server and you want to make a fresh clone or a fresh copy of that particular repository onto your local box more than often many developers or git users find themselves in situations where they would want to go back in history and modify or change some of the files that they recently committed or checked into the repository now from a perspective of get this is a very very simple activity however from a perspective Version Control tool you will have to be really cautious about the chain that you're gonna do in case the changes are pretty local none of the developers or none of the other users who are collaborating with you would be affected by this change you are good to go ahead and make this change however if your repository has been pushed to the server and you're trying to modify something from history this would have a very bad or adverse effect on all the other users who will be using it you need to be really cautious while running these commands in case your repository contents are local you've not pushed your changes to your repository and if at all you made some changes to the repository maybe to fix a defect or to include a new feature and you may few things you can go over and modify this with some of these commands let's say I have this simple repository called undo undo repo and if at all I look at the logs of these I have the commit histories which are like this I have about five commits that is there in my repository and in C1 I've added a file in C2 I added the file in C3 I've also added some file in C4 I made some changes to M1 and M3 and in C5 I have added another file called M4 now this is my comment history now let's say I made these two commits but possibly I want to Club them together along with some other changes because whatever changes I did did not really fix things the way I expected you to fix so there is a git command called git reset this would allow me to go back in history to go back to any of the snapshot get rid of all this commit history on the commit messages but retain the changes that were made as a part of these comments so if at all I need to go back in history and go back to this particular comment and however I just want to get rid of these commits that are there in terms of the commit messages but I want the changes that were there as a part of these commits to still exist I would do command called get reset hyphen iPhone soft now this would go back or unwind my changes back to this particular snapshot getting rid of all the commit messages that were there as a part of these two comments but a soft would ensure that the file changes that were done as a part of these two commits would still exist a full remain so the command for that is git reset hyphen iPhone soft which is the snapshot that I want to get back to if this is the comment that I want to get back to I just copy this and paste this all right it doesn't give me any message but if at all I do a git log pretty one line what it says is 7 C2 now the head which is nothing but the current pointer which is earlier pointing to C5 it's gone back to C3 now what happened to the changes that were there as a part of C4 and C5 if at all you do a git status if you see the changes that were part of C4 was some changes to M1 and M2 and C5 a new file got M4 got added if I don't get status you'll see those changes still exist however from the git history or the git commit history C3 is the topmost comment that exists as of now c4 and C5 have been currently read off but the changes that were there still exist all right now I can make some more changes that I want if as a part of my commit and possibly go ahead and commit this whole thing as a new commit altogether maybe I'll open up a notepad and possibly create a new file called m6.text and say one new file that was missed out I make this change I go to a get status all these files are still there I would send it add dot I'll say get commit hyphen m C4 rewriting history and I do a git blog pretty there you go so C1 C2 C3 still existed and whatever was a part of C4 and C5 the older ones I have gotten rid of this along with those old changes I also added a new file I made some changes and I was able to comment it now if you look at my git history it's all nice and clean however do this activity only when you don't push your git repository to the server one other most powerful and useful command while undoing or resetting the history is something called as revert this is a safe way of undoing some things remember what I talked about when exactly you can do a reset it's only when your changes are local and is not being pushed to the repository take this scenario wherein let's say I have a git server and I have the changes which are C1 C2 C3 and possibly C4 all right so this is a bunch of comments that has happened this was the oldest comment and this is the newest comment and if at all developer 1 is the one who's responsible for pushing these contents in the first place so he has got C1 to C4 on his history I'm going to jump all these C2 and C3 and get into C4 and he's pushed all these contents out here and there are a bunch of people from the server who has pulled this repository and all of them are the same level now imagine a use case where developer 1 then figures out that by mistake he put in the C3 which is a wrong thing to do and possibly you know he could have done a better job of fixing this in a different way so he wants to get rid of the C3 or C4 or any one of these comments if he's going to do that and push it to the server first of all the git server would not allow him to go back in history but assuming that he forcibly pushed something what would happen to all these people who are looking to work ahead from C4 all of them would be affected in an adverse way so in short never go back in history if at all you need to undo something go ahead and put in a new comment to undo something let's take this example maybe if at all a developer one wants to get rid of C3 better way to do that is by adding a new commit which is ahead say let's say C5 and what does this commit to this commit possibly gets rid of C3 instead of going back in history it actually is taking your git history ahead but as a part of this commit you are undoing some part of the work that was done as a part of C3 in short you're removing a commit but to remove a commit you're adding a new commit and this is why you're doing it because there are a bunch of other people who are you're using this repository collaboratively and what plan to get ahead in terms of History they cannot afford to go back in history and modify something in history all right the command to do this is called the get revert command so let me take up this repository which has got a bunch of commits in it and if at all I do a git log pretty one nine I have these bunch of comments and C1 I've added a file in C2 our other file in C3 I've added one more file and in C4 I made some changes and this is the latest commit and this is the oldest commit so if at all I see the contents of the git folder it contains M1 m2m3 and M4 and let's say looking at the previous use case that I mentioned I may want to get rid of this particular comment I want to get rid of the C3 which is nothing but I've added a file called m3.text but I added it by mistake I need a better way of fixing it possibly I look at it later on but for now I don't want this particular commit and the changes that were made as a part of that commit to exist so the better way to do that is using the git revert command so git revert and the commit ID whatever I want to revert so if I want to revert this I copy this particular commit ID and bear in mind that considering a reset revert works on one commit at a time so I can do one commit revert at a time I can Club a bunch of them together and do one commit but otherwise the revert always works on one commit at a time so get revert face this essentially what I'm trying to do I'm trying to undo this particular commit while I'm undoing this commit what does git do it safely adds a new commit and a part of this new commit It undoes Whatever changes that were done as a part of this command so if I say git revert this all right I forgot to mention a message which is why this window is showing up but that message is good for me I say write and quit all right and if I do a git log all right if you see this C1 still is there in history and our history is going ahead from C5 I see another commit but what was there as a part of this comment actually is getting rid of this commit C3 so if at all I see my files here there's only M1 M2 and M4 M3 was added as a part of C3 now I've gotten rid of M3 by getting rid of the revert command for C3 so this is a better way of undoing things now that we have acquainted ourselves with few of those basic git commands let me get into a very very interesting topic regarding git which is about branching the topic is interesting because git deals with branches in a completely different way compared to any other version control tool in most of the Version Control tools whenever you create a new Branch a complete copy or a subset of the whole repository is created which in my opinion is very very cost inefficient because it's a very very expensive operation both in times of time and the disk space however in get you just create a very very lightweight movable pointer whenever you create a branch there is no folder or no copy of the repository that is created in short the branches are very very inexpensive and great so go ahead and create any number of branches that you need however once you create your branches do all the work relating to that branch in that specific branch and once you've completed your task go ahead and merge your branches back into your base branch and then delete the branch let me go ahead and create a repository with some branches in this and give you a demonstration of that so let me go back to my base directory which is git demo and then I'm going to create a new folder here I'll create a folder called Arc details all right this is my folder here all right by navigating to this folder and let me create a repository here I initial as a repository and Dot git folder is created without any files in it so let me create a few files in it say notepad name dot text organization name I create a name.txt within this maybe I'll create one more file in this Notepad employees dot text foreign I just put in some content in this save this content all right so if I try to get status I just see two files in it so let me go ahead and add these two files and let me commit my changes it's okay commit hyphen am or iPhone m First Command so if I do a git log I see only one commit and there are two files and you can see those two files that is here all right so at this stage let me go ahead and create a new branch and the command for creating new branch is get Branch name of the branch so let me name my branches Bangalore Branch blr branch and this created a branch but I don't see any changes in the folder structure or in the file that exists here I've just created this Branch so if it all I do a git branch I see these two branches and whatever star means I am currently in this master Branch so let me go ahead and get into the blr branch at this moment the number of files that exist in this repository is same there's absolutely no change in any of the branches both master and Baylor Branch point to the same snapshot so let me get into the other branch and the command to do that is git checkout plr Branch I've actually switched into Bangalore Branch if you see the content of the files this exactly two files the way it was earlier because I've created a branch from my base branch which is the master branch and when I created a branch there were two files existing here if you see the content of these files they would all be the same all right so let me create a file that is specific to panel branch so I will just say Notepad maybe I want to put down the address of this organization specific to Bangalore in this address dot text so I see notepad address dot text I'll say simply learn Bangor office say located in some arteries I would say Bangalore foreign of this file and I'm creating all this new file in this blr Branch so I would sync it add Dot I would say kit commit hyphen m angalore branch comment all right if you see this now I have an address.txt which is there only in the Bangalore branch that I am presently on now let me switch back to my master Branch the command to do that is git checkout Master Branch if you remember when I forged out or when I created a new Branch from the master Branch I didn't have this address dot text in it so let me go back to git git checkout Master branch and keep noticing this particular what happened to the address dot text it kind of disappeared how did that happen it's all because of this magical git folder so what happened is now git knows that when you are in the master Branch the file that was there in the Bangalore branch which was called the address dot text didn't exist so git is playing around with the file system snapshot so if I come back let's say I come back to git checkout Bangalore branch if you see this the address dot text file appears so this particular file is only present in the branch and git is playing around with the file system snapshot by moving the pointers across and ensuring that whenever I am switching to Bangalore Branch I see this address dot text when I go back to my master Branch I don't say this Bangalore branch contents which is the address dot text all right now that I created this Branch maybe I may need to push this onto the server so let me go ahead and create a particular repository structure on the server similar to whatever we did in our previous exercise so the name of the folder needs to be exactly this one so let me copy this and let me create a repository very quickly go here and say new Repository all right this is the name of the Repository get Branch demo this is what I'm going to put in as a description I would say public I don't want any readme files in it because I already have a repository I will just say create a repository all right let me use the SSH URL yeah this is the SSH URL this is good for me so let me copy this URL let me add it here I'm linking my local repository with the server and I would let me come back to my master branch let me first push my master Branch so I would say git push hyphen U origin master it asks for my passphrase all right push my master Branch it looks good it says New Branch Master is created so if I go back to my server and it will refresh if you see it says only one branch and it says what is the name of the branch master if you see the contents that is employees.txt and name dot text now let's try to push the other branch that is there the command to do that is a little different compared to the way we created it on the track because on the client we created the branch with a git Branch command now this Branch doesn't exist on the server and I need to push this new Branch onto the server so the command would be get push hyphen new origin and the name of the brand that I want to specifically push all right now it created a branch out here let me do a refresh of this all right it says two branches if I get into this Bangalore Branch I will see the address dot text if at all I go back into Master Branch I would not see this under stock text wonderful now let me do one thing let me try to get the changes from the brands that I created back into the master branch that is what I would mean by merging I would want the changes that is there as a part of the Bangalore Branch to come back into my master Branch so if I go to click checkout Bangalore Branch I see this address DOT type this is there only as a part of Bangalore Branch now assuming that I am done with the changes from the Bangalore Branch I want to get those changes back into my master so let me come back to my master branch and the way to merging the changes is get merge so what is that you want to merge I want to merge the Bangalore Branch changes and where will it merge it will merge it into the current brands that I'm on presently I'm in the master Branch so this command would merge the changes from the Bangalore Branch into your master Branch so with this if we'll keep looking here the content that was there that was added as a part of Bangalore branch which is nothing but the address dot text this file should appear as soon as we do a merge all right so it happened quickly and you would see this address dot text that is here so if you see let's check the contents in terms of the git log all right so there's only one branch that we had earlier which had only one commit right now we are at this Bangalore Branch commit which was there as a part of the Bangalore Branch commit so with this change let me push this change onto the server as well git push origin master okay those changes were pushed to the server and now if at all I do a refresh out here okay I mean two branches the princess's Master even if I'm in master plans I should see this address dot text that is there even if I'm in the Bangalore Branch the content is all same because we immerse these two branches now that I've merged this Branch I don't need this Branch anymore so I would go ahead and delete this locally as well as push it under the server so that it gets deleted from the server as well so delete Branch locally the command is get Branch hyphen D and the name of the branch only thing you need to ensure is that you should not be currently on that branch which you're trying to delete so I would say get brands and I'm presently in the master Branch so I can go ahead and delete the other Branch so get Branch hyphen D Bangalore branch all right this got rid of our Pandora Branch locally if at all I don't get Branch iPhone V it shows there's only one branch as of now however this Branch would still exist on the server because Server doesn't know anything about the changes that we have done we need to specifically push our changes onto the server okay there are two branches still here so let me go ahead and push these changes so that the branch gets deleted from the server as well the command to do that is get push origin hyphen hyphen delete and the name of the branch all right so this went ahead and deleted the branch from the server let's check that okay if you see now there's only one branch these are the basic steps that you would need to know before you work with any of the branches within git and as I said before the branches are very very inexpensive in git go ahead and create any number of branches that you want but just ensure that once you create your branch finish off your task and once you finish off your task go ahead and delete the branch and before you delete the brands please ensure that you merge in the branches so that you get in or take in the work that was done as a part of that particular Branch to summarize the git branching commands are as follows to create a new Branch you issue a command git Branch the name of the branch and this would automatically spin off or create a new Branch from the existing Branch from where you issue this command if you are on the master branch and if you execute this command git Branch new underscore Branch it creates a new underscore Branch from the master Branch once you create a new branch in order to get into this Branch you would type in git checkout the name of the brand inch this would automatically switch the file system snapshot between different branches now the branches that are created are by default created on the local repository in case you want to push any specific Branch onto the server you would have to issue a command get push hyphen U origin and the name of the branch so this will ensure that the contents of the new Branch are pushed onto the server at any time if you want to list out all existing branches on your local repositories the command would be git Branch hyphen AV assuming that you did all your work in a new branch and you're happy with the changes you would want to bring back those changes into the master Branch the command to merge the contents of any new Branch into a master Branch would be git merge and the name of the branch this would automatically merge the branch contents that you specified into the branch on which you currently reside if at all you're on the master branch and if you issue a command git merge new underscore Branch this would merge the contents of the new Branch onto the master Branch so once you merge the contents of any branch you would possibly want to go ahead and delete this Branch so the command to delete those branch locally is git Branch hyphen capital D and the name of the branch this again would delete the branch locally in order to make this change on the server which is in order to remove this Branch from the server the command to do that is git push origin hyphen iPhone delete and the name of the branch this would also delete the brands that you created on the server hello this is Matthew from Simply learn and today we're going to Deep dive into what Jenkins is and how it improves your devops continuous integration environments so this is what we're going to be covering in this session we're going to cover what life is like before using Jenkins and the issues that Jenkins specifically addresses then we'll get into what Jenkins is about and how it applies to continuous integration and the other continuous integration tools that you need in your devops team then specifically we'll Deep dive into features of Jenkins and the Jenkins architecture and we'll give you a case study of a company that's using Jenkins today to actually transform how their it organization is operating so let's talk a little bit about life before Jenkins let's see this scenario I think it's something that maybe all of you can relate to as developers we all write code and we all submit that code into a code repository and we all keep working away writing our unit tests and hopefully we're running our unit tests but the problem is that the actual commits are actually gets sent to the code repository aren't consistent you as a developer may be based in India you may we have another developer that's based in the Philippines and you may have another team lead that's based in the UK and another development team that's based in North America so you're all working at different times and you have different amounts of code going into the code repository There's issues with the integration and you're kind of running into a situation that we like to call development how where things just aren't working out and there's just lots of delays being added into the project and the bugs just keep mounting up the bottom line is the project is delayed and in the past what we would have to do is we'd have to wait until the entire software code was built and tested and before we could even begin checking for errors and this just really kind of increased the amount of problems that you'd have in your project the actual process of delivering software was slow there was no way that you could actually iterate on your software and you just ended up with just a big headache with teams pointing fingers at each other and blaming each other so let's jump into Jenkins and see what Jenkins is and how it can address these problems so Jenkins is a product that comes out of the concept of continuous integration that you may have heard of as power developers where you'd have two developers sitting next to each other coding against the same piece of information what they were able to do is to continuously develop their code and test their code and move on to new sections of code Jenkins is a product that allows you to expand on that capacity to your entire team so you're able to submit your codes consistently into a source code environment so there are two ways in which you can do continuous delivery one is through 90 builds and one is through continuous so the approach that you can look at continuous delivery is modifying the Legacy approach to building out Solutions so what we used to do is we would wait for nightly bill and the way that our 90 builds would work and operate is that as code developers we would all run and have a cut-off time at the end of the day and that was consistent around the world that we would put our codes into a single repository and at night all of that code would be run and operated and tested to see if there were any changes and a new build would be created and that would be referred to as the nightly build with continuous integration we're able to go one step further we're able to not only commit our changes into our source code but we can actually do this continuously there's no need to race and have a team get all of their code in at arbitrary time you can actually do a continuous release because what you're doing is you're putting your tests and your verification Services into the build environment so you're always running Cycles to test against your code this is the power that Jenkins provides in continuous integration so let's dig deeper into continuous integration so the content except of continuous integration is that as a developer you're able to pull from a repository the code that you're working on and then you'll be able to then at any time submit the code that you're working on into a continuous integration server and the goal of that continuous integration server is that it actually goes ahead and validates and passes any tests that a tester may have created now if on the continuous integration server a test is a pass then that code gets sent back to the developer and the developer can then make their changes it allows the developer to actually do a couple of things it allows the developer not to break the build and we all don't want to break the builds that are being created but it also allows the developer not to actually have to run all the tests locally on their computer write tests particularly if you have a large number of tests can take up a lot of time so if you can push that service up to another environment like a continuous integration server it really improves the productivity of of your developer what's also good is that if there are any code errors that have come up that may be Beyond just the standard CI test so maybe there's a Code the way that you write your code isn't consistent those errors can then be passed on easily from the tester back to the developer too the goal from doing all this testing is that you're able to release and deploy and your customer is able to get new code faster and when they get that code it's simply just works so let's talk a little bit about some of the tools that you may have in your continuous integration environment so the cool thing with working with continuous integration tools is that they are all open source at least the ones that we have listed here are open source there are some that are private but typically you'll get started with open source tools and it gives you the opportunity to understand how you can accelerate your environment quickly so bamboo is a continuous integration tool that specifically runs multiple builds in parallel for faster compilation so if you have multiple versions of your software that runs on multiple platforms this is a tool that really allows you to get that up and running super fast so that your teams can actually test how those different builds would work for different environments and this has integration with and Maven and other similar tools so one of the tools you're going to need is a tool that allows you to automate the software build test and release process and buildbot is that open source product for you again it's an open source tool so there's no license associated with this so you can actually go in and you can actually get the environment up and running and you can then test for and build your environment and create your releases very quickly so buildbot's also written in Python and it does support parallel execution jobs across multiple platforms if you're working specifically on Java projects that need to be built and test then Apache Gump is the tool for you it makes all of those projects really easy it makes all the Java projects easier for you to be able to test with API level and functionality level testing so one of the popular places to actually store code and create a versioning of your code is GitHub and it's a service that's available on the web just recently acquired by Microsoft if you are storing your projects in GitHub then you'll be able to use Travis continuous integration or Travis CI and it's a tool design specifically for hosted GitHub projects and so finally we're covering Jenkins and Jenkins is a central tool for automation for all of your projects now when you're working with Jenkins sometimes you'll find there's documentation that refers to a product called Hudson Hudson is actually the original version of the product that finally became Jenkins and it was acquired by Oracle when that acquisition happened the team behind um Hudson was a little concerned about the direction that Oracle May potentially go with Hudson and so they created a hard Fork of Hudson that they renamed Jenkins and Jenkins has now become that open source project it is one of the most popular and continuously contributed projects that's available as open source so you're always getting new features being added to it it's a tool that really becomes the center for your CI environment so let's jump into some of those really great features that are available available in Jenkins so Jenkins itself is really comprised of five key areas around easy installation easy configuration plugins extensibility and distribution so as I mentioned for the easy installation Jenkins is a self-contained Java program and that allows it to run on most popular operating systems including Windows Mac OS and Unix you can even run it on Linux it really isn't too bad to set up it used to be much harder than it is today the setup process has really improved the web interface makes it really easy for you to check for any errors in addition you have great built-in help one of the things that makes tools like Jenkins really powerful for developers and consumers integration teams and your devops teams as a whole when you have plugins that you can then add in to extend the base functionality of the product Jenkins has hundreds of plugins and you can go and visit the update Center and see which other plugins that would be good for your devops environment certainly checking out there's just lots of stuff out there in addition to the plugin architecture Jenkins is also extremely extensible the opportunity for you to be able to configure Jenkins to fit in your environment it's almost endless now it's really important to remember that you are extending Jenkins not creating a custom version of Jenkins and that's a great differentiation because the core Foundation remains as the court Jenkins product the extensibility can then be continued with newer releases of Jenkins so you're always having the latest version of Jenkins and your extensions mature with those core Foundation the distribution and the nature of Jenkins makes it really easy for you to be able to have it available across your entire network it really will become the center of your CI environment and it's certainly one of the easier tools and more effective tools for devops so let's jump into the standard Jenkins pipeline so when you're doing development you start off and you're coding away on your computer the first thing you have to do when you're working in the Jenkins pipeline is to actually commit your code now as a developer this is something that you're already doing or at least you should be doing you're committing your code to a git server or to an SVN server or a similar type of service so in this instance you'll be using Jenkins as the place for you to commit your code Jenkins will then create a build of your code and part of that build process is actually going through and running through tests and again as a developer you're already comfortable with running unit tests and writing those tests to validate your code but there may be additional tests that Jenkins is running so for instance as a team you may have a standard set of tests for how you actually write out your code so that each team member can understand the code that's been written and those tests can also be included in the testing process within the Jenkins environment assuming everything past the the test you can then get everything placed in a stage and release ready environment within Jenkins and and finally you're getting ready to deploy or deliver your code to a production environment Jenkins is going to be the tool that helps you with your server environment to be able to deploy your code to the production environment and the result is that you're able to move from a developer to production code really quickly this whole process can be automated rather than having to wait for people to actually test your codes or going through a nightly build you're looking at being able to commit your code and go through this testing process and release process continuously as an example companies Etsy will release up to 50 different versions of their website every single day so let's talk about the architecture within Jenkins it allows you to be so effective at applying a continuous delivery devops environment so the server architecture really is broken up into two sections so on the left hand side of section you have the code the developers are doing and submitting that code to a source code repository and then from then Jenkins is your continuous integration server and it will then pull any code that's been sent to the source code repository and we'll run tests against it it'll use a build service such as MAV into actually then build the code and then every single stage that we have that Jenkins manages there are constant tests so for instance if a build fails that it feedback is sent right back to the developers so that they can then change their code so that the build environment can run effectively the final stage is to actually execute specific test scripts and these test scripts can be written in selenium amp so it's probably good to mention here that both mavin and selenium are plugins that run in the Jenkins environment so before we were talking about how Jenkins can be extended with plugins mavin and selenium are just two very popular examples of how you can extend the Jenkins environment the goal to go through this whole process again it's an automated process is to get your code from the developer to the production server as quickly as possible have it fully tested and have no errors so it's probably important at this point to mention uh one piece of information around the Jenkins environment that if you have different code builds that need to be managed and distributed this will require that you need to have multiple builds being managed Jenkins itself doesn't allow for multiple files and builds to be executed on a single server you need to have a multiple server environment with running different versions of Jenkins for that to be able to happen so let's talk a little bit about the Master Slave architecture within Jenkins so what we have here is an overview of the Master Slave architecture within Jenkins on the left hand side is the remote source code repository and that remote source code repository could be GitHub or it could be a team Foundation services or the new Azure devops code repository or it could be your own get repository the Jenkins server acts as the master environment on the left hand side and that Master environment can then push out to multiple other Jenkins slave environments to distribute the workload so it allows you to run multiple builds and tests and production environments simultaneously across your entire architecture so Jenkins slaves can be running the different build versions of the code for different operating systems and the server Master is controlling how each of those builds operate so let's step into a quick story of a company that has used Jenkins very successfully so here's a use case scenario and over the last 10 or 15 years there has been a significant shift within the automotive industry where manufacturers have shifted from creating complex Hardware to actually creating software we've seen that with companies such as Tesla where they are creating a software to manage their cars we see the same thing with companies such as General Motors with their OnStar program and Ford just recently have rebranded themselves as a technology company rather than just a automotive company what this means though is that the software within these cars is becoming more complex and requires more testing to allow more capabilities and enhancements to be added to the core software so Bosch is a company that specifically ran into this problem and their challenge was that they wanted to be able to streamline the increase increasingly complex Automotive software by adopting continuous integration and continuous delivery best practices with the goal of being able to delight and exceed the customer expectations of the end user so Bosch has actually used Cloud bees which is the Enterprise Jenkins environment so to be able to reduce the number of manual steps such as building deploying and testing Bosch has introduced the use of cloud bees from Jenkins and this is part of the Enterprise Jenkins platform it has significantly helped improve the efficiencies throughout the whole software developments cycle from automation stability and transparency because Jenkins becomes a self-auditing environment now the results have been tangible previously it took three days before a build process could be done and now it's taken that same three-day process and reduced it to less than three hours that is significant large-scale deployments are now kept on track and have expert support and there is clear visibility and transparency across the whole operations through using the Jenkins tools hello and welcome to continuous integration tutorial using Jenkins my name is chidanand and I am with the simply learn team today let's get started with Jenkins Jenkins in my opinion is one of the most popular continuous integration servers of recent times what began as a hobby project by a developer working for Sun Microsystems we're back in early or mid-2000s has gradually and eventually evolved into a very very powerful and robust automation servers it has a wider option since it is released under MIT license and is almost free to use Jenkins has a vast developer community that supports it by writing all kinds of plugins plugins is the heart and soul of Jenkins because using plugins one can connect Jenkins to anything and everything under the sun with that introduction let's get into what all will be covered as a part of this tutorial I will get into some of the prerequisites required for installing Jenkins post which I will go ahead and install Jenkins on a Windows box there are few first time configuration that needs to be done and I will be covering those as well so once I have Jenkins installed and configured properly I will get into the user administrative part I create few users and I will use some plugins for setting up various kinds of access permissions for these users I will also put in some freestyle jobs freestyle job is nothing but a very very simple job and I will also show you the powerfulness of Jenkins by scheduling this particular job to run based upon time schedule I will also connect Jenkins with GitHub GitHub is our source code where source code repository where I've got some repositories put up there so using Jenkins I will connect to GitHub pull up a repository is existing on GitHub onto the Jenkins box and run few commands to build this particular repository that is pulled from GitHub sending out emails is a very very important configurations of Jenkins or any other continuous integration server for that matter Whenever there is any notifications that has to be sent out as a part of either a build going bad or build being good or building propagate to some environment and all these scenarios you would need the continuous integration servers to be sending out notifications so I will get into a little bit details of how to configure Jenkins for sending out emails I will also get into a scenario where I would have a web application a maven-based Java web application which will be pulled from a GitHub repository and I will deploy it onto a tomcat server the Tomcat server will be locally running on my system eventually I will get into one other very very important topic which is the Master Slave configuration it's a very very important and pretty interesting topic where distributed builds is achieved using a Master Slave configuration so I will bring up a sleeve I will connect the sleeve with the master and I'll also put in a job and kind of delegate that particular job to the slave configuration finally I will let you know how to use some plugins to backup your Jenkins so Jenkins has got a lot of useful information set up on it in terms of the build environments in terms of workspace all this can be very very easily backed up using a plugin so this is what I'm going to be covering as a part of this tutorial Jenkins is a web application that is written in Java and there are various ways in which you can use and install Jenkins I have listed popular three mechanisms in which Jenkins is usually installed on any system the topmost one is as a Windows or a Linux Based Services so if at all you have Windows like the way I have and I'm going to use this mechanism for this demo so I would download a MSI installer that is specific to Jenkins and install this service so whenever I installed as a service it goes ahead and nicely installs all that is required for my Jenkins and I have a service that can be started or stopped based upon my need any value of Linux as well one other way of running Jenkins is downloading this generic War file and as long as you have jdk installed you can launch this war file by the command opening up a command prompt or shell prompt differently on Linux box specifying Java hyphen jar and the name of this war file it typically brings up your web application and you know you can continue with your installation the only thing being if at all you want to stop using Jenkins you just go ahead and close this prompt you either do a Ctrl C and then bring down this prompt and the Jenkins server would be down other older versions of Jenkins were run popularly using this way in which you already have a Java based web server running up and running so you kind of drop in this war file into the root folder or the httpd root folder of your web server so Jenkins would explode and kind of bring up your application all user credentials or user Administration is all taken care of by the Apache or the Tomcat server or the web server on which Jenkins is running this was a very older way of running but still some people use it because if they don't want to maintain two servers if they already have a Java web server which it's being nicely maintained and backed up Jenkins can run attached to it all right so either ways it doesn't matter how we're going to bring up your Jenkins instance the way we're going to operate Jenkins is all going to be very very same or similar one with the subtle changes in terms of user Administration if at all you're launching it through any other web server which will take care of the user Administration otherwise all the commands are all the configuration are the way in which I'm going to run this demo it is going to be same across any of these installations all right so the prerequisites for running Jenkins as I mentioned earlier Jenkins is nothing but a simple web application that is run in Java so all that it needs is Java preferably jdk 1.7 or 1.8 2GB Ram is the recommended RAM for running Jenkins and also like any other open source toolsets when you install jdk ensure that you set in the environment variable Java home to point to the right directory this is something very specific to jdk but for any other open source tools that you of install there is always a preferred environment variable that you got to set in which is specific to that particular tool that you're going to use this is a generic thing that is there for you know for any other open source projects because the way open source projects discover themselves is using this environment variables so as a general practice or a good practice always set these environment variables accordingly so I already have jdk 1.8 installed on my system but in case you do not what I would recommend is just navigate on your browser to the Oracle home page and just type in or search for install jdk1.8 and navigate to The Oracle home page you will have to accept the license agreement and there are a bunch of installers that is okay that you can pick up based upon the operating system on which you're running so I have this windows 64 installer that is already installed and running on my system so I'm not going into the details of downloading this or installing it let me show you once I install this what I've done with regard to my path so if you get into environment variables all right so I've just set in a Java home variable if you see this C colon program files Java jdk 1.8 this is where my my Java is located C program files C program files Java okay so this is the home directory of my jdk so that is what I've been I've set it up here in my environment variable so if you see here this is my Java home all right one other thing to do is ensure that in case you want to run Java or Java C from a command prompt and show that you also add that path into this path variable so if you see this somewhere I will see yes there you go see colon program files Java jdk 1.8 pin so with these two I'll ensure that my Java installation is nice and you know good enough so to check that to double check that or to verify that let me just open up a simple command prompt and if I type in Java hyphen version all right and Java C yphen version so the compiler is on the path Java is on the path and if at all I do this even the environment variable specific to my Java is installed correctly so I am good to go ahead with my Jenkins installation now that I have my prerequisites all set for installing Jenkins let me just go ahead and download Jenkins so let me open up a browser and say download Jenkins all right LTS is nothing but the long term support these are all stable versions weeklies I would not recommend that you try these unless until you have a real need for that long term support is good enough and as I mentioned there are so many flavors of Jenkins that is available for download you also have a Docker container wherein you know you can launch Jenkins as a container but I'll not get into details of that in this tutorial all right so what I want is yes this is the war file with a generic War file that I was talking to you earlier and this is the windows MSI installer so go ahead and download this MSI installer I already have that downloaded so let me just open that up all right so this is my downloaded Jenkins instance or other installer this is a pretty maybe a few months old but this is good enough for me before you start Jenkins installation just be aware of one fact that there is a variable called Jenkins home this is where Jenkins would store all this configuration data jobs project workspace and all that specific to Jenkins so by default if at all you don't set this to any particular directory if at all you install an MSI installer all your installation gets into SQL and program files 86 and Jenkins folder if at all you run a war file depending upon the user ID with which you're running your War file the Jenkins folder there's a DOT Jenkins folder that gets created inside the user home directory so in case you have any need wherein you want to backup your Jenkins or you want Jenkins installation so get into some specific directories go ahead and set this Jenkins home variable accordingly before even begin your installation for now I don't need to do any of this thing so I've already downloaded the installer let me just go ahead with the default installation all right so this is my Jenkins MSI installer I would just I don't want to make any changes in into the Jenkins configuration C colon program files is good for me yeah this is where all my destination folder and all the configuration specific to it goes I'm happy with this I don't want to change this I would just say go ahead and click installation okay so what typically happens once the Jenkins installation gets through is it'll start installing itself and there are some small checks that needs to be done so and by default Jenkins Launches on the port 8080 so let me just open up localhost 8080 there's a small checking that will be done as a part of the installation process wherein I need to type in the hash key all right so there's a very very simple hash key that gets stored out here so I'll have to just copy this path if at all you're running as a war file you would see that in your logs all right so this is a simple hash key that gets created every time when you do a Jenkins installation so as a part of the installation it just asks you to do this so if that is not great it'll crib about it but this looks good so it's going ahead all right one important part during the installation so you would need to install some recommended plugins what happens is the plugins are all related to each other so it's like the typical RPM kind of a problem where you try to install some plugin and it's got a dependency which is not installed and you get into all those issues in order to get rid of that what Jenkins recommends there's a bunch of plugins that is already recommended so just go ahead and blindly click that install recommended plugin so if you see there is a whole lot of plugins which are bare essential plugin that is required for Jenkins in order to run properly so Jenkins as a part of the installation would get all these plugins and then install it for you this is a good combination to kind of begin with and mind you at this moment Jenkins needs lots of bandwidth in terms of network so in case your your network is not so good few of these plugins would kind of fail and these plugins are all you know on available on openly or or mirrored sites and sometimes some of them may be down so do not worry in case some of these plugins kind of fail to install you'll get an option to kind of retry installing them but just ensure that you know at least most or 1995 of all these plugins are installed without any problems let me pause the video here for a minute and then get back once all these plugins are installed my plugin installation is all good there was no failures in any of my plugins so after that I get to create this first admin user again this is one important point that you got to remember Keen given any username and password but ensure that you kind of remember that because it's very hard to get back your username and password in case you forget it alright so I'm going to create a very very simple username and password something that I can remember I will that's my name and an email ID is kind of optional but it doesn't allow me to go ahead in case I don't so I just given an admin and I got a password I've got I remember my password this is my full name all right I say seven finish all right that kind of completed my Jenkins installation it was not that tough was it now that I have my Jenkins installed correctly let me quickly walk you through some bare minimal configurations that is required these are kind of a first time configuration that is required so and also let me warn you the UI is little hard for many people to wrap their head around it specifically the windows guys but if at all you're a Java guy you know how painful it is to write UI in Java you will kind of appreciate you know all the effort that has gone into the UI bottom line UI is little hard to you know wrap your head around it but once you start using it possibly you'll start liking it all right so let me get into something called as manage Jenkins this can be viewed like a main menu for all Jenkins configuration so I'll get into some of those important ones something called as configure system configure system this is where you kind of put in the configuration for your complete Jenkins instance few things to kind of look out for this is a home directory this is a Java home where all the configurations all the workspace anything and everything regarding Jenkins is stored out here system message you want to put in some message on the system you just type in whatever you want and it's possibly show up somewhere up here on the menu number of executors very very important configuration this just lets Jenkins know at any point in time how many jobs or how many threads can be run you can you can kind of visualize it like a thread that can be run on this particular instance as a thumb rule if at all you're on a single core system number of executors two should be good enough in case at any point in time if there are multiple jobs that kind of get triggered the same time in case the number of executors are less compared to the number of jobs that have that woken up no need to panic because they will all get queued up and eventually Jenkins will get to running those jobs just bear in mind that whenever a new job kind of you know gets triggered the CPU usage and the memory usage in terms of the disk right is very high on the Jenkins instance so that's something that you got to kind of keep in mind all right but number of executors two for my system is kind of good label for my Jenkins I don't want any of these things usage how do you want to use your Jenkins this is good for me because I only have a primary uh server that is running so I want to use this node as much as possible quite better each of these options I've got some bare minimal help kind of a thing that is that is out there by clicking on these question marks you will get to know as to what are these particular configurations all right so this all look good what I want to show you here is there's something regarding the docker timestamps git plugin SVN email notifications I don't want that what I want the yes I want this SMTP server configuration remember I mentioned earlier that I would Bond Jenkins to be sending out some emails and what I've done here is I've just configured the SMTP details of my personal email ID in case you are in a in an organization you would have some sort of an email IDs that is set up for a Jenkins server so you can specify the SMTP server details of your company so that you know you can authorize Jenkins to kind of send out emails but in case you want to try it out like me I have configured my personal email ID which is on my Gmail for sending out notifications so the SMTP server would be smtp.gmail.com I'm using the SMTP authentication I have provided my email ID and my password I'm using SMTP Port which is 465 and I'm you know reply to address is the same as mine I can just send out an email and see if at all this configuration works again Gmail would not allow you to allow anybody to send out notifications on your behalf so you'll have to lower the security level of your Gmail ID so that you can allow a programmatically somebody to send out email notifications on your behalf so I've done already that I'm just trying to see if I can send a test in email with the configuration that I've set in yes all right so the email configuration looks good so this is how you configure your you know your Gmail account in case you want to do that if not put in your organization SMTP server details which are with a valid username and password and should all be set all right so no other configurations that I'm going to change here all of these look good all right so I come back to manage inkins okay one other thing that I want to kind of go over is the global tool configuration imagine the scenario or look at it this way Jenkins is a is a continuous integration server it doesn't know what kind of a code base it's going to pull in what kind of a tool set that is required or what is the code that is going to pull in and how is it going to build so you would have to put in all the tools that is required for building the appropriate kind of code that you're going to pull in from you know your source code repositories so just to give an example in case your source code is a Java source code and assuming that you know because in this a demo this is my laptop and I've put in all the configurations jdk everything on my laptop because I'm a developer I am working on the laptop but my continuous integration server would be you know a separate server without anything being installed on it so in case I want Jenkins to you know run a Java code I would need to install GDK on it I need to specify the jdk location of this out here this way okay since I already have the jdk installed and I've already put in the Java home directory or rather the environment variable correctly I don't need to do it git if at all I want the Jenkins server to use git a git is a you know command bash or the command prompt for for running git and connecting to any other git server so you would need git to be you know installed on that particular system and set the path accordingly Gradle and Maven if at all you have some mavens as well you want to do this any other tool that you're going to install on your system which is your continuous integration server you will have to come in here and configure something in case you don't configure it venchenkins runs it will not be able to find these tools for building your task and it'll crib about it that's good I don't want to save anything manage Jenkins let me see what else is required yes configure Global Security all right the security is enabled and if you see by default it's the security access control is set to Jenkins own user database so what does this mean you know Jenkins by default it uses file system where it stores all the usernames with hashes up these usernames and kind of stores them so as of now it Jenkins is configured to use its own database assuming that you are running in an organization you would probably want to have a you know some sort of an ad or an ldap server using which you would want to control access to your Jenkins repository rather Jenkins tool so you'd specify your ldap server details the root DN password or the manager DN and the manager password and all these details in case you want to connect your Jenkins instance with your ldap or ad or any of the authentication servers that you have in your organization but for now since I don't have any of these things I'm going to use this own database that's good enough all right so I will set up some authorization methods and stuff like that once I put in few jobs so for now let me not get into any of these details of this just be aware that Jenkins can be connected for authorization to an ldap server or you can have Jenkins managing its own servers which is happening as of now so I'm going to save all this stuff that's good for me so enough of all these configurations let me put in a very very simple job all right so job new item you know little difficult to kind of figure out but then that's the new item so I'll just say you know first job this is good for me I just gave a name for my job I would say it's a freestyle project that's good enough for me I don't want to choose any of that so unless until you choose any of this this particular button would not become active so choose the freestyle project and say okay at a very high level you will see General source code management build triggers build environment build and post build in case you install more and more plugins you will see a lot more options but for now this is what you would see so what am I doing at the moment I'm just putting up a very very simple job and the job could be anything and everything so I don't want to put in a very complicated job for now for the demo WordPress let me just put in a very very simple job I'll give a description this is an optional thing this is my first Jenkins job all right I don't want to choose any of these again there are some helps available here I don't want to choose any of this I don't want to connect it into any source code for now I don't want any triggers for now I'll come back to this in in a while build the environment I don't want any build environment as a part of this build step you know I just want to you know run few things so that I kind of complete this particular job so since I'm on a Windows box I would say execute Windows batch command all right so what do you want to do I let me just Echo something Echo uh hello this is my first Jenkins job and possibly I would want the date and the timestamp pertaining to the job I mean the date and time in which this job was run all right very very simple command that says you know this is my first job it just puts out something along with the date and the time all right I don't want to do anything else I want to keep this job as simple as this so let me save this job all right so once I save this job you know the job names comes up here and then I need to build this job and you would see some build history out here nothing is there as of now because I've just put in a job I have not run it yet all right so let me try to build it now you see a build number you will see a date and a timestamp so if I click on this you will see a console output if I go here okay as simple as that and where is all the job details that is getting into if you see this if I navigate to this particular directory all right so this is the directory what I was mentioning earlier regarding Jenkins home so all the job related stuff that is specific to this particular Jenkins installation is all here all the plugins that is installed the details of each of those plugins can be found here all right so the workspace is where all the jobs that I've created whichever I'm running would you know there will be an individual folders specific to the jobs that has been put up here all right so one job one quick run that's what it looks like pretty simple okay let me do one thing let me put up a second job I would say second job let's see freestyle project all right this is my second job I just want to demonstrate the powerfulness of the automation server and how simple it is to automate a job that is put up on Jenkins which will be triggered automatically remember what I said earlier about Jenkins because at the core of Jenkins is a very very powerful automation server all right so what I'm going to do I will just keep everything else the same I'm going to put in the build script pretty much similar to second job that gets triggered automatically every minute all right let me do that percentage date and I'll put in the time all right so I just put in another job called second job and it pretty much does the same thing as what I was doing earlier in terms of printing the date and the time but this time I'm just going to demonstrate the powerfulness of the automation server that is there if you see here there's a build trigger so the build can be triggered using various triggers that is there so we'll get into this GitHub uh triggering our hook or a web hook kind of a triggering later on but for now what I want to do I want to ensure that this job that I'm going to put in would be automatically triggered on its own let's say every minute I want this job to be run on its own so build periodically is my setting if you see here there's a bunch of help that is available for me so for those of you you have written cron jobs on Linux boxes you'll find it very very simple but for others don't panic let me just put up a very very simple regular expression for scheduling this job every minute all right so that's one two three four five all right come up come up come up all right so five stars is all that I'm gonna put in and Jenkins got a little worried and she's asking me do you really mean every minute oh yeah I want to do this every minute let me save this and how do I check whether it gets triggered every minute or not I just don't do anything I'll just wait for a minute and if at all everything goes well Jenkins would automatically trigger my second job in a minute's time from now this time around I'm not going to trigger anything look there you see it's automatically got triggered if I go in here yep second job that gets triggered automatically you know it was triggered at 42 1642 which is 442 my time that looks good and if everything goes well every one minute onwards this job would be automatically triggered now that I have my Jenkins up and running a few jobs that has been put up here on my Jenkins instance I would need a way of controlling access to my Jenkins server this is wherein I would use a plugin called role based access plugin and create few rules the rules are something like a global Rule and a project role Project Specific role I can have different rules and I can have users who have signed up or the users who might create kind of assigned to these roles so that each of these users fall into some category this is my way of kind of controlling access to my Jenkins instance and ensuring that people don't do something unwarranted alright so first things first let me go ahead and install a plugin for doing that so I get into many Jenkins and manage plugin a little bit of a confusing screen in my opinion there's updates available installed and advanced as of now we don't have the role-based plugin so let me go to available it'll take some time for you to get refreshed all right now these are the available plugins these are the install plugins all right so let me come back to available and I would want to search for my role based access plugin so I would say search for role and hit enter okay role-based authorization strategy enables user authorization using a role-based strategy roles can be defined globally or for particular jobs or notes and stuff like that so exactly this is the plugin that I want I would want to install it without a restart right looks good so far yes go back to the top of the page yes remember Jenkins is running on a Java using a Java instance so typically many things would work the same way and lesson until you want to restart Jenkins once in a while but as a good practice whenever you do some sort of a big installations or big patches on your Jenkins instance just ensure that you kind of restarted otherwise there would be a difference in terms of what is installed on the system and what is there on the file system you will need to flush out few of those settings later on but for now these are all very small plugins so this would run without any problems but otherwise if at all there are some plugins which would need a restart you know kindly go and restart your Jenkins instance but for now I don't need that it looks good I've installed the plugin so where do you see my plugin I install the plugin that is specific to the user control or the access control so let me go into yes Global Security and I would see this role-based strategy showing up now all right so this comes in because of my installation of my role based plugin so this is what I would want to enable because I already have my own database set up and for the authorization part in the sense that who can do what I'm going to install I mean I've already installed a role based strategy plugin and I'm going to enable that strategy all right I would say save okay now I have installed the role based access plugin I would need to just set it up and check that you know I would go ahead and create some roles and ensure that I assign users as per these roles all right so let me go to manage Jenkins configure all right let me see where is this configured configure Global Security is that where I create my rules nope not here yes manage and assign roles okay again you would see these options only after you install these plugins so for now I've just enabled the plugin I have enabled role based access control and I would go ahead and create some rules for this particular Jenkins instance so I would say first manage rules so I would need to create some roles here and the rules are at a very high level these are Global rules and there are some project rules and there are some slave rules I'll not get into details of all of this at a very very high level which is a global role let me just create a row a role can be kind of visualized like a group so I would create a role called developer typically the Jenkins instance or the ca instance are kind of owned up or controlled by QA guys so QA guys would need to provide some sort of a you know limited access to developers so that's why I'm creating a role called developer and I'm adding this role at a global role level so I would say add this here and you would see this developer role that is there and each of these options you can click hover over it you would see some sort of a help on what what are these uh you know permissions specific to so what I want is like you know it sounds a little you know different but I would want to give very very little permissions for the developer so from an Administration perspective I would just want him to have a read uh kind of a role credentials again I would just want to view kind of a role I don't want him to create any agents and all that stuff that's good for me for a job I would want him to just possibly read I don't want him to build I don't want him to cancel any jobs I don't want him to configure any job I don't even want him to create any job I would just want him to read few things I would not give him possibly a role to the workspace as well I mean I don't want him to have access to the workspace I would just want him to read a job I'll check you know have read only access to the job run um no I don't want him to give him any any particular access which will allow him to run any jobs view configure yeah possibly create yeah delete I don't want to read yes definitely and this is the specific role so what I'm doing I'm just creating a global role called developer and I'm giving him very very limited roles in the sense that I don't want this developer to be able to run an agents nor create jobs or build jobs or cancel jobs or configure jobs at the max I would just want him to read a job that is already put up there okay so I would save now I created a rule I still don't have any users that is there on the system so let me go ahead and create some user on the system that's not here I will say configure managing cans manage users okay let me create a new user I would call this user as yeah developer one sounds good some password some password that I can remember okay his name is developer One D at d.com or something like that okay so this is the admin with with which I kind of configured or brought up the system and developer one is a user that I have configured so still have not set any rules for this particular user yet so I would go to manage and gains I would say manage and assign roles I would say assign roles okay so if you see what I'm going to do now is assign a role that is specific to that particular dial I will find the particular user and assign him the developer role that I have already configured the role shows up here I would need to find my user whoever created and then assign him to that particular role so if you remember the user that I created was developer one I would add this particular user and now this particular user what kind of a role I want him to have because this is the global rule that had created so developer I would assign this developer 1 to this particular Global rule and I would go ahead and save my changes now let me check the permissions of this particular user by logging out of my admin account and logging back as developer one if you remember this role was created with very less privileges so there you go I have Jenkins but I don't see a new item I can't trigger a new job I can't do anything I see these jobs however I don't think so I'll be able to start this job I don't have the permission set for that the maximum I can do is look at the job see what was there as a part of the console output and stuff like that so this is a limited role that was created and I added this developer to that particular role which was a developer role so that the developers don't get to configure any of the jobs because the Jenkins instance is owned by a cure person he doesn't want to give developer any administrative rights so the rights that he set out by creating a developer role and anybody who is tagged any user who is tagged as a part of this developer role would get the same kind of permissions and these permissions can be you know fine-grained it can be a Project Specific permissions as well but for now I just demonstrated the high level permission that had set in let me quickly log out of this user and get back as the admin user because I need to continue with my demo with the developer role that was created I have very very less privileges one of the reasons for Jenkins being so popular as I mentioned earlier is the bunch of plugins that is provided by users or Community users who don't charge any money for these plugins but it's got plugins for connecting anything and everything so if you can navigate to or if you can find Jenkins plugins you would see index of over so many plugins that is there all of these are wonderful plugins whatever connectors that you would need if you want to connect Jenkins to an AWS instance or you want to connect Jenkins to a Docker instance or any of those containers you would have a plugin you can go and search up if I want to connect Jenkins to big bucket bitbucket is one of the git servers there's so many plugins that is available okay so bottom line Jenkins without plugins is nothing so plugins is the heart of Jenkins for you to connect or for in order to connect Jenkins with any of the containers or any of the other tool sets you would need the plugins if you want to connect or you want to build a repository which has got Java and Maven you would need to install Maven and GDK on your Jenkins instance if at all you're looking for a net build or Microsoft build you would need to have MS build installed on your on your Jenkins instance and the plugins that will trigger Ms build if at all you want to listen to some server side web hooks from GitHub you would need GitHub specific plugins if you want to connect Jenkins to AWS you need those plugins if you want to connect to a Docker instance that is running anywhere in the world as long as you have the URL which is publicly reachable you just have a Docker plugin that is installed on your Jenkins instance sonar cube is one of the popular static code analyzers so you can connect a Jenkins build you can build a job on Jenkins and push it to sonar Cube and get sonar Cube to run analysis on that and get back the results in Jenkins all of these works very well because of the plugins now with that let me connect our Jenkins instance to GitHub I already have very very simple Java repository up on my GitHub instance so let me connect Jenkins to this particular GitHub instance and pull out a job that is put up there all right so this is my very very simple uh you know repository that is there called hello Java and this is what is there in the report there is a hello hello.java application that is here or a simple class file that is there it's got just one line of system dot out so this is already present on github.com at this place and this would be the URL for this Repository if I pick up the https URL this is my https URL so what I would do is I would connect my Jenkins instance to go to GitHub provide my credentials and pull out this repository which is on the cloud hosted github.com and get it to my Jenkins instance and then build this particular Java file I'm keeping the source code very very simple it's just a Java file how do I build my Java file how do I compile my Java file I just say Java C and the name of my class file which is hello.java and how do I run my Java file I would say Java and hello okay so remember I don't need to install any plugins now because what it needs is a git plugin so if you remember when we were doing the installation there was a bunch of recommended plugins so git is already installed on my system so I don't need to install it again so let me put up a new job here this is a git job let it be a freestyle project that's good for me I would say okay all right so the source code management remember in the earlier examples we did not use any source code because we were just putting up some Echo kind of jobs we did not need any integration with any of the source code systems so now let me connect this so I'm going to put up a source code and git put sure because the plugin is already there SVN performs any of those additional source code management tools if at all you would need just install those plugins and Jenkins connects wonderfully well to all this particular Source control tools okay so I would copy the https URL from here I would say this is the URL that I'm supposed to go and grab my source code from but all right that sounds good but what is the username and password so I'll have to specify a username and password all right so I would say the username this is my username and this is my https credential for my job okay so this is my username and this is my password I just save this I say add and then I would say you know use these credentials to go to GitHub and then on my behalf pull out a repository all right if at all at this stage if there's any error in terms of not able to Jenkins not able to find git or the git.exe or if my credentials are wrong somewhere down here you would see a red message saying that you know something is not right you can just go ahead and kind of fix that for now this looks good for me I'm going to grab this URL what am I going to do this step would pull the source code from the GitHub and then what would be there as a part of my build step because this repository just has a Java file correct hello.java so in order to for me to build this I would just say execute Windows batch command and I would say Java C Hello dot Java that is the way I would build my Java code and if I have to run it I would just say Java hello pretty simple two steps and this would run after the repository contents are fetched from GitHub so Java C Java that sounds good I would say save this and let me try to run this okay if you see there's a lot of you know it executes git on your behalf it goes out here it provides my credentials and says you know it pulls all my repository and by default it will pull up the master branch that is there on my repository and it kind of builds this whole thing Java C hello to Java and it runs this project Java hello and there you see this is the output that is there and if at all you want to look at the contents of the repository if you can go here this is my workspace of my system hang on this is not right okay good job if you see here this is my hello.java this is the same program that was there on my GitHub repository okay so this is a program that was there on the main GitHub repository all right so this was the same program that was here and Jenkins on our behalf went over all the way to GitHub pulled this repository from there and then you know it brought it down to my local system or my Jenkins instance it compiled it and it ran this particular application okay now that I've integrated Jenkins successfully with GitHub for a simple Java application let me build a little bit on top of it what I will do is I have a maven-based web application that is up there as a repository in my GitHub so this is the repository that I'm talking about it's called AMV and web app it's got It's a maven-based repository as you would know my one is a very very simple Java based uh build tool that will allow you to run various targets and it will compile it will based upon the goals that you specify it can compile it can run some tests and it can you can build a war file and even deploy it into some other server for now what we're going to use Maven is just for building and creating a package out of this particular web application it contains a bunch of things and what is important is just the index.jsp it just contains an HTML file that is there as a part of this web application so from a perspective of requirements now since I'm going to connect Jenkins with this particular repository git we already have that set we only need two other things one is Maven because Jenkins will use Maven so in order to use Maven Jenkins would have to have a maven installation that is there on the Jenkins box and in this case the Jenkins box is this laptop and after I have my Maven installed I also need a tomcat server Tomcat is a very very simple web server that you can freely download I'll let you know how to quickly download and install the Tomcat alright so download Maven first at the various ways in which you can kind of download this my one there is zip files binary zip files and archive files so what I've done is I've just already downloaded Maven and if you see I've unzipped it here so this is the folder with which I have unzipped my maven so as you know Maven again is is a one open source build tool so you'll have to set in a few configurations and set up the path so mvn hyphen iPhone version if I specify this after setting my path my one should work and if at all I Echo M2 home which is nothing but the variable environment variable specific to mavenode it is already set here so once you unzip my oven just set this M2 home variable to the directory where you unzip your Maven also just set the path to this particular directory slash bin because that is where your Maven executables are all found all right so that's with Maven and you know since I've set the path and the environment variable mav1 is running perfectly fine on my system I've just verified it okay next one is a tomcat server download Apache Tomcat server 8.5 is what I have on my system so I'm just going to show you where to download this from this way you download Tomcat server and I already have the server downloaded again this doesn't need any installation I just unzip it here and it kind of has a bin and configuration I've made some subtle changes in the configuration first and foremost Tomcat server also by default runs on port 8080 since we already have our Jenkins server running on port 8080 we cannot let Tomcat run on the same port there will be a port Clash so what I've done I have configured Tomcat to use a different port so if I go to this configuration file here there is a server.xml let me open this up here all right okay so this is the port by default it will be 80 80. I've just modified it to 8081 so I've changed the port on which my Tomcat server would run all right so that's one chain second change when Jenkins kind of tries to get into my tomcat and deploy something for someone he would need some authentications so that he'll be alloy deployment by Tomcat so for that I need to create a user on tomcat and provide this user credentials to my Jenkins instance so I would go to tomcatusers.xml file here I have already created a username called deployer and the password is deployer and I've added a role called manager hyphen script manager hyphen script will allow programmatic access to the Tomcat server so this is the role that is there so using this credentials I will enable or I'll Empower Jenkins to get into my Tomcat server and deploy my application all right only these two things that is required let me just start my Tomcat server first so I get into my bin folder I open a command prompt here and there's a startup.bat it's pretty fast it just takes a few seconds yes there you go don't get service up and running now this is running on Port 8081 so let me just check if that looks good so localhost 8821 okay my Tomcat server is up and running that sounds good the user is already configured on this that's also fine so what I'll do as a part of my first job my one is also installed on my system so I'm good to use Maven as a part of my Jenkins so I will put up a simple job now I will say job NBN web app I call this freestyle job that's good okay so this will be a git repository what is the URL of Market repository is uh this guy https URL okay that's this URL I will use a credentials the old credential that I set up will work well because it's the same git user that I'm kind of connecting into all right so now the change happens here where after I get this since I said this is a simple Maven repository I will have some Maven targets to run so the simple Target first is let me run Maven package this creates a war file okay so mvn package is the target package is the target so when whenever I run this package it kind of creates it it builds it it tests it and then creates a package so this is all that is required maybe let me try to save this and let me first run this and see if it connects well if there's any problem with my War file or the war file gets created properly okay wonderful so it built a war file and if you see it all shows you what is the location where this war file was generated so this will be the workspace if you see this this Warfare was successfully built now I need to grab this particular War file and then I would need to deploy it into Tomcat server again I would need a small plugin to do this because I need to connect Tomcat with my Jenkins server let me go ahead and install the plugin for the container deployment so I will go to manage plugins available type in container continue container deploy to container okay so let's put this the plugin that I would need I would install it without a restart right since we're very fast nope sorry it's still installing okay it installed the plugin so if at all you see this if you go to my workspace okay in the Target folder I would say this web application War file that is already built so I would need to configure this plugin to pull up this war file and deploy it onto the Tomcat server for deploying onto the Tomcat server I will use the credentials of the user that I've created okay so let me go to configure this particular project again and um okay all this is good so the package is good I'm going to just create a package that's all fine now add a post build step so after the war file is built as a part of this package directive let me use this deployment to container now this will show up after you install the plugin so deploy this one to the container now what is that you're supposed to specify you're supposed to specify the what is the location okay so this is a global uh you know configuration that is there that will allow you to from the root folder it'll pick up the war file that is there so star star forward slash star dot War that's good for me okay what is the context path context path is nothing but just the name of an application that you know under which it will get deployed into the Tomcat server I will just say mvn web app that's the name of my thing now I need to specify what kind of a container that I'm talking about all right so the deployment would be for the stormcat 8.5 is what I need okay because the server that we have is a tomcat 8.5 server that I have so this would be the URL so the credentials yes I need to add a credential for this particular server so if you remember I created a credential for my web application so let me just find that my Tomcat server yes configuration of this okay so deployer and deployer username is deployer password is deployed okay so let me use that credential I would say I would say add a new credential Jenkins credential the username is deployer and the password is replyer so I would use this deploy credentials for that and what is the URL of my Tomcat instance so this is the URL of my Tomcat instance so take the war file that is fine found in this particular folder and then you know context path is MN web app use the deployer deployment credentials and get into this localhost which is there 1881 this is the Tomcat server that is running on my system and then go ahead and deploy it okay so that is all that is required so I would say just save this and let me run it now okay it built successfully built the war file it is trying to deploy it and looks like a deployment went ahead perfectly well so the context path was mvn web app so if I type in this all right if at all I go ahead into my Tomcat server there would be a web apps folder you would see the you know the date time stamp so this is the file that get got recently copied and this is the Explorer version of our application so the application was built the source code of this application was pulled from the GitHub server it was built locally on the Jenkins test and then it was pushed into a tomcat server which is running on a different port which is 808.1 now for this demo I am running everything locally on my system but assuming that you know this particular Tomcat instance was running on some other server with some other different IP address all that you got to go and change is the URL of the server so this would be the server in case you you already have that uh you know if you have a tomcat server which is running on some other machine that's all fine with a different IP that's all good enough the whole bundle or the war file that was built as a part of this Jenkins job gets transferred onto the other server and gets deployed that's the beauty of Jenkins and automatic deployments or other deployments using Jenkins and my own the four components are as follows the docker clients and server which we've already done a deeper dive on Docker images Docker containers and the docker registry so if we look at the structure that we have here on the left hand side you see the relationship between the docker client and the docker server and then we have the rest API in between now if we start digging into that rest API particularly the relationship with the docker Daemon on the server we actually have our other elements that form the different components of the docker ecosystem system so the docker client is accessed from your terminal window so if you are using Windows this can be Powershell on Mac it's going to be your terminal window and it allows you to run the docker Daemon and the registry service when you have your terminal window open so you can actually use your terminal window to create instructions on how to build and run your Docker images and containers if we look at the images part of our registry here we actually see that the image is really just a template with the instructions used for creating the containers which you use within Docker the docker image is built using a file called the docker file and then once you've created that Docker file you'll store that image in the docker Hub or registry and that allows other people to be able to access the same structure of a Docker environment that you've created the syntax of creating the image is fairly simple it's something that you'll be able to get your arms around very quickly and essentially what you're doing is your create meeting the option of a new container you're identifying what the image will look like what are the commands that are needed and the arguments for and then those commands and once you've done that you have a definition for what your image will look like so if we look here at what the container itself looks like is that the container is a standalone executable package which includes applications and their dependencies it's the instructions for what your environment will look like so you can be consistent in how that environment is shared between multiple developers testing units and other people within your devops team now the thing that's great about working with Docker is that it's so lightweight that you can actually run multiple Docker containers in the same infrastructure and share the same operating system this is its strength it allows you to be able to create those multiple environments that you need for multiple projects that you're working on interestingly though within each container that container creates an isolated area for the applications to run so while you can run multiple containers in an infrastructure each of those containers are completely isolated they're protected so that you can actually control how your Solutions work there now as a team you may start off with one or two developers on your team but when a project starts becoming more important and you start adding in more people to your team you may have 15 people that are offshore you may have 10 people that are local you may have 15 Consultants that are working on your project you have a need for each of those developers or each person on your team to have access to that Docker image and to get access to that image we use a Docker registry which is an Open Source server-side service for hosting and distributing the images that you have defined you can also use Docker itself as its own default Retreat and Docker Hub now something that has to be bear in mind though is that for power publicly shared images you may want to have your own private images in which case you would do that through your own registry so once again public repositories can be used to host the docker images which can be accessed by anyone and I really encourage you to go out to Docker and see the other Docker images that have been created because there may be tools there that you can use to speed up your own development environments now you will also get to a point where you start creating environments that are very specific to the solutions that you are building and when you get to that point you'll likely want to create a private repository so you're not sharing that knowledge with the world in general now the way in which you connect with the docker registry is through simple pull and push commands that you run through terminal window to be able to get the latest information so if you want to be able to build your own container what you'll start doing is using the pull commands to actually pull the image from the docker repository and the command line for that is fairly Sim simple in terminal window you would write Docker pull and then you put in the image name and any tags associated with that image and use the command pause so in your terminal window you would actually use a simple line of command once you've actually connected to your Docker environment and that command will be Docker pull with the image name and any Associated tags around that image what that will then do is pull the image from the docker repository whether that's a public repository or a private one now in Reverse if you want to be able to update the docker image with a new information you do a push command where you take the script that you've written about the docker container that you define and push it to the repository and as you can imagine the commands for that are also fairly simple in terminal window you would write Docker push the image name any Associated tags and then that would then push that image to the docker repository again either a public or a private Repository sorry so if we recap the docker file creates a Docker image that's using the build commands Docker image then contains all the information necessary for you to be able to execute the project using the docker image any user can run the code in order to create a Docker container and once a Docker image is built it's uploaded to a registry or to a Docker Hub where it can be shared across your entire team and from the docker Hub users can get access to the docker image and build their own new containers welcome to all our viewers I'm Anjali from Simply learn and today I'll be taking you through the installation process for Docker so before we begin let me give you an intro to Docker so what is Docker Docker is a tool which is used to automate the deployment of applications in lightweight containers so when I say containers I mean a software package that consists of all the dependencies required to run the particular application and when you deploy containers you're basically providing the capability for the application to run in any kind of environment so Dockers have multiple features some of which are that multiple containers can run on the same Hardware it has very high productivity when compared to Virtual machines it maintains isolated applications and has a very quick and easy configuration process so now let's begin with the demo we'll be installing Docker on an Ubuntu system so this is my system I'll just open the terminal so the first thing you can start with is removing any Docker installation that you probably already have present in your system if you want to start from scratch so this is the command to do so sudo app get removed docker dock engine Docker dot IO enter your password and Docker is removed so now we'll start from scratch and we'll install Docker once again before that I'll just clear my screen okay so before I install Docker let me just ensure that all these softwares on my system currently is in its latest state so sudo app get update great so that's done next thing we'll actually install our docker so type in sudo apt get install docker now as you can see here there's an error that's occurred so sometimes it's possible that due to the environment of the machine that you're working in this particular command does not work in which case there's always another command that you can start with just type Docker install and that by itself will give you the commands you can use to install docker so as it says here sudo apt install Docker dot IO is a command that we will need to execute to install docker and after that we'll execute the sudo snap install Docker so sudo apt install Docker dot IO first and this will install your docker after that's done we will have sudo snap install Docker so snap install Docker installs a newly created snap package there are basically some other dependencies for Docker that you'll have to install of course since this is the installation process for the entire Docker i o it will take some time great so our Docker is installed the next thing we do as I mentioned earlier is that we need to install all the dependency packages so the command for that is sudo snap install talk enter your password so with that we have completed the installation process for Docker but we'll perform a few more stages where we will test if the installation has been done right so before we move on with the testing for Docker let's once again just check the version that we have installed so for that the command is Docker version and as you can see Docker version 17.12.1 CE has been installed next thing we do is we pull an image from the docker hub so Docker run hello world now hello world is a Docker image which is present on the docker Hub Docker Hub is basically a repository that you can find online so with this command the docker image hello world has been pulled onto your system so let's see if it's actually present on your system now the command to check this is sudo Docker images and as you can see here hello world repository this is present on our system currently so the image has been successfully pulled onto the system and this means that our Docker is working now we'll try out another command pseudo Docker PF minus a this displays all the containers that you have pulled so far so as you can see here there are three hello world images displayed and all of them are in exited state so I did this demo previously too which is why the two hello worlds which is created two minutes ago is also displayed here and the first hello world which has been created a minute ago is the one we just did for this demo now as you have probably noticed that all the hello world images over here all these containers are in the exited state so when you give the option for Docker PS minus a where minus a stands for all it displays all the containers whether they are in exited or running state if you want to see only those containers which are in their running State you can simply execute sudo Docker PS pseudo docker yes and as you can see no container is visible here because none of them are in running State and with that we come to an end of our Docker installation welcome everyone to our one another demo on which we are going to see that how exactly we can do the installation of Docker on the Windows platform specifically on Windows 10. now Docker is something which is available for most of the operating systems different different platforms so it supports both the Unix and the windows platform as such so Linux through various commands we can do the installation but in the case of Windows you have to download the exe file and a particular installer from the docker Hub websites you can simply Google it and you will get a kind of link from where you will be able to download the package so let's go to the Chrome and try to search on for the windows string particular installer you will get a link from Docker Hub you download it you get the stable version you get the edge version whichever version you want you wish to download you can download it so let's go back to the Chrome so here you have the dock next stop for Windows so you can go for this table or you can go for the edge right so you also have the comparison that what is the difference between these two versions right so um the particular Edge version is something which is getting releases every month and uh the uh stable version is getting the releases every quarter so they are not doing much of the changes to the stable version as compared to the edge there so you just have to double click on the installer and that will help you to do the installation of the process so let's get started so you just click on the get in stable version so when you do that the particular installer is going to install now it's going to take like around 300 MB there so that's the kind of installed which is available so once the installer is downloaded so what you can do is that you can actually go ahead and you can proceed with the doing the double click on this installer when you double click on that you have to proceed with some of the steps like you know from the GUI itself you are going to proceed with these steps so we'll wait for 10 to 20 seconds more and then the installer will be done and then we can do the double click and the installation will proceed so another thing is that uh there is a huge difference between the installer like for example in case of Unix they installed is a little bit less but in case of windows it's a GUI is also involved and there are a lot of binders which is available there so that's the reason why you know the Hue size is there now it's available for free that's for sure and it also requires the Windows 10 professional or Enterprise 64-bit there so um if you are working on some previous version of operating systems like Windows 7 and all you have the older version called Docker toolbox so they used to call it as like Docker toolbox earlier but now they are calling it as in Docker desktop with a new Docker Windows 10 support as such here so another couple of seconds and then the installer will be done and then we will be able to proceed with the installation so let's see that how much progress is there to the download so we'll click on the downloads and here still we have some particular installations or some download going on so we'll wait for some time and once the installation is done then we'll go back and we'll proceed with installation so couple of seconds so it's almost done so I'll just click on this one you can go to the directory to the downloads and you can double click on that also but if you want to do the installation you can click on this one also and it will ask for the approval yes or no you have to provide now once that is done so um a desktop a kind of a GUI component will open there so it will start proceeding with installation so it's asking whether you want to add a desktop the shortcut to desktop so you can say okay I'm going to click on OK so it will unpack the files all the files which is required for Docker to successfully install that is getting unpacked over here so it will take some time to do the installation because it's doing a lot of work here so you can just wait for till the execution of the installer to be completed and once the installer is done you can open your command line and start working on the docker so taking some time to extract the files now it's asking us to you know do the close and do the restart so once that is done you will be able to proceed further and you can just you know run the command line and any Docker command if you can run so that will give you the response whether the docker is installed or not so you can see here that Docker is you know something which is installed so you can run like Docker version you will be able to get a version of the client when you do the restart of the machine then at that moment of time the docker server will also be started and then this particular error message will go off right now the docker demon is not up and running because the installation requires a restart and when you close on this one and go for the restart the machine will be restarted here so this is the way that how exactly we can go for a Docker installation and we can go on that part welcome everyone to this topic on Docker tutorial for the beginners here now in this one we are going to talk about the various components for example what is the devops here why the devops is important what is Docker the advantages of Docker installation of Docker architecture of Docker components of Docker game so these are the different things which we are going to talk about over here now let's start with what exactly isn't devops all about here now devops is nothing but a collaboration between the development and the operations teams over here which enables continuous delivery of the applications and services to the end user so it's a combination or collaboration between the two teams development and operations which helps us to go for a further implementation of various processes like continuous integration continuous deployment continuous delivery all these things are possible with this collaboration over here now why exactly Docker is important for us as such in a city far away from this one so a developer decided to develop a tons of software on a single VM now he can put up applications he is having a big server and he wants to host the the multiple application on a single machine but for him these virtual machine got a crash because definitely if you are putting up more uh different different applications as such on the on a single VM it's not able to take up that much kind of load over there so in order to avoid this situation how exactly he can do the various options or what he can Implement to have this uh issue result so ultimately Docker is something which is the solution of him so if he really wants to host multiple applications uh on a single virtual machine or he want to have different different applications then Docker is the ultimate solution for him to do that and by doing so uh there will be no issues as such when we talk about that uh the errors or the virtual machines cannot be crashed and well there will be no impact on the virtual machine as such so why exactly Docker is important so let's talk about the comparison between the docker and virtual machine there are very few reasons why exactly the docker is being chosen overnight virtual machine because virtual machine is a full-fledged component which can take up any kind of load but some of the reasons are still that in which case we will be preferring the docker instead of the virtual machine here the very first one is the memory users so Docker is having a very less memory users because there is no concept of guest operating system whereas on the other hand the virtual machines are going to take a huge amount of ram resources in order to keep that particular operating system up and running second one performance because uh if definitely if you have less uh utilization and the users is less the performance will be better but for some reason if you are taking up more resources and you will not be able to deliver that amount of performance as such over there and then again we have portability so portability is also quite easy when we talk about the virtual machines in case of Docker it's quite portable as compared to the virtual machine and then the last time is that the boot up time so virtual machine bottom time is probably in minutes but uh in case of Docker it's in second so if we summarize it on this one so the memory utilization the boot up time the performance everything is better in case of Docker as compared to the virtual machines right for a classic example let's say that what about the memory user 9gb of memory uh you know is something if you can see that I have one app to and app3 is there so green and red color represents the occupied memory and the wasted memory now there is a particular uh utilization of let's say that um you know a server which is available there and uh you know uh 3gb of uh RAM is something which is wasted and 4GB of RAM is something which is being occupied there so app2 is using like 2GB and uh then you can see that the specific 2GB of RAM is occupied and 3gb is occupied and the wasted one is 1GB so VM can see over there that uh you know 9 GB of RAM is used and the remaining 6gb of uh unused Ram cannot be reused because if for example I allocate like 4GB of ram to the virtual machine it's there itself it gets logged on so it's not like okay uh I'm using only 4GB Ram out of this 9gb of ram then the next 5gb of ram will be totally wasted so you cannot reuse it for any purpose so this is a problem with the virtual machine that if you allocate some memory to the specific virtual machine if you are not using that much resources or that much memory the remaining memory will be simply pasted because this is hard coldly allocated to a virtual machine and you cannot utilize this unused memory as such over there so this is the limitation from the memory perspective but in case of Docker definitely if uh you know out of the 9 GB of RAM if we are using uh only 3gb of RAM so whatever the unused memory is there so we can actually reuse it to for the new containers so it's not dedicately allocated to the virtual machines if you are using out of 9gb if you are using 1GB 2GB 3gb or 4GB whatever the remaining uh memory is available that you can simply use it for the other containers also this ensures that you always go for an optimal utilization of your resources and unnecessarily there is no wastage of the different different resources or the different components over here in this one so that's the reason why the docker containers are quite uh efficient in terms of memory when we compare it with the virtual machine next one is the performance now in a virtual machine we know that you know on the infrastructure on the physical infrastructure on top of that you put up a kind of operating system a base operating system is there and then on top of that you have the hypervisor and this hypervisor then is used to create the virtual machines which is having a kind of just operating system inside it now the issue over here is that you have two operating systems over here so the guest operating system is also going to take some particular resources and uh for example the memory or a space a processor so in every terms in every resource terms it is going to go ahead and say that okay I'm going to use the particular sum of the resources Now problem is that if you're hosting a very small application you will end up in having a more conception because of this guest operating system will take more of more of the resources and then you are just using some small very small resources for your application in an real world so Docker container on the other hand does not have a concept of the guest operating system and that's where what happens that the docker engine is more than sufficient to handle the docker containers and inside the docker containers there is the application which is being hosted over there now with this one definitely the performance is quite better as compared to any virtual machine because of the terms that there is no unnecessarily wasted of the resources because in the previous case virtual machines are full-fledged gas operating system is up and running you need some resources to be allocated to that but in case of containers you don't need to have that particular functionality implemented then portability so um you know having a portability or moving an application which works on a system a does not mean that it will work on the system B also this is a similar problem which we face that you know usually the application works perfectly fine in a lower environment and when it goes to the production environment it sometimes fails and it does not comes up properly so this isn't portability issue which we have when we talk about the virtual machines but when in case of containers when we got a container up and running in a lower environment and once the container is prepared you don't have to do the changes or modification to that depending on the environment the same container can be deployed till production environment and it's very easy so whatever a container prepared in the lower environment in a Dev environment the same will work in production environment without any issues because it contains all the dependencies and executables everything inside it so you don't have to do any kind of modifications or changes to these containers as such over here and the last one is a boot up time so uh the boot up time of the virtual machines is in minutes and whereas the boot up time of the docker containers is in seconds or probably in milliseconds so these are the reason why we consider the virtual machine as in resources based uh environment but on the other hand Docker is something which is quite light and fast as compared to watch a message so let's see that what exactly is in Docker all about here so in this one you can see we have the physical infrastructure on which we have the host operating system now we don't have the hypervisor and the guest operating system instead of that we have the docker engine which is taking care of managing the docker containers and inside these Docker containers you have the applications and all the required binaries and the libraries here so there is no concept of the gift operating system these Docker engine is the uh person or is the process which is taking care of all the requests which is going on and if this Docker engine is up and running then all the containers will also be accessible and will be available now Docker is nothing but a OS level virtualization software platform that enables developers and it administrators to create deploy and run applications in a Docker container with all its dependencies so Docker is a very important tool nowadays if you want to look forward to the host your application because of its portability uh benefit that you don't require much of the changes uh depending on the platform or depending on the environment so whatever the developers are able to build up the same thing will go till the production government now let's talk about what exactly is in Docker container here now Docker container is a lightweight software package that includes all the dependencies whether it's a base operating system uh some framework sample libraries it's it's something which is uh part of this one so that if you will be if you want to host your application you will be able to do that so it's a complete package so both the source code and the dependencies is what you put up inside the docker containers now within this Docker containers you can have multiple applications there multiple programming languages multiple Frameworks can be hosted there in fact the databases also can be hosted within this Docker containers you can host them as such over there now one of the different advantages of Docker let's talk about that now the first one is the rapid deployment so a rapid deployment is the very first advantages which we are getting over here second quick and easy configuration better efficiency portability scalability and security over here now there are different ways in which we can understand the uh benefits or we can have the understanding about the docker here so uh these are some couple of benefits now let's take a uh take them one by one the first the first one is the deployment part so uh it's it supports the rapid deployment now just like a web application for instance I want to deploy to Webster so there are some couple of steps which needs to be done across different different environments like for example if I have five environments I have to do the deployment the same steps in all the five environments at the different period of time now when we talk about Docker containers we don't have to run those steps multiple times because once those steps are executed in one Docker container the same Docker container goes till the production environment so you don't have to actually do a rebuild of the docker containers and you can simply say that okay I have got the docker container here so you can just run one or two couple of commands and that will help you to host your Docker containers so you don't have to redo or rework on any kind of the deployment work into the production environment or into the higher environments and then we go for the quick and easy configurations so uh the configurations of Docker containers is quite easy you can have your own custom Docker images built out the docker files so these processes can be integrated within the Jenkins also so cicd is quite possible with the help of Docker if you want to really do that and then we have better efficiency yes they have a better efficiency and resources management as compared to the virtual machines portability uh we have already thought about that that it's easy to move from one environment from one system to another in fact it's easy from the developer also so from one machine from one developer to another developer it's very easy to move these uh different Docker containers then we have scalability scalability is also possible with the Stocker containers just like multiple virtual machines you can also create multiple containers and you can share the traffic out of that so but that scalability and all is possible with the orchestration tools then security is there so you can have your best practices of the security implemented because um it's just a kind of a virtualization just like a virtual machine so you have all the uh different uh possibilities which is available which you can explore when you talk when you want to make this stock container secure right so these were some of the benefits which was there for the docker here now let's talk about the installation of Docker here now installation of Docker is pretty common and whatever platform you want it's available over there so it's available for Ubuntu windows it's available for Mac operating system so for Ubuntu you have to run the commands on the Ubuntu part operating system to download it and for Windows it's actually having a a particular exe file so you can go for the installation of exe file and then we have the Mac operating system so for Mac operating system you have the DMG files and when you do the DMG file so it will help you to install the component over there so Mac you can also use the home view wants to do the installation of that particular component now next one is the architecture of Docker now the architecture of Docker is pretty straightforward there are key components which is available there in the architecture of Docker here the very first one is the Azure client facing is the docker client now Docker client or you can say the docker CLI is available there now this is the component it's a command line interface which takes up the request from the end user like for example I'm running the command so it will take all the requests from me as an end user and it will send those uh particular things or the requirements to the docker daemin to the server using the rest TV API interface so this Docker demon is something which keeps on running on the server and it's a background process it gets deployed as in service there in case this Docker demon is not available or it's not up and running you will not get the proper response from the docker command line here so in order to get your activity or your procedures implemented you need to have the docker server or Docker demon up and running in fact this Docker CLI can be configured to connect to any other servers or any other different server Docker Daemon also so it's a connectivity which is happening over the rest API interface where all the requests are processed are sent to the docker demand and whatever response is coming back that is given back to the end user so what are the different components of Docker here let's talk about that so Docker client consists of the CLI component like we have already discussed so that is something which is going to take up all the requests from the end user then it's using the rest API interface which is there to have the connectivity with the docker Daemon so Docker demon is uh something uh which is always up and running so if you run a couple of commands like Docker pull position also that activity you are actually sending the request to the docker demon and Docker demon is actually doing all the work so it's not a Docker client which is doing it just taking its kind of interface which will take the things up and send it back or send it forward to the docker day right so it's a backend process which keeps on running on the server on the operating system and uh it's the same process across all the infrastructure all the different platforms whether you use Mac operating system use Windows or you use Linux everywhere this is the basic functionality and it keeps on listening to the rest API interface so uh whatever the request new request comes in it immediately process it and whatever the response whether you are stopping a container whether you're running a container according to that the action will be taken and the response will be sent back to the rest API to the docker client so a dockerd is a processor it's a command line which is used to start the docker demon when you run the command like service Docker status on the server you will be able to see the command which is getting executed in the back end so there you can see that Docker is the process or is the command line which is used to initialize the docker Daemon and of course there is a kind of configurations which you have to pass it on as a parameters so even if you run psyphen EF command you will be able to know that what process is running from the docker perspective and the third component is a particular Docker registry so we have a ready-made uh cloud service which is available known as the docker Hub so you can actually store the docker images on these Docker registry so there are a lot of ready-made Docker images which is available there but of course you can have your own Docker image also over there or you want you can have your own custom registry Docker Registries also for example in jfrog you can have your private hosted Registries also so all these uh different repositories you can use with your Docker container so Docker containers is having the flexibility that whichever Registries or repositories you want to connect or you want to establish the connectivity you can always do that now let's talk about the docker image over here in detail now a Docker image is nothing but a template of instructions it's having a couple of instructions in which you will be using these instructions to create a final doc container so Docker image is uh nothing but uh for example I want to have a tomcat Apache Docker image so what are the instructions I have to do so the very first thing is that I have to pick up a base operating system then on top of that I will be installing the jdk awesome pre-request packages and then on top of that we will be configuring on to the Target Apache there so these are the steps these are the different instructions step by step procedure which we are implementing so that at the end of the activity we will be able to have a Docker container ready which we can deploy to host Tomcat Apache so it's a template which is having all the particular instructions in order to build a Docker image you require a file called Docker file which is uh of course having all these different uh instructions embedded that so when you run the docker file it acts like a build tool so whatever the instructions you are putting up over there all those instructions are getting a processed and you are able to get a end result and then these Docker images are actually comprised of multiple layers so multiple layers are available there so step by step so you will be first of all having a base layer from which the image starts on top of that another layer for example you did the installation of jdk so another layer will be there and then you did the configuration of of the term KET Apache and then another layer will be there so it's a multi-layered in uh component which is available and these layers are in incremental order so definitely the oldest one will be known as the base layer and then on top of that all the latest changes will be done in the latest layers there so this is an example now here you can see that there are four instructions and these four instructions are also known as the four layer Docker image so whatever over the final Docker image you will get in there you will be able to see four layers of the docker image here now the very first is Ubuntu so Ubuntu 18.04 we are pulling up and then we are pulling like uh the file over here and uh then we are running the make utility and uh once that is done then we are running that file dot uh python file over here in this one so these are four instructions which is available right so uh which will definitely help us to do that how exactly we can make or we can perform the different activities right from the base so in this case the base layer will be like uh the Ubuntu one so one to one will be the base image and the moment the final CMP attribute or CMD uh attributions are processed and executed after that you will be able to get a final full flash Docker image over there so this will be a full complete Docker image or Docker container which you will get once the whole build process is completed now in this ways you have you can see that the previous Docker file which we processed so the base layer is Ubuntu 18.04 we are not building that actually it's automatically it's uh it's by default getting downloaded as in full base layer after that only we have like layer one layer 2 layer 3 added up because those modifications we are doing it after the uh because from does not means that you are doing any kind of layer modifications it's just like you are importing some base operating system or base image so the modification or adding up the layers is only done after the base layer there so whenever a user creates a container a new layer is something which is stopped on top of the image these layers are known as the containers layers so one to one the very first instruction does not comprises it it is it's not a com kind of a container layer we can call it as an base image layer there so every container has a separate read write container layer and any modification in a container is reflected upon the container layers only so layer 1 may be having some different of uh instructions layer to maybe having different instructions so like that it's a step-by-step processor in case you know whenever our container is deleted the top layer also gets deleted over there in that one so if I delete some uh particular containers and that is something which is gone so it was suppose having like three layers so the top layer will be simply deleted because the container is also deleted for that particular part now Docker uses a copy and write strategy with both Docker images and Docker containers so uh whenever we uh perform any activity or whenever we copy on uh write a strategies and strategy to share and copy files for better efficiency so it's something which is being used to increase the overall performance of the docker containers it also makes Docker efficient by reducing the uses of disk space because whenever uh required okay whenever is needed only at that moment of time it's actually performing the copy and uh it it definitely reduces the overall disk utilization and this is very important also because when you have so many document containers up and running so you have to worry or you have to make the utilization of a disk or your disk users or resources also minimum so that is also you need to make sure that it does not exceed a particular value so once the utilization is also less then definitely the overall performance will also be better and improved over here in this case so uh let's talk about some of the facts here so the base layers are something which is purely read-only format so you cannot do the modifications Union file system saves the memory uh you know by avoiding the duplications of the files the layers can be combined in a union file system to create a single image so all these layers are combined and got a final product are called you know you can call it as Docker image or Docker containers and this allows a file system to appear as a writeable which is known as copy on right so copy and write strategy is being used so to do the changes or to do the perform the modifications of this Union file system file creating the different different layers so now let's talk about the docker registry very important component because even if you prepare the docker image locally and you don't have the collaborative tool or you don't have the tool where you can share these images definitely it's not easy for you to Port these Docker images from one system to another so Docker industry is very important for the portability feature of Docker so Docker registry is a service to host and distribute Docker images among users so uh it can be built up by one user and multiple users can use it for example I'm using a Jenkins one so um shenkins image is prepared only once but it's being used by different people as such Docker registry again it's a repository is a collection of Docker images so in a repository you will be having like different different uh Docker images as such so in registry a user can distinguish between the docker images with their tag names so uh for example Jenkins is an a kind of a single image they might be different type of images there let's say that okay let's talk about uh Jenkins uh GDK so there will be image called Jenkins the name of the image will be same but the tags will be different so one tag could be like jdk8 one tag would be like jdk 9 and one time would be jdk 10. so the difference between all these three Docker images is that that jdk81 is prepared on gdk8 it's running on jdk8 and if you want to use jdk 9 in that case you have to pull down the the docker image of Jenkins with the jdk 9 installed so depending on the framework so depending on the runtimes so with these tags different tags are processed so you can have the distinguish between the uh Docker images and whichever Docker image you feel you want to use you can Implement according to your requirement and uh Docker by default is having its own uh cloud-based registry called as Docker Hub which gets automatically connected when the moment you do the docker installation but in case you are working into your private infrastructure so in that case you may use the repositories uh Docker repositories on the specific jfrog artifactory or the private repositories also can be hosted but Docker Hub is something which is having a public and private repositories both so you can create either the public and private repositories you can use uh push and pull activities to interact with these Docker images you want to download a Docker image so you can run pull if you want to push your Docker image to the docker registry in that case you can run the push activity so in both the cases you are doing the interaction with the docker Hub so when you run some build process or when you run the containers there may be some Docker containers which is getting pulled down from the specific registry from the docker Hub and uh if that specific image and tag is available definitely pull command will be successful but in case of push command you are pushing your custom Docker images uh you cannot push the ready-made Docker images which is available you can publish your own Docker image only to the docker Hub and then you can further on do the pull and push activity accordingly to those repositories so whatever the ready made and third party images which is available you can only do the poll activities to them you cannot do the push activities push command scenario will only come when you are looking when you are interacting with your custom Docker image next one is the docker container now Docker container is an executable package of an application and its dependencies together so when we prepare a Docker container we actually combine both the dependencies and the application source code inside it so the docker container don't have to look for the dependencies on the host machine on the server so this is one of the benefit where we see that the portability is very easy because whatever I'm building up on one system everything is inside the docker container so I don't really need to do any further activity on other machines so whatever works on one system can be easily worked on another system so it's very lightweight also so it can be easily deployed and executed on you know on their particular uh servers because it does not take that much utilization like a virtual machine so if you are looking forward for having your application hosted into your local system you can easily do it within the dock container so the utilization is very less as compared to the virtual machine so you don't have to worry about your operating host operating system and the qualifications so Docker containers can run in totally isolation just like a virtual machine so there is no interaction or there is no interface so they share the OS called uh kernel but uh the thing is they don't have the interface as such over there so it works totally in isolation uh two different containers will not have any interaction unless and until we do we set up that window containers to have the interactions now there is uh data volumes which can be shared and uh be used among the multiple containers so um you have the capability over here that you know if uh the there are some on used uh space that can be easily shared across the multiple containers here so uh it is uh built using the docker images so uh using the docker images the docker containers are built up and uh Docker run is a specific command which is used to build a Docker container so if you got a Docker image let's say that I have prepared a Docker image a custom Docker image using the docker file now if I need to create a Docker container out of a Docker image I have to run a docket on command so Docker run command is nothing but it initializes or creates an instance of a Docker image now next component is the docker compose now typically when we work with the the docker containers so we had to run like if I want to run like five six containers I have to initialize like five six times or I have to run Docker and commands five six times there so and if you want to create a relationship between the containers then definitely that is something which you need to manage using the docker commands but sometimes it's not an easy task for you to run those commands manually or remember those commands so Docker compose is a utility over here using which you can actually run multiple containers with a single service so you just got off a kind of a single file you just run it and all the containers definitions and relationships everything is inside this Docker file here so you want to run two containers that's totally fine instead of running two commands now you just have to run one file and that will do take care of that now you may be having more than two containers five six seven whatever number of containers you have that can be easily deployed without any manual activity or without any maintenance onto the server so these containers totally run in isolation but yes you can establish the interaction that how they want to uh interact with each other and that interaction you can put up in these uh Docker compose files so Docker compose file is something which is written in a viable format and in this you define that okay I need two containers and this in this image should be there these are the parameters which you need to run and if you want to have the interaction or the linkage between the images that also you can establish within this yml files or this in within this Docker compose file here right so let's say for example I want to have an application which requires Apache web server and mySQL database now I can create a Docker compose file which can run both containers and uh without you know you don't require to restart or start them separately or initialize them separately so I will be having a final Docker compose file which is having like a uh you know containers like Apache containers and mysq containers both inside a single Docker compose file here so that's the benefit you are going to get because now you don't have to run them separately separate Apache and separate viable MySQL so you can get it done with the help of a single Docker compose file here now what exactly is the docker spam now Docker spam is a in build or it's a default oxygen tool which is being provided by the docker itself it's a service for containers which allows the it administrators and the developers to create and manage a cluster of swam nodes so um in a typical world when you do the installation of Docker there is a limitation that you some of the features like high availability multiple uh instances or scalability all these things are not possible as it is with the code Docker now on top of that you require the organization tools like Docker spam where you have a swam of node machines uh five six known machines are there on which you can have your highly available application infrastructure implemented now this one means that if I want to run my applications five containers six containers and seven containers for that purpose we are using the docker spam so each node of Docker swam is having a Docker demon uh which is uh used to for performing the interaction so there will be a master node which is available so uh it will be interacting with all the known machine on the all the client machines and this interaction will be in such a way that it will let them know okay I want to deploy the container or onto the Note 3 so it will perform that activity and the container will be up and running on the Note 3 there so a swam consists of two type of nodes one is the manager node and one is a worker node so manager is the master node which you can refer and the worker node is the one which is actually doing most of the work so uh the manager node is not something which is expected to host uh anything apart from the main component but yes uh you can always configure if you feel that you want to only deploy the container on the worker node so that also you can do during the deployment of the containers so manager will will keep on connecting to the different worker nodes there and whenever new request comes in so it will basically try to connect and see that okay I just want to go ahead and deploy a container so it will do the deployment of the container on the worker node so the main work will be done by the worker nodes and manager is just doing the management of all these resources the worker nodes will be definitely receiving the request from the manager and it will be proceeding and Performing those activities and uh if for some reason worker node is not active or online so the master node will not be able to deploy the containers on that server so right now let's talk about a demo in which we have to create a Docker image using a Docker file on the open220 so if you want to use a base operating system as an Ubuntu 20 there and using that we will we are going to create a custom Docker image so let's connect to our virtual machine and see that how we can do that image of Ubuntu so let's take an operating system of Ubuntu 20 20 is the latest version of the Ubuntu operating system so I have got a server so let's see so first of all we have to install the docker.io package which will install the docker package and then we can enable the uh service we can make sure that the docker service is up and running because if the service itself is down then we will not be able to perform any activity and then our sample if you run Docker run hello hyphen volt so that is going to pull a Docker image from the docker Hub and a particular activity will be done to you so once that is done so we can say that okay the container is up and running and everything has got uh you know over there up and running and then you can also try some of the Ubuntu Docker image also if you feel that you want to try you can do that but the main uh demo which we need to do over here is that we need to prepare a custom Docker image and in this one we will be having like we will be using an Ubuntu base operating system on top of that we will be installing PHP nginx all these different components so we are going to have a PHP based website hosted onto the Apache web server as an nginx and then we will be able to access that application so the very first thing which we need to do is that we need to create a folder let's say that I just created a particular folder called nginx iPhone image and then I create a Docker file over there so Docker file uh content we are going to take from this one so let's go to the server and see that how we can go ahead with the activity now uh what we can do is that uh What uh we have got a server over here now in this server what I'm going to do is that I am going to proceed with the installation of Docker first now when I do the docker installation first so what will happen that I will be able to have my Docker installation completed and then further on I can take the further steps of having the accessibility of going through the understanding that what is this particular platform or how to prepare different different kind of containers so I have got a particular container over here so what I can really do is that I can simply say that okay I have got a server so this is a one two server so I'm going to run the command called app update first now this will do the update locally so all the links and everything will be updated now the whole idea about doing the updates is that so that I will be able to have establish the fact that I'm trying to connect I'm trying to connect to the Ubuntu URL so that at any moment of time I should be able to fetch on the latest packages now once that is done so I'm going to run the command called apt install Docker dot IO now the moment I do this what will happen that it will try to download some packages it will ask for the confirmation you can give your confirmation and I can say yes the moment you give the confirmations the download of the packages will proceed over here step by step it will download the packages and the installation will go on now once the installation is done so what you can do is that that you can see that Docker is deployed into your system or not you can run Docker version command or you can also use the service command to check to make sure to ensure that the docker service is always up and running if the service is not up and running you have to make sure you have to enable it you have to start that service so these are some couple of steps which you need to do in order to make sure that your connectivity or your environment is ready for with the docker installation and once the docker installation is done then only you will be able to proceed further with the further activities like preparing the containers preparing the images and all that stuff but if you don't have the docker installed prior to that then definitely it's going to be a problem so just make sure that before even doing any kind of activity before even doing any kind of steps we ensure that the docker is fully up and running so it's going to take some time to download the packages and we will wait for that and once the installation is done we will continue further on that part okay now once the installation is done so you can run the command called service Docker status and when you do that uh it will let you know that whether the docker service is available there or not so uh you can start the service in case you don't get uh the proper response and uh now you can see that it's showing up that a process is running so earlier it was not uh up and running it was inactive but now it's active and you can see here the dock ride is the process which is running over here as in a process to initialize the docker demand so Docker demon is up and running now and we are good to go here so now uh if we go and say like Docker run hello hyphen world so we are doing a validation to our Docker instance that whether our Docker instance is working fine as expected or not because uh if there is any kind of problem with the docker demon and Docker demon it's not up and running properly we would be able to get an error message over there but right now there is no error message which represents that uh yes you are pretty much running the docker image and it's giving you the proper response so you will be able to see the uh Docker images when you run so you will be able to see that the docker image is pulled down to your system and definitely you have executed that and it got you the response now the next thing which you need to do is that we need to run a particular Ubuntu one so for example if I go for docker run hyphen ID 0.2 bash so what exactly we are doing in this one is that we are pulling down a Ubuntu Docker image and we are running the bash executable on top of that the whole purpose of doing that activity is that so that if I want to do some uh process or I want to perform some activity I should be able to connect to that Docker image so the moment I do this first of all uh if this Docker image is available locally that's great it will immediately create the container if the docker image is not available locally the very first thing which it will perform over here is to pull down the docker image the correct one and as you can see here that it uh definitely pulls down the docker image and once it is done so it got a terminal for you now you can see it shows that root and the ID root at the rate one ID is there this ID is nothing but it's a container ID which we are getting here so definitely you are inside the docker container and when you run the docker when you run the exit command you will see that you have come out of the docker container over here and the moment you run Docker PS hyphen a command you can actually see also that how many container you deployed so far so you run hello world you run the Ubuntu One both of them are stopped there you can see the exit 0 in the uh status over here because both of them are something which you ran in the interactive mode so they are done so if you want to keep your container up and running you need to have a container which runs in the dash mode and always up and running but for that you have to go ahead to the level where we need to create a custom Docker image and do the setup accordingly so let's see that how we can do that now I'm going to create a particular folder called nginx hyphen image you can pretty much give any name whatever name you feel you want to perform you can do that now in this one I'm going to put up a file called Docker file so let's see that what content we need to put as such when we talk about the docker file because whatever content you put over here that is what according which the docker image will be prepared now here we have to talk about so first we have to see a base image so we are saying that okay Ubuntu 20.04 is what I want to use then we are giving some labels now these labels are used to give some information like uh what is the version Who is the maintainer what is the description uh you can do the customizations to this one so these labels are actually defined and utilized so that I will be able to give an information that okay so and so developer is something which we can contact so if there is any kind of problem with the uh Docker file or will this build script we have to reach out to that specific person so this is what we typically do with the labels here now um we also have to set up a parameter like devian underscore fronted as in non-interactive because whenever we talk about any build process in Docker file it it cannot be there in the interactive session it should be there in the non-interactive and definitely a typical uh scenarios like you can run app update command and then we are installing the PHP nginx super supervisor all these things we are doing we are doing the installation uh we are using the Run attribute run a particular step and in there whatever the commands are executed these statements will be executed when we will be doing the build process and then we are using some environment variables okay so nginx via virtual host PHP underscore conf nginx underscore conf and supervisor underscore cons these are the different environment variables which we configuring over here now next thing which we have is that uh we have to transfer some files like probably nginx and the virtual host file we may have to transfer from a Linux server to the destination to the docker container so for that we are going to run some copy commands copy statements so that we can do that stuff and uh then definitely we are transferring the supervisor configuration file also to the server so that we can have that configurations set up over there and uh these uh config supervisor underscore conf and all these different variables we have already established where we are saying that what exactly configuration we are looking forward over here and then we are you know creating in some folder structure we via WWE HTML folder structure we are creating so that we will be able to push some content over there and then we have the volume configurations so whatever the volume configurations are there we are putting up so that uh you know we will be able to have all the uh specific direct directories mounted on the host machine on the host server so that we can even talk about locally also and then the star.sh is something which we need to run because that's uh that will go into the command statement which needs to be executed because when you want to run your process when you want to run your nginx process or your Docker container some process should be up and running that's where uh if any interactive process will run then only the docker container will be hold up so that's what you can do do or you can configure over there so I'm going to copy this whole content now this whole content represents the complete Docker file so uh we have already gone through this content one by one but uh if we combine all these content together till the exposed part that's where we come up into these ones so exposed we are doing 80 and 443 over here right after this one we have to create a host file virtual host file supervisor D file and the star.sh file so let's go back to the server here now in this one you have to uh you can use whatever editor you feel you want to use you can do that uh so this is what I will my master file will look like you can see I'm running some run commands uh these are the environment variables I'm using then further on this variables I'm using just to define the paths okay so you can see that this default file we are copying from the host machine from the server to the docker container and then second file is supervisor D not confile and then we have star.sh file so these are the different files which we want to configure here so once this file is prepared we have to uh type in or we have to create a default file over here in this current directory so in this current directory we have to put up a file called default now in this one we will be having the default virtual host configuration config so we need to put our server attribute saying that we have to look forward for index.html or index.php and these kind of files from the Via ww HTML directory there so I'm just going to copy this content and uh put it up into this one so this is how my final file look like so I'll save it and uh then we have a supervisor d.com file so I'll just copy the content of this one I'll copy it and we have to create a file called supervisor D dot confile now names we can double check because we are referring these names into our Docker file so we can double check that so once this is done save it and we can check the docker file you can grab water so you can see okay this one so super visor D so the name is correct that's cool so we got default we got supervisor D now we have to create the star.sh file because in this one we will be configuring that how we want to start our process of this nginx or this web server so in this one again we have to run the supervisory process over here and a configuration file so we are going to put this one and we'll go here and put this content right so we got the supervised config also available so that's also there and the moment is done so you are going to give the execute permission to this one because um you can see that into our Docker file that we are having the attribute where we want to run this into the dots slash there so if we want to refer in that mode you have to change the permissions in case so that uh change mode you have to give the execute permission to the start.sh here now once this is done so what you got is that you got a particular components like you got a some couple of commands you have uh you know uh got Docker file supervisd.com default all these files are there now the only thing which we need to do next is that we need to build to the build process so uh if we look forward for the build process what happens the docker build is the command which we need to use and uh nginx iPhone image is the name which you want to refer now you can totally use this uh custom name you whatever name you feel that you may want to use you can always use that there's no restriction on that part and once that is on so you will be able to list down those images that you've got a new custom Docker image here so you can see that you got the Ubuntu as on base image which is built down and the nginx iPhone image is also prepared over there and once that is done so you are going to create a Webroot folder and uh you can run the docker run command to see that if you really got connected with that particular one so let's go back and uh see that how we can create uh the docker custom Docker image here so the moment I do this so it will pick up the docker file over here and start performing the activities uh the package installations all those configurations ones will be performed over here in a non-interactive mode there is no confirmation which will be asked from the user because because you cannot put that confirmations so the command should be in such a way that it should not take any kind of confirmations at all from the end user so because tomorrow you may be putting up these things in the Jenkins so that will not be a feasible thing for you to put up the commands and put up the confirmations during the execution so so it's trying to do the package installation for the PHP so fpm packages are getting installed and um here we go so we caught the successful message now you can see it says successfully tag this one this means that there is no issues at all with the docker image and when you run the docker images or Docker images LS you can run these two couple of commands so when you do that you got a 203 37 MB of the size of this image that represents that this Docker image is fully ready to run to execute so now what we are going to do is that we are going to map a volume over here with the VAR WWE HTML directory during the execution of this container because I really want to deploy some pages and web pages there if you have the web pages now as part of the build process as part of the docker find you can do a git clone also of some repository but I don't want to do that uh during the preparation of Docker image I just now got an environment where the nginx is set up properly and it's up and running there now it should be upon me that how I want to put up my content and all so that's the reason why I'm putting up a kind of a folder structure over here and sharing it with the docker container so that the moment I put the the HTML files or the PHP files into that local folder I should be able to get some web page up up and running there so that's what I'm going to do over here in this one so I'm going to create a folder structure saying slash web root here okay so there is a folder structure which is created now in this Webroot I will be putting up some files and then same files will be redirected inside the docker container with the help of Docker volume so let's go back to the document now okay you have to run uh the uh once like I said that you are going to connect on that part okay so uh you can use this port as an at also but here it is you so whatever Port you feel you want to use you can do that and we are giving the container name as test and iPhone container that also you can change nginx iPhone image is the one which we are using so if you use some different uh particular image you can also use that and uh you can see that the web root is something which we are connecting with the Via WWE HTML directory so the moment you put some file in the web root the same file will be reflected into the WW HTML directory there so I'm going to run this command and once this is done so you can see that for the testing purpose we are putting up some content in the HTML one so I'll come to that point also and we'll validate as of now we should be having our container up and running over here in this one now I'm going to copy that command okay and hit on so if uh the 80 Port is used so you have to make sure that photo gets uh you know gets stopped automatically or it should not be up and running so just double check whatever code you are using that should be free now when you run the docker PS command so uh you will be able to see a Docker container up and running but if you see like Docker psyphony that gives also these stopped and running both the containers are there but the main thing which we want to run is that we want to run the docker PS command and you can see that this is a container which is running so 8080 is the container which is running and test hyphen container is the name now you got a container up and running so you can actually do a curl on localhost colon 8080 so um this is a way to get the content uh you can get the public IP also and uh in the browser you can put up and you can have the content there but right now we are getting the response at least uh it's showing as 403 error forbidden error but that's fine we will be putting up some web pages there in the Via WWE HTML tree or we can say in local Webroot directory so that we will be able to have some content so what we can do is that we can have some Eco commands to execute it so that we can have some added some content added up into the uh particular uh files there so let's see that what different contents we can do so uh you can use the vi mode also to do a uh you know couple of files so I'm just using this Eco command so um the moment I go for this execution so what happens that a file will be created into that uh Slash web root there so um I'm expecting that I have put up a index.html page so if you feel that you want to put some different page you can always do and the moment you do the curl now you will be able to see the same response so whatever the things you put up in the index.html file the same content is going to pop up when you run the curl command now um you want to use some PHP files you want to put up some PHP content all that stuff you can do it now what I'm going to do is that rather than just a HTML static contain I'm going to put some PHP content that is going to help me to get a file so my Webroot will have now two files so uh if if you feel uh that index.html file should have some uniform content you can always do that you can pull some of the websites also you can put that into the you can do a git clone in the brute and the whole application will be started and now if you feel that I got a info.php file so I can use the NAT name into the URL I have to modify the URL because uh index.html file is the what as in default name page you do if you want to put up some different name you can change the server configuration of nginx and according to that it will work so um when I do this so you will start getting the output so this is the PHP content output which is there it's a complete information now it may look like a very bulky content to you but when you put up this one into the browser into the web browser that will be a meaningful information probably in terms of some tables or something like that right so um let me scroll to the top here so you can see that it's showing an error sorry it's showing in the output and this output represents it you've got a particular PHP content here so this is how we will be able to have a container and uh you know this container is up and running you can modify it at any level you feel that you want to prepare a Docker image with the code embedded also you can have the fit Chrome or you can have this configuration into some git repository have the source code also there and then prepare a complete bundle all together so this is what we have in this demo how we can deploy a container and a container up and running in case of the docker yes today we're going to cover what is a Docker container why you would want to consider using it in your devops environment so let's have a look at what we have in our current environment so today when you actually have your standard machine you have the infrastructure you have the host operating system and you have your applications and then when you create a virtual environment what you're actually doing is you're actually creating virtual machines but those virtual machines actually are now sitting within a hypervisor solution that sits still on top of your host operating system and infrastructure and with a Docker engine what we're able to do is we're able to actually reduce significantly the different elements that you would normally have within a virtualized environment so we're able to get rid of the the bins and the so we're able to get rid of the guest OS and we're able to eliminate the hypervisor environment now this is really important as we actually start working and creating environments that are consistent because we want to be able to make it so it's really easy and stable for the environment that you have within your Dev and Ops environment now critical is getting rid of that hypervisor element it's just a lot of overhead so let's have a look at a container as an example so here we actually have a couple of examples on the right hand side we have different containers we have one container that's running Apache Tomcat with Java a second container is running SQL server and microsoft.net environment the third container is running python with mySQL these are all running just fine within the docker engine and sitting on top of a host OS which could be Linux it really could be any host OS within a consistent infrastructure and you're able to have a solution that can be shared easily amongst your teams so let's have a look at an example that you'd have today if a company is doing a traditional Java application so you have your developers working in JBoss on his system and he's coding away a and he has to get that code over to a test and now what will happen is that tester will then typically in your traditional environment then have to install JBoss on their machine and get everything running and cool and hopefully set up identically to the developer chances are they probably won't have it exactly the same but they're trying to get it as close as possible and then at some point you want to be able to test this within your production environment so you send it over to a system administrator who would then also have to install JBoss on their environment as well yeah this just seems to be a whole lot of duplication so why go through the problem of installing JBoss three times and this is where things get really interesting because the challenge you have today is that it's very difficult to almost impossible to have identical environments if you're just installing software locally on devices the developers probably got a whole bunch of development software they could be conflicting with the JBoss environment the tester has similar testing software but probably doesn't have all the development software and certainly the system administrator won't have all the tools of the developer and test to have their own tools and so what you want to be able to do is kind of get away from The Challenge you have of having to do local installations on three different computers and in addition what you see is that this uses up a lot of effort because when you're having to install software over and over again you just keep repeating doing really basic foundational challenges so this is where Docker comes in and Docker is the tool that allows you to be able to share environments from one group to another group without having to install software locally on device you install all of the code into your Docker container and simply share the container so in this presentation we're going to go through a few things we're going to cover what Docker actually is and then we're going to dig into the actual architecture of Docker and kind of go through what Docker container is and how to create a Docker container and then we'll go out through the benefits of use using Docker containers and then the commands and finalize everything out with a brief demo so what is darker so Docker is as you'd expect because all the software that we cover in this series is an open source solution and it is a container solution that allows you to be able to containerize all of the necessary files and applications needed to run the solution you're building so you can share it from different people in your team whether it's a developer a tester or system administrator and this allows you to have a consistent environment from one group to the next so let's kind of dig into the architecture so you understand why Docker runs effectively so the docker architecture itself is built up of two key elements there is the docker client and then there is a rest API connection to a Docker Daemon which actually hosts the entire environment within the docker host and the docker demon you have your different containers and each one has a link to a Docker registry the docker client itself is a rest service so as you'd expect a rest API and that sends command line to the docker Daemon through a terminal window or command line interface window and we'll go through some of these demos later on so you can actually see how you can actually interact with Docker the docker Dem then checks the request against um the docking components and then performs the service that you're requesting now the docker image itself all it really is a collection of instructions used to create a container and again this is consistent with all the devops tools that we have the devops tools that we're looking to use throughout this series of videos are all environments that can be scripted and this is really important because it allows you to be able to duplicate and scale the environments that you want to be able to build very quickly and effectively the agile container itself has all of the applications and the dependencies of those applications in one package you kind of think of it as a really effective and efficient zip file it's a little bit more than that but it's one file that actually has everything you need to be able to run all of your Solutions the actual Docker registry itself is an environment for being able to host and distribute different Docker images among your team so say for instance you had a team of developers that were working on multiple different solutions so say you have a team of developers and you have 50 developers and they're working on five different applications you can actually have the applications themselves the containers shared in the docker registry so each of those teams at any time check out and have the latest container of that latest image of the code that you're working on so let's dig into what actually is in a container so the important part of a docking container is that it has everything you need to be able to run the application it's like a virtualized environment it has all your Frameworks and your libraries and it allows the teams to be able to build out and run exactly the right environment that the developer intended what's interesting though is the actual applications then will run in isolation so they're not impacting other applications that using dependencies on other libraries or files outside of the container because of the architecture it really uses a lot less space and because it's using less space it's a much more lightweight architecture so the files and the actual folder itself is much smaller it's very secure highly portable and the boot up time is incredibly fast so let's actually get into how you actually create a Docker container so the docket container itself is actually built through command line and it's built of a file and Docker image so the actual Docker file is a text file that contains all the instructions actions that you would need to be able to create that Docker image and then we'll actually then create all of the project code with inside of that image then the image becomes the item that you would share through the docker registry you would then use the command line and we'll do this later on select Docker run and then the name of the image to be able to easily and effectively run that image locally and again once you've created the document you can store that in the docker registry making it available to anybody within your network so something to bear in mind is that Docker itself has its own registry called Docker Hub and that is a public restrict so you can actually go out and see other doc images that have been created and access those images as your own company you may want to have your own private repository so you want to be able to go ahead and either do that locally through your own repository or you can actually get a licensed version of Docker Hardware you can actually then share those files now some something it's also very interesting to know is that you can have multiple versions of a Docker image so if you have a different version control different release versions and you want to be able to test and write code for those different release versions because you may have different setups you can certainly do that within your Docker registry environment okay so let's go ahead and we're going to create a Docker image using some of our basic Docker commands and and so they're essentially really you know kind of just two commands that you're going to be looking for one is the build command another one is to actually put it into your registry which is a push command so if you want to get a image from a Docker registry then you want to use the pull command and a pull command simply pulls the image from the registry and in this example using ngi next as our registry and we can actually then pull the image down to our test environment on our local machine so we're actually running the container within our Docker application on a local machine we're able to then have the image run exactly as it would in production and then you can actually use the Run command to actually use the docker image on your local machine so just say you know a few interesting tidbits about the docking container once the container is created a new layer is formed on top of the docker image layer is called the container layout each container has a separate read write container layer and any changes made in that docking container is then reflected on the particular container layout and if you want to delete the container layer the container layer also gets deleted as well so you know why would using Docker and containers be of benefit to you well you know some of the things that are useful is that containers have no external dependencies for the applications they run once you actually have the container running locally it has everything it needs to be able to run the application so there's no having to install additional pieces of software such as the example we gave with JBoss at the beginning of the presentation now the containers are really lightweight so it makes it very easy to share the containers amongst your teams whether it's a developer whether it's a tester whether it's somebody on your operations environment it's really easy to share those containers amongst your entire team different data volumes can be easily reused and shared among multiple containers and again this is another big benefit and this is a reflection of the light weight nature of your containers the container itself also runs in isolation which means that it is not impacted by any dependencies you may have on your own local environment so it's a completely sandboxed environment so some of the questions you might ask us you know can you run multiple containers together without the need to start each one individually and you know what yes you can with Docker compose docking compose allows you to run multiple containers in a single service and again this is a reflection on the lightweight nature of containers within the docker environment so we're going to end our presentation by looking at some of the basic commands that you'd have within Docker so we have here on the left hand side we have a Docker container and then the command for each item we're actually going to go ahead and use some of these commands and the demo that we're going to do after this presentation you'll see that in a moment but just you know some of the basic commands we have are committing the docker image into the Container kill is a you know standard kill command to you know terminate one or more of the running containers so they stop working then restart those containers but suddenly you can look at all the image all of the commands here and try them out for yourself so we're going to go ahead and start a demo of how to use the basic commands to run Docker so to do this we're going to open up terminal window or command line now depending whether you're running Linux PC or Mac and we're going to go ahead and the first thing we want to do is see what our Docker image lists are so we can go sudo Docker images and this will give us well first of all enter in our passwords let's go enter that in and this one that'll give us a list of our Docker images and here are the docker images I have already been created in the system and we can actually go ahead and actually see the processes that are actually running so I'm going to go ahead and open up this window a little bit more but this will show you the actual processes and the containers that we actually have and so on the far left hand side you see under names we have learned simply learn be unscore cool these are all just different ones that we've been working on so let's go ahead and create a Docker image so I'm going to do sudo docker run Dash D Dash p 0.0.0.0 go on a t colon 80 Ubuntu and this will allow us to go ahead and run an Ubuntu image and this will run the latest image and what we have here is a hash number and this hash number is a unique name that defines the container that we've just created and we can go ahead and we can check to make sure that the container actually is present so we're going to do pseudo.go.ps and this actually show us some there so it's not in a running state right now but that doesn't mean that we don't have it so let's list out all the containers that are both running and in the exit state so let's do sucker PS Dash a and this lists all the containers that I have running on my machine and this shows all the ones that have been in the running State and in the exit State and here we see one that we just created about a minute ago and it's called learn and these are all running Ubuntu and this is the one that we had created just a few seconds ago let's open it up and there we go so let's change that to that new dock container to a running state so scroll down and we're going to type sudo docker run Dash it Dash Dash name my um so this is going to be the new container name it's going to be my Docker so this is how we name our Docker environment and we'll put in the image name which is Ubuntu and dash bin Dash Bash and it's now in our route and we'll exit out of that so now we're going to go ahead and start the new my Docker container so sudo docker start my and we'll get the container image which will be my docker my docker return and that started that Docker image and let's go ahead and check against the other running Docker images to make sure it's running correctly so sudo docker PS and there we are underneath name on the right hand side here you see my Docker along with the other Docker images that we created and it's been running for 13 seconds quite fast so we want to rename the container let's use the command sudo docker rename we can take another Docker image this to scrub this one and we'll put it in rename and we'll rename I'm putting the old container name which is image and then we'll put in the new container name and let's call it purple so now the container image that had previously been called image is now called Purple so do sudo Docker PS to list all of our Docker images and we scroll up and there there we go purple how easy is that to rename an image and we can go ahead and use this command if we want to stop container so we're going to write sudo docker stop and then we have to put in the container name and we'll put in my Docker the container that we originally created and that image has now stopped and let's go ahead and prove that we're going to list out all the docker images and what you see is that it's not listed in the active images it's uh not on the list on the far right hand side but if we go ahead and we can list out all of the docker images so you actually see it's still there as an image it's just not in an active stages what's known as an exit state so here we go and there's my Docker it's in an exited state so that happened 27 seconds ago so if we want to to remove a container we can use the following command so sudo docker RM will remove my docker and that will remove it from the exit state and we're going to go ahead and we're going to double check that and yep yep it's not not listed there under exit State anymore it's gone and there we go there that's where it used to be all right let's go back so if we want to exit a container in the running state so we do sudo kill and then the name of the container I think one of them is called yellow let's just check and see if that's going to kill it oops no I guess we don't have one called yellow so let's find out name of container that we actually have so sudo docker kill oh we're going to list out of the ones they're running oh okay there we go now yellow isn't in that list so let's take I know this takes simply learn and so we can actually go ahead and let's write sudo Docker kill simply learn and that will actually kill an active Docker container boom there we go and we list out all the active containers you can actually see now that's they simply learn container is not active anymore and these are all the basic commands for Docker container hey everyone welcome to this on kubernetes full course in this full course we will learn everything about kubernetes in detail we will begin this full course video with understanding what exactly is kubernetes is next we will see how to install kubernetes and the differences between kubernetes and Docker then we will look into the services and components of communities in detail after that we will be discussing everything about the architecture of kubernetes then we will see how kubernetes can be implemented on AWS finally we will conclude this video by discussing the important kubernetes interview questions and answers for beginners and experienced candidates by the end of this video I can assure you that all your kubernetes career related queries will have been answered so let's start with an exciting video on the configuration management tool before we begin make sure to subscribe to our YouTube channel and hit that Bell icon to never miss an update from Simply learn hello and welcome to Simply learn in this session we're going to cover what kubernetes is and why you would want to be using it within your devops team but before we get started remember to hit the like button if you like this video and the Subscribe button if you want to get notified about more of these videos as they come out and as always if you have any comments about the content that we're covering or questions that you have on the subject please post them in the comments below so let's get started we're going to break up this presentation into four key areas we're going to talk about life before kubernetes which some of you are probably experiencing right now what is kubernetes the benefits that kubernetes brings to you particularly if you are using containers in a devops environment and then finally we're going to break down the architecture and working infrastructure for kubernetes so you understand what's happening and why the actions are happening the way that they are so let's jump into our first section of life before kubernetes so the way that you have done work in the past or you may be doing work right now it's really building out and deploying Solutions into two distinct areas one is a traditional deployment where you're pushing out code to physical servers in a Data Center and you're managing the operating system and the code that's actually running on each of those servers another environment that you may potentially be using is to Blind code out to Virtual machines so let's go through and look at the two different types of deployment that you may be experiencing when you have applications running on multiple machines you run into the potential risk that the setup and configuration of each of those machines isn't going to be consistent and your code isn't going to work effectively and there may be issues with uptime and errors within the infrastructure of your entire environment there's going to be around problems with resource allocation and you're going to have error issues where applications may be running effectively and not look not effectively and not load balance effectively across the environment the problem that you have with this kind of infrastructure is that it gets very expensive you can only install one piece of software one service on one piece of Hardware so your Hardware is being massively underutilized this is where virtual machines have become really popular with a virtual machine you're able to have better resource utilization and scalability a much less cost and this allows you to be able to run multiple virtual machines on a single piece of Hardware the problem is is that VMS are for virtual machines are not perfect either some of the challenges you run with VMS is that the actual hardware and software need needed to manage the VM environment can be expensive there are security risks with virtual with VMS there are security risks with VMS there have been data breaches recorded about solutions that run in virtualized environments you also run into an issue of availability and this is largely because you can only have a finite number of virtual machines running on a piece of hardware and this results in limitations and restrictions in the types of environment you want to be running and then finally setting up and managing a virtualized environment is time consuming it can take a lot of time and it can also get very expensive so how kubernetes well kubernetes is a tool that allows you to manage containerized deployment of solutions an inherently kubernetes is a tour that is really a Next Level maturity of deployment so if you can think of your maturity curve as deploying code in directly to Hardware in a Data Center and then deploying your solutions to Virtual machines the next evolution of that deployment is to use containers and kubernetes so let's kind of go through and look at the differences between a virtual machine and kubernetes we've got a few here that we want to highlight and you'll get an understanding of what the differences are between the two so first of all with virtual machines there is inherently security risk and what you'll find is we get dig through the architecture later in the presentation is that kubernetes is inherently secure and this is largely because of the Legacy code the Legacy of kubernetes and where it came from we will talk about that in just a moment but kubernetes is inherently secure virtual machines are not easily portable now with that said they they are technically portable they're just not very easily portable whereas with kubernetes it's working with Docker container Solutions it is extremely portable that means that you can actually spin up and spin down and manage your infrastructure exactly the way that you want it to be managed and scale it on the demands of the customers as they're coming in to use the solution from a time consuming point of view kubernetes is much less time consuming than with a virtual machine a few other areas that we want to kind of highlight from differences virtual machines use much less isolation when building out the encapsulated environment then kubernetes does for instance with a virtual machine you have to run hypervisor on top of the OS and hardware and then inside of the virtual machine you also have to have the operating system as well whereas in contrast on a kubernetes environment because it's leveraging a Docker container and or container like Technologies it only has to have the OS and the hardware and then inside of each container it doesn't need to have that additional iOS layer it's able to inherit what it needs to be able to run the application this makes the whole solution much more flexible and allows you to run many more containers on a piece of Hardware than versus running virtual machines on a single piece of Hardware so as we highlighted here VMS are not as portable as kubernetes and kubernetes is portable directly related to the use of containerization and because kubernetes is built on top of containers it is much less time consuming because you can actually script and automatically allocate resource to nodes within your kubernetes environment because it allows the infrastructure to run much more effectively and much more efficiently so this is why if we look at our evolution of the land of time before kubernetes while we are running into a solution where kubernetes had to come about because the demand for having more highly scalable solutions that are more efficient was just really a natural evolution of this software deployment model that started with pushing out code to physical hardware and then pushing code out to Virtual machines and then needing to have a solution much more sophisticated kubernetes would have come about at some point in time I'm just really glad it came back when it did so what is kubernetes does this dig into the history of kubernetes and how it came about so in essence kubernetes is an open source platform that allows you to manage and deploy and maintain groups of containers and a container is something like Docker and if you're developing code you're probably already using Docker today consider kubernetes as the tool that manages multiple Docker environments together now we talk a lot about DACA and as a container solution with kubernetes the reality is is that kubernetes can actually use other container tools out there but Docker just simply is the most popular container out there both these tools are open source that's why they're so popular and they just allow you to be able to have flexibility in being able to scale up your Solutions and they were designed for the post-digital world that we live and exist in today so a little bit of background a little bit of trivia around kubernetes so kubernetes was originally a successor to a project at Google and the original project was Google book Google Borg it does exactly what kubernetes done does today but kubernetes was Rewritten from the ground up and then released as an open source project in 2014 so that people outside of Google could take advantage of the power of kubernetes containerization management tools and today it is managed by the cloud native Computing foundation and there are many many companies that support and manage kubernetes so for instance if you're signing up for Microsoft Azure AWS Google Cloud all of them will leverage kubernetes and it's just become the the de facto tool for managing large groups of containers so let's kind of Step through some of the key benefits that you'd experience from kubernetes and so we have nine key benefits the first it is highly portable and is 100 open source code and this means that you can actually go ahead and contribute to this code project if you want to through GitHub the ability to scale up the solution is incredible um what's the the history of kubernetes being part of a Google project for managing the Google network and infrastructure kind of really sets the groundwork for having a solution that is highly scalable the out of the high scalability also comes the need for high availability ability and this is the desire to be able to have a highly efficient and highly energized environment that also you can really rely on so if you're building outer kubernetes management environment you know that it's going to be available for the solutions that you're maintaining and it's really designed for deployment so you can script out the environment and actually have it as part of your devops model so you can scale up and meet the demands of your customer then what you'll find is that the load balancing is extremely efficient and it allows you to distribute the load efficiently across your entire network so your network remains stable and then also the tool allows you to manage the orchestration of your storage so you can have local storage such as an SSD on the hardware that the kubernetes is maintaining or if the kubernetes environment is pulling storage from a public Cloud such as Azure or AWS you can actually go ahead and make that available to your entire system and you can inherit the security that goes back and forth between the cloud environments and one of the things you'll find consistent with kubernetes is that it is designed for a cloud-first environment um kubernetes as well is that it's it's really a self-healing environment so if something happens or something fails uh kubernetes will detect that failure and then either restart the process kill the process or replace it and then because of that you also have automated rollouts and rollbacks in case you need to be able to manage the state of the environment and then finally you have automatic bin packaging so you can actually specify the compute power that's being used from CPU and RAM for each container so let's dig into the final area which is the actual kubernetes architecture I'm going to cover this at a high level there's actually another video that you can that simply learn has developed which digs deeper into the kubernetes architecture and so the kubernetes architecture is a clustered based Arc texture and it's really about two key areas you have the kubernetes master which actually controls um all of the activities within your entire kubernetes infrastructure and then you have nodes that actually are running on Linux machines um outs that are controlled by the master so let's kind of go through some of these areas so if we look at the kubernetes master to begin with um we'll start with uh Etc is this is a tool that allows for the configuration and the information and the management of nodes within your cluster and one of the key features that you'll find with all of the tools that are managed within either a the master environment or within a node is that they are all accessible via the API server and what's interesting about the API server is that it's a restful based infrastructure which means that you can actually secure each connection with SSL and other um security models to ensure that your entire infrastructure and the communication going back and forth across your infrastructure is tightly secured scheduler goes ahead and actually as you'd expect actually manages the schedule of activities within the actual cluster and then you have the control and the controller is a Daemon server that actually manages and pushes out the instructions to all of your nodes so the other tools really are the the infrastructure and you can consider them the administration site um of the master whereas controller is the management it actually pushes out all of the controls via the API server so let's actually dig into one of the actual nodes themselves and there are three key areas of the nodes one is the docker environment which actually helps and manage and maintain the container that's actually inside of the node and then you have the Kubler which is responsible for information that goes back and forth and it's going to do most of the conversation with the API server on the actual health of that node and then you have the actual kubernetes proxy which actually runs the surfaces actually inside of the node so as you see all of these infrastructures are of extremely lightweight and designed to be very efficient and very available for your infrastructure and so here's a quick recap of the different tools that are available and it really breaks down into two key areas you have your kubernetes master and the kubernetes node the kubernetes Mazda has the instructions of what's going to happen within your kubernetes infrastructure and then it's going to push out those instructors to an indefinite number of nodes that will allow you to be able to scale up and scale down your solution in a dynamic way hi guys in today's video I am going to show you how to install Ubuntu on kubernetes kubernetes is an open source platform used for deployment and management of containers so let's get started now the first step is to install the necessary dependencies using two of the commands first is sudo app get update which gets all the updates and then you have to enter password so this will take a few seconds that's done the next step is to install app transport https this is basically used to make repositories via https so let's go ahead with that sudo okay that's done going ahead the next thing we have to do is to install the docker dependency using the command sudo apt install Docker dot IO you have to choose why and this will take a few minutes okay that's done so after installing the docker we have to start and enable the docker using the command sudo system CTL start Docker and pseudo system CTU enabledoc so now the docker is enabled so now we are done with the first step of installation the next step is we have to install the necessary components for kubernetes before that we have to install the curl command because current command is used to send data using a URL syntax let's install the curl command using the command sudo apt-get install code so you have to select Y and this will take few minutes [Music] so moving on the next step we have to do is download an add key for kubernetes installation from a URL so let's go ahead with that pseudo curl so this is where we get the key and then we have to add it using sudo app key add and it's done so the next step is we have to add a repository in a certain location so before doing that we have to change the permission of that file so we do that using C or sudo CH mode command okay so the power machine has changed now let's go ahead and save this file in that location first we have to enter this command into the file this is a URL save as so we have to save it in this location let's rename this to kubernetes dot list now let's see okay so here it is now moving forward now we have to check for any updates available in the next command we are going to install the kubernetes components that is cubelet cubadian Cube vector and kubernetes CNA so let's go ahead with that so this is going to take a few minutes now that's done so the next step we have to initialize the master node and to do this we have to first use a swap of command to disable the swapping on other devices so let's do that sudo swap off hyperly now let's go ahead with the initialization sudo qadian in it this is going to take a few minutes okay so before going to the next step to start the cluster we have to use these three commands let's just copy paste it okay that's done so moving forward the next step is to deploy paths using the following command [Music] now the parts have been deployed to our Network to see all the pods we have to use the command pseudo cubicle get pause they exceed the possible hello and in this video we're going to cover a common conversation which is kubernetes versus darker but before we jump into that I want you to hit the Subscribe button so you get notified about new content as it gets made available and If you hit the notification button that notification will then pop up on your desktop as a video is published from Simply learn in addition if you have any questions on the topic please post them in the comments below we read them and we do reply to them as often as we can so with that said let's jump into kubernetes versus Docker so let's go through a couple of scenarios let's do one for kubernetes and then one for Docker and we can actually go through and understand what the problem specific companies have actually had and how they're able to use the two different tools to solve them so our first one is with Bose and Bose had a large catalog of products that kept growing and their infrastructure had to change so the way that they looked at that was actually establishing two primary goals to be able to allow their product groups to be able to easier more easily catch up to the scale of their business so after going through a number of solutions they ended up coming up with a solution of having kubernetes running their iot platform as a service inside of Amazon's AWS cloud service and what you'll see with both these products is they're very Cloud friendly but here we have um Bose and kubernetes working together with AWS to be able to scale up and meet the demands of their product catalog and so the result is that we were able to increase the number of non-production deployments significantly by taking the number of services from being large bulky Services down to small micro Services being able to handle as many as 1250 plus deployments every year an incredible amount of time and value has been opened through the use of kubernetes now let's have a look at marker and see what a similar problem that people would have so the problem is with PayPal and PayPal and processes something in the region of over 200 payments per second across all of their products and PayPal doesn't just have PayPal they have Braintree and venmo so the challenge that PayPal was really being given is that they had different architectures which resulted in different maintenance cycles and different deployment times and an overall complexity from having a decades-old architecture with PayPal through to a modern architecture with a venmo through the use of Docker PayPal was able to unified application delivery and be able to centralize the management of all of the containers with one existing group the net net result is that PayPal was able to migrate over 700 applications into Docker Enterprise which consists of over 200 000 containers this Ultra ultimately opened up a 50 percent increase in availability for being able to add in additional time for building testing and deploying of applications just a huge win for PayPal now let's dig into kubernetes and Docker and so kubernetes is an open source platform and it's designed for being able to maintain a large number of containers and what you're going to find is that your argument for kubernetes versus Docker isn't a real argument it's kubernetes and Docker working together so kubernetes is able to manage the infrastructure of a containerized environment and Docker is the number one container management solution and so with Docker you're able to automate the deployment of your applications being able to keep them in a very lightweight environment and being able to create a nice consistent experience so that your developers are working in the same containers that are then also pushed out to production so with Docker you're able to manage multiple containers running on the same Hardware much more efficiently than you are with a VM environment the productivity around Docker is extremely high you're able to keep your applications very isolated the configuration for dark here is really quick and easy you can be up and running in minutes with Docker once you have it installed and running on your development machine or inside of your devops environment so if we look at the deployment between the two and the differences kubernetes is really designed for a combination of PODS and services in its deployment whereas with Docker it's around about deploying services in containers so the the difference um here is that kubernetes is going to manage the entire environment and then and that environment consisting of PODS and inside of a pod you're going to have all of your containers that you're working on and those containers are going to control the services that actually power the app applications that are being deployed kubernetes is by default an auto scaling solution it has it turned on and is always available whereas a Docker does not and and that's not surprising because Docker is a tool for building out Solutions whereas kubernetes is about managing your infrastructure kubernetes is going to run health checks on the liveness and Readiness of your entire environment so not just one container but tens of thousands of containers whereas Docker is going to limit the health check to the services that it's managing within its own containers now I'm not going to kid you kubernetes is quite hard to set up it's it's if of the tools that you're going to be using in your devops environment it's it's not an easy setup for you to use and for this reason you want to really take advantage of the services within Azure and other similar Cloud environments where they actually will do the setup for you Docker in contrast is really easy to set up you can as I mentioned earlier you can be up and running in a few minutes as you would expect the fault tolerance within kubernetes is very high and this is by Design because the architecture of kubernetes is built on the same architecture that Google uses for managing its entire Cloud infrastructure in contrast Docker has lower fault tolerance but that's because it's just managing the the surfaces within its own containers what you'll find is that most public Cloud providers will provide support for both kubernetes and Docker here we've highlighted Microsoft Azure because they were very quick to jump on and support kubernetes but the realities is that today Google Amazon and many other providers are having first level support for kubernetes it's just become extremely popular in a very very short time frame the companies using both kubernetes and Docker is vast and every single day there are more and more companies using it and you should be able to look and see whether or not you can add your own company to this list hello and welcome to Simply learn today we're going to be giving you a kubernetes tutorial we'll provide the overview of how and why you should be using kubernetes but before we get started don't forget to hit the Subscribe button and the like button if you want to get notified about additional content from the cover what kubernetes is and why you should be using it along with the features of kubernetes we're also going to take a little bit of a look at kubernetes versus Docker swarm as the two technologies do tend to get referred to quite frequently and then we're going to do a deep dive into the actual kubernetes architecture itself and how kubernetes and runs and operates within your network and then finally we're going to look at some kubernetes use cases of how you can actually apply best learnings that other companies have used with kubernetes we'll finish everything off with a demo where we actually step through and use some of the actual codes that you can actually use to run from your terminal window to actually activate kubernetes itself so with that said let's jump in so why kubernetes well kubernetes is really being built from allowing you to be able to have a more reliable infrastructure the tool itself is designed to manage the containers you have within your network it's very modular in architecture which allows it to be very easy to be able to maintain and to be able to scale and meet the demands that your current customers are offering you today the actual ability for you to be able to deploy and update software at scale is really at the heart and center of kubernetes it's really well there's one of the reasons why kubernetes was created was to be able to scale up the deployment of software into tens of thousands of networks and servers within a Data Center and then finally when you start working with kubernetes really what you're doing is laying the foundation for building Cloud native applications and part of this comes from the architect history of kubernetes which we'll get to in a little bit but it's good to know that as you're using kubernetes and containerized solutions that you're really building out an infrastructure that's designed for cloud-centric solution delivery so let's dig into why what kubernetes is so the history of kubernetes is that it was originally developed by Google Google actually used the precursor to kubernetes to manage and maintain the data structured and data infrastructure that they had within their Network they really liked it and one of the big benefits that kubernetes that Google provided is that they converted their code into an open source program that they released as kubernetes in 2014 and made an open source this means that any company can use kubernetes there's no charge for it all you have to do is know how to use the command line to actually get up and running the benefit of kubernetes being open source is that most Cloud providers now support kubernetes so whether you're using Microsoft Azure you're using Google cloud or AWS you'll find that you can actually use kubernetes and kubernetes and edit core really helped eliminate manual processes so it allowed for deploying and scaling your container-based Solutions much more effortless within a cloud environment so let's jump into the features of kubernetes and why you'd want to be using it and we have a number of features that we want to kind of Step through again the first thing that kubernetes is a strong benefit of is that it eliminates those manual processes and allows you to automate them the benefit you have from this is that you can then scale up your automation across your entire network so whether you are managing 10 nodes and within your network or whether you're managing 10 000 doesn't matter you have the ability to scale that something you simply didn't have before kubernetes is also will manage the actual containers themselves including the security and the storage and the networking within the those containers the great benefit of working with kubernetes is that not only is it a cloud-centric solution but it was really built from a core need of a secure solution so security is at the heart and center of all the work you're doing which really helps benefit any teams scaling out containers your kubernetes environment will be constantly monitored so that when you have nodes that have your containers the health of those nodes is being constantly communicated back to uh the kubernetes environment so that the entire environment has always been managed and checked so if you have any additional hosting and launching that needs to be done kubernetes will help you automate those manual processes if something does go wrong you can actually set up your kubernetes environment to do an automatic rollback again this is just allowing you to have that confidence that when you're deploying a solution that everything is going to work within that and then finally kubernetes will allow you to mount and add the storage needed to run your apps so whether you have storage such as a local SSD storage device physically on the hardware or if you're connecting to storage fire the cloud kubernetes will allow you to access the storage you need how you're accessing it so let's look a little bit into the differences between kubernetes and Docker swarm so if we just step back a little bit kubernetes and Docker often get connected with each other and largely because kubernetes manages containers and Docker is really famous for creating the most popular container management solution which is called Docker if you're a developer you're probably already using Docker today so one of the things that kubernetes is really really good at is that it manages containers at scale and just massive volume of containers Docker swarm is docker's alternative to kubernetes is a tool that allows you to also manage large number lowest kubernetes and yes the original product was by developed by Google but Google has been great stewards and they've allowed the community to drive the future evolution of the product uh Docker swarm is actually managed and developed by Docker um there is a smaller Community but it is also a very passionate community and the reason being is that both products are actually pretty good and you just really kind of have to choose which one you want to go with without doubt there is more customization and extensions built for kubernetes simply because the community is just so much bigger than the docker swarm Community but with that said Docker swarm is really easy to set up so if you are used to working within the docker environment you'll be able to set up Docker swarm very very quickly whereas kubernetes does require some heavy lifting to get up and running now with that said if you are using Microsoft Azure or Google Cloud very often those environments will actually set up kubernetes for you so you don't have to do it and you can actually then focus on writing the instructions to maintain your kubernetes environment kubernetes has extremely high voltage tolerance again it's because kubernetes is has been Road tested the foundation for kubernetes is based on managing sites such as Gmail google.com YouTube I mean just incredibly robust environment compared to Docker swarm which has lower fault tolerance um and this really kind of leads into the final areas where kubernetes does guarantee strong clusters and management just a really reliable environment Docker swarm however is able to take containers deploy them into large casters very very quickly two interesting differences is that with kubernetes the load balancing is still manual and be interesting to watch how kubernetes matures that skill set in contrast load balancing is automatic within Docker swarm so let's dig into the kubernetes architecture and just kind of expose some of those key elements um if you're interested in actually having a very deep dive into kubernetes architecture we have already got another video and we should have a link in the comments that you can select that will take you to that video as well but this will provide you an overview as well um so the way that kubernetes runs is that it uses nodes to actually manage all the individual Hardware within a specific unit and so a node is really a physical machine in a data center or a virtual machine and when you're setting up an environment on a cloud environment such as Microsoft Azure or Google cloud or AWS kubernetes will be managing the containers within that node so it's important to know that kubernetes itself doesn't work with the individual no modes but actually works with the cluster as a whole and and why that's important is that it's a tool to manage your entire environment it's not going to come down and manage the individual node itself it's going to match the entire environment the actual control of data within a cluster um it's very very flexible within kubernetes you can have local storage such as an SSD card or a traditional local storage or you can actually connect to the cloud and use cloud-based storage and the benefit that's cloud-based storage offers you is that you are increasingly Moving Your solution into more of a hardware free environment cloud-based architecture where you're not reliant on a physical piece of Hardware within your data center and why that's important is that it gives you the opportunity to scale up your Solutions much more effectively so if you need to increase the hard drive space you can do do that very easily from the cloud whereas local hardware is a little bit harder to manage great news kubernetes gives you both options to be able to choose from and so the the next layer down is the actual management of the application and that's within a container and as mentioned earlier Docker is the most proper container solution out there and again if you're a developer you're probably already using Docker as it was a natural maturity from using a virtual machine but kubernetes itself will actually manage the containers that you have within your nodes and so you have a collection of containers working together and then you pull those together into what's called a pod and what's great is kubernetes will actually manage the health of that pod so if a container or a pod fails kubernetes will automatically deploy out a replica so your entire environment can stay running and being efficient and so the pods themselves will actually be managed by kubernetes and you really want to let kubernetes do this because it's allowing you to to step away from the manual management of Parts your environment is just going to be healthier relying on kubernetes to do the actual management of all the parts you have out there so you are able to allow access to from Services outside of your kubernetes and this is done through a process called Ingress and going back earlier we talked about how security is a key part of all the work that goes into kubernetes and this is a great example of where you can use an external service but still have that layer of security that is vital for your solution because to be able to provide the Ingress connection kubernetes requires that SSL connection which automatically allows for a secure data transfer absolutely fantastic so here we have an overview of the actual architecture on the right hand side we have our nodes and the pods within our nodes and within those pods are the containers on the left hand side we have the the master architecture and the master architecture is the actual brains behind the work that you're doing it's actually managing the schedules the controllers for all of the nodes and then and the bottom left hand side we have commands which are written on a local workstation and those commands are pushed up to the master so let's kind of dig a little bit further into the architecture of the master so the master itself is comprised of a number of key components again this is the tool that's going to manage your entire kubernetes infrastructure the key components that you have are the etcd control manager scheduler API server and the key for components are Etc or the cluster store the controller manager the scheduler and the API server the cluster store is the tool where all the details of the work that you're doing is stored now it's going to manage the Raw rules can manage all the instructions and this is where you're going to post that information the controller is actually going to then perform the tasks that are stored in the cluster store and as you'd expect the schedule is the tool that actually controls when instructions and tasks are pushed out to all the pods and nodes within your network and then finally you have an API server which allows you to be able to control all the operations the actual operations themselves are all instructions written as rest instructions and which allows you to take advantage of the security that has been built into rest and so you end up with a worker slave architecture and slaves are all the different nodes and pods within your network and so a pod itself will have multiple one or more multiple containers and then each container and then each pod will have a Docker container a kublet and a kubernetes proxy and the these are tools to allow you to communicate back up to the master layer so the docker is the actual container itself and this is going to actually run your application and the instructions here are to pull down the correct container from the docker images that you have within your library the cubelet is going to manage your containers and your container instructions that will be deployed to the cubelet are going to be written in either yaml or Json format yaml is just great if you haven't had a chance to start working with yaml I certainly encourage you to do so it's very easy to pick up if you know any XML you'll pick up yaml very quickly and finally the kubernetes proxy is the tool that interacts with the master environment within the kubernetes network and allows all the instructions for that are pushed to the couplet and to Docker to be able to perform effectively so let's have a look at some of the companies using kubernetes there are a lot and if you're doing any work today in the cloud and even if you're just starting off and doing some simple work in the cloud there's a really good chance that you are using kubernetes for some of the companies that you would are probably not surprised at using kubernetes are Spotify sap eBay and of course uh some of the the big companies out there Google with their entire network of solutions and Microsoft with office and their tools so let's take a step into a you but now let's have a look at a kubernetes use case and in this example we're going to look at the New York Times and how they use kubernetes to solve a problem that their IIT Department was struggling with so when New York Times started working after their data centers their their actual deployments were small and the applications were managed across VMS but the problem is is that as New York Times became more of a digital company they started building more digital Solutions and at some point they realized that what they needed to do was to embrace the cloud and move their solutions to the cloud however their first step of using the cloud and this is I think typical for a lot of companies is that they treated like the cloud hosted provider in the same way as they would with a Data Center and they at some point they realized they had to take a change and this is where the development team came in and they said look how about we stop thinking of our Solutions as a data center even though they're being run in Amazon and start thinking of them as Cloud first Solutions and so what they proposed and what they ended up doing was using Google Cloud platform with kubernetes as servers and to provide greater efficiency and so by using kubernetes they have the following advantages they're able to deliver Solutions at a much faster speed and in the world of digital content speed is everything they were able to reduce the time for deployment from minutes to seconds and by doing this they were able to increase the amount of time that uptime for the actual infrastructure um updates began to be deployed independently and when required so you didn't have to schedule an event when you do the deployment it just happened and there was more unified approach to deployment from the engineering staff which just made the whole solution more portable so the fundamental end line for the New York Times is that by leveraging kubernetes they were able to automatically manage their deployments and scheduling of solutions instead of having to do Solutions through a more traditional ticket based system it just really speed up the efficiency of the New York Times website so that's finally kind of look into some of the important terminologies before we take it our time to do a demo so we have six terms that we're going to look through and we'll let you uh read through this but uh we have a cluster which is a set of machines physical vertical in which applications are managed and run we have a node that are the working machines that run containerized applications and other workloads there is the pod which is within in a node and there's a group of containers that are deployed together on the same host we have replication controllers and a replication controller is used to define a pods life cycle rather than to create the pods directly we have a selector which is an expression matched label to filter specific resources uh labels are key value pairs that are attached to objects such as pods and the key value pair or the label can be filtered organize and perform operations on resources replication sense Define how many replicas of each pod will be running and managed when you want to either replace a pod when it dies annotation is a label with much larger data capacity a name is a resource as it is identified volume is the direction which data is accessible to a container namespaces provide additional qualification to resources name and then finally service is an abstraction of a top-tier pods which provide a single IP address and DNS Name by which the pods can be accessed so this time and what we want to do is take you through kubernetes demo actually runs through some of these basic controls so we're going to take you through a video of how to actually install kubernetes um and to use some of the basic instructions so kubernetes is the most wildly used platform for container management the first step is to actually install the necessary dependencies so let's see if there's any updates first so the first so to check if there's any updates we're going to use the command sudo app dash get update we're entering our password and this looks like it's going to take a few minutes because we've got a few updates that need to be running all right we're done all right next we're going to write sudo app get install Dash y apt-transport https and this will install all of the dependencies we need to actually start installing the solutions needed for kubernetes all right that was quick now we're going to put in the docker dependency and the way we do that is bring to the command sudo apt install Docker dot IO I'll go typo correct that quickly and put in y so vs and we continue and it's going to take a while because we're going to download and install Docker off the internet and it's quite a large installation now would be a good time to go get a cup of coffee because it'll take us about a minute to get this all done once we actually have Docker installed we'll let you go ahead and we'll start Docker and enable Docker using our sudo commands all right so that's all done so let's start enable Docker we're going to be using so we're going to use sudo systemctl start Docker and sudo systemctl enable Docker this will be quite quick so the next step is to install all the components for kubernetes and we're going to install the curl command and then kubolate Cube C and then cubeadm and we're going to install all the necessary tools we'll do the call command cubelet cubic C2 and Cube ADM and so let's go ahead and put in the sudo apt-get install Curl we're going to get a prompt we can say yes for that and then after this we'll go ahead and install kubernetes so the first what we're going to do is we're going to actually put in the kubernetes key so that's going to be pseudocal dash s and then we have a long URL which is https packages.cloud google.com apt dark apt-key.gpg pseudo apt-key there we go create my typo there okay now we have to go ahead and change permissions of a folder so we do that with sudo chmod 777 slash Etc apt sources.list.d all right we're good okay now you want to go ahead and open up your text editor and we're going to make an actual edit and we're going to create a simple file and so what we're going to type in is the the following command DB hcgbt apt-kubernetes dot IO slash kubernetes Dash zenial Main and we're going to save this file in the ETC folder and then look for and then a APT which is going to be down a little bit and there we have it and we're going to save it in the apt-get updates and we're going to name this file as kubernetes dot list and we'll save that and then we can close out the notepad and go back to the terminal window and let's look and see see if there are any updates again so we're going to go ahead and run apt-get update and it's going to run any updates from what we've created and we want to check what's in the folder and so we do cat slash X Etc apt sources.d kubernetes dot list and there we are it shows what we have in that text file and so we're going to go ahead and we're going to install cubelet cubic cubic 2 and Cube ATM but first we're going to have to initialize the master I'm going to show you what an error you would get if you were to install cubelet and the other files too quickly you get a fine error all right so we have installed cubelet qbadm cubic tool and kubernetes cni and so we're going to go ahead and use sudo swap up Dash a and now we're going to initialize the masternode and we're going to do that with sudo Cube ADM in it and this will take a few minutes for it to initialize all right so that's all done all right so we're going to deploy the Pod to a network and we have two commands that we're going to use we use sudo cuboo Cube CTL apply and then we have a long path we have to type out which is dash out of https raw githubs content.com Korea's final Master documentation Cube Dash file Dot yml what we we have to do is we have to actually go ahead and deploy the nodes first so we're going to do that by now creating um mktr Dash p and then we have dollar sign home dash dot Cube and create that new directory and sudo cp-i and then the path will be Etc kubernetes admin.conf and then dial sign home slash dot Cube dot slash config and then final one will be pseudo c-h-o-w-n dollar sign open Prime ID Dash U closed paren colon doci brain ID slash Dash G close paren door sign home slash dot Q slash config all right now we're going to go ahead and deploy and uh let's see and oh yeah I think I have a typo my command here yep oh yes I did have a typo and so um I've got that incorrect now so documentation is actually spelled with a t not an S and then we're going to go ahead and we're going to put in the next line which is pseudo cubic Cube CTL apply dash dash F and then we're just going to change a little bit of the actual link here and that runs great and now we want to be able to see all of the pods that we have in our Network and so we're going to do that by writing the commands sudo cubic TCL get pods Dash or Dash Dash namespaces and it should list all of our pods we only have one there we are there's all of our content now we're going to so there's only one node so let's see what we have here so we can use the command sudo you know cubic TL get nodes there we are just one node so our final step is to actually go ahead and deploy a service and so what we're going to go ahead is deploy the nginx as service and we're going to do that with the command sudo cubic CTL run Dash Dash image equals nginx and we're going to put this into Port 80 so it's going to be nginx Nash app dash dash Port equals 80 dash dash environment EMV equals uh ubunt open quota domain equals cluster close quotes and we're going to write one more command which is sudo Cube CTL exposed deployments nginx Dash app dash dash Port equals 80 dash dash name equals nginx Dash HTTP and once we've done this we will be able to then run the command to actually see our services and there'll be quite a few of them and our final command is sudo docker ps-a we have quite a lot of lists quite a lot of services listed so there you are that's the instructions on how to get Docker up and running and uh with some basic commands and it is worth going through a few times to again get right um but once you actually have the kubernetes environment up and running and managing your document environment and containers it's extremely powerful if getting your learning started it's half the battle what if you could do that for free visit skill up by simply learn click on the link in the description to know more welcome to kubernetes architecture from Simply learn we're going to take a deep dive into the kubernetes architecture if you're new to kubernetes we already have an introduction that goes and provides an overview of kubernetes and we have the link below so you can go straight to that if you want to have a refresher otherwise hang around and join us in this presentation and if you want to see more of these videos hit the Subscribe button and if you want to get notified hit the notification Bell and every time we post up a new video you'll get the notification right onto your phone or on your desktop that the new video has been uploaded and as always if you have any questions please post them in the comments below we do read the questions or we try to respond to all of them as quickly as we can with that said let's jump into kubernetes architecture so we're going to break out the presentation into the following areas we're going to cover what kubernetes is and why you'd want to be using it we're going to introduce the actual architecture and provide an overview of how it contains the containers and the other components that you'd have within the architecture we'll also compare 80s with Docker swarm and one of the things that you hear with kubernetes almost in the same breath as Docker and we have to be careful not to confuse Docker from a container point of view and Docker swarm which is a container management tool and kubernetes is also a container management tool so compare the two of those and then we'll look at the hardware and software components and then finally do a deep dive into the architecture of kubernetes and provide a case study of where kubernetes is being used successfully in the past so let's jump into this so what actually is kubernetes so kubernetes is a tool that was designed to actually help manage and contain large groups of containers it was developed originally by Google out of the Google Borg solution and then Open Source by Google the actual environment allows you to manage large groups of containers so if you're a developer in devops and you're working with Docker then you're used to the concept of the container and you'll also know that containers and Cloud technology these tend to be almost breathed in the same breath so the work that you're doing with kubernetes is to be able to manage large groups of containers inside of the cloud the thing that's great about working with kubernetes is that it is incredibly flexible and it allows you to have incredibly complex applications run efficient efficiently so let's step into why you'd want to use kubernetes so a couple of key points kubernetes itself is an open source solution originally developed by Google but it is available on most Cloud platforms today so AWS Google Cloud Microsoft Azure all of them support kubernetes and what you'll find is as you're setting up your infrastructure particularly if you're using containers you'll see that the support for kubernetes is flooded up very very efficiently and effectively by all three vendors it's extremely um useful for being able to manage large modular environments and that's really kind of these I think one of the big benefits of kubernetes is its modular T is that it really breaks down containers into their smaller parts and once you do that it becomes much more efficient for you as an administrator to be able to manage that entire environment the reproductibility of kubernetes is extremely high you can build out infrastructures very quickly and have them being able to imagine and have containers coming on and off and killed and created to be able to help load balance your entire environment and so again you know we keep talking about containers but that's really what kubernetes is all about is being able to manage your applications that are managed through containers and being able to do that in a virtualized infrastructure and the thing that's also really good with kubernetes is it's really easy to deploy Solutions you just have to use a simple curl call and you can actually push out your kubernetes infrastructure so let's have an overview of the kubernetes architecture so kubernetes has really broken up into three key areas so you have your workstation where you develop your commands and you push out those commands to your master and Master is comprised of four key areas which essentially control all of your nodes and the node contains multiple parts and each pod has your Docker container built into it so consider that you could have a really almost an infinite number of PODS I'm sorry infinite number of nodes are being managed by the master environment so you have your cluster which is a collection of servers that maintain the availability and the compute power such as RAM CPU and disk utilization and you have the master which is really components that control and schedule the activities of your network and then you have the node which actually hosts the actual Docker virtual machine itself and be able to actually control and communicate back to the master the health of that pod and we'll get into more detail on the architecture tech chat later in the presentation so you know you keep hearing me talk about containers but they really are the center of the work that you're doing with could kubernetes and can the concept around kubernetes and containers is really just a natural evolution of where we've been with internet and digital Technologies over the last 10 15 years so before kubernetes you had tools where you either running virtual machines or you're running data centers that had to maintain and manage and notify you of any interruptions in your network kubernetes is the tool that actually comes in and helps address those interruptions and manages them for you so the solution to this is the use of containers so you can think of containers as that Natural Evolution from you know uh 15 20 years ago you would have written your code and posted it to a Data Center and more recently you probably posted your code to a virtual machine and then move the virtual machine and now you actually just work directly into a container and everything is self-contained and can be pushed out to your environment and the thing that's great about containers they're they're isolated environments very easy for developers to work in them but it's also really easy for operations teams to be able to move a container into production so let's kind of step and back and look at a competing product to kubernetes which is Docker swarm now one of the things that we have to remember is that Docker containers which are extremely popular I'm built by the company Docker and made open source and Docker actually has other products one of those other products is Docker swarm and Docker swarm is a tool that allows you to be able to manage multiple containers so if we look at some of the uh the benefits of using Docker swarm versus kubernetes and one thing that you'll find is that both tools have strengths and weaknesses but it's really good that they're both out there because it helps keep it really kind of justifies the importance of having these kind of tools so kubernetes was designed originally from the ground up to be the auto scaling whereas took a swarm isn't the load balancing is automatic on Docker Swan whereas with kubernetes you have to manually configure load balancing across your nodes the installation for Docker swarm is really fast and easy I mean you can be up and running within minutes kubernetes takes a little bit more time is a little bit more complicated eventually you'll get there um I mean it's not like it's going to take you days and weeks but it's it is a tool that's a when you compare the two Docker swarms much easier to get up and running now what's interesting is that kubernetes is incredibly scalable and it's you know that's its its real strength is its ability to have strong clusters whereas with Docker swarm it's cluster stream isn't um as strong when compared to kubernetes now you compare it to anything else on the market it's really good um so this is kind of a splitting hairs kind of comparison but kubernetes really does have the advantage here if you're looking at the two compared to each other for scalability kubernetes was designed for by Google to scale up and support Google Cloud Network infrastructure they both allow you to be able to share storage volumes with Docking you can actually do it with any container with that is managed by the docker swamp whereas with kubernetes it manages the storage with the pods and I probably can have multiple containers within it but you can't take it down to the level of the container interestingly uh kubernetes does have a graphical user interface for being able to control and manage the environment the reality however is that you're likely to be using terminal to actually make the controls and Commands to control your either Docker swarm or kubernetes environment and it's great that it has a GUI and to get you started but once you're up and running you're going to be using terminal window for those fast quick administrative controls that you need to make so let's look at the hardware components for kubernetes so and what's interesting is that kubernetes is extremely light of all the systems that we're looking at is extremely lightweight um it allows you to have you compare it to like a virtual machine which is very heavy you know kubernetes is extremely lightweight and other uses any resources at all interesting enough though is that if you are looking at the usage of CPU it's better to actually take it for the cluster as a whole rather than individual nodes because the nodes will actually combined together to give you that whole compute power again this is why kubernetes works really well in the cloud where you can do that kind of activity rather than if you're running in your own data center um so you can have persistent volumes and such as a local SSD or you can actually attach to a cloud data storage again kubernetes is really designed for the cloud I would encourage you to use cloud storage wherever possible rather than relying on physical storage the reason being is that if you connect to cloud storage and you need to flex your your storage the cloud will do that for you I mean that's just an inherent part of why you'd have cloud storage whereas if you're connecting to physical storage you're always restricted to the limitations of the physical Hardware so that's um kind of pivot and look at the software components as compared to the hardware components so the main part of the components is the actual container and all of the software running in the container runs on Linux so if you have Docker installed as a developer on your machine it's actually running inside of Linux and that's what makes it so lightweight and really one of the things that you'll find is that most data centers and Cloud providers now are running predominantly on Linux inside of the um the the container itself is then managed inside of a pod and a pod is really just a group of containers bundles together and um the kubernetes scheduler and proxy server then actually manage what um how the pods are actually pushed out uh into your kubernetes environment the the parts themselves can actually then share resources both networking and storage so pods aren't um pushed out manually they're actually managed through a layer of abstraction and part of their deployment and and this is the strength of kubernetes you use to find your infrastructure and then kubernetes will then manage it for you and there isn't that problem of manual management of PODS if you have to manage the deployment of them and you that's simply taken away and it's completely automated and the the final area of software Services is on Ingress and this is really the secure way of being able to have communication from outside of the cluster and passing of information into that cluster and again this is done security through SSL layers and allows you to ensure that security is at the center of the work that you have within your kubernetes environment so let's dive now into the actual architecture before we start looking at a use case of how kubernetes is being employed so kubernetes again is we looked at this diagram at the beginning of the presentation and there were really three key areas there's the workstation where you develop your commands and then you have your master environment which controls the scheduling the communication and the actual commands that you have created and pushes those out and manages the health of your entire node Network and each node has various pods so we like break this down so the master node is the most vital component with the master you have four key controls you have Etc the controller manager scheduling API server the cluster store Etc this actually manages the details and values that you've developed on your local workstation and then we'll work with the out the control scheduling API server to communicate that out those instructions of how your infrastructure should look like to your entire network the control manager is really an API server and again um this is all about security so we use wrestle apis which can be packaged in SSL to communicate back and forth across your pods and the master and indeed the services within each of them as well so at every single layer of extraction the communication is secure the schedule as the name would imply really schedules and when tasks get sent out to the actual nodes themselves the nodes themselves are are dumb nodes they just have the applications um running on them the master and the scam is really doing all of the work to make sure that your entire network is running efficient shipping and then you have the API server which has your rest commands and the communication back and forth across your network so that is secure and efficient so your node environment is where all the work that you do with your containers gets pushed out too so um a work is really a it's a combination of containers and each container will then logically run together on that node so you'd have a collection of containers uh on a node that all make logical sense to have together within each node you have a Docker and this is your isolated environment for running your container you have your cubelet which is a service for conveying information back and forth to the service about the actual health of the kubernetes node itself and then finally you have the proxy server and the proxy server is able to manage the nodes the volumes the the creation of new containers and actually helps pass the community the the health of the container back up to the master to see whether or not the containers should be either killed stop started or updated so finally let's look at see where kubernetes is being used by other companies so you know kubernetes is being used by a lot of companies and they're really using it to help manage complex existing systems so that they can have greater performance and with the end goal of being able to Delight the customer increase value to the customer and enhance increased value and revenue into the organization so example of this is a company called BlackRock where they actually went through the process of implementing kubernetes and so they could so BlackRock had a challenge where they needed to be able to have much more Dynamic access to their resources and they were running complex installations on people's desktops and it was just really really difficult to be able to manage their entire infrastructure so they actually went and pivoted to using kubernetes and this allowed them to be able to be much more scalable and expansive in the in the management of their infrastructure and as you can imagine kubernetes was then hooked into their entire existing system and has really become a key part of the success that BlackRock is now experiencing of a very stable infrastructure and the bottom line is that BlackRock is now able to have confidence in their infrastructure and be able to give that confidence as back to their customers through the implementation and more rapid deployment of additional features and services because what it is specific to AWS Cloud platform so what's a part of this tutorial and demo what's in store for you at our offset I would like to cover the basics of orchestration tools as most of you would know kubernetes is one of the most popular orchestration tools in recent times specifically for applications which are Cloud native and deployed on some or the other types of containers but what are these orchestration tools why would one need to use orchestration tools what are the facilities or features provided by these orchestration tools that's the first thing that I'm going to cover after that I will pick two orchestration tools specific to container management Docker swarm versus kubuntis I am going to compare them with regard to the features that they provide the use cases that they cover what facilities that they provide and when to use what after that I will get into little bit details of the kubernetes architecture what exactly is required for setting up a kubernetes cluster what runs in a controlled plane or a master node what runs as a part of the worker node after that I will end the tutorial by running you through a demo by setting up a three node kubernetes cluster on AWS platform I will use something called as cops which is one of the admin tools for setting up production grade kubernetes cluster so I will use this to set up a three node cluster on AWS all right now that we set the context right let me get started so what are orchestration tools the application development and the life cycle management from the time the application is developed connecting the source code to your continuous integration to testing them all along your development process and eventually moving it down to production managing your production servers the dependencies that your software has the requirement that it has in terms of the hardware the features of fault all runs self-healing capabilities Auto scaling all this as complicated over the last few years as a part of devops one thing that everyone is interested in is managing all these application dependency or lifecycle management using some or the other kind of a tool so that's where these orchestration tools have become very very popular so what kind of features that they provide is you'll have to just let them know what are the kinds of tool sets that is required what are the fault tolerance mechanisms that it has to be adopted to what kind of a self-healing capabilities that application will have to need and if at all there is any auto scaling features that is required if at all you can bake in all these specific parameters into your tool your orchestration tool becomes a One-Stop shop for all your deployment and configuration management needs that's where this orchestration tools have become a very very popular and relevant specifically these days when people are more and more interested in adopting devops practices all right so having said that about orchestration tools let's concentrate on two of these orchestration tools specifically for container management Docker swarm and kubuntus many of the applications that are written for these kind of containers are called or kind of fall into a category of cloud native where the application need not know much about the underlying infrastructure or the platform where these applications are running so these two along with Apache me Source there are many other container management systems but I am pick Docker swarm and kubernetes for comparing the features that is provided by these two Frameworks for the sake of comparison I picked up Docker swarm and kubuntase just because of the reason that both of them operate in the space of container management they do something very very similar to Containers that's the similarity that exists between these two orchestration tools however there's a big difference that exists when it comes to the types of containers that both of them cater to and also the capacity of workloads they can be used for Docker swarm is a cluster management solution that is provided by Docker container Docker is one of the very very popular containers of recent times and in case you have your applications that is totally powered by only Docker containers if you have a need where you want to run a cluster of servers which are running only Docker containers Docker swarm should be your choice for your orchestration tool kubernetes this on the other hand can cater to any other types of containers other than Docker and including Docker as well so in case you have your applications which have got Docker in them which I've got rkt in them which they have got Alex in them or any other type of container kubernetes should be your choice of orchestration too a Docker swarm can manage up to 40 50 or 60 Max nodes so in case your application is totally written in Docker containers and the load or the expected cluster size is about 50 60 nodes docus swarm should be the choice of orchestration tool on the other hand kubernetes is something that was open sourced by Google and the kind of the scale of operations that kubernetes can cater to is Google scale so if in case your applications are more than a thousand nodes that is where kubernetes comes into play that's the big difference that exists between Docker swarm and kubernetes putting those differences aside let me compare docus warm and kubernetes based upon other features which are similar in nature the first and important feature that kubernetes provides is something called as Auto scaling if at all the load on your cluster is too high and your application is experiencing more load so kubernetes can add new nodes to your cluster of course you'll have to configure kubernetes in order to have those capabilities where it can spin up new VMS or new nodes if at all you do that configuration correctly kubernetes has got the capacity or it has got the feature where it can bring up a new node and add it to the cluster on the Fly based upon the load that exists at that moment on similar lines if at all the load is not too high it can identify a few of those nodes which have got less number of replicas or less number of application containers running on it it can move them to some other node and also scale down by deleting few nodes from your cluster so that is the of less of Auto scaling which is provided by kubernetes and this unfortunately does not exist in Docker swarm the other feature of load balancing specifically application load balancing so Docker swarm gives you an application Level Auto load balancing however kubernetes gives you the flexibility of manually configuring any other type of load balancer of your choice for application load balancing installation as I mentioned earlier Docker swarm is a loose cluster of containers which are running on nodes it is very easy to spin up a new node and then connect them to a swarm and this can be done in a very very loose coupled way where you can create swarms of your choice notes are allowed to connect to swarms and quit or leave the Swarms on the fly so in and all the installation is pretty easy and fast kubernetes on the other hand the configuration of kubernetes is the way to spin up a big cluster specifying the size of the node how many Master known so many config planes that you want how many nodes that you want it's a pretty tough thing to bring up a kubernetes scalability kubernetes strength is very very strong they are very tightly coupled and even they have the capability where on the cluster size things or the nodes can be increased on the Fly based upon the requirement on a Docker swarm the cluster strength is weak as compared to kubernetes worker nodes can be added to a swarm worker notes can be asked to leave us form or can be taken out of a swarm based upon the requirement so this is kind of a Loosely coupled architecture for Docker swarm while kubernetes the cluster is very tightly coupled since Docker swarm runs only Docker containers containers find it very easy to share data and a lot of other things with each other because they all have the same signature they are all from the same family so it's very easy for them to share not just volumes but lot of other things however for kubernetes since it manages containers of different types if application has to share some data across different containers there's a little bit of a complexity in how you would want your containers to share data also when it comes to kubernetes kubernetes groups containers in something called as pods while a pod can have either one container that's a preferred choice or multiple containers and the idea of pod is like each pod can run in any other node it's not guaranteed that you would want to have two or three parts to be running on the same node that makes data sharing a little bit of a different thing compared to Dockers form uh GUI so there's not a good enough UI tool in Dockers form at least the CE edition of it which will allow you to get to know what is running on your containers what is the size of the containers what is the volume of the containers and all that stuff there are some free and open source tools like portaner which gives you a good visibility into your running Docker containers however there's nothing at a swarm level that is provided by docker kubernetes gives you an out of the box dashboard which is easy to configure and set up you can also Club it with some metrics services and you can also get information about the size of the cluster that is running what is the load on the nodes and stuff like that all this across your cluster in a very beautiful dashboard perspective so that way this is little bit of a good UI for kubernetes file for Dockers form there isn't anything that is provided out of the box let me spend some time in explaining a little bit about the kubernetes architecture what comprises of the kubernetes cluster what resides in a master or a control plane and what resides in a worker node this is a very high level depiction of a kubernetes cluster so this is the master node which has got something called as a cluster store a controller a scheduler component and an EPA server and these are the bunch of nodes that are connected or administered by the master node the master node can be 1 3 Phi typically an odd number this is as per the typical cluster management thing where you would need the master nodes to be in odd numbers so that whenever there's any contention in terms of what needs to be deployed where or where to give the job to whom and stuff like that all the Masters would cast their vote and they would decide on the outcome so that's the master node and there are a bunch of nodes which can be connected and administered by the master node cubic CTL or the command from using which anybody can assign some workloads to the cluster can also be run either on the same node or on a separate node so Cube CTL is the command line tool that we will be installing and using in order to fire up our commands to our cluster so if at all I have to put up a job on our cluster if I have to give out any specific commands to my cluster all this is done using cubic CTL there's a bunch of rest apis which is used by Q CTL and Cube CTL will talk to the API server and fire up the commands specific to my cluster what runs in the master node Master node without saying it's the most important component of any of the cluster if at all you're running um Thruster with only one must node if your master node goes down there's no other way that you know you can or any user can talk to the different nodes in the cluster so Master node is the most vital component responsible for the complete kubernetes cluster there's always one node that should be running as a master node that's the bare minimum requirement for your cluster so what are the other components in the master node the most important one is called etcd so this is nothing but a data store a value of key value pair which is stored which contains all information about your cluster so all the configuration details which node is up which worker node is down all this information is stored in the cluster store so all the managers would access this cluster store be 4 they go and decide any other work item or anything else that has to be done specific to your cluster what's in the controller as the name says controller is something it's like a Daemon server that is running in a continuous loop all the time it is responsible for controlling the set of replicas the kind of workload that is running based upon the configuration that the user has set in so if at all if you are aware of something called as replication controllers and point controllers namespace controllers all these controllers are managed by the controller component if any user asked for some three replicas of a particular pod or a service to be running and if at all any of the nodes goes down or the Pod goes down for whatever reason the controller is the one who wakes up and assigns this particular job or the Pod to some other available node by looking up for the details using in the cluster store that's the importance of the control manager or the controller the scheduler the scheduler assigns the tasks based upon whoever is asked for any job to be scheduled based upon a time frame or based upon some criteria it also tracks the working load as to what what exactly is the load who is running what and in the cluster and places the workload on whoever is the available resource at that time all right APA server this is one other important component in in our kubernetes cluster where how would the end user deploy or give out any sort of a workload onto your cluster all the requests come to the API server so this is an entry point for all your requests that come into your cluster anybody wants to deploy something anybody want to scale up a controller anybody who wants to bring down few Services anybody wants to put in a service all this will have to come in as a part of a rest API endpoint and API server is the entry point in case you don't want want you know somebody to access your cluster in case you want only specific people or specific users to be running some specific workloads you can set all those role-based access control for this API server so this is the entry point for anyone who wants to submit any job to your cluster so that's a quick overview about the master node what what are the important components of a master node now let's go over to what runs in a worker or a slave node as I mentioned earlier slave nodes are something where the job that is submitted to your cluster is eventually run as a pod as a service as a container so in kubernetes world there's something called as a pod a pod is nothing but a combination of a container it is a wrapper around your running containers so slave nodes are the worker nodes typically run these parts so that is where the whole workload typically gets run but there are a bunch of other components in the in the node which also manages what runs in the Pod who has to have access to the Pod what is the state of the Pod is it in a running state is it going down for some reason and all this stuff so let's quickly go over those components that is there in the slave node the most important component in my opinion that should be on the on the Node should be the container runtime as I mentioned earlier kubernetes can run any different types of containers not just Docker so in case you want to have a node which wants to have Docker running on it rkt running or at LKC running on it or any other container environment that is running on it you have to ensure that the specific container runtime environment is available on that specific node so that whenever a job is submitted description of what needs to be run as a part of the Pod what should be the image with it should be powered and what kind of a container to spin up so that's what you know when the job is finally assigned to this particular node it would definitely need a container runtime to be up and running so that exists only if at all the container runtime is installed and running on your kubernetes worker node okay now that we know what our kubernetes node can run how would somebody assign a job to our node that is using something called as a cube alert as the name says cubelet or cube alert is a small subset which talks to the cube API server so any node which has to run any kind of a pod all these instructions are passed around by the qbps server to the cubelet and cubelet is capable of processing whatever job that is assigned to it and ensuring that so many parts and so many services are spun up based upon the requirement the last component that exists in the worker node is the cube proxy or kubernetes proxy this plays a very very important role acting as a load balancer and a network proxy so what typical happens is whenever the pods are running in nodes the parts can be typically running in any node there is no um Affinity towards any node on which these parts are running because pods or containers are something called as ephemeral they can run anywhere so how would somebody reach out to these applications that is running in some pod which is running in one container now and running in probably another container another node all together tomorrow so that is where your kubernetes proxy contains a picture and this component will ensure that any container that is spun up or any pod that is spun up it keeps track of all these spots and kind of connects the end points or acts like a DNS server so that it knows when somebody is trying to reach out to this particular service which is the pod on which the service is typically running so that plays a very very important role by acting as a load balancer and a network proxy now that at a very high level we know what are all the components that make up of our kubernetes cluster let's go ahead and create a small cluster we'll spin up this cluster on AWS platform finally we get to the demo section of my tutorial now that you guys are aware as to what is kubernetes what is the use of kubernetes you also know at a very very high level what are the components of a cluster what is a master node or a control plane what are the components of a master node and what should be existing in a worker node probably you're thinking that is maybe pretty hard to kind of set this up so let me demystify that by running you with a quick demo of how to set up a simple three node kubernetes cluster using a tool called cops and I will be doing this whole setup on my AWS Cloud okay so what is cops cops is nothing but a simple tool is an admin tool that allows you to bring up production grade kubernetes environments as I said setting up a kubernetes cluster on a bare metal is little challenging so that is why I would use something called as a cops and I will use this on my cluster which I'll spin up on my AWS instance so I have an AWS account and what I'll do first is I I will first pin up an ec2 server I will power this with one of the Linux Amis and I will install cops on it now this will be my starting point from where I'm going to trigger running up a big cluster in our case I'll try to keep it little small I don't want too many nodes in my cluster I'm going to have one master node and two worker nodes this whole operation would be done programmatically by installing this cops admin tool so as you may know AWS is little hard for me to programmatically run other components or rather bring up servers it's not pretty simple so what I would do is I will create an IAM role which will attach to my S2 server so that my ec2 server gets powered with all the required role so that it can go ahead and spin up cluster based upon my configuration that I'm going to specify all right so once I have my cluster set up I will also install cubic CTL which you probably know by now is nothing but command line utility using which I can kind of connect to my cluster give out some jobs put some parts and stuff like that so I will use Cuba CTL I will also have a pair of SSH keys so that from this machine I would be able to successfully get onto the master node and submit jobs on my behalf so let me Begin by first logging into my AWS account all the steps that I will be following to bring up my AWS cluster kubernetes cluster will be documented and I'll be sharing you this document in a GitHub repository so that in case anybody wants to try you'll be more than happy to find all the required instructions in one place okay now the first thing that I would need is to launch an AWS ec2 instance now this is an instance where I am going to install cops and I will use it as a base server from where I'm going to find some commands which will bring up my kubernetes cluster so let me log into my AWS account and let me stick to one particular region and let me bring up some a one ec2 instance all right so let me launch my instance let me choose um it doesn't matter in the ways but in case you want to try it on your free tire I would recommend choose free tire only and choose an Ami of your choice let me choose this instance I will make it T2 micro that is good for me configuration instance details all right I don't want to change anything here so that looks good maybe I will change this 30 gig that's good for me let me add a tag to my instance I will name this as my cops server alright I would need a bunch of ports to be opened up so what I will do is I will create a new security group called open to the word I would not recommend that you do this since I don't want to it will take a long time for me to specifically uh pick and choose the ports that has to be open for running my cluster I don't want to do that I will open up up all the HTTP and https ports so that it's quicker so I will say all TCP I would say from anywhere just to be on the safer side also open HTTP and https and I will make this anywhere and anywhere this is good for me I will say review on launch all right so I chose rt2 micro instance just for the sake of easiness I've opened up all the ports and the instant details are you've seen what is that have chosen all right now important part is like I would need a keypad I need to create a new keypad I will call this as simply learn keys I will download this key pair and then I will launch the instance it will take some time for my ec2 instance to come up in the meanwhile what I'll do is I will convert I have this pem file which is the pair of keys the SSH keys with which I need to log into my ec2 instance so I'll need to convert this because I'm trying to connect from my Windows box I would need to convert convert this into a PPK file so I have putition I am going to load my set of keys here and convert that key into a ppk3 all right open all right I would say save and save Private key yes I would say simply learn or simply learn private private key I'll save this here and that that's all so this is my public key which is the pem file and this is my PPK or my private key Now using the private key I will log into my ec2 instance okay now my ec2 instance is up and running it is running in this Ohio region and this is the IP address of my public IP address of my machine so let me connect to it so I use mobile exterm which is nothing but an SSH emulator you can use putty or any of these sessions which allow you to do an SSH this is my IP address of my machine and since I've spun up an Amazon Ami the user this is the default user I would need to specify the private keys for my session and this is the private key for my session say okay I am in my Amazon ec2 instance so let me look at what are the other things that I need to do all right so as I said I just brought up my ec2 instance but I would need my ec2 instance to run few things on my behalf so that it can spin up easy to other ec2 instances it can also talk to an S3 bucket where I'm going to store the instant state of my kubernetes cluster also some sort of an auto scaling group because I want to spin up more and more instances I also want to have a private hosted Zone using my Route 53 so I would need my ec2 instance to have all these sort of permissions for my server so what I would do is I will go and create a permission a role in the ec2 or rather in the AWS IM role and I will attach this I am role to my ec2 instance all right so let me go to my IAM which is nothing but the identity and access management I will create a role called as possibly cops role I will create a role this role will be used by my ec2 so I'm going to click on ec2 and say next permissions they would need a lot of permissions uh specific permissions to be very honest and the permissions are all listed out here S3 ec2 and all this stuff just to keep it pretty simple what I would do is I'll create a role with the administrative access all right so I don't want any tags I will review I'll create a real name I will say cops fold for ec2 all right so I'm going to create a rule which has got administrative access rules I'm going to create a role so this would ensure that my ec2 instance from which I am going to run a my cluster would have all the prerequisite permission so that it can go and spin up instances talk to S3 buckets and all that stuff all right now let us have our running instance get powered by this role so I will connect this role to my running instance alright so this is my easy to instant that is running I will say action attire Branch attach attach attach replace IM rules so there is no rule as of now I would want my cops role for ecos rule that I created so I want this new room to be assigned to my ec2 all right great now my ec2 instance which is my cop server has got all the required permissions so that he can create a cluster on my behalf great now before this let me do a sudo gem update hyphen y so that any of the because this is a newly provisioned VM I just want to ensure that all my library system libraries are all updated so that when I do install cop and Cube CTL and all those things none of them will fail because of some dependency or package issues so I'm just running a sudo update iPhone y so that all the libraries are updated okay looks like it is done so let me go ahead and install cops cops is a pretty simple installation it is available in a particular location I have to just copy this and ensure that I do a curl and I install this particular stuff all right so it is fetching the cops tool it's installing for me once it is copied down let me change the mode so that I can execute it and then also let me move it to my user local bin so that it is available for me in my path okay that's pretty much the cops installation let me also do one other thing let me install something called as cubic CTL so this is what would be you know a tool using which I'm going to be firing my commands to my kubernetes cluster once it comes up right this is a pretty smaller executable so keep sitting here all right so I have Cube CTL as well as cops installed on my ec2 server now okay now what I'll do next is I'm going to create an S3 bucket in AWS so that the equivalent is a cluster state is persisted in this bucket so let me create a simple bucket with this name S3 bucket with this name so let me go to S3 and let me create a simple bucket with this name I will just say create a bucket okay so the simply run dot kubernetes is the bucket that I created so this bucket will store all information about my cluster let me now go and create a DNS entry or a DNS zone or a AWS Route 53 hosted zone for my cluster so this is required so that you know I can give a cluster name for my kubernetes cluster that I'm coming up so I will create a private hosted Zone and possibility possibly I will call this as simply learn.in so this will be my name of my private hosted Zone in case you already have any other public hosted zones in your name you can always put a Route 53 hosted Zone specific for your domain name since I don't have any one I'm just going to create a simple private hosted zone so let me head down to Route 53 click on any of these and you will get these hosted zones out here so I'm going to create a hosted Zone here I'm going to create a hosted Zone click on create hosted Zone um this would be simply learn.in I want to create a public no I don't want public I want a private hosted Zone and I'm going to associate my VPC ID for this hosted Zone since I'm going to try out all my exercises in the Ohio region I will associate the VPC of the Ohio region for this all right so this is the one that is specific to the Ohio region so I will say this one I'll say create a hosted so great now let me come back to my ec2 box my ec2 is all powered with whatever it needs in order to go out with the installation of my cluster only a few things that I need to take care of is that you know now that I put a name for my cluster I also have an uh S3 bucket that I've configured for my cluster I will have to ensure that I need to put in these two configurations as a part of my cluster building activity so I will open my bash RC file and I'm going to export these two variables if at all you remember well these two variables are nothing but the cluster name which is nothing but the public or sorry the private hosted Zone that it created and the S3 bucket where I'm going to create or rather I'm going to store my cluster state so let me copy these two and open up my bash RC file all right so I will just add these two and copy these and Export them out as my variable I'm going to save this here now let me ensure that this gets picked up all right I've got these two that I've configured and I've also ensure that this environment variables are set now let me create a pair of SSH keys this is to ensure that I can log into the box that I'm going to be provisioning as a part of my cluster SSH iPhone Keygen I don't want to give any parameters let it go to the home directory with the default name and without any passphrase all right so I created a bunch of my keypad now I'm all set to go ahead and run my cops now cops will ensure that this will take some time for the cluster to come up but this is exactly how I would Define my cops command I already got a copy here so cops create cluster the state of the cluster will be stored in this particular variable which is what I have exported out the number of node count is two node count is the worker node count if at all I don't specif there's a configuration for specifying the master or the control pane as well if I don't specify that the default one is one so I'm going to create one primary or the master node and two worker nodes uh size of my master Note size of the worker nodes or what are the zones where I want this to be created and where do I store the cluster information and okay I've already added this must account this is actually not required but this is the command that I'm going to fire off so that I bring up my cluster all right that went off very fast but this actually did not create a cluster it is just the definition of a cluster and that's why this came out very fast now that everything looks good with whatever configuration that is specified now let me go ahead and create the cluster by seeing cops update cluster hyphen iPhone 8 yes now this will take some time a good 5-10 minutes for it too because this will actually start provisioning the servers and as I mentioned earlier based upon my configuration I'm going to come up with one master server and uh two nodes or the worker nodes all right so this is the command for me to validate the cluster and I'm going to try it out first I'm pretty sure that will fail because all my servers are not up yet it is taking some time the validation everything failed but let me try to look at my ec2 instances and see how many servers do I have running as of now if you see I had only one server that had started which was my cop server and automatically the other three instances one is called notes dot simply not in and this these two are the nodes and this is the master node so these three got automatically provisioned by the cops command that I ran all right so this may take a while for it to get validated Creator a good five to ten minutes so what I'll do is I'll pause the video now and I'll come back once on the server the cluster is up and running it's been a good uh eight to nine minutes since I started my cluster so let me validate now okay seems good so this is a master node uh minimum One Max one some nodes which are nothing but the slave nodes or the worker nodes there are two of them due to micro uh subnet so my clusters seem to be up and running and my cluster name is billon.in so what I would want to do is now let me just log into the master and see if I can run some parts so let me get back to my installation steps here the variates and cluster is done so let me log into my since my cluster name is simply learn.n this is the way to log into my box so let me get into my box so if you see here this is the host name of the machine that I'm currently in if you see this this is nothing but our this cop server this is a seventh now from here I'm trying to get into the master box all right so if you see the IP address has changed I was in a different box I'm in a different box now if I see host name you'll find our different host name so this is 153 which is nothing but the master node yep that's 153 this is the particular house so I'm getting into this machine now so I started the cluster from my cop server here it ran and brought up three nodes so I'm actually getting into my master node and see if I can run some parts on it alright so let me try Cube CTL get cluster info right Cube CTL get nodes all right so there are three nodes here Master node and one master node and two worker nodes so let me see if I have some pods here give the CTL get pods so there's no pod as of now so what I'll try to do is let me just spin up a very very simple Pawn just to check if my connections is everything is correct or not correct I have a simple node.js application that I have built and I've got a container for that this is already pushed to the docker up registered or simply run Docker Hub and the image name that I have here is called my node app and I will use this image to power one of my parts that I will run so this is the way in which I am going to launch my pod let me show you these commands Cube CTL run the name for my deployment that I'm going to run hyphen hyphen image and this is the image that is going to be powering my container so this is simply learn Docker Hub forward slash my node app hyphen F1 replica is equal to I want two replicas to be running and the port that I want to expose this particular part is on 8080. so let me run this as soon as this is run it creates something called as a deployment all right now let me just say I'm not really interested in the deployment I'm interested in checking if at all the parts are up and running so I will say Cube CTL get what all right so that was pretty fast so I have two parts that are running here these are two replicas because I asked for two replicas these parts are running so I can run Cube CTL describe pod pickup all right I can pick up up any of the Pod name and see what is the Pod what is the information does it contain from what is the image that is pulling what is the image image Name ID this is actually running my thought which is actually spinning up a container great so first of good so in kubernetes the pods can are ephemeral so they can be there at any time cannot be runtime if I need to expose my pod outside I will have to create a service for my pod so what I'll do is I'll create a very very simple service by exposing this particular deployment before that you check Cube CDL get deployment so there's a deployment that is created the deployment name is simply an app so I will expose my deployment simply learn app as um yeah I'll just expose this let me see all right I am not specifying what type and all I don't want to get into the complications of what are the different types of exposing this service and all that stuff so if I don't specify anything it gives me something project cluster ID so this is where my pod is actually exposed so let me just check if at all this is up and running I'll just try a simple curl command curl HTTP colon this is my cluster IP and the port is 8080 if at all I hit this it's actually hitting my application and giving me uh whatever I put a simple uh sys dot out kind of a thing where I'm just printing um the container ID of whatever pod is serving uh mix out of so I'm actually eating my container and I'm getting this output from the content so everything looks good my content is up and running so let me just go and clean it up I will say Cube CTL delete deployment simply learn app this should get rid of all the parts that I created Cube CTL get pod all right these parts are interminating things let me just check Cube CTL get Services there's one service I don't want this service let me delete that keep CTL delete service I want this all right that's good so let me come back to my host my cop server from where I'm running so I managed to successfully verify this power see if everything is up and running so what I'll do is I'll just go ahead and complete this demo by going and getting rid of my cluster so cops delete cluster hyphen iPhone yes all right so this will ensure that it will clean up my complete cluster that I created so I had three or four running instances if you see that all the three are shutting down because you know the cop server which had cops installed on it uh has now got a mandate to go ahead and shut down and clean up the whole instances and all those things that are created as a part of my deployment hey everyone I welcome you all to this puppet Phil course by simply learn in this complete course we will learn everything that we need to know to excel in puppet we have Matthew and Anjali to guide us through this journey we shall begin with having a basic understanding of puppet and what is its relevance today after that we shall dive deeper and understand the components of puppet followed by its architecture after checking its architecture we shall look at the process to install puppet and configure it in our local system moving forward we shall see companies that have adopted puppet and all about writing manifest and puppet after that we will check regarding the puppet community and puppet DSL postswitch we shall see the Hands-On demo that will explain the working of puppet from scratch after we have ample knowledge regarding the tool we shall understand the difference between Chef puppet ansible and salt stack finally we shall take a look at some of the interview questions that might increase your chances of cracking any puppet interview so before we begin make sure that you have subscribed to our YouTube channel and don't forget to click that Bell icon to never miss an update from Simply learn now without further Ado let's begin with our introduction hello this is Matthew from Simply learn we're going to focus specifically on puppet so in this session what we can do is we're going to cover what and why you would use puppets what are the different elements and components of public and how does it actually work and then we'll look into the companies that are adopting puppet and what are the advantages that they have now received by having puppet within their organization and finally we'll wrap things up by reviewing how you can actually write and manifest in puppet so let's get started so why puppet so here is a scenario that as an administrator you may already be familiar with you as an administrator have multiple servers that you have to work with and manage so what happens when a server goes down it's not a problem you can jump onto that server and you can fix it but what if the scenario changes and you have multiple servers going down so here is where puppet shows its strap with puppets all you have to do is write a simple script that can be written with Ruby and write out and deploy to the servers your settings for each of those servers the code gets pushed out and to the servers that are having problems and then you can choose to either roll back to those servers to their previous working States or set them to a new state and do all of this in a matter of seconds and it doesn't matter how large your server environment is you can reach to all of these servers your environment is secure you're able to deploy your software and you're able to do this all through infrastructure as code which is the advanced devops model for building out Solutions so let's dig deeper into what puppet actually is so puppet is a configuration management tool maybe similar tools like Chef that you may already be familiar with it ensures that all your systems are configured to a desired and predictable state public can also be used as a deployment tool for software automatically you can deploy your software to all of your systems or to specific systems and this is all done with code this means you can test the environment and you can have a guarantee that the environment you want is written and deployed accurately so let's go through those components of Puppets so here we have a breakdown of the puppet environment and on the top we have the main server environment and then below that we have the client environment that would be installed on each of the servers that would be running within your network so if we look at the top part of the screen we have here our puppet master store which has and contains our main configuration files and those are comprised of manifests that are actual codes for configuring their clients we have templates that combine our codes together to render a final document and you have files that will be deployed as content that could be potentially downloaded by the clients wrapping this all together is a module of manifest templates and files you would apply a certificate authority to sign the actual documents so that the clients actually know that they're receiving the appropriate and authorized modules outside of the master server where you'd create your masses templates and files you would have public clients as a piece of software that is used to configure a specific machine there are two parts to the client one is the agent that constantly interacts with the master server to ensure that the certificates are being updated appropriately and then you have the fact that the current state of the client that is used and communicated back to through the agent so now that we know what puppet is let's look into the puppet architecture so puppet has a Master Slave architecture as you can see here on the top we have the puppet master and Below we have the puppet clients so the Puppet Master is a system where your system administrator sits the clients are those systems which require configuration you can have n number of clients connected to the puppet master so don't worry we look into each of these components in detail firstly we have the puppet master so the Puppet Master is where the system administrator sits and he or she types in the codes here so the codes are not only written in the system but they are also stored here and whenever the client requires these codes for their configuration they simply request it so manifest is the first sub-competent of the Puppet Master these are the actual files where the codes are written so in case of puppet these are Ruby files modules tie manifest templates and files into a single unit now these codes dictate the instructions for performing various tasks on your client the next component is templates so if manifests are your code templates are the data so let me explain this to you with an example consider you want to create a file called index.html on one of your client machines so what you do is you write the code for creating this file in your manifest and the content of this file is written in your templates then you simply link them but here's the thing when you divide your code and data in this way it's a very clean structure so it's preferred to do it in this manner files are your static contents so the files are basically send to your client and they are simply downloaded at the client there is no execution or compilation involved with your files now this is where files and templates are extremely different because with templates they are never sent over to your client the templates are simply read by the passer at the master itself modules ties the Manifest templates and files into a single unit so modules is not a separate component as such it just provides a very clean hierarchy this is the most important component of the Puppet Master CA which stands for certificate Authority now as we will see later in this video The Puppet Master and the puppet client communicate through an exchange of SSL certificates so when the client sends over the certificate to the master CA provides the master the authority to sign these certificates the minute the master signs a certificate the master is basically saying that yes this client is registered with me and I agree to exchange data with this client a next component puppet client of course these are the systems that require to be configured agent is a service installed on the client the agent is solely responsible for all the communication between the master and the client and finally we have Factor so factor is responsible for collecting all the data about the present state of our clients and sending this data over to the Puppet Master at the master this data is compared against the Manifest which are your code files and then these manifests are sent over to the client depending upon the results of this comparison so now that we had a look at the various components of puppet let's see how these work together so we have a master and we have three clients now before any information can be transferred between the master and the client it's important that these components recognize each other also there needs to be an encrypted channel of communication between each pair of Master and client now all the data which is transferred between the master and the client it flows through this channel so let's see how this entire procedure of recognizing each other works the core of this procedure are the SSL certificates SSL stands for secure socket layer secure socket layer is a security protocol and it forms an encrypted Channel between two components what this encrypted Channel ensures is that all the data that is passed through this channel is not accessible by any other system outside so it provides complete Security okay so we will consider a very general scenario we have a master and a client here this is not the puppet master or the puppet client they're just two systems and we're just going forward with Master Slave texture so the client so far is not a client it is just a system and we will refer to the system as system a so system a sends over a client certificate to the master associated with this client certificate is the host name of Systema now Master goes through the certificate it sees the host name and then it decides whether it wants to form a communication link with the system a if it does it signs the certificate and the signed certificate is sent back to system a now the system a is officially a client the minute the master signed the client certificate it said that yes I am willing to exchange information with this system and I do take the system as one of my clients so now that the system is officially a client it can request for data from the master and the master will simply send over the data now we are here to talk about puppet so if this was a puppet master and this was a puppet client then the first three stages will be the same so the puppet client sends over the certificate to the master the master Sciences certificate sends it back and the client request for data now by data here of course we mean the configuration data those are the manifests or even could be your files so what happens after this request after this request the factor service which is present on your clients collects all the information regarding the present state of your client so this information could include the operating system that's installed on them the various number of software present on them the versions of these software now all this information is collected it's called the fact and it's sent over to the master at the master this information that is the facts is compared against the Manifest so why does the master do this the master basically compares the facts against the Manifest to know if what the Manifest is about to perform is already performed on these systems so if the Manifest has some additional functionality they are compiled into catalogs and then these catalogs are sent over to the client now remember those agent Services which existed in your clients here's where they come in the agents take the catalog and execute them on the client once executed these catalogs make the required changes on the client and a report is generated this report is then sent back to the master so the master keeps record of all the changes that's made in the system and this comes very handy when one of the systems fail and it's required to be rolled back to its previous working state so this entire procedure can be repeated n number of times and can be repeated at regular intervals so if you have a scenario where a system needs to be configured every 30 minutes what you can do is you can set your interval and automatically every 30 minutes your systems will be configured if getting your learning started is half the battle what if you could do that for free visit skillup by simply learn click on the link in the description to know more so today I'll be showing you how you can use puppet to install Apache on one of your client machines so we'll be using two machines here we'll have a server and we'll have a node machine the server is essentially your master the node is your client I'm just using the term server and node instead because I want to keep it very close to as they are used in the command now both these machines are going to be sent to as seven machines and I have Oracle VM virtualbox installed on my machine so on that I've simply created two virtual boxes both Center S7 of course so before we move on to the actual demo I'll just take you through the various steps so first we'll install the puppet server on our server or Master machine we'll then start this server service we then shift over to our node or the client machine and we'll install the puppet agent service we will then start the service to and the minute you start the puppet agent service and SSL certificate is created so this certificate will have the host name of your you're not attached to it and this is how the master or the server will recognize this certificate and then it'll sign this certificate so once your master or the server signs the certificate you can write your manifest so the Manifest has just the codes for configuration and once that too is done you shift back to your client or the node system and you run the Asian service so once you run the agent service your manifest or your cat logs because as you know the manifests are compiled into catalogs these are executed on your node and this will make the required changes so that's how this entire process is going to work now let's move on to the actual demo this is my Oracle VM virtualbox manager the last two machines you see here are the ones we'll be using for this demo the puppet server is your puppet master puppet client of course it's the client so let's just have a look at these two machines this is my puppet server so just so you can differentiate guys the terminals have been set accordingly puppet server has a black background with white text and puppet client here has a black background with green text so we'll move on to server machine or the Master machine because that's the one we'll be working on first so before we begin we need to set the hostname of our Master to puppet so this is a very essential step in the configuration of puppet if you missed this out there are quite a lot of other changes you'll have to make in certain files so let's do this first so that's the command to set the hostname the last word you see there that's the new hostname you want okay so now we'll just check if our hostname is set so just type host name and yeah it's puppet so next we will check the IP of this machine so for that you just type if config enter so this is your IP guys under enp 0s3 in its value that is 192.168.2.183 of course the IP will differ for you guys needless to say foreign so note that down somewhere because we'll require it when we are configuring our client okay so now you want your puppet server and your client to communicate so for this we need to remove certain firewalls so the command for this is sudo because we want the root permission system CTL stop firewall d and put in the password okay so all your firewalls are removed so the next step now we start on with a puppet process so we download the Puppet Labs at first now guys ensure that your systems your virtual boxes in this case are connected to the net also you won't be getting an IP if it's not for the first time so here's the command to download the puppet lab foreign again okay so the Puppet Labs is downloaded now you need to install this so installation guys we will be installing the puppet server on the Master machine or the puppet server machine so this may take a while so that took a while so I just skipped to the last part where it's completed the installation process is over now we need to make a small change in our system config puppet server file so let's just move to the file first this file will open in VI editor so this line here it basically allocates memory for your puppet server now you do not need 2GB so we are going to reduce the size 512 MB should be enough so guys notice here I did not type the B okay so I just changed this to XMS 512 m let's save that always remember to open this file using sudo otherwise saving the changes you made could be quite Troublesome okay so now we can start our puppet server service we enable the service okay so that's done so basically a puppet server service has started it's running presently so for a while we are done with the server machine let's move on to our client machine okay first things first we open our Etc host file with sudo because you need to make a few changes and save it open this in the vi terminal thank you okay guys so here you have to add this one line at the end type in the IP for your puppet server machine foreign puppet Dash master so let's save this file now we'll move on to downloading Puppet Labs on our puppet client so the command is the same that we use for puppet server thank you so Puppet Labs is downloaded now we need to install the puppet agent service here again this will also take some time puppet servers installed let me just clear this window now that our puppet agent Services installed we'll start this service so I'll just type in the command for starting the puppet agent service as you can see here ensure changed stop to running so basically the service was not running before and now it's changed to the running state and while we executed this command not only did the servers start running but an SSL certificate was also created so this certificate should have the host name of this machine that's a perfect line machine attached to it so let's just check what the host name of our machine is that's localhost.local domain so a certificate should have this attached to it so guys the first time your client sends a certificate the name of the certificate itself is usually the host name of the machine so a certificate's name should be localhost.local domain we'll check this on our Master System and that's where we'll sign the certificate to so first we list out all the certificates in our case since we have just one client and we just started the client we should have technically just one certificate and it will be so yeah so this is a certificate localhost.local domain we'll now sign the certificate okay sorry about that just need to remove these brackets yeah as you can see here the certificate is signed signed Certificate request here so once the certificate is signed the request for signing the certificate is removed that's this line now that a certificate is signed we can begin to write a manifest so let's move into a manifest folder and we'll create a placeholder file so we'll create a file called sample.pp This is a sample file we won't be writing anything in this file as of now so we are just checking here we are just creating this file and then we'll go to our client machine and just check the if we can access this file from there thank you so now we'll move to our client machine so let's test the connection so as you can see here it says applied catalog in 0.03 seconds so our catalog is executed on a client machine now our catalog was just a placeholder basically so the file that we created which is sample.pp had no content and we were just testing our client now let's go back to our Master server put some content into this file and see how the execution takes place so we'll open our filesample.pp which we created earlier in the vi editor let's type our code here so node and then within the brackets we enter the hostname of our client machine so this ensures that our manifest is executed on this particular machine now the Apache service package on Centos is called the httpd so this line that I just typed in is the action it'll ensure that the httpd package is installed on your client and that's it so this is our code we'll save it now let's go back to our client machine and test it so the suckline machine and our catalogs applied on the client so I just cleared the screen now we'll start the Apache service okay so how do we see if it's running we just need to go to our browser and add localhost you must have the testing page for your Apache so we type localhost until and yeah this is the testing page for Apache so as you can see we have successfully used puppet to install the Apache service on our puppet client and this is the end of our demo so let's have a look at companies that are using puppets today there are a number of companies that have adopted puppet as a way to manage their infrastructure so companies that are using puppy today include Spotify Google ATT so why are these companies choosing to use puppet as their main configuration management tool the answer can be seen if we look at a specific company Staples so Staples chose to take and use puppet for their configuration management tool and use it within their own private Cloud the results were dramatic the amount of time that the it organization was able to save in deploying and managing their infrastructure through using puppets enabled them to open up time to allow them to experiment with other and new projects and assignments a real tangible benefit to a company so let's look at how you write a manifest in puppets so so manifests are designed for writing out in code how you configure a specific node in your server environment the manifests are compiled into catalogs which are then executed on the client each of the manifests are written in the language of Ruby without dot PP extension if we step through the five key steps for writing a manifest they are one create your manifest and that is written by the system administrator to compile your manifest it's compiled into a catalog three deploy the catalog is then deployed onto the clients for execute the catalogs are run on the client by the agent and then five and clients are configured to a specific and desired state if we actually look into how manifest is written it's written with a very common syntax if you've done any work with Ruby or really configuration of systems in the past this may look very familiar to you so we break out the the work that we have here you start off with a package file or service as your resource type and then you give it a name and then you look at the features that need to be set such as IP address then you're actually looking to have a command written such as present or start the Manifest can contain multiple resource types if we continue to write our manifest and pop it the default keyword applies a manifest to all clients so an example would be to create a file path that creates a folder called sample in a main folder called Etc the specified content is written into a file that is then posted into that folder and then we're going to say we want to be able to trigger an Apache service and then ensure that that Apache service is installed on a node so we write the Manifest and we deploy it to a client machine on that client machine a new folder will be created with a file in that folder and an Apache server will be installed you can do this to any machine and you will have exactly the same result results on those machines now let's have a look at the puppet community the puppet Community is a rich community of a lot of people officially there is no count but counting the various channels where people collaborate there are a little over 2 900 committers to GitHub projects who are directly or indirectly associated with Puppet Labs there are 4893 modules on the official puppet Forge these include modules from the Puppet Labs and also from open source contributors to GitHub so that's nearly 5 000 modules of ready-made code available on the puffet forge so before you write any code for puppet you can check to see if it's already been written by someone now there's a mailing list which has 6 588 participants apart from this on slack overflow you have a big tag of Puppet Labs in puppet which gives you a list of only puppet questions that's a good playground for you to interact with people who are already using puppet and who are contributing and helping others now let's have a look at puppet Forge so puppet Forge is a GitHub client for a repository which is hosted by Puppet Labs that has almost 5 000 modules of puppet code so these are both Puppet Labs and Community contributed the best thing about this is that they're easy to customize so when you have a requirement for anything you're doing with your infrastructure you can just grab one of those 5000 modules and then you can customize it according to your needs one of the other benefits is they have built-in tests so they follow all of the best practices now if you write your own module you also have to write your own tests and you have to spend extra time writing best practice code all of that has been taken care of and you can just download any one of those and customize it according to your needs and you're good to go so that's the benefit you get from such a large community where people contribute to the modules and the code and share with the community the puppet DSL puppet DSL is a puppet domain specific language it's the puppet's primary programming language it's just like plain English and you don't really need to be a programmer to write code in it it's kind of a sub-child of Ruby because it approaches and uses is equal to and greater than marks to Define things so it's very easy to learn you can specify packages that you want to install you can specify services to be managed you could do things based on the operating system that you have so we'll be writing a small puppet module which will be using this puppet DSL we will do that after we install the puppet server and the client and then we will write a simple puppet module to show how to use the puppet DSL let's have a look at the demo we are going to do so you will be setting up the machines using sudo DNS because we want to have a resolvable DNS because only by having a resolvable DNS can we use puppet because puppet uses the SSL certificates now we'll be installing the puppet server and the client we'll be signing the certificates of the client on the server side and then we'll proceed with writing an ntp module which we'll be using for installation of ntp on the client and then we'll look at the puppet programming language or DSL now we are going to look at a demo here so we will be setting up the machines we'll set up a pseudo DNS locally because puppet requires a fully qualified domain name in order to run since there is a certificate communication taking place here we will be installing the puppet server the puppet client we'll be signing the certificate and finally we will write an ntp module and make it run [Music] so now we will be starting a puppet master and a client I have two machines here one is the puppet master and the second is the puppet client the puppet master has got almost four gig of RAM and four vcpus which I can show you here because it's a Ruby on Rails application it requires a lot of CPU and RAM for the client I don't need that much RAM and CPU so I've allotted one gigabyte of RAM and one vcpu I will start both of these machines and find out the IP addresses right click normal start both of these machines are starting up the puppet client has started up and the first thing I'll do is log in using root and red hat password and then I will change the host name so I'll say Vim Etc host name this will open a hostname file in a text editor which is vim then we'll do a DD which will delete that existing hostname then I press I for insert mode and finally I'll type the puppet hyphen client which will be the host name of this machine I'll save this file by doing Escape semicolon WQ exclamation mark and I'll reboot the machine similarly going to the Puppet Master I'll say root red hat I'll open the hostname file say DD then I'll say puppet hyphen master I'll save this file and reboot the machine for the hostname change to take effect the hostname change does not take effect unless you reboot the machine now both of these machines are ready I'm going to find out the IPS and use the Mac terminal to log into these so this is 23 and 21. so 23 is my server and 21 is my client I'll SSH to 23 first now in order for the Puppet Master to be installed I need a fully qualified domain name here and a fake DNS which means a fully resolvable DNS name so if I do hostname here you see my host name is Puppet Master in order to set up a fake DNS I need to go to the ETC host file go to the end then I say IP address hostname local then space hostname now this is one of the techniques which makes the Linux server believe that there's a fully qualified domain name on this machine I'll save this then I do a hostname hyphen f you see it has a fully qualified domain name now so now we will be downloading the Puppet Labs repository for Ubuntu 16.04 which is zenial so I'll say curl hyphen 0 https apps.puppet labs .com Puppet Labs hyphen release hyphen pc1 hyphensannial.deb you can see there's a Debian file which has been downloaded here I will import this using the dpkg command so I'll say dpkg hyphen I Puppet Labs hyphen release hyphen pc1 hyphen xenial.dap and then I'll say apt-get update which will update my apt package indexes on this machine and now I'll install the puppet server so I'll say apt-get install puppet hyphen puppet server hyphen y so the puppet server is now installed and we will be configuring the memory allocation for the puppet server so I'll say Etc default puppet server so by default it has only got two gigabyte allocated we'll give it three gigabyte because it's jvm I'll make this three and I'll change this to three as well and I'll save this file I'll make sure the unconfigured firewall allows Port 8140 which is the puppet lab puppet server port the rules have been updated to allow that I'll now start the puppet server so I'll say a system CTL start puppet server it takes some time because it's jvm so it takes a couple of minutes to start the puppet server so the puppet server is now running I'll also enable it at boot time so I'll say systemctl enable puppet to server it will enable at boot time now I also need to configure a fully qualified domain name here so okay so I'll do that by going into the hosts file I'll go to the end and I'll say 192.168.1.23 and then puppet hyphen server.local and then puppet hyphen server if I do a hostname hyphen f put hyphen master.local because the current hostname is puppet hyphen master so the trick is to have the hostname.local in front of the IP address in the host file having a fully qualified domain name now now we will go to the client setup the client setup is almost the same with just a small difference the package which we need to install on the client is puppet agent so we will go to the client so you see this has got a host name now we also need to set up the fake DNS here so I'll copy this hostname open the host line go to the end write the IP address and then dot local and then hostname again so this will give me a fully qualified domain name so when I do a host name it says puppet Ivan client when I do a hostname hyphen f it says puppet client local to install the puppet agent on this machine I will need to download the zenio packages so I'll say wget https apt.puppetlabs.com Puppet Labs hyphen release hyphen pc1 hyphen xenial.deb I'll download this and then I'll say dpkg hyphen I Puppet Labs hyphen release hyphen pc1 hyphen xeniel.dap so this is the repository package which I'll be downloading here and then I'll say apt-get update hyphen y this will update my apt caches on this machine and now I'll finally install the puppet agent so I'll say apt-get install puppet agent hyphen agent hyphen y now the puppet agent is here I'll start the puppet agent and I'll enable it at boot time so I'll say system control start puppet then system control enable puppet now puppet is enabled now we have set up the fully qualified domain name on the client and the server but each of them should know each other so we'll have to do a cross certificate host mapping so we will be copying the host file entry of the client and adding it to the server so I'll go to the ETC hosts on the server side go to the end paste it there okay and I'll be adding the entry of the master on the client side okay now both of the servers will recognize each other now because one of them has the host entry of the other so if if I do from the master ping puppet hyphen agent dot local if I ping the agent from the server hyphenclient.local it is able to Ping it and if I ping the master from the client if I say ping puppet hyphen master.local both of them will be able to Ping each other now the next step would be to create a certificate signing request so I'll go to my to this directory which is opt Puppet Labs puppet bin and I'll say puppet agent hyphen T hyphen hyphen server is equal to and I'll give the fully qualified domain name of the Puppet Master then I'll say hyphen f when I do this it will create a certificate signing request you see now if I go to the server and I do an opt I'll check and see how many certificates are pending to be signed you'll see there is a public client certificate which is pending to be signed which is denoted by the absence of a plus sign here I'll be signing the certificate so I'll say puppet search sign and then the name so puppet hyphen client dot local this will sign the certificates and when I go to the server on the client side if I run the puppet agent again you see it's actually pulling down the catalog as of now it has an empty catalog that says there is nothing now we have configured the puppet agent and the master now we will be taking a module from Puppet Forge puppet Forge is a big collection of Open Source modules which are free to use as a first step we will be using a ready-made module and that would be a demonstration for how to actually get up and running with a module so search for Apache module here this is the Puppet Labs Apache module I'll be using the puppet command line puppet module install this is the command for that but I will make sure I run this command from the puppet home so I'll go to the master I'll go to Etc Puppet Labs code environments production in this I'll do modules now I will run it from the binary which is present at the opt Puppet Labs bin puppet and then module install Puppet Labs hyphen Apache and then this version so we are specifying the latest version and I'll hit enter when I press enter it starts downloading from the forge API so you see the module is downloaded in addition to the dependencies so there is concat STD lib and Apache so now I'll go to a folder here which is one level up called manifest so this is the file here Etc Puppet Labs code environments production manifest so in this there's a file which I'll be creating called site.pp the site.pp will specify and this is the main controller of how the infrastructure looks in the site PP I specify that I want puppet hyphenclient.local node to have the Apache module included I'll save this file and the only thing I need to do now is since all is set on the server side I need to go to the client side and run puppet agent it will automatically download the Apache module and compile it and may then make sure the server is in a state so when I press enter we see a lot of output it tries to configure Apache on this machine and done now when I go to the front end of this machine on this port I should be able to see apache's default page so this is apache's default page so the module which we downloaded from the forge was actually written in a way that Apache would be installed and run at boot time so that is why it is running here and that is all we have from the uh puppet module and this concludes the demo today we're going to go through a real tough battle we're going to decide which is better for your operations environment is it Chef puppet and support or salt stank all four are gonna go head to head so let's go through the scenario of why you'd want to use these tools so let's meet Tim he's our system administrator and Tim is a happy camper putting and working on all of the systems in his network but what happens if a system fails if there's a fire a server goes down well Tim knows exactly what to do he can fix that fire really easily the problems become really difficult for Tim however if multiple servers start failing particularly when you have large and expanding networks so this is why Tim really needs to have a configuration management tool and we need to now decide what would be the best tool for him because configuration management tools can help make Tim look like a superstar all he has to do is configure the right codes that allows him to push out the instructions on how to set up each of the servers quickly effectively and at scale all right let's go through the tools and see which ones we can use the tools that we're going to go through are Chef puppet and support and salt stacks and we have videos on most of these software and services that you can go and view to get an overview or a deep dive in how those products work so let's go and get to know our contestants so our first contestant is Chef and Chef is a tool that allows you to configure very large environments it allows you to scale very effectively across the entire ecosystem and infrastructure Chev is by default an open source code and one of the things that you find is a consistent metaphor for the tools that we recommend on simply learn is to use open source code code the code itself is actually written in language of Ruby and erlang and it's really designed for heterogeneous infrastructures that are looking for a mature solution the way that Chef works is that you write recipes that are compiled into cookbooks and those cookbooks are the definition of how you would set up a node and a node is a selection of servers that you have configured in a specific way so for instance you may have Apache Linux servers running or you may have a MySQL server running or you may have a python server running and Chef is able to communicate back and forth between the nodes to understand what nodes are being impacted and we need to have instructions sent out to them to correct that impact you can also send instructions from the server to the nodes to make a significant update or a minor update so there's great communication going back and forth if we look at the pros and cons the pros for Chef is that there is a significant and following for chef and that has resulted in a very large collection of recipes that allow you to be able to quickly stand up environments there's no need for you to have to learn complex recipes the first thing you should do is go out and find the recipes that are available it integrates with Git really well and provides for really good strong Version Control some of the consoles are really around the learning speed it takes to go from beginner user with Chef to being an expert there is a considerable amount of learning that has to take place and it's compounded by having to learn Ruby as the programming language and the main server itself doesn't really have a whole lot of control it's really dependent on the communication throughout the whole network alright let's look at our second Contender puppet and puppet is actually in many ways very similar to Chef there are some differences but again puppet is designed to be able to support very large heterogeneous organization it is also built with Ruby and uses DSL for writing manifests so there are some strong similarities here to Chef as with a chef there is a Master Slave infrastructure with puppets and you have a master server that has the manifests that you put together in a single catalog and those catalogs are then pushed out to the clients over an SSL connection some of the pros with public is that as with Chef there is a really strong Community around puppies and there's just a great amount of information and support that you can get right out of the gate it is a very well developed reporting mechanism that makes it easier for you as an administrator to be able to understand your infrastructure one of the cons is that you have to really be good at learning Ruby again as with shav you know the more advanced tasks really need to have those Ruby skills and as with Chef the server also it doesn't have much control so let's look at our third Contender here ansible and so ansible is slightly different it is the way the ansible works is that it actually just pushes out the instructions to the server environment there isn't a client server or Master Slave environment where ansible would be communicating backwards and forwards with its infrastructure it is merely going to push that instructions out the good news is that the instructions are written in yaml and yowel stands for yaml eight markup language yaml is actually pretty easy to learn if you know XML and XML is pretty easy if you know XML you're going to get yaml really well ansible does work very well on environments where the focuses are getting servers up and running really fast it's very very responsive and can allow you to move quickly to get your infrastructure up quick very fast and we're talking seconds and minutes here really really quick so again the way that ansible works is that you put together a Playbook and an inventory or you have a Playbook so the way that ansible works is that you have a Playbook and the Playbook it then goes against the inventory of servers and will push out the instructions for that Playbook to those servers so some of the pros that we have for ansible we don't need to have an agent installed on the remote notes and servers it makes it easier for the configuration yaml is really easy to learn you can get up to speed and get very proficient with yaml quickly the actual performance once you actually have your infrastructure up and running is less than other tools that we have on our list now I do have to add a Proviso this is a relative less it's still very fast it's going to be a lot faster than individuals manually standing up servers but it's just not as fast as some of the other tools that we have on this list and yaml itself as a language while it's easy to learn it's not as powerful as Ruby Ruby will allow you to do things that at an advanced level that you can't do easily with the ammo so let's look at our final Contender here salt stack so salt stack is a CLI based tool it means that you will have to get your command line tools out or your terminal window out so you can actually manage the entire environment via salt sack the instructions themselves are based on python but you can actually write them in yaml or DSL which is really convenient and as a product it's really designed for environments that want to scale quickly and be very resilient now the way that sort stat works is that you have a master environment that pushes out the instructions to what they call grains which is your network and so let's step through some of the pros and cons that we have here with salt stag so salt stack is very easy to use once it's up and running it has a really good reporting mechanism that makes your job as an operator in your devops environment much much easier the actual setup though is a little bit tougher than some of the other tools and and it's getting easier with the newer releases but it's just a little bit tougher and related to that is that sort stack is fairly late in the game when it comes to actually having a graphical user interface for being able to create and manage your environment other tools such as ansible have actually had a UI environment for quite some time all right so we've gone through all four tools let's see how they all stack up next with each other so let the race begin let's start with the first stage architecture so the architecture for most of our environments is a server client environment so for Chef puppet and salt snack so very similar architecture there the one exception is ansible which is a client Only Solution so you're pushing out the instructions from a server and pushing them out into your network and there isn't a client environment there isn't a two-way community education back to that main client for what's actually happening in your network so let's talk about the next stage ease of setup so if we look at the four tools there is one tool that really stands out for ease of setup and that is ansible it is going to be the easiest tool for you to set up and if you're new to having these types of tools in your environment you may want to start with ansible just to try out and see how easy it is to create automated configuration before looking at other tools now and so with that said Chef puppet and sauce stack aren't that hard to set up either and you'll find there's actually some great instructions on how to do that setup in the online community let's talk about the languages that you can use in your configuration so we have two different types of language with both chef and ansible being procedural in that they actually specify at how you're actually supposed to do the task in your instructions with puppet and salt stack it's decorative ways plus 5 only what to do in the instructions let's talk about scalability which tools scale the most effectively and as you can imagine all of these tools are designed for scalability that is the driver for these kind of tools you want them to be able to scale to massive organizations what do the management tools look like for our four contenders so again we have a two-way split with ansible and salt stack the management tools are really easy to use you're going to love using them they're just fantastic to use with puppet and Chev their management tools are much harder to learn and they do require that you learn some either the public DSL or the Ruby DSL to be able to be a true master in that environment but what does interoperability look like again as you'd imagine with the similar to scalability interoperability with these products is very high in all four cases now let's talk about Cloud availability this is increasingly becoming more important for organizations as they move rapidly onto cloud services well both ansible and salt stack have a big fail here neither of them are available in the most popular Cloud environments and puppet and Chef are actually available in both Amazon and Azure now we've actually just haven't had a chance to update our Chef link here but Chef is now available on Azure as well as Amazon so what does communication look like with all of our four tours so the communication is slightly different with them Chef has its own knife tool and whereas puppet uses SSL to secure sockets layer and ansible and soul stack use secure socket hashing and SSH as their communication tool bottom line all four tools are very secure in their communication so who wins well here's the reality all four tools are are very good and it's really dependent on your capabilities and the type of environment that you're looking to manage that will determine which of these four tools you should use the tools themselves are open source so go out and experiment with them there's a lot of videos our team has done a ton of videos on these tools and so feel free to find out other tools that we have then covered so you can learn very quickly how to use them but consider the requirements that you have and consider the capabilities of your team if you have Ruby developers or you have someone on your team that knows Ruby your ability to choose a broader set of tools becomes much more interesting if however you're new to coding then you may want to consider yaml based tools again the final answer is going to be up to you and we'll be really interested on what your decision is please post in the comments below which tool you have decided to go with and what are the challenges that you're coming up with because we'll help answer those challenges for you and someone provides you with one of the easiest ways to automate your application and it infrastructure on that note hi guys welcome to this complete answerable tutorial by simply learn in this video we'll be covering some of the most important Concepts relating to the devops tool ansible we take you through how you can install ansible and how you can use the tool to automate your processes first we have Anjali to tell us how to install ansible on your system a very warm welcome to all our viewers I'm Anjali from Simply learn and today I'll be showing you how you can install the configuration management tool ansible so let's have a brief about why one would use ansible and what exactly is ansible so if you consider the case of an organization it has a very large infrastructure which means it has more than probably hundreds of systems and giving one or even a small team of people the responsibility to configure all these systems makes their work really tough repetitive and as you know manual work is always prone to errors so ansible is a tool which can automate the configuration of all these systems with ansible a small team of system administrators can write simple codes in yaml and these codes are deployed onto the hundreds and thousands of servers which configures them to the desired States so ansible automates configuration management that is configuring your systems it automates orchestration which means it brings together a number of applications and decides an order in which these are executed and it also automates deployment of the applications now that we know what ansible does let's move on to the installation of ansible so here is my Oracle VM virtualbox manager I'll be using two systems there's the node system which is basically my client system and there's a server system or the Master System so let's begin at data service system so this is my Master System guys so the first thing we do is we download our ansible tool so one thing we must remember with ansible is that unlike Chef or puppet ansible is a push type of configuration management tool so what this means is that the entire control here lies with your master or your server system this is where you write your configuration files and these are also responsible for pushing these configuration files onto your node or client system as and when required Creator ansible tool is installed now we need to open our ansible host file and there we'll specify the details of our node or client machine so this is ansible host file as you can see here the entire file is commented but there's a certain syntax that you'd observe for example here we have a group name web servers under which we have the IP addresses or certain host name so this is about how we'll be adding the details for our client system first we need to give a group name under this group basically we add all the clients which require a certain type of configuration since we are using just one node we'll give only the details for that particular node first we need to add the IP address of our client machine so let's just go back to our client machine and this here is the IP address 192.168.2.11 once you have typed in your IP address give a space and then we'll specify the user for a client machine so all Communications between the server or the Master System and the client or the node system takes place through SSH ssh basically provides a secure channel for the transfer of information follow this up with your password in my case it's the roots password and that's it we are done so now we save this file and go back to our terminal so now that our host file is written the next thing we do is we write a Playbook so Playbook is the technical term used for all the configuration files that we write in ansible now playbooks are written in yaml yaml's extremely simple to both write and understand it's in fact very close to English so now we'll write our Playbook The Playbook or any code in yaml first starts with three dashes this indicates the beginning of your file next thing we need to give a name to our Playbook so name and I'm going to name my playbook sample book we next need to specify our host systems which is basically the systems at which the configuration file or the playbook in our case will be executed so we'll be executing this at the client machines mentioned under the group ansible servers so we had just one client machine under it we'll still mention the group name we next need to specify the username with which we'll be logging into our client machine which is Root in my case and become true specifies that you need to become the route to execute this Playbook so becoming the roots called a privilege escalation next we need to specify our tasks so these are basically the actions that the Playbook will be performing so you would have noticed everything so far is aligned that is name host remote user become and task because these are at one level now whatever comes under task will be shifted slightly towards the right although yamil is extremely simple to understand and read both it's a little tricky while writing because you need to be very careful about the indentations and the spacing so my first task is install httpd which is basically a Apache server so now my command yum and this will be installing the httpd package and the latest state of it will be installed so that's our first task now our second task could be running our Apache service so name run httpd and the action which is service will be performed on httpd hence the name httpd and state must be started now we come to our third task so here we'll create a very simple web page that will be hosted so create content is the name of our task and the content that we are creating here will basically be copied to a node system at a particular file location that will provide a Content will be congrats and then we'll provide the destination at which this file will be copied so this is the default location for all our HTML files and that's it we are done writing our Playbook just save this and go back to your terminal before we execute the Playbook or push the Playbook onto our node system let's check the syntax of our Playbook so the command for doing so is and if everything is fine with your playbook the output would be just your playbook name so our syntax is perfectly fine now we can push on the Playbook to unnode or the client machine and that's the Syntax for doing so now as your playbook is being sent over to the client machine you can see that first the facts are gathered that is the current state of your client machine is first fetched to check what all is to be changed and what is already present so the first thing is installing httpd so our system already had httpd so it says okay because this does not need to be changed and next task was running httpd now Although our system had the Apache service it was not running so that is one thing that was changed the next was there was no content available so the content was also added so two tasks were changed and four things were okay now everything seems fine and before you move any forward it is very important that you check this one line of documentation provided by ansible you have all kind of information available here regarding which all tasks were executed if your client machine was reachable or unreachable and so on so now that that everything is fine here we can move on to our node system and we'll just go to our browser so if our Playbook has been executed here what should happen is that the httpd service must be in the running State and the web page that we created should be hosted so let's just type localhost and great everything's working fine so our web page is displayed here so we come to an end for our installation and configuration video for the configuration management tool ansible if you have any doubts please post them in the comment section below and we'll definitely get back to you as soon as possible thanks Anjali now we have Matthew and Anjali to take us through how to work with ansible ansible today as one of the key tools that you would have within your devops environment so the things that we're going to go through today is we're going to cover why you would want to use a product like ansible what ansible really is and how it's of value to you in your organization the differences between ansible and other products that are similar to it on the market and what makes ansible a compelling product and I'm going to dig into the architecture for ansible we're going to look at how you would create a Playbook how you would manage your inventory of your server environments and then what is the actual workings of ansible there's a little extra we're going to also throw in ansible Tower one of the secret Source solutions that you can use for improving the Speed and Performance of how you create your ansible environments and finally we're going to go through a use Case by looking at HootSuite social media management company and how they use ansible to really improve the efficiency within their organizations so let's jump into this so the big question is why ansible so you have to think of ansible as another tool that you have within your devops environment for helping manage the servers and this definitely falls on the operations side of the devops equation so if we look here we have a picture of Sam and like yourselves Sam is a system administrator and he is responsible for maintaining the infrastructure for all the different servers within his company so some of the servers that he may have that he has to maintain could be web servers running Apache it could be database servers running my Sequel and if you only have a few servers then that's fairly easy to maintain I mean if you have three web servers and two database servers and let's face it will we all love just to have one or two servers to manage it would be really easy to maintain the trick however is as we start increasing the number of servers and this is a reality of the environments that we live and operate in it becomes increasingly difficult to create consistent setup of different infrastructures such as web servers and databases for the simple reason that we're all human as if we had to update and maintain all of those servers by hand there's a good chance that we would not set up each server identically now this is where ansible really comes to the rescue and helps you become an efficient operations team ansible like other system Solutions such as chef and puppet uses code that you can write and describe the installation and setup of your servers so you can actually repeat it and deploy those servers consistently into multiple areas so now you don't have to have one person redoing and re-following setup procedures you just write one script and then each script can be executed and have a consistent environment so we've gone through why you'd want to use ansible let's step through what ansible really is so you know this is all great but you know how do we actually use these tools in our environment so ansible is a tool that really allows you to create and control three key areas that you'd have within your operations environment first of all there's it automation so you can actually write instructions that automate the it setup that you would typically do manually in the past the second is the configuration and having consistent configuration imagine setting up hundreds of Apache servers and being able to guarantee with Precision that each of those Apache servers is set up identically and then finally you want to be able to automate the deployment so that as you scale up your server environment you can just push out instructions they can deploy automatically different servers the bottom line is you want to be able to speed up and make your operations team more efficient so let's talk talk a little bit about pull configuration and how it works with ansible so there are two different ways of being able to set up different environments for Server Farms one is to have a key server that has all the instructions on and then on each of the servers that connect to that main Master server you would have a piece of software known as a client installed on each of those servers that would communicate to the main Master server and then would periodically either update or change the configuration of the slave server this is known as a pull configuration an alternative is a push configuration and the push configuration is slightly different the main difference is as with a pull configuration you have a master server where you actually put up the instructions but unlike the pull configuration where you have a client installed on each of the services with a push configuration you actually have no client installed on the remote servers you simply are pushing out the configuration to those servers and forcing a restructure or a fresh clean installation in that environment so ansible is one of those second environments where it's a push configuration server and this contrasts with other popular products like chef and puppet which have a master's slave architecture with a master server connecting with a client on a remote slave environment where you would then be pushing out the updates with ansible you're pushing out the service and the structure of the server to remote hardware and you are just putting it onto the hardware irrelevant of the structure that's out there and there are some significant advantages that you have in that in that you're not having to have the extra overhead weight of a client installed on those remote servers having to constantly communicate back to the master environment so let's step through the architecture that you would have for an ansible environment so when you're setting up an ansible environment the first thing you want to do is have a local machine and the local machine is where you're going to have all of your instruction and really the power of the control that you'd be pushing out to the remote server so the local machine is where you're going to be starting and doing all of your work connect it from the local machine are all the different nodes pushing out the different configurations that you would set up on the local machine the configurations that you would write and you would write those in code within a module so you do this on your local machine for creating these modules and each of these modules is actually consistent playbooks the local machine also has a second job and that job is to manage the inventory of the nodes that you have in your environment the local machine is able to connect to each of the different nodes that you would have in your Hardware Network through SSH clients so a secure client let's dig into some of the different elements within that architecture and we're going to take a first look at playbooks that you would write and create for the ansible environments so the core of ansible is the Playbook this is where you create the instructions that you write to define the architecture of your Hardware so the Playbook is really just a set of instructions that configure the different nodes that you have and each of those set of instructions is written in a language called yamo and this is a standard language used for configuration server environments did you know that yaml actually stands for yaml 8 markup language well it's just a little tidbit to hide behind your ear so let's have a look or one of these playbooks it looks like and here we have a sample yaml script that we've written so you start off your yamla script with three dashes and that integrates the start of a script and then the script itself is actually consistent of two distinct plays at the top we have play one and below that we have play two within each of those plays we Define which nodes are we targeting so here we have a web server in the top play and in the second play we have a database server that we're targeting and then within each of those server environments we have the specific tasks that we're looking to execute so let's step through some of these tasks we have an install patchy task we have a start Apache task and we haven't installed my SQL task and when we do that we're going to execute a specific set of instructions and those instructions can include installing Apache and then setting the state of the Apache environment or starting the Apache environment and setting up and running the MySQL environment so this really isn't too complicated and that's the really good thing about working with yaml is it's really designed to make it easy for you as an operations lead to be able to configure are the environments that you want to consistently create so let's take a step back though we have two hosts we have web server and database server why do these names come from well this takes us into our next stage and the second part of working with ansible which is the inventory management part of ansible so the inventory part of ansible is where we maintain the structure of our Network environment so what we do here is part of the structure and creating different nodes is we've had to create two different nodes here we have a web server node and a database a server note and under web server node we actually have the names that were actually pointed to specific machines within that environment so now when we actually write our scripts all we have to do is refer to either web server or database server and the different servers will have the instructions from the yamascript executed on them this makes it really easy for you you to be able to just point to new services without having to write out complex instructions so let's have a look at how ansible actually works in real world so the real world environment with is that you'd have the ansible software installed on a local machine and then it connects to different nodes within your network on the local machine you'll have your first your playbook which is the set of instructions for how to set up the remote nodes and then to identify how you're going to connect to those nodes you'll have an inventory we use secure SSH connections to each of the servers so we are encrypting the communication to those servers we're able to grab some basic facts on each server so we understand how we can then push out the Playbook to each server and configure that server remotely the end goal is to have an environment that is consistent so this asks you a simple question what are the major opportunities that ansible has over chef and puppet really like to hear your answers in the comments below pop them in there and we'll get back to you and really want to hear how you feel that ansible is a stronger product or maybe you think it's a weaker product as it compares to other similar products in the market here's the bonus we're going to talk a little bit about ansible Tower so ansible Tower is an extra product that red hat created that really kind of puts the cherry on the top of the ice cream or is the icing on your cake ansible by itself is a command line tool however asphalt Tower is a framework that was designed to access ansible and through the ansible tower framework we now have an easy to use GUI this really makes it easy for non-developers to be able to create the environment that they want to be able to manage in their devops plan without having to constantly work with the command prompt window so instead of opening up terminal window or a command window and writing out complex instructions only in text you can now use drag and drop and mouse click actions to be able to create your appropriate playbooks inventories and pushes for your nodes alright so we've talked a lot about ansible let's take a look at a specific company that's using ansible today and in this example we're going to look at HootSuite now HootSuite if you not already use their products and they have a great product HootSuite is a social media management system they are able to help with you managing your pushes of social media content across all of the popular social media platforms they're able to provide the analytics they're able to provide the tools that marketing and sales teams can use to to be able to assess a sentiment analysis of the messages that are being pushed out really grateful and very popular but part of their popularity drove a specific problem straight to HootSuite the challenge they had at HootSuite is that they had to constantly go back and rebuild their server environment and they couldn't do this continuously and be consistent there was no standard documentation and they had to rely on your memory to be able to do this consistently imagine how complex this could get as you're scaling up with a popular product that now has tens of thousands to hundreds of thousands of users this is where ansible came in and really helped the folks over at HootSuite today the devops team at HootSuite write out playbooks that have Specific Instructions that Define the architecture and structure of their Hardware nodes and environments and are able to do do that as a standard product instead of it being a problem in scaling up their environment they now are able to rebuild and create new servers in a matter of seconds the bottom line is ansible has been able to provide HootSuite with it automation consistent configuration and free up time from the operations team so that instead of managing servers they're able to provide additional new value to the company I've very warm welcome to all our viewers I am Anjali from Simply learn and today I'll be taking you through a tutorial on ansible so ansible is currently the most trending and popular configuration management tool and it's used mostly under the devops approach so what will you be learning today you learn why you should use ansible what exactly is ansible the ansible architecture how ansible works the various benefits of ansible and finally we'll have a demo on the installation of Apache or the httpd package on a client systems we'll also be hosting a very simple web page and during this demo I'll also show you how you can write a very simple playbook in Yemen and your inventory file so let's begin why should you use ansible let's consider a scenario of an organization where Sam is a system administrator Sam is responsible for the company's infrastructure a company's infrastructure basically consists of all its systems this could include your web servers your database servers the various repositories and so on so as a system administrator Sam needs to ensure that all the systems are running the updated versions of the software now when you consider a handful of systems this seems like a pretty simple task Sam can simply go from system to system and perform the configurations required but that is not the case with an organization is it an organization has a very large infrastructure it could have hundreds and thousands of systems so here is where Sam's work gets really difficult not only does it get tougher Sam has to move from system to system performing the same task over and over again this makes Sam bored not just that repeating the same task leaves no space for Innovation and without any ideas or innovation how does the system grow and the worst of it all is manual labor is prone to errors so what does Sam do well here is where ansible comes in use with ansible Sam can write simple quotes that are deployed onto all the systems and configure them to the correct States so now that we know why we should use ansible let's look at what exactly is ansible ansible is an I.T engine that automates the following tasks so first we have orchestration orchestration basically means bringing together of multiple applications and ensuring an order in which these are executed so for example if you consider a web page that you require to host this webpage stores all its values that it takes from the user into a database so the first thing you must do is ensure that the system has a database manager and only then do you host your web page so this can of an order is very crucial to ensure that things work right next ansible automate configuration management so configuration management simply means that all the systems are maintained at a consistent desired state are the tools that automate configuration management include puppet and Chef and finally ansible automates deployment deployment simply means the deploying of application onto your servers of different environments so if you have to deploy an application on 10 systems with different environments you don't have to manually do this anymore because ansible automates it for you in fact ansible can also ensure that these applications or the code are deployed at a certain time or after regular intervals now that we know what exactly ansible is let's look at ansible's architecture ansible has two main components you have the local machine and you have your node or the client machine so the local machine is where the system administrator sits he or she installs ansible here and on the other end you have your node or the client systems so in case of ansible there's no supporting software installed here these are just the systems that required to be configured and they are completely controlled by the local machine at your local machine you also have a module a module is a collection of your configuration files and in case of ansible these configuration files are called playbooks playbooks are written in yaml yaml stands for yaml ain't a markup language and it is honestly the easiest language to understand and learn since it's so close to English we also have the inventory the inventory is a file where you have all your nodes that require configuration mentioned and based on the kind of configuration they require they are also grouped together so later in the demo we'll have a look at how the Playbook and the inventory is written and that will probably make it clearer so of course the local machine needs to communicate with the client and how is this done this is done through SSH ssh is your secure shell which basically provides a protected Communication in an unprotected environment okay so we saw the various components of ansible now how does ansible exactly work you have your local machine on one end this is where you install ansible if you've gone through any previous material on ansible you would have come across the term agentless often being associated with this tool so this is what agentless means you're installing ansible only on your local machine and there's no supporting software or Plugin being installed on your clients this means that you have no agent on the other end the local machine has complete control and hence the term agentless another term that you would come across with ansible is push configuration so since a local machine has complete control here it pushes The Playbook books onto the notes and thus it's called a push configuration tool now the playbooks and the inventory are written at the local machine and the local machine connects with the nodes through the SSH client this step here is optional but it's always recommended to do so it's where the facts are collected so facts are basically the current state of the node now all this is collected from the node and sent to the local machine so when the Playbook is executed the tasks mentioned in the Playbook is compared against the current status of the note and only the changes that are required to be made further are made and once the playbooks are executed your nodes are configured to the desired States so as I mentioned before ansible is currently the most trending tool in the market under the configuration management umbrella so let's have a look at the various benefits of ansible which gives it this position well ansible is agentless it's efficient it's flexible simple and important and provides automated reporting how does it do all this let's have a look at that agentless as I already mentioned before you require no supporting software or Plugin installed a new node or the client system so the master has complete control and automatically this means that ansible is more efficient because now we have more space in our client and node systems for other resources and we can get ansible up and running real quick ansible is also flexible so an infrastructure is prone to change very often and ansible takes no amount of time to adjust to these changes ansible cannot get any simpler with your playbooks written in a language such as yaml which is as close to English as you can possibly get it important basically means that if you have a Playbook which needs to be run n number of systems it would have the same effect on all of these systems without any side effect and finally we have automated reporting so in case of ansible your playbook has number of tasks and all these tasks are named so whenever you run or execute your playbook it gives a report on which tasks ran successfully which failed which clients were not reachable and so on all this information is very crucial when you're dealing with a very large infrastructure and finally we reach the most exciting part of our tutorial the Hands-On before we move on to the actual Hands-On let me just brief you through what exactly we'll be doing so I'll be hosting two virtual boxes both Centos 7 Operating Systems one would be my local machine and other my node or the client machine so on my local machine first I'll install ansible we then write the inventory and the Playbook and then simply deploy this Playbook on the client machine there's just one thing that we need to do is that we need to check if the configurations that we mentioned in our Playbook are made right so we'll now begin our demo this is my Oracle virtual box here I have my master system which is the local machine and this is the client machine so let's have a look at these two machines this is my client machine the terminal is open right now so the client machine terminal has a black background with white text and the Master machine terminal has a white background with black text just so you can differentiate between the two so we'll start at the Master machine the first thing to do is we need to install our ansible so yum install ansible hyphen Y is the command to do so so this might take some time yeah so ansible is installed the next step we go to our host file so host file here is basically the inventory it's where you'll specify all your nodes in our case we just have one node that's the path to your host file as you'll see everything here is commented so just type in the group for your client notes so I'm going to name it ansible clients [Music] and here we need to type the IP address of a client machine so my client machines IP address is 192.168.2.127 so before you come to this it's advice that you check the IP address on your client machine the simple command for that is ifconfig now once you type the IP address put a space and here we need to mention the username and the password for our client so I'll be logging in as the root user so this is the password and then the user which is Root in my case that's it now you can save this file foreign next we move on to our Playbook we need to write the Playbook so the extension for our Playbook is yml which stands for yaml and as you can see here I have already written my playbook but I'll just explain to you how this is done so a yaml file always begins with three dashes this indicates the start of your yaml file now the first thing is you need to give a name to the entire Playbook so I have named it sample book host is basically where this would be executed so as we saw earlier in our inventory I mentioned client's group name as ansible clients so we use the same name here the remote user is the user you'll be using at your client so in my case that's root and become true is basically to indicate that you need to set your privileges at root so that's called the privilege escalation now A playbook consists of tasks so we have here three tasks the first task I've named it to install httpd so what we are doing here is we are installing our httpd package which is basically the Apache server and we are installing the most latest version of it hence the state value is latest the next task is running httpd so for the service the name is httpd because that's the service we need to start running and the state is started our next task is creating content so this is the part where we are creating our web page so copy because this is the file that will be created at the client the content will be welcome and the destination of the file will be via www.html index.html as you know this is like a default path that we use to store all our HTML files now as you can see here there's quite a lot of indentation and when it comes to yaml although it's very simple to write and very easy to read the indentation is very crucial so the first Dash here represents the highest stage that is the name of the Playbook and all the dashes under tasks are slightly shifted towards the right so if you have two dashes at the same location they basically mean that they are siblings so the priority would be the same so to ensure that all your tasks are coming under the tasks label make sure they are not directly under name so yeah that's pretty much it so when you write your yaml file the language is pretty simple very readable indentations absolutely necessary make sure all your spaces are correctly placed we can now save this file next thing we need to check if the syntax of a yaml file is absolutely right because that's very crucial so the command to check the syntax of the yaml file is ansible Playbook the name of your playbook syntax check so we have no syntax errors which is why the only output you receive is sample dot yml which is the name of your playbook so our Playbook is ready to be executed the command to execute the Playbook is ansible Playbook and the name of your playbook so playbooks executed as you can see here Gathering facts that's where all the facts of the know that the present state of the note is collected and sent to the local machine so it's basically to check that if the configuration changes that we are about to make is already made so it's not made we do not have the httpd package installed on our node so this is the first change that's made also if it's not installed of course it's not running that's the second change that's made so it's put into the running state and a final task which is create content is under the OK State this means that the content is already present in the client machine so I made it this way so that you can at least see the different states that's present so over here we have okay four so four things are all fine the facts are gathered two things are changed and one is already present two changes are made zero clients are unreachable and zero tasks have failed so this is the documentation that I was referring to previously that ansible provides automatically and is very useful as you can see so our next step we need to just check on a client machine if all the changes that we desired are made so let's move to our client so this is my client machine so to check this since we are installing the httpd package and hosting a web page the best way to do it is open your browser and type in localhost so there you go your Apache server is installed and your web page is hosted hey there are you exploring test automation are you wondering why everyone is talking about selenium around you do you want to know what selenium is all about if yes this is the video for you simply learn brings to you a very simple tutorial on selenium what should you expect from this video before you start understanding any automation tool it's good to look back into what manual testing is all about what are its challenges and how automation tool overcomes these challenges challenges are always overcome by inventing something new so let's see how selenium came into existence and how did it evolve to become one of the most popular web application automation tool selenium Suite of tools selenium is not a single tool it has multiple components so we will look into each of them and as you know every automation tool has its own advantages and limitations so we will be looking at what the advantages are and the limitations of selenium and how do we work around those limitations all right so let's get started manual testing a definition if you can say a manual testing involves the physical execution of test cases against various applications and to do what to detect bugs and errors in your product it is one of the Primitive methods of testing a software this was the only method which we knew of earlier it is execution of test cases without using any automation tools it does not require the knowledge of a testing tool obviously because everything is done manually also you can practically test any application since you are doing a manual testing so let's take an example so say we have a use case you are testing say a Facebook application and in Facebook application let's let's open the Facebook application and say create an account this is your web page which is under test now now as a tester what would you do you would write multiple test cases to test each of the functionalities on this page you will use multiple data sets to test each of these fields like the first first name the surname mobile number or the new password and you will also test multiple links what are the different links on this page like say forgotten account or create a new page so these are the multiple links available on the web pages also you look at each and every element of the web page like your radio buttons like your drop down list apart from this you would do an accessibility testing you would do a performance testing for this page or say a response time after you say click on the login button literally you can do any type of tests manually once you have this test cases ready what do you do you start executing this test cases one by one you will find bugs your developers are going to fix them and you will need to rerun all these test cases one by one again until all the bugs are fixed and your application is ready to ship now if one has to run test cases with hundreds of transactions or the data sets and repeat them can you imagine the amount of effort required in that now that brings us to the first demerit of the manual test manual testing is a very time consuming process and it is very boring also it is very highly error-prone why because it is done manually and human mistakes are bound to happen since it's a manual executions tester's presence is required all the time one is to keep doing manual Steps step by step again all the time he also has to create manual reports group them format them so that we get good looking reports also send this reports manually to all stakeholders then collection of logs from various machines where you have run your test consoliding all of them creating repositories and maintaining them and again since it's all as a manual process there is a high chance of creating manual errors there scope of manual testing is limited for example let's say regression testing ideally you would want to run all the test cases which you have written but since it's a manual process you would not have the luxury of time to execute all of them and hence you will pick and choose your test cases to execute that way you are limiting the scope of testing also working with large amount of data manually is Impractical which could be the need of your application what about performance testing you want to collect metrics on various performance measures as a part of your performance testing you want to simulate multiple loads on application under test and hence manually performing these kind of test is not feasible and to top it all up say if you are working in an agile model where code is being churned out by developers testers are building their test and they are executing them as and when the bills are available for testing and this happens iteratively and hence you will need to run this test multiple times during your development cycle and doing this manually definitely becomes very tedious and burning and is this the effective way of doing it not at all so what do we do we automate it so this tells us why we automate one for faster execution two to be less error prone and three the main reason is to help frequent execution of our test so there are many tools available in the market today for automation one such tool is selenium birth of selenium much before selenium there were various tools in the market like say rft and qtp just to name a few popular ones selenium was introduced by gentlemen called Jason Huggins way back in 2004. he was an engineer at thoughtworks and he was working on a web application which needed frequent testing he realized the inefficiency in manually testing this web application repeatedly so what he did was he wrote a JavaScript program that automatically controlled the browser actions and he named it as JavaScript test run later he made this open source and this was renamed as the selenium core and this is how selenium came into existence and since then selenium has become one of the most powerful tool for testing web applications so how does selenium help so we saw all the demerits of manual testing so we can say by automation of test cases one selenium helps in Speedy execution of test cases since manual execution is avoided the results are more accurate No human errors since your test cases are automated Human Resources require to execute automated test cases is far less than manual testing because of that there is a lesser investment in human resources it saves time and you know time is money it's cost effective as selenium is an open source it is available free of cost early time to Market since you save effort and time on manual execution your clients will be merrier as you would be able to ship your product pretty fast lastly since your test cases are automated you can rerun them any point of time and as many times as required so if this tool offers so many benefits we definitely want to know more detail about what selenium is selenium enables us to test web applications on all kind of browsers like Internet Explorer Chrome Firefox Safari Edge Opera and even The Headless browser selenium is an open source and it is platform independent the biggest reason why people are preferring this tool is because it is free of cost and the qtp and the rft which we talked about are chargeable selenium is a set of tools and libraries to facilitate the automation of web application as I said it is not a single tool it has multiple components which we'll be seeing in detail in some time and all these tools together help us test the web application you can run selenium scripts on any platform it is platform independent why because it is primarily developed in JavaScript it's very common for manual testers not to have in-depth programming knowledge so selenium has this record and replay back tool called the selenium ID which can be used to create a set of actions as a script and you can replay the script back however this is mainly used for demo purposes only because selenium is such a powerful tool that you should be able to take full advantage of all its features so in name provides support for different programming languages like Java python c-sharp Ruby so you can write your test scripts in any language you like one need not know in-depth or Advanced knowledge of these languages also selenium supports different operating systems it has supports for Windows Macs Linux even Ubuntu as well so you can run your selenium test on any platform of your choice and hence selenium is the most popular and widely used automation tools for automating your web applications selenium set of tools so let's go a little more deeper into selenium as I said selenium is not a single tool it is a suite of tools so let's look at some of the major components or the tools in selenium and what they have to offer so selenium has four major components one selenium ID it's the most simplest tool in the suite of selenium it is integrated development environment earlier selenium IDE was available only as a Firefox plugin and it offered a simple record and Playback functionality it is a very simple to use tool but it's mainly used for prototyping and not used for creating Automation in the real-time projects because it has its own limitations like any other record and replay tool selenium RC this is nothing but selenium remote control it is used to write web application test in different programming language what it does it basically interacts with the browser with the help of something called as RC server and how it interacts is its uses a simple HTTP post get request for communication this was also called as selenium 1.0 version but it got deprecated in selenium 2.0 version and was completely removed in 3.0 and it was replaced by Webdriver and we will see in detail as why this happened selenium Webdriver this is the most important component in the selenium Suite it is a programming interface to create and execute test cases it is obviously the successor of the selenium RC which we talked about because of certain drawbacks which RC hat so what Webdriver does is it interacts with the browsers directly unlike RC where the RC required a server to interact with the browser and the last component is the selenium grid so selenium grid is used to run multiple test scripts on multiple machines at the same time so it helps you in achieving parallel execution since the selenium Webdriver with you can only do sequential execution grid is what comes into picture where you can do your parallel execution and why is parallel execution important because in real time environment you always have the need to run test cases in a distributed environment and that is what grid helps you to achieve so all this together helps us to create robust web application test Automation and we will go in detail about each of these components so before that let's look at the history of selenium version so what did selenium version comprised of it had an IDE RC and grit and as I said earlier there were some disadvantages of using RC so RC was on its path of deprecation and web driver was taking its path so if you look at selenium 2 version it had an earlier version of Webdriver and also the RC so they coexisted from tree thought onwards RC was completely remote and Webdriver took its place there is also a four dot version around the corner and it has more features and enhancements some some of the features which are talked about are w3c Webdriver standardization improved ID and improved grid now let's look at each of the components in the selenium Suite selenium IDE is the most simplest tool in the suite of selenium it is nothing but an integrated development environment for creating your automation scripts it has a record and Playback functionality and is a very simple and easy to use tool it is available as a Firefox plugin and a Chrome extension so you can use either of this browser to record your test scripts it's a very simple user interface using which you can create your scripts that interact with your browser the commands created in the scripts are called selenis commands and they can be exported to the supported programming language and hence this code can be reused however this is mainly used for prototyping and not used for creating automation for your real-time projects why because of its own limitation which any other record and replay tool has so a bit history of selenium ID so earlier selenium ID was only a Firefox extension so we saw that IDE was available since the selenium version one selenium ID died with the Firefox version 55 that was ID was stopped supporting from 55 version onwards and this was around 2017 time frame however very recently all new brand selenium ID has been launched by apply tools and also they have made it a cross browser so you can install the extension on Chrome as well as as an add-on on Firefox browser so they completely revamped this IDE code and now they have made it available on the GitHub under the Apache 2.0 license and for the demos today we'll be looking at the new ID now with this new ID also comes a good amount of features reusability of test cases better debugger and most importantly it supports parallel test case execution so they have introduced a utility called selenium side Runner that allows you to run your test cases on any browser so you can create your automation using IDC on Chrome or Firefox but through command prompt using your site Runner you can execute this test cases on any browser thus by achieving your cross browser testing control flow statement so initially in the previous versions of idea there were control flow statements available however one had to install a plugin to use them but now it is made available out of box and what are this control flow statements these are nothing but your if else conditions the while Loops the switch cases so on it also has an improved locator functionality that means it provides a failover mechanism for locating elements on your web page so let's look at how this ID looks and how do we install it and start working on that so for that let me take you to my browser so say let's go to the Firefox browser so on this browser I already have the ID installed so when you already have an ID installed you will see an icon here which says selenium ID and how do you install this you simply need to go to your Firefox add-ons here whether it says find more extension so just type in selenium ID and search for the sixth section so in the search results you see the selenium ID just click on that and now since I've already installed here it says remove otherwise for you it is going to give you an add button here just click on the add button it will install this extension once it is installed you should be able to see this selenium ID icon here okay so now let's go ahead and launch this ID so when I click on that it is going to show me a welcome page where it's going to give me few options the first option is it says record a new test case in a new project so straight away if you choose this option you can start recording a test case in which case it's going to just create a default project for you which you can save it later then open an existing project so you can open if you already have a saved project create a new project and close so I already have an existing project with me for the demo purpose so I'll go ahead and open that so I'll say open existing project and I have created a simple script what the script does is it logs me into the Facebook using a dummy user mail sorry username and password that's all it's very simple script with few lines and this is what it's going to do so what we will simply do is we'll just run the script and see how it works for that I'm just going to reduce the test execution speed so that you should be able to see every step of execution here all right so so what I'll do now here is I'll just adjust this window and I'll just simply say run current test all right so I'll just get this side by side so that you should be able to see what exactly the script is doing okay so now you are able to see both the windows okay so now it's going to type in your user email here there you go and now it will enter the password and it has clicked on the login button so it's going to take a while to say login and since these are the dummy IDs it is you are not able to log in here and you're going to see this error window fine that is what is the expected output here now on the ID if you look here after I execute the test case every statement or every command which I have used here is color coded in green so that means this particular step was executed successfully and then here in the log window it will give you a complete log of this test case right from the first step till the end and your end result is it says FB login which is my test case name completed successful let's look at few components of this ID the first one is the menu bar so let's go to our ID all right so the menu bar is right here on the top so here is your project name so either you can add a new project here or rename your project so since we already have this project which is named as Facebook and then on the right you have options to create a new project open an existing project or save the current project and then comes on toolbar so using the options in this toolbar you can control the execution of your test cases so first one here is the recording button so this is what you use when you start recording your script and then on the left you have two options here to run your test cases the first one is run all tests so in case you have multiple test cases written here you can execute them one by one sequentially by using this run all test else what you can do is if you just want to run your current test this is what you would use then idea has this debugger option which you can use to do a step execution so say for example now whenever I run the script it's going to execute each and every command here sequentially so instead if I just select the first command and say do step execution all right so what it does is the moment it finishes the First Command which is opening of Facebook right I think which is already done here yeah all right so once this is done it is going to wait immediately on the second command and it says paused debugger so from here you can do whatever you would like to do in case you want to change the command here you can do that you can pause your execution you can resume your execution here right you can even completely stop your test execution or you can just select this to run the rest of the test case so if we say run the test case what it is going to do is it's just going to Simply go ahead and complete the com complete the test case now there is another option here which is you see the timer there which says test execution speed so to execute your test cases in the speed you want say whenever you're developing an automation script right and say you want to give a demo so you need to control the speed sometimes so that the viewer is able to exactly see all the steps which is being performed and this gives you an option to control that complete execution right so do you see the grading here so we have somewhere from Fast to completely slow execution so the previous demo which I showed was I control the speed and then I executed it so that we could see every command how it is being executed all right so what's the next this is called as an address bar so whichever Wherever Whenever you enter an URL here that is where you want to conduct your test and another thing what it does is it gives a history of all the URLs which I've used for running your test then here is where your script is recorded So each and every instruction is displayed here in the order in which you have recorded the script and then if you look here you have something called as log and reference so now log is an area where it records each and every step of your command as in when they get executed right so if you see here it says open https Facebook book.com and ok so that means this command was executed successfully and after the complete test case is done it gives you whether the test case passed or filled so in case there is a failure you'll immediately see this test case is filled in red color also there is something called as reference here for example say if I click on any of this command the reference tab what it is going to show me is the details of this command which I have used in the script it gives you the details of the command as well as what the arguments have been used or how how is that you need to be using this particular command okay so now what we'll do is let's go ahead and write a simple script using this ID so with this you'll get an ideas how do we actually record scripts in ID so for that I have a use case here a very very simple use case so what we will do is we will open amazon.in then we'll search simply search for say a product iPhone and once we get that search page where all your iPhones are displayed we will just do an assert on the title of the page simple all right so let's do that so first thing what I need is a new URL okay so first let me go to my Firefox browser here and say amazon.in so why I'm doing this just to Simply get the right URL absolute URL path here and so that I don't make any mistakes while typing in the URL okay so I got this so let me close all this windows I don't need any of this let's minimize this all right so here what I will do in the tests tab I'll say add a new test and name this test as uh Amazon search done I'll say add now I'll enter this URL which I just copied it from my browser okay and then I'll just say start recording so what it did was since I have entered the URL in this address box it just opened the amazon.in URL now let's do the test case so in my test case what I said was I want to search for iPhone once I have this I'm just going to click on my search button so now this gives me a list of all iPhones and then I said I want to add an assertion on the title of the this page so for me to do that what id gives me an option is I have to just right click anywhere on this page and you'll see the selenium ID options here so in this I will select assert title and then I will close this browser so that kind of completes my test case so now take a look at all the steps which has created for me so it says open slash because I've already provided the URL here so either you can replace it with your regular URL or you can just leave it as it is so what I will do is since this is going to be a proper script and I might be using this to run it from a command prompt also so I'll just replace this target with the actual URL and then what it is doing it is setting a window size then there are whatever I did on that particular URL on that website it has recorded all the steps for me so this is where it says type into this particular text box which is my search box and what did it type iPhone this was the value which I entered now there was one more feature which I told you in this new ID which had which I said it has a fail our mechanism for your locating techniques now that is what this is now if you look here this ID is equal to two tab search text block this is nothing but the search box where we entered the text iPhone and it has certain identification through which this IDE identifies that web element and that has multiple options to select that particular search box so right now what it has used is ID is equal to two tab search box however if you know the different locating techniques you will be able to see here that it has other techniques also which it has identified like the name and the CSS and the XPath so how does this help in failovers say tomorrow if Amazon dot in website changes the ID of this element right you are not going to come and rewrite the scripts again instead by using the same script what it will do is if this particular ID fails if it is unable to find the element using the first locator which is the ID it simply moves to the next available ones and it tries to search for that element until one of this becomes true that is what was the failure mechanism which has got added it's a very brilliant feature because most of our test cases break because of element location techniques well let's come back to this so then we added an assert title right so what is assert Title Here it simply captures the title of that particular page and it checks this is all a very simple test case so what we will do now is we will stop the recording and then I've also given a closed browser so right now what I will do is I'll just comment this out why because if I just run this test case it's going to be very fast and you might not be able to catch the exact command execution what has happened all right so right now I'll just disable it so that it will just do all these test cases and it just stays there without closing the browser so now I'll just say run the current test so your Amazon in is launched okay it is typed in the iPhone it's also clicked on the search so it is done so now if you look here since we are in the reference tab it is not able to show so let's go to the log and now let's see the log so it's going to be running log so if you notice here the previous examples which we have run for Facebook is also in the same lock so we all have to see the log from running Amazon search because that's our test case so if you see here every command line right it was executed successfully assert title was also done and your test case was executed successfully so it passed now what we will do is on this assert title I'll just modify this and let's say just add some text I'll just add double s here now this by intentionally I am going to fail this test case just to show you that whenever there is a test case failure how does the ID behaves and how do you get to know the failures all right so I'll just run the test test case again so before that let's close the previous window all right done and now here I'll also uncomment the close because anyway it's a failure which I'm going to see which I should be able to see it in the logs so I'll close the browser after the execution of test case Okay so let's simply go and run the test case Okay amazon.in is launched it should search for iPhone now yeah there you go all right now it should also close the browser yes let us close the browser and it has failed now see here now this is the line where our Command fit why because the expected title was not there and if you look in the logs it says your assert title on amazon.in failed actual result was something different and it did not match with what we had asked it for so this is how simple it is to use your ID to create your automation scripts so we saw all the components of ID we saw the record button then I showed you the toolbar I showed you the editor box and also the test execution lock so now let's come to what are the limitations of this ID with IDE you cannot export your scripts your test scripts to web drivers scripts this support is not yet added but it is in The Works Data driven testing like using your Excel files or reading data from the CSV files and passing it to the script this capability is still not available also you cannot connect to database for reading your test data or perform any kind of database testing with selenium Webdriver yes you can also unlike selenium Webdriver you do not have a good reporting mechanism with the ID like say for example test NG or report NG so that brings us to the next component of The Suite which is selenium RC selenium remote control so selenium RC was developed by Paul Hammond he refactored the code which was developed by Json and was credited with Json as a co-creator of selenium selenium server is written in Java it is used to write web application test in different programming languages as it supports multiple programming languages like your Java C sharp Pearl Python and Ruby it interacts with a browser with the help of an RC server so this RFC server uses a simple HTTP get and post request for communication and as I said earlier also selenium RC was called as selenium 1.0 overshill but it got deprecated in selenium 2.0 and was completely removed in 3.0 and it got replaced by what Webdriver and we'll see why this happened and what was that issue which we had with the RC server so this is the architecture of selenium remote control at a very high level so when Jason Huggins introduced selenium you know the tool was called as JavaScript program and then that was also called as a selenium core so every HTML has a JavaScript statements which are executed by web browser and there is a JavaScript engine which helps in executing this command now this RCA had one major issue now what was that issue say for example you have a test script say test dot JavaScript here which you are trying to access elements from anywhere from the google.com domain so what used to happen is every element which is accessible is are the elements which can belong only to google.com domain like say for example mail the search or the drive so any elements from this can be accessible through your test scripts however nothing outside the domain of say google.com in this case was accessible say for example if your test scripts wanted to access something from yahoo.com this was not possible and this is due to the security reasons obviously now to overcome that the testers what they had to do was they had to install the selenium core and the web server which contained your web application which is under test on the same machine and imagine if you have to do this for every machine which is under test this is not going to be feasible or even effective all the time and this issue is called as the same origin policy now what the same origin policy issue says is it prohibits a JavaScript from accessing elements or interacting with scripts from my domain different from where it is launched and this is purely for the security measure so if you have written manuscripts which can access your google.com or anything related to google.com these scripts cannot access any elements outside the domain like as we said in the example yahoo.com this was the same origin policy now to overcome this what this gentleman did was he created something called as selenium remote control server to trick the browser in believing that your course your selenium core and your web application under test are from the same domain and this is what was the selenium remote control so if you look at again a high level architecture or how did this actually work first you write your test scripts which is here right in any of the supported language like your PHP or your Java or Python and before we start testing we need to launch this RC server which is a separate application so this selenium server is responsible for receiving the selenis commands and these Salinas commands are the ones which you have written in your script it interprets them and reports the result back to your test so all that is done to you RC server the browser interaction which happens through RC server right from here to your browser so this happens through a simple HTTP post and get request and that is how your RC server and your browser communicate and how exactly this communication happens this RC server it acts like a proxy so say your test scripts asked to launch a browser so what happens is this commands goes to your server and then your RC server launches the browser it injects the JavaScript into the browser once this is done all the subsequent calls from your test script right from your test scripts to your browser goes through your RC and now upon upon receiving these instruction your selenium core executes this actual commands as JavaScript commands on the browser and then the test results are displayed back from your browser to your RC to your test scripts so the same cycle gets repeated right until the complete test case execution is over so for every command what you write in your Java JavaScript here or your test script here goes through a complete cycle of going through the RC server to the browser collecting the results again from the RC server back to your test scripts so this cycle gets repeated for every command until your complete test execution is done so RC had definitely a lot of shortcomings and what are those so RC server needs to be installed before running any test scripts which we just saw so that was an additional setup since it acts as a mediator between your commands which is your Cellini's commands and your browser the architecture of RC is complicated why because of its intermediate RC server which is required to communicate with the browser the execution of commands takes very long it is slower we know why because every command in this takes a full trip from the test script to your RC sir takes a full trip from the test script to your RC server to the core engine to the browser and then back to the same root which makes your overall test execution very slow lastly the API is supported by RC are very redundant and confusing so RC does have a good number of apis however it is less object oriented so they are redundant and confusing say for example say if you want to write into a text box how and when to use a type key command or just a type command is always confusing another example is some of the mouse commas using a click or a mouse dot both kind of you know almost providing a similar functionality so that is the kind of confusion which developers use to create hence selenium RC got deprecated and is no more available in later selenium versions it is obsolete now now to overcome these shortfalls web driver was introduced so while RC was introduced in 2004 web driver was introduced by Simon Stewart in 2006. It's a class platform testing platform so Webdriver can run on any platform like say Linux Windows Mac or even if you have a Ubuntu machine you can run your selenium scripts on this machine it is a programming interface to run test cases it is not an ID and how does this work actually so test cases are created and executed using web elements or objects using the object locator and the web driver method so when I do a demo you will understand what this Webdriver methods are and how do we locate the web elements on the web page it does not require a core engine like RC so it is pretty fast why because web driver interacts directly with the browser and it does not have that intermediate server like the RC hat so each browser in this case what happens is each browser has its own driver on which the application runs and this driver is responsible to make the browser understand the commands which you will be passing from the script like say for example click of a button or you want to enter some text so through your script you tell which browser you want to work with say Chrome and then the Chrome driver is responsible for interpreting your instructions and to execute it on the web application launched on the Chrome browser so like RC Webdriver also supports multiple programming languages in which you can write your test Scripts so another advantage of web driver is it supports various Frameworks like test NG junit and unit and report and change so when we talk about the limitations of Webdriver you will appreciate how this support for various Frameworks and Tool help in making the selenium a complete automation solution for web application so let's look at the architecture of Webdriver at a high level what is in Webdriver so Webdriver consists of four major components the first one is we have got client libraries right or what we also call it as language bindings so since selenium supports multiple language and you are free to use any of the supported languages to create your automation script these libraries are made available on your selenium website which you need to download and then write your scripts accordingly so let's go and see from where do we download this so if I go to my browser so seleniumhq.org right so if you're working with selenium this website is your Bible so anything and everything you need to know about selenium right you need to come here and use all the stabs here in this website so right now what we are going to look at is what are those language binding so for that I'll have to go to this download tab here okay and if you scroll down here you will see something like selenium client and web driver language bindings and for each of the supported language of selenium you have a download link right so say for example if you're working with Java here what you need to do is you need to download your Java language binding so let's go back to the presentation so this is where your language bindings are available next so selenium provides lots of apis for us to interact with the browser and when we do the demo I'll be showing you some of this APS right and these are nothing but the rest apis and everything whatever we do through the script happens through the rest calls then we have a Json wire protocol what is Json JavaScript object notation it is nothing but a standard for exchanging data over the web so for example you want to say launch a web application through your script so what selenium does it it creates a Json payload and posts the request to the browser driver that is here and then we have this browser drivers themselves and as I said there is a specific driver for each browser as you know every tool has its own limitation so does selenium so let's look at what these limitations are and if there are any workarounds for them cannot test mobile applications requires framework like APM selenium is for automating web application it cannot handle mobile applications mobile applications are little different and they need its own set of automation tool however what selenium provides is a support for integrating this APM tool which is nothing but a mobile application automation tool and using APM and selenium you can still achieve mobile application automation and when do you usually need this when your application under test is also supported on mobile devices you would want a mechanism to run the same test cases on web browser as well as your mobile process right so this is how you achieve it the next limitation so when we talked about the components of selenium I said that with Webdriver we can achieve only sequential execution however in real time scenario we cannot just live with this we need to have a mechanism to run our test cases parallelly on multiple machines as well as on multiple browsers so though this is a limitation of web driver but what selenium offers is something called as grid which helps us achieve this and we will see in shortly what the selenium grid is all about also if you want to know more details as how do we work with the grid how do you want to install that grid so do check out our video on simply learn website on selenium grid third limitations so limited reporting capability so selenium Webdriver has a limited reporting capability it can create basic reports but what we definitely need is a more so it does support some tools like say test engine report NG and even extend reports which you can integrate with selenium and generate beautiful reports powerful isn't it also there are other challenges with selenium like selenium is not very good with image testing especially for the ones which are designed for web application automation but then we have other tools which can be used along with selenium like autoit and securely so if you look at all this selenium still provides a complete solution for your automation needs and that's the beauty of selenium and that is why it makes the most popular tool of today for automation okay let's do a quick comparison between the selenium RC and the web driver so RC has a very complex architecture you know why because of the additional RC server whereas due to direct interaction with the browser web driver architecture is pretty simple execution speed it is slower in RC and much faster than Webdriver why because in Webdriver we have eliminated the complete layer of selenium server right that the RC server and we established a direct communication with the browser through browser drivers it requires an RC server to interact with the browsers we just talked about it and whereas Webdriver can directly interact with the browser so RC again we talked about this as one of the limitations that we have lot of redundant ABS which kept developers guessing as which API to use for what functionality however Webdriver offers pretty clean apis to work with RC did not offer any support for headless browser whereas in Webdriver you do have a support for using headless browsers let's see the web driver in action now now for the demo we will use this particular use case and what this use case says is navigate to the official simply learn website then type the selenium in search bar and click on it and click on the selenium 3.0 training so we are basically searching for selenium 3.0 training on the simply learn website first let's do the steps manually and then we will go ahead and write the automation script so let's go to my browser on my browser what I'll do is let me first launch the simply learn website okay and here what my use case step says is I need to search for selenium and click on the search button so once I do that it is going to give me a complete list of all kind of selenium trainings which is available with simpler and what I'm interested in is the selenium 3.0 training here once I find this on the web page I need to go and click on that all right so this is all the steps which we are going to perform in this use case okay now so for writing the test cases I'll be using an ID which is Eclipse I've already installed my eclipse and also I have installed selenium in this instance of my Eclipse all right so if you can see the reference library folder here you will see all the jars which are required for the selenium to work next another prereq which is required for selenium and that is your driver files now every browser which you want to work with has its own driver file to execute your selenium scripts and since for this demo I'll be working with the Firefox browser I will need a driver file for Firefox now driver file for Firefox is the gecko driver which I have already downloaded and placed in my folder called drivers now where did I download this from let's go ahead and see that so if I go back to my browser and if you go to your selenium hq.website you have to go to this download tab here in the download tab when you scroll down you will see something like third party drivers bindings and plugins in this you'll see the list of all the browsers which is supported by selling it and against each of this browser you will find a link which has the driver files now since we'll be using the gecko driver this is the link where you need to go to and depending on which operating system which you're working on you need to download that particular file now since I am working on Mac this is the file which I'm using if you're a Windows user you need to download this ZIP file and unzip it so once you unzip that you would get a file called gecko driver for your Firefox or a chrome driver for your Chrome browser and then what you do is you just create a directory called drivers under your project and just place the driver files here so these are the two prereqs for your selenium one is importing your jar files like this and then having your drivers downloaded and keep them under a folder where you can reference to okay so now we'll go ahead and create a class I already have a package created in this project so I'll use this project and create a new class so I'll say create new Java class and let's call this as search training I'll be using a public static wordman and I'll click on finish so let's remove this Auto generated lens as we do not need all right now the first statement which you need to write before even you start writing the rest of your code is what you need to do is you need to define or declare your driver variable using your class web driver so what I would do is I'll say Webdriver driver done all right now you will see that this ID is going to flash some errors for you that means it is going to ask you to import certain libraries which is required by the web driver so simply just go ahead and say import web driver from org dot open sq.cellaneous this is the package which we will need all right so you have a driver created which is of the class web driver and now after this I am going to create three methods all right so first method I will have for launching the Firefox browser okay and then I will write a simple method for searching selenium training and clicking on it this is the actual use case what we'll be doing and then third method I'm going to write is just to close the browser which I'm going to be opening right so these are the different methods which I'll be creating and from the public static wordman I will just call these methods one after the other okay so let's go ahead and write the first method now my first method is launching the Firefox browser so I'll say public void since my return type is null or there is no return type for this let's call it as launch browser okay all right now in this for launching any browser I need to mention two steps now the first step is where I need to do a system.set property okay let's do that first and then I'll explain what this does I'll just say system dot set property so this accepts a key and a value pair so what is my key here my key here is Webdriver dot gecko dot driver and I need to provide a value so value is nothing but the path to the gecko driver and we know that this gecko driver which I'm going to use here is right here in the same project path under the drivers folder okay and that is what the path which I'm going to provide here so here simply I need to say drivers slash gecko driver gec KO all right done and let me close this sentence all right now since I'm a Mac User my gecko driver installable is just the name gecko driver if your windows use user and if you are running your selenium scripts on the Windows machine you need to provide a complete path to this including dot exe because driver executable on your machines is going to be gecko driver.exe all right so just make sure that your path which you mentioned here in the system.set property is the correct path okay then the next thing what we need to do is I need to just say driver is equal to new Firefox driver okay so this command new Firefox driver creates an instance of the Firefox browser now this is also flagging me error why because again it's going to ask me to import the packages where the Firefox driver class is present okay we did that now these two lines are responsible for launching the Firefox browser form so this is done so what's my next step in the use case now I need to launch the website simply learn so for that we have a command called driver dot get driver dot get what it does is whatever URL you're going to give it here in this double quotes as an argument it is going to launch that particular website and for us it's a simply learn website so what I do as a best practices instead of typing out the URL I go to my browser launch that URL which I want to test and I simply copy it come back to your eclipse and just simply paste it so this ensures that I do not make any mistakes in the URL okay so done so our first method is ready where we are launching the browser which is our Firefox browser and then launching the simply learn website now the next method what is my next method in my next method I need to give the search string to search selenium training on this particular website now for that we need to do few things what are those few things let's go to the website again all right so let me relaunch this let's close this okay let me remove all this and let's go to the home page first okay this is my home page so as you saw when I did a manual testing of this I entered the text here so now since I have to write a script for this first I need to identify what this element momentous for that what I'm going to do is I'm just going to say right click here and I'll say inspect element all right now this element let's see what attribute it has which I can use for finding this element so I I see that there is an ID present so what I'm going to do is I'm just going to Simply use this ID and then I'll just copy this ID from here go back to Eclipse let's write a method first so I'll say public void and what do we give the method name say search training or just search all right now in this I need to use a command called driver dot find element by ID is what I'm going to use as a locating technique and in double quotes the ID which I copied from the website is what I'm going to paste here okay and then what am I going to do on this element is I need to send that text the text which I'm going to search for which is selenium so I'll just say send keys and whatever text I want to send I need to give it in double quotes So for that selenium so this is done so now I've entered the text here and after entering the text I need to click on this button so for that I need to first know what that button is so let's inspect that search button okay now if you look at the search button other than the tag which is span and the class name I do not have anything here all right so what I can do is I can either use the class name or I can write an X path since this is a demo which we have already used ID locating technique I would go ahead and use the X path here so for me to construct an X path I will copy this class first okay and then I already have a cropath installed on my Firefox so I'll use the crow path and first test my X bar so I'll just say double slash let's see what was that element it has a span tag okay so I'll have to use pan and at class equal to and I'll just copy the class name here and let's see if it can identify that element yeah so it is able to identify so I'll just use this x path in my code so I'll go back to eclipse and I'll say driver dot find element by dot X path and the X path which I just copied from cropath is what I'm going to paste here and what is the action I need to do here I need to say click done so I have reached a stage where I have entered this selenium okay and then I have clicked on the search button once I do this I know that expected result is I should be able to find this particular link here selenium 3.0 training okay and I should be able to click on that so for that again I need to inspect this so let's inspect this selenium 3.2 all right so now what are the elements this has now this particular element has attributes like it has a tag Edge too then it has got some class name and some other attributes so I would again would like to use a x path here now this time while using the X path I am going to make use of a text functionality so that I can search for this particular text so I'll simply copy this I'll go to my Crow path the tag is H2 so I'll say simply H2 okay and here I'll say text equal to and this is the text which I copied I missed out that yes there so I'm just going to add an S okay so let's first test here whether it is able to identify that element yeah so it is able to identify so can you see your blue dotted line it is able to show us which element it is identified so I'll copy this x path now and let's go to my ID Eclipse so now here what I need to do is I'll have to again simply say driver dot find element by dot X path and paste the X path which we just did and then again I have to do a click operation done all right so technically we have taken all the steps of the use case and we have written the commands for that now let's add an additional thing here say after coming to this page after finding this we want to uh say print the title of this page now what is the title of this page if you just hover your mouse on this it says online and classroom training for professional certification courses simply now so what I will do is after doing all these operations I will just print out this page title on our console so for that I have to just do this drive dot so let's do a sys out so I'll set this out system.out.println okay and here I would say let's add a text here the page title is and then let's append it with driver dot get title so this is the command which we'll be using to fetch the page title done now what is the last method I need to add just to close the browser all right so let me add a method here I'll say public void close browser and it's a one single command which I need to call I'll say driver dot quit Okay and then I need to call all these methods from my public static void main so I'll let me use my class name which is this so I'm going to create an object obj is equal to new class name and then using this object first is I need to call the method launch browser and then I'll call the method search right and then I'll call the method close browser so technically our script is is ready with all the functionality which we wanted to cover from our use case now there are few other tweaks which I need to do this and I'll tell you why I need to do this now for example after we click here right after we click on the search if you observed on your website it took a little while before it listed out all the selenium trainings for us and usually when you are actually doing it you wait for the selenium 3.0 training to be available and then you click on that now same thing you also need to tell your scripts to do that you need to tell your scripts to wait for a while until you start seeing the selenium 3.0 training or it appears on your web page there are multiple ways to do that in your script and it is a part of overall synchronization what we call where we use kind of implicit and explicit kind of favorites now since this is a demo for demo purpose what I'm going to do is I am going to use a command curve thread.sleep and I'm just going to give an explicit weight of say three seconds so you can use this many only for the demo purposes you can use a thread.sleep command now this thread.sleep commands needs us to handle some exceptions so I'm just going to click on ADD throws declaration and say interrupted exception now same thing I'll have to do it in my main function also okay so let's do that and complete it all right so this is done so by doing this what am I doing I'm ensuring that before I click on the selenium 3. training we are giving enough time for the script to wait until the web page shows this link to the selenium 3.0 training that's one thing I'm doing all right and also now since you are going to be seeing this demo through the video recording the script when it starts running it is going to be very fast so you might just miss out saying how it does the send keys and how did it click on the search button for us to enable us to see it properly I'll just add some explicit weights here just for a demo purpose so after entering the keys right so what I'll do is I'll just give a simple thread.sleep here yeah okay so probably a three seconds or two seconds which should be good enough okay three seconds weight should be good enough here so that we should be able to see how exactly this works on your browser when we execute this okay now our complete script is ready so what I'll do is I'll just save the script and then we will simply run the script so to run the script I just say right click run as Java application okay it says ask me to select and save I have saved the script now so let's observe how it runs okay the simply learn.com the website is launched so the selenium text has been entered in the search box it has clicked on the search okay all right so now it did everything whatever we wanted it to do all right so since we are closing the browser you are unable to see whether the selenium three dot training was selected or not however what I have given here is to fetch the title after all these operations were complete and if you see here the complete operations was done and we were able to see the page title here okay so now what I'll do since we are unable to see whether it clicked on the selenium 3.0 training or not I'll just comment out the closed browser uh the command okay so we will not call the close browser so that the browser remains open and we get to see whether did it really find the training link or not okay so let me close this window we don't need this Firefox window close all tabs and then I'll just re-execute this script so I'll say run as Java application so save the file okay simplylearn.com is launched so search text is entered now it's going to click on the search button yes all right so we've got the search results it should click on selenium 3.0 training and yes it is successfully able to click on that all right so now it's not going to close the browser because we have commented on that line however it did print us the title here all right so this is a simple way of using the selenium Scripts selenium grid so grid is used to run multiple test scripts on multiple machines at the same time with Webdriver you can only do sequential execution but in real time environment you always have the need to run test cases in distributed environment and that is where selenium grid comes into picture so grid was conceptualized and developed by Patrick the main objective is to minimize test execution type and how by running your test parallelly so design is in such a way that commands are distributed on multiple machines where you want to run tests and all these are executed simultaneously what do you achieve by this methodology of course the parallel execution on different browsers and operating system grid is pretty flexible and can integrate with many tools like say you want a reporting tool integrated to pull all the reports from the multiple machines where you're running your test cases and you want to present that report in a good looking format so you have an option to integrate such report okay so how does this grid work so grid has as a hub and node concept which helps in achieving the parallel execution let's take an example say your application supports all browsers and most of the operating system like as in this picture you could say one of them is a Windows machine one of them is a Mac machine and another one is say a Linux machine so your requirement is to run the test on all supported browsers and operating system like the one which is depicted in this picture so what you have to do is first thing is you configure a Master machine or what you also call it as a hub by running something called a selenium Standalone server and this Talent Standalone server can be downloaded from the selenium HQ website using the server you create a hub configuration that is this node and then you create nodes specific to your machine requirement and how are these nodes created you again use the same server which is your Standalone selenium server to create the node configuration so I'll show you where the selenium server can be downloaded so if we go back to our selenium HQ website so you can see here right on the top it says selenium Standalone server and this is the latest version available so you just need to download once you download this keep a version of this one on your master machine that is your Hub and this should also be present on your node machines and there is a certain command which you can use to install the server as a node and as a master depending on your configuration so once that is done you have your master and your master or the Hub and the notes created your master can control on which nodes you want to execute test now say if this is a Mac machine this is a Windows machine through your master or your Hub you control on which machine you want to execute what is then also you could have a multiple combination of tests like say for example smoke test to be run on all these nodes that's the kind of configuration you can create or a certain component and test to be run on say only Windows 10 with an edge browser or say regression test on Mac machine so any such things whatever configurations you have for your node what happens is the Hub picks the right node which confirms to your requirements or the configurations which you have set up and then sends a command to the node to execute those test cases on the Node everything is controlled through your master machine only here what happens on the Node so once the node gets the command from the master the node actually runs all the test cases which is it which it intends to run on the specific browser which you have mentioned in your configuration file so this is a very high level architecture and how a grid works so if you want to know more details of how the grid works you can refer to our video on simply learn all right again limitations so what limitation does is grid have now the first one it has relatively poor scalability compared to many modern applications it is pretty static each server is configured in advance with a subset of required browser instances available so if you want to vary this you'll need to reconfigure so any configuration tomorrow on any machines you want to change it cannot be done automatically you need to completely reconfigure that node and then add it back to the grid thirdly although it is available to run on Virtual servers it is actually not designed as a cloud native application and because of as a result of this it isn't optimized to take advantage of things like say distributed storage or dynamic scaling or even automatic failover so these are some of the limitations of the grid advantages of selenium speed and accuracy of course since there is no manual execution your test can run faster and with grid it helps in parallel execution and you can run or execute large volumes of test within a very short time frame let's take an example say before just before the release of your product say during the last lap of your testing you find a bug and of course it has to be fixed so the developer fixes the bug and you want to run all the test cases pertaining to that area now can you imagine that if you have to do this testing manually how long it's going to take so by automation you can achieve this in a very short duration of time and it will also help you release your product in time with very minimal or absolutely no human errors the second Advantage it's an open source so it's available free of cost so anyone can download it and start using it it supports wide spectrum of programming languages we have been seeing throughout this presentation it is not restricted to any particular programming language so whatever language you are comfortable with you have an option to use that and write your selenium scripts selenium has support for almost all browsers and operating system we also talked about headless browser support So this helps you create test cases once using any of the browser and run them across all browsers and all operating system thus saving you lot of time from manual execution and helping in achieving a very broad test coverage and we also saw that really easy to use tool right so you can check out all our videos and you can it's just a matter of time that you can Master this killer writing your automation scripts reusability of course like any programming language it provides you a mechanism to reuse your code and avoid redundancy well again let's see an overall what are the limitations of selenium itself now since selenium is open source that is one of the biggest Advantage now there is a little disadvantage which comes with that and what is that you do not have much technical support because it's an open source code however there are loads and loads of documentations and forums available which you can definitely refer to and selenium as we said it is only for automating web applications it cannot handle the mobile applications or even the desktop application however it does provide support for integrating tools like APM for mobile testing selenium is not very good with image testing especially designed for your web application tools again we have tools like autoit and securely which can be integrated with selenium selenium Webdriver has limited a limited reporting capability but again it does provide us support for integrating tools like test NG report NG and extend report it does not have test management capabilities it need not right not one tool need not have each and everything what is required but there is always a way to integrate it so selenium does provide a way to integrate any of the test management tools now since selenium supports multiple programming language the developer of test automation will require to have some basic knowledge of any of the programming language it supports to write effective automation scripts right so this you could look at it as an advantage or as a disadvantage so overall if you look at all this right selenium still provides a complete solution for our automation need and that is why we can still say that selenium is one of the most popular tool used in the industry today for your web application automation how to install selenium on Windows system selenium installation is a three-step process so it has certain prereqs the first prerequis you need to have Java on your system so we will be installing Java first and then we will be working with Eclipse ID so we will be installing eclipse and then we will install Selena for Java we will install the version Java 8 and for Eclipse we have a version 4.10 this was the last stable version which was released in December last year so I'll be using that version and selenium we will download the latest 3.14 version okay so let's get started with our first step which is the Java installation so to install Java let's go to the browser and simply just search for Java 8 download so now you will see that there is an oracle site which is listed there and that is where you would be downloading all your Java package so go ahead and click on that and for you to download any jdk package from the Oracle site you need to create an account so if you already have one you just need to log in using that account and then you can download any of the jdkits and if you do not have one please go ahead create a new account on the Oracle log into that account and then you can just download the Java 8. so since I already have an account and I have already downloaded the package but I'll show you how and where to download it from so in this page if you scroll down so you will see this Java development kit 8211 so this is the version we'll be downloading it so click on the accept license agreement and then since we are working on the Windows system today so we will be downloading this the windows package so just click on that and it will get downloaded in your downloaded folder and as I said I've already downloaded the packages so here it is what I've done is I've just created a directory called installers and I'm going to be keeping all my installables here so here I have a folder called Java installer and this is where my installable is so now that we have this file so we will just go ahead double click on it and launch this installer the installer is launched and just click on run so this will take a few minutes to install Java the installer is launched now just click on the next button here so here for the installation directory you can change the directory to the choice of whatever drive and the folder structure you want to I would like to leave it as default here and we'll just go and click on next and then the Java installation is in progress so let's wait until this is completed it really shouldn't take too much time maybe just a few more minutes here okay accept the license term just click on next we leave the destination folder as it is so jdk8 is successfully installed on your system so close the installer now and let's go ahead and check whether the installation is done properly so for that what I'll do is I'll go to my command prompt and I'll suggest say Java minus version so it says Java version 1.8 and this tells us that the Java is installed successfully now after this installation there are couple of configurations which we need to do and what are those configuration one is you need to set the path variable and then we are also going to set a Java home directory so for that first let's go ahead and check where is the Java installed actually let's figure out the directory first so if you remember the directory structure where the Java got installed was in program files Java I have there are certain previous versions which are installed and then uninstalled it so that is why you see some residuals here sitting here that's not worry too much about that instead let me go to the latest one what I have installed which is this okay and there is a bin folder here and this is the path which we need to set in our path variable so so what I will do is I'll just copy this path and then go to your control panel here go to your where is my system yeah so click on the system go to Advanced system setting and here in the environment variables find the PATH variable okay and then say edit now what are we doing here in the path variable is we are going to add the Java bin directory to the path be very careful whenever you are editing your path variable do not overwrite anything always go into the edit mode go towards the end here and then just say Ctrl V paste the path which you have just copied from the Explorer window that's it now just say okay done so your path setting is done so what's the next one we need to do we need to add a new environment variable called the Java now what I'll do for that is I just say new I'll just type Java home here and what is the value of this we need to set we need to set the same path but without the print directory so we just need to set the path till your Java directory that is this so we'll just copy the path again and paste it here that is up just say Okay click on Okay click on OK here and we are done so again let's go to our Command Prompt and just say Java minus version so everything seems to be fine so now successfully we have installed Java on the system so what is our next installation step what we have now we need to install the eclipse so let's go back to the browser again so to download Eclipse we will be downloading the package from the eclipse.org so when you go here to eclipse.r you can see the latest version which is available and the latest version available when this video was made was 20906 so especially with Eclipse since it's an open source I prefer to work with the last stable version and so does most of the developers do and hence that is the reason why I have picked up the version which is like last year's version which is 4.10 which was released in last December so you can always choose to go with the latest version but then if there are any issues and if you are like first time working with the eclipse you're going to get confused as where these issues are coming from right so I would still recommend that you use the last stable version which is available with your records so now to get the last stable version what you need to do is go and click on this download packages and here if to scroll down this page you can see here more downloads so there is a list of all the previous releases of app Clips which is available and this is what we need to download so just click on that 4.10 version and then click on the OS on which you want to install eclipse for us it is Windows so I'll just click here on the 64-bit windows and then click on the download and you will be downloading the complete package so once you download this is what it will look like so let's go back to our directory of installers so this is the installer for the Eclipse which I got now what's the next step I need to do just launch this installer and install Eclipse so I'll just say double click on this I'll say run so here you will see multiple options here for Eclipse installation so depending on your requirement you can go ahead and install any of these packages so for us we just need an eclipse ID for Java developers so I'll select this and I'll say install so again you'll have a choice of directory where you want to install so I have chosen D drive here this is a default directory name it takes which is okay we can leave it as it is and then also you have an option to create a start menu entry and desktop shortcut so just leave the default selection as it is and go ahead and click on install so this will take a while to install the eclipse this is select all you can close this window just say select all and accept it okay so the installation has been completed successfully so let's go and click on this launch and let's see the first window what opens when you launch the eclipse you need to specify a workspace directory now what is this workspace directory so this is a directory or a folder wherein all the Java files or any programs or any artifacts which are going to create through Eclipse will be stored in this particular folder so this could be any location on your system so this is you can go ahead browse the location and change it so for in our case what we will do is I'll go to the D drive and I already have a directory so here I'll create I'll just create select this folder and then create a folder called workspace I'll say my workspace and then I'll say launch so every time I open the eclipse right so if this is going to take as my default workspace and all my programs all my javascripts or my automation scripts are getting are going to be stored in this particular location so we'll say launch so this is a welcome window which opens we can just close this and there we go the eclipse is open with a certain perspective so there are certain Windows here which we do not need let's just close them so now the first thing what you do after launching the eclipse is go ahead and create a new project so I'll say file new and since I'm going to be using Java with selenium I'll say create Java project so give a project name let's say my first project now you have an option here to select the JRE which you want to use so we just install this jdk 1.8 okay so I'm going to click on use default JRE otherwise you also have an option to use a Project Specific gr for example I could have two different projects where one project I'm going to be working with GRE 1.8 and there is another project which I want to work with the latest Java maybe Java 12 and I can have more than one Java installed on the machine so this gives me an option to select whichever Java I want to work with so if you have another Java installed here it will show up in this list so and you can just go ahead and select them now since we have only one Java installed on our machine which is Java 1.8 I will say use default GRE which is 1.8 and I will click on finish now if You observe this folder structure the project which is created see all the reference libraries to this particular Java have been created here now we are ready to create any kind of java programs in this project so now we have successfully done the second step of our installation which is the eclipse installation after this we need to install the selenium so again let's go back to the browser and see what files we need to download to install selling so let me go to my browser and here I will be going to the seleniumhq.org so if you are working with selenium this particular website the seleniumhq.org is going to be a Bible everything and anything related to selenium is available in this website whether you want to download the files whether you want to refer to the documentation anything regarding to selenium is available here so what we want now is the installables for selenium so here go to the the download tab now for you to install selenium and start working with selenium there are three things which are required for you to download one is a standalone selenium server so this is not required immediately when you get started with selenium however when you start working with remote selenium Webdriver you would be requiring this when you have a grid setup you will be requiring the Standalone server so for that what you can do is you can just download the latest version available here so when you click on that it'll download the file into your download folder so this is one particular file which you need to keep next selenium client and Webdriver language mine takes now in today's demo we will be looking at selenium with Java so that means my client package of java is what I need to download so whatever programming language selenium supports we have respective downloadables available with that so if you're working with python then you need to download your client library for Python and since we are working with Java you need to download this package so simply what you need to do click on this link and it will download the Java package file which are basically the jar files so we have client libraries now and then there is another component what we need now with a linear you are going to be automating your web browser applications correct and you also want your applications to run on multiple browsers so that means your scripts the automation scripts which you create should be able to run on any browser selenium works with multiple browsers like Edge Safari Chrome Firefox and other browsers even it has a support for headless browser now every browser which it supports comes with its own driver file now say for example we want to say work with Firefox driver so that means for us to start working with Firefox browser we need to download something called as a gecko driver here and if you want to work with Chrome browser you need to install the Chrome driver so depending on what browsers you'll be testing with go ahead Click on each of this link and download the latest driver files now since we are going to be working with Firefox in this demo what I need to do is I just need to click here on the latest link so when I click on the latest link it is going to take me to this driver files so driver files are specific to each of the operating system so if you go down here you will see there is a separate driver file available for Linux for mac and for Windows so depending on which operating system where you'll be running your test download that particular driver file and this is the driver file I need because we are working on Windows machine so these are the three different packages which we need to download from the selenium hq.org for us to install sending so let me show you the folder where I have already downloaded all this so if you see here selenium Java 3.141.59 okay this is nothing but our client Library which we saw here let's go back to the main page here that is this so once I download this this is a ZIP file after I unzip the file this is the folder structure I see and let's see what is there in this folder selector so there are two jar files here and then in the lips there are multiple jar files and we will need all this to work with selling it and then we also downloaded the driver files so what I did was after downloading those driver files for the browser I created a directory here called drivers and have kept all my browser drivers here so I have a driver file downloaded for Chrome I want to work with Firefox so I have a gecko driver here and then for Internet Explorer that's it so this is all we need so once we have all this what you need to do is go to your eclipse in the eclipse right click on the project which you have created and then go to the build path and say configure build path go to the libraries tab here now do you see this JRE libraries here this is what got installed first and now similarly we are going to add the selenium Jazz to this library and how do we add that on your right you can see this add external jars click on ADD external Jazz go to your folder where you have downloaded your selenium which is this select all the jar files which is available so I have two jar files here I'll just say click open again I will click on ADD external jars now from the lips folder I will select all this five so select all the five jars and click on open so you should see all the seven jar files here so once you have this just say apply and close now if you look into your project directory here you'll see some a folder called referenced library and this is where you will see all the selenium charts here this is a very simple installation in Eclipse when you want to install selenium you just need to export all the jars of the selenium into eclipse and now your system is ready to start working with selenium scripts so for us to install Java let us go to our browser and simply what I'll do is I'll just Google for Java 8 download so we would select the Oracle site for downloading the SD okay so once you go here you can see that we have something called as Java software development kit 8 u211 so this is the version we'll be working with so in order for you to download you need to have an oracle account so since I already have one I have directly been logged into that so if you do not have an oracle account please go ahead create an account for yourself so that you can download any of the Java installers now let's go for and download our package so I'll be downloading this package for that I need to click on accept license agreement and then since we are going to be working on Mac this is the package which I'm going to download so let's go ahead and click on that and now the download is going to start however we will not wait for the complete download to happen since I have already downloaded the package and I have kept it in one of my folders so let me show you where that is so let me go to my finder so what I have done is I have created a folders called installers and all the software which we are going to require today for us to work with selenium is what I have downloaded it and kept it here so we'll go step by step so that you understand what each of these packages are so as you see here I have the jdk8 which I've already downloaded so let's go ahead and install jtk so how do we install jdk so you have the DMG file here you just go ahead and double click on that okay and then I'll double click on this package and then we will say continue and install so it's going to ask me for my password okay I've given my password and I'll say install the software it's probably it will just take a couple of minutes to install Java okay there you go so now the Java has been installed so let me close this window let me close this window too and then now I'll go to my terminal window so let's open my terminal and here let's say Java minus version bingo so there we have Java 1.8 installed on our system now there are couple of other things which you need to do apart from installation of your jdk you also need to set some system variables right so in Mac how do you set the system variables or the environment variables you do that in your bash profile so let's see how we do that all right so what I will do is I'll use an editor which is my VI okay and then I'll open my bash profile so what is Bash profile it's a config file for your bash cell this is mainly used to export variables in your shell like in Windows we are going to do the system variable setting from the advanced setting right so similarly we are going to do that here so I'll say VI Dot bash underscore profile so this is the file where I will be adding Java home path now I have other parts here already written because I'm working with other multiple software on this machines and now I'll go to the end of this file and I'll say export Java home equal to now what is the path we need to give for Java home we need to give the path where the Java has been installed on your system okay first let's go and find out that path so let's go back to our finder and then I have a folder called Library here so here there is a Java and then there is something called as Java virtual machines so if you have a single Java or multiple Java versions installed so all those versions you are going to see under this directory which is Java virtual machines and Java one dot is is what we installed just now so let me go and open that folder there is something called as content and then home and in this you have your JRE and Bin folder so that means I need to give my Java Home Path until here till this home directory so simply what I'm going to do is I'll just go here say get info because I want to just copy the path and what you can do is you copy this path okay and then go back to your shell and here I'm going to paste this path plus I want to tell the home directory so I'm just going to type out homie home that is it so you have now set your Java home environment variable so simply you do save the file Escape colon WQ and come out of it and now what you need to do for this batch profile uh to be activated you just exit out of this terminal okay close this terminal window and I'll go ahead and open another terminal so let me close this info text so here let me again give Java minus version okay so everything seems to be working fine so this completes our Java installation now your system is ready and you have your Java up and running other thing which you also can do is go to your system preferences okay here you will see this Java control panel okay this is another way of finding out that your Java has been installed successfully on the machine so you can just open this Java control panel and if there is any settings which you want to do here you can do it through your control panel also all right let's close this let's minimize this window and this folder Windows so we have successfully done our step one so what's our step two we need to go ahead and install the eclipse now same thing what I will do I'll go to my browser okay and now I'll open another tab here and I'll say Eclipse download and we'll go to this eclipse.org now here whenever you're going to do this installation you will see the latest version appears here my recommendation to you is instead of if you're working first time instead of working on the latest version available I would suggest that you work with the last stable version and that's exactly what we are going to do today so for me to go to the previous versions I just need to go here on this download packages click on it okay so once you click on that on this page if you scroll down you will see something called as more downloads right in this you will see all the bills of Eclipse which is available all your previous versions So since today I'll be working with 4.10 version I will go ahead and click on this 4.10 this 4.10 was released last year this is 2018 December okay and now when you look at this installer you have multiple packages available here so depending on your need you can choose which package you want to work with so today for our demo what I would be using is Eclipse ID for Java developers so since we are going to be working on Mac I will go ahead and click on this downloadable and then I'll click on this download which is going to download this particular file for me I've already downloaded this file and kept it in my folder let's go to my folder okay it's not opening so let me click from here and then let's go to our selenium demo folder and in that installer okay so now do you see this Eclipse Java 2018 12 are blah blah blah dot DMG so this is the file which we're going to be using to install Eclipse now as simple as what we did for jdk I'm going to double click on this and install the eclipse let's see how the eclipse gets installed all right so now I'll just move this folder into applications done so my eclipse is ready now so simply let's go ahead and launch so what I'll do I'll just simply search for that on my system Eclipse there you go and I'll just double click this is an application downloaded from the internet are you sure you want to open it yes please go ahead and open it okay it is launching it so did you see how simple the eclipse installation is it's just downloading the package and just installing it on your Mac machine all right when you start the eclipse for the first time you need to set your workspace so workspace is place where you are going to keep all your folders all your Java code or whatever you're going to do in Eclipse all under that particular directory so right now it gives me a default directory rather what I will do is since I have created a separate folder for our demos I will go ahead and select that folder which is my simply demo and then in this I will add a folder called workspace okay so that means in this particular location whatever files I'm going to create whatever code I'm going to write in selenium is going to be stored at this location and then I'll go ahead and say launch so it'll take a minute to load your workbench yeah there it goes so when you launch your Eclipse right so let me just make this window bigger the first screen what you are going to see is the welcome screen just go ahead and choose the welcome skills and this is the first view of your eclipse what you're going to get so let's close out all the windows which we are not going to be working today and I just need this main screen here now what is the first step you're going to do once you have the eclipse I would go ahead and create a project first so I'll simply say open file new Java project let's give a nice name to this since this is our first project which is going to be created so I'll say selenium first project now if You observe here so we have a jdk already installed and Eclipse needs a jdk to work with and what it has done while I create my new project it has already given me an option to use the execution JRE which you already have on the system okay that's why it is already chosen Java one dot it in case you have multiple Javas installed in in that case what you can do is you can always go here select this and select the Java of your choice so say if you've got the latest Java which is Java 12 installed on your system it will give you an option here along with Java software 8 it will give you Java 12 year so you can go ahead and select that for this particular project that option is also available now since we have only one jtk installed here I will go ahead and create this project so I'll just click on finish and it's going to create a projection so there are few default things which it creates created one is definitely the Jr system libraries right this is what we told Eclipse when we created project that my Java is 1.8 so what it has done is it has imported all this jar file which it requires for our Java programs to work in our Eclipse ID and then it creates a SRC which is your Source directory wherein where we'll be creating all our Java code or our selenium test scripts okay good so far so this is a simple way of installing your eclipse and getting your Eclipse ready to work with your Selena all right so now let's go back so we have done the installation of java we have done the installation of eclipse now what's left just the selenium right so what is that we need for selenium let's go back to our browser okay and this time we will go to selenium HQ dot or so this is going to be a Bible for you if you're going to be working with selenium so in this website what I do is I go to downloads and there are two things which we need to download for us to work with Selena one is the selenium Standalone server right so when you're going to be working with remote selenium Webdriver is when you would require this however I would suggest that we download whatever is required for us even though we might not use it immediately so what I just need to do is I need to click on this and it's going to download This Server form okay so I have already downloaded it and I'll show you that and then what we need is the selenium client and Webdriver language findings okay we need a client so what is the client which you are going to work with we are going to working with Java client with the selling it and for that I need to go ahead and download this if you're working on say C sharp or a ruby or python then you have the respective packages which you can go and download now since we are going to be working with Java here I would go ahead and click on this and this is going to download the Java client formula now as I said since I've already downloaded it so let's go ahead and look at what are those downloadables what we have so let's close out this window now if you see here one is the selenium server Standalone which we just downloaded and then when you download the client libraries what it does it it creates a zip file it downloads a zip file and it unzips it and what is contained in that particular folder so if you look here selenium Java 3 in this folder you have multiple jar files with which is what we will be working with Selena so here you have two jar files here and then under the folder Labs you have other jar files now these are the files which we will need and now let's see how do we utilize this file in our eclipse so for that let's go back to our Eclipse okay so let me maximize this window so in Eclipse first thing what you need to do is click on your project okay right click on that and then you have something called as build path go to the build path and say configure build path now here in this configure build path what you need to do is go to your libraries okay go to add external jars now when you click on that this is where you need to select all your jar files from what you have downloaded from the selenium site so that is present in our selenium Java 3. so what I'll do I'll simply select these two first okay then again you click on the external jar file so what we have is in Libs so select all the jar files in the list say open okay you have all the jar files what we need identified it go ahead and click on apply and close once you do that how do you know whether all those jars have been imported correctly go to the reference libraries here you should be able to see all the selenium jars which we have downloaded to be listed out here and so it lists here so that means we have successfully imported selling hello this is Matthew from Simply learn and today we're going to go through part one of a two-part series on interview questions that you are likely to be asked if you are interviewing for a job that focuses on devops so there are approximately seven sections that we cover in devops we go from general devops questions source code management with tools such as git continuous integration and here we'll focus on Jenkins continuous testing with tools such as selenium and then you also have the operation side of devops which is your configuration management with tools such as Chef puppet and ansible containerization with Docker and then continuous monitoring with tools such as nagios so let's just get into those general devops questions so one of the questions that you're going to be asked is how is devops different from agile and the reality is is that devops is a cultural way of being able to deliver Solutions that's different from an agile if we look at the evolution of delivery over the last five to ten years we've gone from waterfall based delivery to Agile delivery which is on Sprints to where we are with continuous integration and continuous delivery around devops the whole concept of devops is culturally very very different from agile and the difference is is that you're looking at being able to do continuous releases what does that mean the difference is is that you want to be able to send out code continuously to your production environment that means the operations team the development team have to be working together that means that any code that gets created has to be able to go to production very quickly which means you need to be testing your code continuously and then that production environment must also be able to be tested continuously and any changes or any errors that come up have to be communicated effectively and efficiently back to the dev and opt team another area in which I see that devops is different is really the response that we have for how we engage with the customer so the customer is coming to your website to your mobile app to your chatbot or any digital solution that you have and has an expectation when you're going through and actually doing a devops paradigm the old model would be that you would capture requirements from the customer then you do your development then you do your testing and there'd be these barriers between each of those as we move faster through from waterfall to Agile what we saw is that with agile we're able to respond much faster to customer months so instead of it being weeks or months sometimes in some cases years between releases of software what we saw it would was a transition to weeks and months for releases on software now we see with devops is that the release cycle has shrunk even further with the goal of continuously delighting the customer how further has that release cycle shrunk to there are companies that have gone from having releases of once a week or once every two weeks or once a month to now having multiple releases a day indeed some companies have up to 50 releases a day this isn't something to also bear in mind is that each of those releases are tested and verified against test records so that you can guarantee that the code that's going to production is going to be good continuous code so what are the differences between the different phases of devops so effectively there are two main phases of devops there's the planning and coding phase and then there's the deploying phase and you have a tool such as Jenkins that allows you to integrate between both environments some of the core benefits that you may have to devops are going to be some technical benefits and some business benefits so when somebody asks you what are the benefits of devops you can reply that from a technical point of view you're able to use continuous software delivery to constantly push out code that has been tested and verified against scripts that have been written and approved you can be able to push out smaller chunks of code so that when you have an issue you're not having to go through massive blocks of code or massive projects you're going through just very small micro services or small sections of code and you're able to detect and correct problems faster on the business side the benefits are absolutely fantastic from a customer that's coming to your website and or to your a mobile app they're going to see responses happening continuously so that the customer is always aware that you as a company are listening to their demands and responding appropriately you're able to provide a more stable environment and you're able to scale that environment to the demands of the number of customers that are using your services so how you approach a project that needs to implement devops so this is really an exciting area for you to be in so there are effectively three stages when it comes to actually working in a devops the first stage is an assessment stage and think of this as the back of the napkin ideation stage this is where you are sitting with a business leader and they're giving you ideas of what they would like to see from feedback that they've had from their customers this is Blue Sky opportunity this is thinking of Big Ideas that second stage and this often comes as a fast follow to stage one it's actually prove moving out that concept so developing a proof of concept and a proof of concept can actually be a multiple different things so it could be something as simple as a wireframe or it could be something that is as complex as a mini version of the final application depending on the scope of the work that you're delivering will really depend on how complicated you want the POC to be but with that in mind whatever choice you make you have to be able to deliver enough in the POC so that when you present this to a customer they're able to respond to that creation that you've developed and able to give you feedback to be able to validate that you are going with the right solution and able to provide the right product to your customers that third stage is where you get into your devops stage and this is just the exciting part this is where the rubber hits the road and you start releasing code based on a backlog of features that are being requested for the solution in contrast to doing agile delivery where you just continuously work through a backlog with devops what you're also looking at is putting an analytics and sensors to be able to validate that you are being successful with the solution that's being delivered so that once you've actually start delivering out code that customers can interact with you want to be able to see what are the pieces of the solution that they are using what do they like what is complicated where are the failure points and you want to use that data and feed that back into your continuous integration and have that as a means to be able to backfill the demand of work that gets completed in the bank log so what is the difference between continuous delivery and continuous deployment so continuous delivery is based on putting out code that can be deployed safely to production an insurance that your businesses and functions are running as you would expect them to be so it's going through and completing the code that you would actually see continuous deployment in contrast is all about ensuring that you're automating the deployment of a production environment so you're able to go through and scale up your environments to meet the demands of both the solution and the customer this makes software development and release processes much more faster and more robust so if we look here we can actually see where continuous integration and continuous deployment come hand in hand so when you actually start out with the initial pushes of your code that's where you're doing your continuous integration and your continuous delivery and then at some point you want to get very comfortable with deploying the code that you're creating so it's been pushed out to your production environment so one of the things that's great about working with the tools that you use in a devops continuous integration and can continuous delivery model is that the development tools that you use the containerization tools the testing tools should always reflect the production environment what this means is that when you actually come to deploying solutions to production there are no surprises because your development team have been working against that exact same environment all the way through so a question that you'll also be asked is you know what is the role of a configuration Management in Dev apps and so the role of configuration management really has three distinct areas and the first and this is really obvious one and this is the one where you probably already have significant experiences is the ability to manage and handle large changes to multiple systems in seconds rather than days hours or weeks as that may have happened before the second area is that you want to also demonstrate the business reasons for having configuration management and the business reason here is that it allows I.T and infrastructure to standardize on resource configurations and this has a benefit in that you're able to do more with fewer people so instead of having a large configuration team you can actually have a smaller more highly skilled team that's able to actually manage an even larger operational environment and thirdly you want to be able to highlight the ability to scale so if you have configuration management tools you're able to manage a significant number of servers and domains that may have multiple servers in it allows you to effectively manage servers that are deployed on cloud or private cloud and allow you to do this with high accuracy so how does continuous monitoring help in maintain the entire architecture of the system so when this question comes up you want to dig in and show your knowledge on how configuration and continuous monitoring is able to control an entire environment so the number one topic that you want to bring up when it comes to continuous monitoring is that with being able to effectively monitor your entire network 24 7. for any changes as they happen you're able to identify and Report those thoughts or threats immediately and respond immediately for your entire network instead of having to wait as it happens sometimes for a customer to email or call you and say hey your website's down nobody wants that that's an embarrassing thing the other three areas that you want to be able to highlight are the ability to be able to ensure that the right software and the right services are running on the right resources that's your number one takeaway that you want to be able to give of continuous monitoring the second is to be able to monitor the status of those servers continuously this is not required hiring manually monitoring but having a agent that's monitoring those servers continuously and then the third is that by scripting out and continuously monitoring your entire environment you're creating a self-audit trail that you can take back and demonstrate the effectiveness of the operations environment that you are providing so one of the cloud companies that is a strong advocate for devops is Amazon's web services AWS and they have a really five distinct areas them that you can zero in on board services so when the question comes up what is the role of AWS in devops you want to really hold out your hand and list of five areas of focus using your thumb and finger so you want to have flexible Services built for scale automation secure and a large partner ecosystem and having those five areas will really be able to help demonstrate why you believe that AWS and other Cloud providers but AWS is 70 the leader in this space are great for being able to provide support for the role of devops so one of the things that we want to be able to do effectively when we're releasing any kind of solution is to be able to measure that solution and so kpis are very important so you will be asked for three important Dev of kpis and so three that really come to mind that are very effective the first one is mean time to failure recovery and what this talks about is what is the average time does it take to recover from a failure and if you have experience doing this then look at the experience you have and use a specific example where you were able to demonstrate that mean time to failure recovery the second is deployment frequency and with deployment frequency you want to be able to discuss how often do you actually deploy Solutions and what actually happens when you're actually doing those deployments and what does the impacts to your network look like when you do doing those deployments and then the third one is really tied to that deployment frequency which is around what is the percentage of failed deployments and so how many times did you deploy to a server and something happened where the server itself failed what we're looking for when you're going through and being asked for these kpis is experience with actually doing a devops deployment and being able to understand what devops looks like when you're pushing out your infrastructure and then the second is being able to validate that self-auditing ability and one word of caution is don't go in there and say that you have a hundred percent success the reality is that servers do degrade over time and you maybe want to talk about a time when a server did degrade in your environment and use that as a story for how you were able to successfully get over and solve that degradation so one of the terms that is very popular at the moment is infrastructure as code and so you're going to be asked to explain structure as code and really it's it's something that actually becomes a byproduct of the work you have to do when you're actually putting together your devops environment and infrastructure's code really refers to the writing of code to actually manage your environment and you can go through many of the other tools that we've covered in this series but you'll see that XML or Ruby or yaml are used as languages to describe the configuration for your environment this allows you to then create the rules and instructions that can be read by the machines that are actually setting up the physical Hardware versus a traditional model which is having software and installing that software directly onto the machine this is really important when it comes to cloud computing there really is a strong emphasis of being able to explain infrastructure as a service and infrastructure as code is fundamental to the foundation to infrastructure as service and then finally allows you to be able to talk about how you can use scripted languages such as yaml to be able to create a consistent experience for your entire network all right so let's now get into the next section which is source code management and we're going to focus specifically on git the reason being is that get is really the most popular source code management solution right now there are other Technologies out there but for the types of distributed environments that we have as source code management with Git is probably the most effective so the first question you'll be asked when it comes to get is to talk about the difference between centralized and distributed Version Control and if we look at the way that the two are set up older Technologies such as older versions of Team Foundation server though the current version does actually have git in it but older versions required a centralized server for you to check in and check out of code the developer in the centralized system does not have all the files for the application and if the centralized server crashes then you actually lose all of the history of your code now in contrast a distributed model actually we do check in our code to a server however for you to be effective and building out your solution you actually check out all of the code for the solution directly onto your local development machine so you can actually have a copy of the entire solution running on your local machine this allows you to be able to work effectively offline it really allows for scalability when it comes to building out your change so if you have a team that may be in Europe you can actually then scale that team with people from Asia from North America or South America very easily and not have to worry about whether or not but they have the right code or the wrong code and in addition to that if the actual main server where you're checking in your code does crash it's not a big deal because you actually have each person has a copy of the code so as soon as the server comes back up you have to check back in and everybody's running back as if there was nothing had happened at all so one of the questions you'll be asked is to give the answer to some of the commands you use for working with Git so if you were to be asked the question is what is the git command that downloads any repository from GitHub to your computer on the screen you have four options we have git push get Fork get clone and get commit the answer in this instance would be get clone now if you want to be able to push code from your local system to a GitHub repository using git then first of all you want to be able to do is connect the local repository to a remote repository and in the example you may want to talk about using the command get remote ad origin and then the actual path to get Hub repository you could if you wanted to actually at this point also talk about other repositories such as get lab that you can also work with or a private git repository that would be used just for the development team once you've actually then added the local repository into your local computer then the second action you want to use is a push which is to actually push your local files out to the master environment so you use the command git push origin master so one question you may be asked is what is the difference between a bear repository and a standard way of initializing a git repository so let's look through what is a standard way so the standard way using get init allows you to create a working directory using the command git in it and then the folder that creates is the folder that creates all the revision history related to the work that you're doing in contrast using the bear way you have a different command for setting that up so it would be git init Dash bear and it does not contain any working or checked out source files locally on your machine in addition the revision history is actually stored in the root folder versus a subfolder that you'd have with the normal get in it initialization so which of the following CLI commands would you be used to rename a file so we have get RM get MV git RM Dash r or none of the above well in this instance it would be gate MV a question that you'll be asked around commit is going to be what is the process to revert a commit that has already been pushed and made public and there are two ways you can address this the first is to actually address the bad file in the new commit and you can use the command git commit Dash M and then put in a comment for why that file is being removed the second is to actually create a new commit that actually under does all the changes that were made with the bad commit and then to do that you would use git revert and then the commit ID and the commit ID and it could be something such as 560e0938f but you'd have to find that from the the commit that you had made but that would allow you to revert any bad files that you had submitted so the two ways of being able to get files from a git repository and you're going to be asked to explain the difference between git Fetch and get Paul so get fetch allows you to fetch and download only new data from a new repository it does not integrate any of the new data into your working files and it can be undone at any time if you want to break out the remote tracking branches in contrast get pull updates the current head Branch with the latest changes from the remote server so you get all of the files and downloaded it downloads new data and integrates it with the current working files you have on your system and it tries to merge remote changes with your local ones so one of the questions you'll get asked about get is what is a get stash so as a developer you'll be working on the con Branch within a solution but what happens if you come up with an idea where it's something that will take a different amount of time for you to be able to complete but you don't want to interrupt the mainline Branch so what you can actually do is you can actually create a branch that allows you to start working on your own work outside of the mainline branch and this is called git stash allows you to be able to modify your files without interrupting the mainline Branch so you once you start talking about branching and git be prepared to answer and explain the concept of branching so essentially what it allows you to do is have a Mainline Master brand that has all the code that the team is checking in and checking out against but allows you to have an indefinite number of branches that allows for new features to be built in parallel to the mainline branch and then at some point be reintroduced to the mainline Branch to allow the team to add in new features and so if we look through the merge and get rebase these are the two features that you'd be using continuously to be able to talk about how you take a branch and merge it back into the mainline Branch so on the left hand side we have get merge which allows you to take the code that you're creating and merge it back into the master on the right hand side what you have is a slightly different approach this is for projects where you reach a point in a project where you go okay we're going to effectively restart the project at this point in time and we want to ignore the complete history that's happened before that and that's called get rebase and that would allow you to rewrite the project history by creating a brand new Mainline branch that ignores all other previous branches that have happened before it you can if you want to very quickly and easily find out all the files that have been used to make a particular commit so when somebody asks you the question how do you find a list of files that's been changed in a particular commit you can actually say that all you have to go is find the command get div tree dash R and then the hash that you would use for the commit and that would actually then give you a breakdown of all the files that have been made with that particular commit a question that you'll be asked when you're talking about merging files is what is a merge conflict in git and how can it be resolved so essentially a merge conflict is when you have two more branches that are competing with commits in git and you have to be able to determine which is the appropriate files that to be submitted and this is where you would go in and to actually help resolve this issue you'd actually go in and manually edit the conflicted files to select the changes you want to keep in the final merge so let me go through the steps that you would take to be able to illustrate this when you're talking about this particular question in your interview now there are essentially four stages the first would be under the repository name you want to select a pull request and you're going to be able to show how that pull request would be demonstrated inside of GitHub So within the pull request there's going to be a highlight of conflict markers and you'll be able to select which conflicts you want to keep and which you want to merge and which ones you want to change so if it's a step through how you would actually resolve a merge conflict and the first step would be under GitHub you want to be able to pull the repository name and then the pull request around that repository in the pull request list click the pull request with a merge conflict and that you'd like to be able to resolve now pull up a file that would list out all of the conflicts for you near the bottom of that file will be a list of the requests that need to be resolved and then if you need to make a decision on which branches you want to keep or which ones you want to change that will have to be something you have to put in instructions inside of the file you'll actually see that there are conflict markers within the instructions which are going to ask you which files you want to change which ones you want to keep if you have more than one merge conflict in your file scroll down to the next set of conflict markers and repeat steps four and five until you resolve all of the conflicts you will want to mark your file as resolved in GitHub so that the repository knows that you are having everything resolved if you have more than one file with a conflict then you want to go then on to the next file and start working on those files and just keep repeating the steps we've done up to this point until you have all of the conflicts resolved and then once you have of all of the resolutions created then you want to select the button which is commit merge and then merge all your files back into GitHub and this will take care and manage the resolution of the merge conflict within GitHub so you can also do this through command line and with the command line you want to use a get bash and so you want to as a first step open up get bash and then navigate to the local git repository in command line by using the CD change directory and then list out the actual folder where you actually are putting all of your code and then you want to be able to generate a list of the files that are affected with the merge conflict and in this instance here you can actually see the file style guide.md has a merge conflict in it and as before with working with GitHub you actually go through and use a text edit and use any text editor but as you go through and edit out what what you want to keep and what you want to manage in your conflict so you actually have a resolution that's been created so that you'll be able to then once you're using the conflict markers you can actually merge your files together so that the solution itself will allow you to incorporate your commits effectively into the resolution once you've gone through and applied your changes you're able to then merge the conflicted commits into a single commit and able to push that up to your remote repository all right let's talk about the next section which is continuous integration with Jenkins so the first question you'll be asked about with Jenkins is explain a master sleeve architecture of Jenkins so the way that Jenkins has set up is that the Jenkins Master will pull code from your remote git repository such as GitHub and we'll check that repository every time there is a code commit it will distribute the workload of that code and the the tests that need to be applied to that code to all of the Jenkins slaves and then on requests the Jenkins master and the slaves will then carry out all the builds and tests to be able to produce test reports the next question you'll be asked is what is a Jenkins file and simply put a Jenkins file as a text file that has a definition of the Jenkins Pipeline and is checked into a source code repository and this really allows for three distinct things to happen one it allows for a code review and iteration of the pipeline it permits an audit Trail for that Pipeline and also provides a single source of troop for the pipeline which can be viewed and edited so which of the following commands runs Jenkins from the command line is it Java Dash jar jenkins.war Java Dash War jenkins.jar java.jar Jenkins jar Java Dash War Jenkins dot wall and the answer is a Java Dash jar jenkins.war so when working with Jenkins you're going to be asked what are the key Concepts and aspects of working with a Jenkins Pipeline and you want to really hold out your fingers here and go through four key areas and that is pipeline node step and stay so pipeline refers to the use-defined model of a CD continuous delivery pipeline node are the machines which is which are part of that Jenkins environment within the pipeline step is a single task that tells Jenkins what to do at that particular point in time and then finally stage defines a conceptually distinct subset of tasks performed through the entire Pipeline and tasks could be build test and deploy so which of the following file is used to Define dependency in Maven and do we have a build.xml b palm.xml c dependency Dot XML or D version.xml and the answer is palm.xml working with Jenkins you're going to be asked to explain the two types of pipeline used in Jenkins along with the syntax and so a scripted pipeline is based on groovy script as the domain-specific language for Jenkins and there are one or more note blocks that are used throughout the entire Pipeline on the left hand side you can actually see what the actual script would look like and the right hand side shows what the actual declaration for each section of that script would be the second type of Jenkins pipeline is a declarative Pipeline and a decarative pipeline provides a simple and a friendly syntax to Define what the pipeline should look like and then you can actually at this point use an example to actually break out how blocks are used to define the work completed in a decorative pipeline so how do you create a copy and backup of Jenkins well to create a backup periodically back up Jenkins to your Jenkins home directory and then create a copy of that directory it's really as simple as that the question you'll be asked as well is how can you copy Jenkins from one server to another well there are essentially there are three ways to do that one is you can move a job from one installation of Jenkins to another by copying the corresponding job directory the second would be to create a copy of an existing job directory and making a clone of that job directory but with a different name and the third is to rename an existing job by renaming a directory so security is fundamental to all the work that we do within devops and Jenkins provides the center core to all the work that gets completed within a devops environment there are three ways in which you can apply security to authenticate users effectively and when you're asked about this question of security within Jenkins the three responses you you want to be able to provide is a Jenkins has his own internal database that uses secured user data from and user credentials B is you can use a ldap or lightweight directory access protocol server to be able to authenticate Jenkins users or C you can actually configure Jenkins to authenticate by using such as oauth which is a more modern method of being able to authenticate users you're going to be asked how to deploy a custom build of a core plugin within Jenkins and essentially the four steps you want to go through are first of all copying the dot HPI plugin file into the Jenkins home plugins sub directory you want to remove the plugins development directory if there is one you want to create an empty file called plugin.hpi dot pinned and once you've completed these three steps restart Jenkins and your custom build plugin should be available how can you temporarily turn off Jenkins security if the administrative user has locked themselves out of the admin console this doesn't happen very often but when it does it's good to know how you can actually get into Jenkins and be able to resolve the problems of authenticating effectively into the system as an administrator so when you want to be able to get into a Jenkins environment what you want to be able to do is locate the config file you should see that it's set to True which allows for security to be enabled if you then change the user security setting to a false Security will disabled allow you to make your administrative changes and will not be re-enabled until the next time Jenkins is restarted so what are the ways in which a build can be scheduled and run in Jenkins well there are four ways in which you can identify the way a build can be scheduled on Ryan Jenkins the first is when source code management commit new code into the repository you can run Jenkins at that point the second can be the after the completion of other builds so maybe you have multiple builds in your project that are dependent to each other and when so many other builds have been executed then you can have Jenkins run you can schedule build to run at a specified time so you may have nightly builds of your code that illustrate the changes in the solution you're building and then finally you also can manually build a environment on request occasionally you will want to also restart Jenkins and so it's good that when a question around how do you restart Jenkins manually comes up that you have the answers and there are two ways in which you can do it one is you can force a restart without waiting for bills to complete by using the Jenkins URL that you have for your environment slash restart or you can allow all running bills to complete before restart are required in which case you would use the command of the your or else your Jenkins environment slash safe restart so let's go into the fourth and final section of this first video which talks about continuous testing with selenium so the first question you will be asked most likely around selenium are what are the four different selenium components and again you will want to hold open your fingers because there are four distinct environments you have selenium integrated development environment or selenium IDE you have selenium remote control or selenium RC you have selenium Webdriver and then selenium grid you'll be asked to explain each of those areas in more detail but let's start off with by looking at selenium driver what are the different exceptions in selenium Webdriver so it's useful to remember that an exception is an event that occurs during the execution of a program that disrupts the normal flow of that program's instructions and so we have 4 we have a timeout exception an element not visible exception no such element exception and a session not found exception and each of those if we step through them are the four different types of exceptions that can be thrown up when using the selenium Webdriver so as we evolve in our digital world with the different types of products that are available for us to be able to build Solutions onto multiple platforms you're going to be asked can selenium and other devops tools run in other environments and so a good question around this is cancellium test and application in an Android web browser and the short answer is absolutely yes it can we have to use the Android driver for it to be able to work so you want to be able to talk about the three different types of supported tests within selenium so when the question comes up what are the different test types supported by Salam you can answer that and there are three different types of tests first is a functional test second is a regression test and third is a load testing test the functional test is a kind of Black Box testing in which test cases are based on a specific area or feature within the software a regression test helps you find any specific areas that functional tests or non-functional areas of the code wouldn't be able to detect the load testing test allows you to monitor the response of a solution as you increase the volume of hits and how you're using the code are put onto it an additional question you'll be asked is how can you get a text of a web element using selenium but the get command is due to retrieve tax of a specific web element and it's important to remember however that the command does not return any parameters but just returns a string value so you want to be able to capture that stream value and discuss about it a question you'll be asked around selenium is can you handle keyboard and mouse actions using selenium and the answer is yes you can but you have to make sure that you're using the advanced user interaction API and the advanced user interaction API is something that can be scripted into your test and it allows you for capturing methods such as a click and hold and drag and drop Mass events and then keyboard down or keyboard up key release events so that if you want to to capture say the use of control shift or a specific function button of the keyboard you'd be able to capture those of the following four elements which of these elements is not a web element method a get text B size C get tag name d send keys and it's B size you're going to be asked to explain what is the difference for when we use find element or find elements and so if we look at find element find element finds the first element in the current web page that matches the specified locator value in contrast find elements finds all of the elements on the web page that matches the specified value when using Webdriver what are the driver close and Driver quit commands and these are two different methods used to close a web browser session in selenium so driver close will close the current web browser which your focus is set and Driver quick closes all the browser windows and ends the web driver session completely the final question that you are likely to be asked in using selenium is how can you submit a form using selenium well in this instance that's relatively easy the following lines of code will let you submit a form in selenium which would be web element El equals driver dot find element and then you put in the ID and the element ID and then L submit open close parentheses semicolon so let's just get into the first action which is configuration management so one of the questions that you'll get asked right away is why do you have SSL certificates used for Chef really fundamentally your immediate answer should be security SSL provides for a very high level of private security and private and public key pairing this really is essential to ensure that you have a secure environment throughout your entire network the second part should be that if you're using SSL and you're using the private public key model within SSL you're able to guarantee the systems on your network that the chef that you'll be able to validate that the nodes within your network the chef is validating against actually are the real nodes themselves not imposters so you will also be asked some questions such as the following which is the following commands which you use to stop or disable the HTTP service when the system boots and you'll typically get four responses and there'll be hashtag system CTL disable httpd.service or is it system disable http.service system disable httpd or the final option which is systemctl disable httpd.service your answer should be the first one which is hashtag systemctl disable http.service so Chef comes with a series of tools that allow it to function effectively and one of the tools that you're going to be asked about is what is Test Kitchen and Test Kitchen is essentially a command line tool that allows you to be able to test out your cookbook before you actually deploy it to a real node so some of the commands that you would use are for instance if you want to create an instance of Test Kitchen you would do kitchen create if you want to destroy an instance after you've created it you do kitchen destroy and if you want to be able to combine multiple instances you would do kitchen converge so a question you'll get is around Chef is how does Chef apply differ from Chef client has a fundamentally the difference between them is that Chef apply will validate the recipe that you're working on whereas Chef client looks to apply and validate the entire cookbook that's run in your server environment so one is focused on the recipe and the other is focus on the entire cookbook so there are some differences when you're working with different command lines so for instance when you're working with puppets and you're working with one version of puppets and you want to do what is the command to sign a requested certificate the top example here is for puppet version 2.7 whereas the lower option here is for puppet version three that's something to bear in mind when you're going through your interview process is that the tools that are used within a continuous integration can continuous delivery devops model do vary and so you want to be able to talk knowledgeably about the different versions of the tools so that when you're talking to your interviewer you're able to show you the Deep knowledge that you have which open source or Community tools do you use to make puppet more powerful and essentially this question is going to be asking you to look beyond the core Foundation of puppet itself and so the three options you have is uh being able to track configurations with jira which you should be doing anyway but it's a great way to be able to clearly communicate the work that's been done with puppet our Version Control can be extended with get and then the changes should be passed through Jenkins so the three tools you want to be looking at integration with jira Git and Jenkins so what are the resources in public well fundamentally there are four the resources are basic units of any configuration management tool they are the features of the nodes they are the written catalog and the execution of the catalog on a node so as we dig deeper into puppets one other thing things that you are likely to be asked regarding puppet is what is a class and puppets and so a class in public is really the name blocks in your manifest that contain the various configurations and this can include Services files and packages and we have on the screen here example of what a class would look like when you write it out and you may want to memorize just one class don't memorize just a whole set of classes just memorize one the person that's interviewing you is just really looking for someone who has a working knowledge they're not looking for you to have memorized complete massive classes but having one small class to be able to illustrate the experience you have is extremely valuable to the interviewer and particularly if it's a technical interview so as we move into ansible one of the things that you're going to be asked around ansible is what is ansible role so a role is an independent block of tasks and variable files and templates embedded with inside of the Playbook so exam example we have here on the screen actually shows you one role within a Playbook and in this role it is to install Tomcat on a node again as with previous question within puppet of a class it's probably good to have memorized just one or two roles so you can talk knowledgeably about ansible when you're having your interview so when you're working with ansible when should you be using the curly brackets and so just as a frame of reference uh there's often two different ways that these kind of brackets are referred to now they're either referred to as French brackets or curly brackets either way and what you'll be wanting to ask is when would you use these specific types of brackets within ansible and really the answer comes down to two things one is that it makes it easier to distinguish strings and undefined variables and the second is for putting together conditional statements when you are actually using variables and the example we have here is this principle value of and we have Foo and we have to then put in the variable conditional statement of Foo is defined as something so what is the best way to make content reusable and redistributable with ansible and there's really essentially three the first is to include a sub module or another file in your playbook the second is to import an improvement of an include which ensures that a file is added only once and then the third is rows to manage the tasks within the Playbook so a question you will be asked is provide a differences between ansible and puppets so if we look at ansible it's a very easy agentless installation it's based on python you can configure it with yaml and there are no support for Windows in contrast puppet is an agent based installation it's written in Ruby the configuration files are written in DSL and it has support on all popular operating systems so we dig deeper into the actual Arc texture ansible it has a much more simple architecture and it's definitely a push only architecture in contrast to puppet it's a more complicated but more sophisticated architecture where you're able to have a complete environment managed by the puppet architecture so let's get on to our next section which is containerization so let's go through and you're going to be asked to explain what the architecture of Docker is and Docker really is the most popular containerization environment so Docker uses a client server architecture and the docker client is a service which runs in a command line and then the docker Daemon which is run as a rest API within the command line will accept the requests and interacts with the operating system in order to build the docker images and run the docker containers and then the docker image is a template of instructions which is used to create containers the docker container is an executable package of applications and its dependencies together and then finally the docker registry is a service to host and distribute Docker images among other users so you'll also be asked to provide what are the advantages of Docker over a virtual machine and and this is something that comes up very consistently in fact um you may want to even extend it as having what are the differences between having a dedicated machine a virtual machine and a Docker or docker-like environment and really the the arguments for Docker are just absolutely fantastic you know first of all Docker does contain an occupy docket containers occupy significantly less space than a virtual machine or a dedicated machine the boot up time on Docker is significantly faster than a VM containers have a much better performance as they are hosted in a single Docker image Docker is highly efficient and very easy to scale particularly when you start working with kubernetes easily portable across multiple platforms and then finally for space allocation Docker data value games can be shed and reused among multiple containers the argument against virtual machines is significant and particularly if you're going into an older environment where a company is still using actual dedicated hardware and haven't moved to a cloud or cloud-like environment your Arguments for Docker are going to be very very persuasive be very clear on what the advantages are for Docker over a virtual machine because you want to be able to succinctly share them with your team and this is something that's important when you're going through the interview process but also equally important particularly if you're working with a company that's transitioning or going through a digital transformation where they aren't used to working with the tools like Docker you need to be able to effectively share with that team what the benefits are so how do we share Docker containers with different nodes and in this instance what you want to be able to do is Leverage The Power of Docker swarm so darker swarm is a tool which allows the it administrators and developers to create and manage clusters of swarm nodes within the the darker platform and there are two elements to the node there's the manager node and then there's the the worker node the manager node as you'd assume manages the entire infrastructure and the working node is actually the work of the agent as it gets executed so what are the commands to create a Docker swarm and so here we have an example of what a manager node would look like and once you've created a swarm on your manager node you can now add worker nodes to that swarm and again when you're stepping through this process be very precise in the execution part that needs to be taken to be able to effectively create a swan so start with the manager node and then you create a worker node and then finally when a node is initializes a manager node it can immediately create a token and that token is used for the worker nodes and associating the IP address with the worker nodes question 17 how to run multiple containers using a single service it is possible to run multiple containers as a single service by using docker compose and Duck and compose will actually run each of the services in isolation so that they can interact with each other the language used to write out the compose files that allow you to run the service is called yaml and yaml stands for yet another markup language so what is the use of a Docker file so a Docker file actually is used for creating Docker images using the build command so let's go through and show on the screen what that would look like and this would be an opportunity where if you're actually in a technical interview you could potentially even ask hey can I draw on a whiteboard and show you what the architecture for using the build command would look like and what the process would look like and again when you're going through an interview process as someone who interviews a lot of people one of the things I really like is when an interview candidate does something that's slightly different and in this instance this is a great example of where you can stand up to the Whiteboard and actually show what can actually be done through actually creating images on the Whiteboard very quickly little square boxes where you can actually show the flow for creating a build environment as in architect this should be something that you're comfortable doing and by doing it in the interview and suddenly you want to ask permission before you actually do it but doing this in the interview really helps demonstrate your comfortable feelings of working with these kind of architecture drawings so back to the question of creating a Docker file so we go through and we have a Docker file that actually then goes ahead and creates the docker image which then in turns creates the docker container and then we are able to push that out up to a Docker Hub and then share that Docker Hub with everybody else as part of the docker registry with the whole network so what are the differences between Docker image and Docker containers so let's go through the docker image so the docker images are templates of a Docker container an image is built using a Docker file and it stores that Docker file in a Docker Repository or a Docker Hub and and you can use Docker Hub as an example and the image layer is a read-only file system the docker container is a collection of the runtime instances of a Docker image and the containers are created using Docker images and they are stored in the docker Daemon and every container is a layer is a read write file system so you can't replace the information you can only append to it so while you can actually use yaml for writing your so a question you can be asked is instead of yaml what can be an alternate file to build Docker compose so yaml is the one that is the default but you can also use Json so if you are comfortable working with Json and my that is something that should be get comfortable with is you want to be able to use that to name your files and it's a frame reference Json is a logical way of being able to do value paired matching using a JavaScript like syntax so you're going to be asked to how to create a Docker container so let's go through what that would look and we'll break it down task by task so the task is going to be create a MySQL Docker container so to do that you want to be able to build a Docker image or pull from an existing Docker image from a Docker repository or Hub and then you want to be able to then use Docker to create a new container which has my SQL from the existing Docker image simultaneously the layer of read write file system is also created on top of that image now below at the bottom of the screen we have what the commands lines look for that so what is the difference between a registry and a repository so let's go through that so for the docker registry and repository for the registry we have Docker registry Is An Open Source server-side service used for hosting and distributing Docker images whereas in contrast for repositories a collection of multiple versions of a Docker image in a registry a user can distinguish between Docker images with their tag names and then finally on the registry Docker also has its own default registry called Docker hub for the repository it is a collection of multiple versions of Docker images it is stored in a Docker registry and it has two tabs a public and private registry so you can actually create your own Enterprise registry so you're going to be asked you know what are the cloud platforms that support Docker really you know lifts them all and we have listed here Amazon web services Microsoft Azure Google Cloud Rackspace but you could add in their IBM bluemix you could put in Red Hat really any of the cloud service providers out there today do support Docker it's just become an industry standard so what is the purpose of Expose and publish commands in Docker so if we go through expose is an instruction used in Docker file whereas publish is used in Docker run command for expose it is used to expose ports within a Docker Network whereas with Publishers can be used out side of a Docker environment for expose it is a documenting instruction used at the time of building an image and running a container whereas with publishes users to map a host port to a running container port for expose is the command used in Docker whereas for publish we use the command Dash p for when we're doing our command line used in Docker and examples of these are exposed 8080 or with Docker we would put in all for publish we would do the example Docker run Dash Dash p and then 0.0.0.80 colon 80 as our command line so let's look at continuous monitoring so with continuous monitoring how does nagos help in continuous monitoring of systems applications and service and so this is really just a high level question of using now yes within your environment and you should be able to just come back very quickly and say negus allows you to help manage the servers and check if they've been sufficiently utilized and if there are any task failures that need to be addressed and so there are three areas of utilization and risk that you want to be able to manage this is being able to verify the status and services of the entire network the health of your infrastructure as a whole and if applications are working properly together with our web services and apis that are reachable so the second question you'll be asked is how does negos help in continuous monitoring of systems applications and services so it's able to do this by having the initial negative process and scheduler and the additional plugins that you would use for your network connect with the remote resources and the negia's web interface to be able to run status checks on a predefined schedule so what do you mean by nagus remote plugin executor or the mpre of nine years so mpre allows you to execute plugins on links Unix machines that allow you to do additional monitoring and machine metrics such as disk usage CPU load Etc what are the ports used by nagios for monitoring purposes in this example there are three and they're easy to remember so I would memorize these three but they're essentially ports five six six five six six seven and five six six eight so the were two types of checks within that year so you will be asked for what is an active and passive check in nagios so an active check and is initiated by the nagios process and is run on a regular schedule a passive check is initiated and formed by an external application or process so this may be where a system is failing and checks our results are submitted to netgears for processing and to continue with this for what is an active and passive check active checks are initiated by the check logic within the nagios Daemon now yes we'll execute a plug-in and pass information about what needs to be checked plugin will then check the operational state of the host or service and then process the results of the host or service check and send out notifications in contrast with the passive check it is an external application that initiates the check it writes the results of the check to an external command line file and I guess reads the external command file and places the results of all passive checks in a queue for later processing so you can go back and revalidate and then negos may send out notifications log alerts Etc depending on the results that they get from checking the information so you're going to be asked to explain the main configuration file and its location in nag year so the main configuration file consists of a number of directives that affect how many years operate so consider this as the configuration file that it's read by both now us processor and the cgis so this will allow you to be able to manage the main configuration file that's placed into your settings directory so what is the netgears network analyzer and again hold out your four fingers because there are four options here so the nagis network analyzer R1 provides an in-depth look at all your network traffic source and security threats two allows system admins to gather high level information on the healthier Network Three provides a central view of your network traffic and bandwidth container and then four allows you to proactive in resolving outages abnormal behavior and threats before they affect critical business processes so what are the benefits of HTTP and SSL certificate monitorings with nag years so with HTTP certificate monitoring it allows you to have increased server and services and application availability obviously very important fast detection of network outages and protocol failures and allows web transaction and web service performance monitoring the SSL certificate monitoring allows you for increased website availability frequent application availability and provides increased security so explain virtualization with nagios so in responses the first thing you should be able to talk about is how 9 years itself can run on many different virtualization platforms including Microsoft Visual PC VMware Zen Amazon ec2 et cetera Etc so just make sure you get that right off the and I guess it was able to provide capabilities tomorrow and so much of metrics on different platforms it allows for ensure for quick detection of service and application failures and has the ability to be able to monitor against many metrics including CPU usage memory networking and VM status so name the three variables that affect recursion inheritance in nagias and it is name use and register so name is a template name that can be referenced in other object definitions use specifies the name of the template object that you want to inherit as properties and variables from and register indicates whether or not the object definition should be registered to nagios and on the right hand side of the screen we have an example of what that script would look like again you may want to be able to memorize this as it's something that you can achieve write down and show someone if you're going through a technical interview so why is nagyo said to be object oriented it fundamentally comes down to the object configuration format that you can use in your object definitions it allows you to inherit properties from other object definitions and this is typical of object-oriented development and is now applied for the nagios environment so some of the objects that you can inherit are Services host commands and time periods so finally explain what is State talking in nagios and so there are really four options here when you're talking about State stalking so State stalking is used for login purposes in nagios it allows you to enable for a particular host or service that nagos will watch over very carefully it will log any changes it sees in the output of the check results and then finally it helps the analysis of log files and so with that we come to the end of this devops tools full course if you have any doubts or queries let us know in the comment section below so watch out for more interesting videos like these until then keep learning and stay tuned to Simply learn hi there if you like this video subscribe to the simply learn YouTube channel and click here to watch similar videos to nerd up and get certified click here foreign

Channel: Simplilearn

Views: 23,432

Keywords: devops tools, devops tools explained, devops tools 2021, devops tools tutorial, devops tools and architecture clear explanation, devops tools to learn, devops tools overview, devops tools list 2021, jenkins tutorial, maven tutorial, docker tutorial, gradle tutorial, selenium tutorial, git tutorial, puppet tutorial, kubernetes tutorial, ansible tutorial, learn devops tools, devops, devops tutorial, devops tools full course, simplilearn devops, simplilearn, 2022

Id: BpkEy0TwqiE

Length: 683min 15sec (40995 seconds)

Published: Wed Feb 08 2023