DevOps Tools Full Course 2022 | DevOps Tools Explained | DevOps Tools Tutorial | Simplilearn

[Music] hey everyone welcome to this devops tools full course by simply learn in this complete course we learn everything that we need regarding several devops tools we have matthews and chidanand to guide us through this journey so let's look at what's in store for us we shall begin with having a basic understanding of devops with a short animated video after that we'll dive deeper and understand the basics of devops like its relevance in the current scenario and several devops phases once we know devops we shall begin with learning different devops tools in detail we shall learn about git followed by github and in the process we shall learn about various git commands second we shall see what jenkins is jenkins pipeline and how we install jenkins after that we'll move on to docker and check out what docker is and docker container is all about we'll also learn about the different processes and how to install docker moving further we'll understand the basics of kubernetes the difference between kubernetes and docker and kubernetes on aws the following tool in the list will be puppet where we shall know what puppet is and why we use puppet and the process to install it moving forward we learn about ansible and its installation process and what are the benefits of using ansible the last tool on our list is selenium where we shall learn about selenium and all about its sweet tools followed by its installation processes while we're learning all these tools we shall understand them with the hands-on demo making the understanding much more effective and more accessible finally when we have a piece of deep knowledge regarding devops and several devops tools we'll look at some of the most prominent devops interview questions that will help you excel in your forthcoming devops interviews but before we begin if you haven't subscribed to our channel already make sure you hit the subscribe button and the bell icon to never miss an update so without further ado let's begin right from the start software development comprise two different departments the development team that develops the plan designs and builds the system from scratch and the operation team for testing and implementation of whatever is developed the operations team gave the development team feedback on any bugs that needed fixing and any rework required invariably the development team would be idle awaiting feedback from the operations team this undoubtedly extended timelines and delayed the entire software development cycle there would be instances where the development team moves on to the next project while the operations team continues to provide feedback for the previous code this meant weeks or even months for the project to be closed and final code to be developed now what if the two departments came together and worked in collaboration with each other what if the wall of confusion was broken and this is called the devops approach the devops symbol resembles an infinity sign suggesting that it is a continuous process of improving efficiency and constant activity the devops approach makes companies adapt faster to updates and development changes the teams can now deliver quickly and the deployments are more consistent and smooth though there may be communication challenges devops manages a streamlined flow between the teams and makes the software development process successful the devops culture is implemented in several phases with the help of several tools let's have a look at these phases the first phase is the planning phase where the development team puts down a plan keeping in mind the application objectives that are to be delivered to the customer once the plan is made the coding begins the development team works on the same code and different versions of the code are stored into a repository with the help of tools like get and merged when required this process is called version control the code is then made executable with tools like maven and gradle in the build stage after the code is successfully built it is then tested for any bugs or errors the most popular tool for automation testing is selenium once the code has passed several manual and automated tests we can say that it is ready for deployment and is sent to the operations team the operations team now deploys the code to the working environment the most prominent tools used to automate these phases are ansible docker and kubernetes after the deployment the product is continuously monitored and nagios is one of the top tools used to automate this phase the feedback received after this phase is sent back to the planning phase and this is what forms the core of the devops life cycle that is the integration phase jenkins is the tool that sends the code for building and testing if the code passes the test it is sent for deployment and this is referred to as continuous integration there are many tech giants and organizations that have opted for the devops approach for example amazon netflix walmart facebook and adobe netflix introduced its online streaming service in 2007. in 2014 it was estimated that a downtime for about an hour would cost netflix two hundred thousand dollars however now netflix can cope with such issues they opted for devops in the most fantastic way netflix developed a tool called the simeon army that continuously created bugs in the environment without affecting the users this chaos motivated the developers to build a system that does not fall apart when any such thing happens so on this note here is a quiz for you match the devops tool with the phase it is used in a b c [Music] d none of the above today more and more companies lean towards automation with the aim of reducing its delivery time and the gap between its development and operations teams to attain all of these there's just one gateway devops and if you're also looking forward to doing the same and excel in devops check out simply learn's postgraduate program in devops design and collaboration with caltech ctme the program can help you master several in-demand devops skills and tools like continuous integration and delivery devops on cloud kubernetes docker and much more hello this is matthew with simplylearn and today we're going to go through and introduce you to devops we're going to go through a number of key elements today the first two will be reviewing models that you're already probably using for delivering solutions into your company and the most popular one is waterfall followed by agile then we'll look at devops and how devops differs from the two models and how it also borrows and leverages the best of those models we'll go through each of the phases that are used in typical devops delivery and then the tools used within those phases to really improve the efficiencies within devops finally we'll summarize the advantages that devops brings to you and your teams so let's go through waterfall so waterfall is a traditional delivery model that's been used for many decades for delivering solutions not just it solutions and digital solutions but even way before that it has its history that goes back to world war ii so waterfall is a model that is used to capture requirements and then cascade each key deliverable through a series of different stage gates that is used for building out the solution so let's take you through each of those stage gates the first that you may have done is requirements analysis and this is where you sit down with the actual client and you understand specifically what they actually do and what they're looking for in the software that you're going to build and then from that requirements analysis you'll build out a project planning so you have an understanding of what the level of work is needed to be able to be successful in delivering the solution after that you've got your plan then you start doing the development and that means that the programmers start coding out their solution they build out their applications to build out the websites and this can take weeks or even months to actually do all the work when you've done your coding and development then you send it to another group that does testing and they'll do full regression testing of your application against the systems and databases that integrate with your application you'll test it against the actual code you'll do manual testing you do ui testing and then after you've delivered the solution you go into maintenance mode which is just kind of making sure that the application keeps working there's any security risks that you address those security risks the problem you have though is that there are some challenges however that you have with the waterfall model the cascading deliveries and those complete and separated stage gates means that it's very difficult for any new requirements from the client to be integrated into the project so if a client comes back and if the project has been running for six months and they've gone hey we need to change something that means that we have to almost restart the whole project it's very expensive and it's very time consuming also if you spend weeks and months away from your clients and you deliver a solution that they are only just going to see after you spend a lot of time working on it they could be pointing out things that are in the actual final application that they don't want or are not implemented correctly or lead to just general unhappiness the challenge you then have is if you want to add back in the client's feedback to restart the whole waterfall cycle again so the client will come back to you with a list of changes and then you go back and you have to start your programming and you have to then start your testing process again and just you're really adding in lots of additional time into the project so using the watford model companies have soon come to realize that you know the clients just aren't able to get their feedback in quickly effectively it's very expensive to make changes once the teams have started working and the requirement in today's digital world is that solutions simply must be delivered faster and this has led for a specific change in agile and we start implementing the agile model so the agile model allows programmers to create prototypes and get those prototypes to the client with the requirements faster and the client is able to then send the requirements back to the programmer with feedback this allows us to create what we call a feedback loop where we're able to get information to the client and the client can get back to the development team much faster typically when we're actually going through this process we're looking at the engagement cycle being about two weeks and so it's much faster than the traditional waterfall approach and so we can look at each feedback loop as comprising of four key elements we have the planning where we actually sit down with the client and understand what they're looking for we then have coding and testing that is building out the code and the solution that is needed for the client and then we review with the client the changes that have happened but we do all this in a much tighter cycle that we call a sprint and that typically a sprint will last for about two weeks some companies run sprints every week some run every four weeks it's up to you as a team to decide how long you want to actually run a sprint but typically it's two weeks and so every two weeks the client is able to provide feedback into that loop and so you were able to move quickly through iterations and so if we get to the end of sprint two and the client says hey you know what we need to make a change you can make those changes quickly and effectively for sprint three what we have here is a breakdown of the ceremonies and the approach that you bring to agile so typically what will happen is that a product leader will build out a backlog of products and what we call a product backlog and this will be just a whole bunch of different features and they may be small features or bug fixes all the way up to large features that may actually span over multiple sprints but when you go through the sprint planning you want to actually break out the work that you're doing so the team has a mixture of small medium and large solutions that they can actually implement successfully into their sprint plan and then once you actually start running your sprint again it's a two-week activity you meet every single day to with the actual sprint team to ensure that everybody is staying on track and if there's any blockers that those blockers are being addressed effectively and immediately the goal at the end of the two weeks is to have a deliverable product that you can put in front of the customer and the customer can then do a review the key advantages you have of running a sprint with agile is that the client requirements are better understood because the client is really integrated into the scrum team they're there all the time and the product is delivered much faster than with a traditional waterfall model you're delivering features at the end of each sprint versus waiting weeks months or in some cases years for a waterfall project to be completed however there are also some distinct disadvantages the product itself really doesn't get tested in a production environment it's only been tested on the developer computers and it's really hard when you're actually running agile for the sprint team to actually build out a solution easily and effectively on their computers to mimic the production environment and the developers and the operations team are running in separate silos so you have your development team running their sprint and actually working to build out the features but then when they're done at the end of their sprint and they want to do a release they kind of fling it over the wall at the operations team and then it's the operations team job to actually install the software and make sure that the environment is running in a stable fashion that is really difficult to do when you have the two teams really not working together so here we have is a breakdown of that process with the developers submitting their work to the operations team for deployment and then the operations team may submit their work to the production servers but what if there is an error what if there was a setup configuration error with the developers test environment that doesn't match the production environment there may be a dependency that isn't there there may be a link to an api that doesn't exist in production and so you have these challenges that the operations team are constantly faced with and their challenge is that they don't know how the code works so this is where devops really comes in and let's dig into how devops which is developers and operators working together is the key for successful continuous delivery so devops is as an evolution of the agile model the agile model really is great for gathering requirements and for developing and testing out your solutions and what we want to be able to do is kind of address that challenge and that gap between the ops team and the dev team and so with devops what we're doing is bringing together the operations team and the development team into a single team and they are able to then work more seamlessly together because they are integrated to be able to build out solutions that are being tested in a production-like environment so that when we actually deploy we know that the code itself will work the operations team is then able to focus on what they're really good at which is analyzing the production environment and being able to provide feedback to the developers on what is being successful so we're able to make adjustments in our code that it's based on data so let's step through the different phases of a devops team so typically you'll see that the devops team will actually have eight phases now this is somewhat similar to agile and what i'd like to point out at time is that again agile and devops are very closely related that agile and devops are closely related delivery models that you can use with devops it's really just extending that model with the key phases that we have here so let's step through each of these key phases so the first phase is planning and this is where we actually sit down with a business team and we go through and understand what their goals are the second stage is as you can imagine and this is where it's all very similar to agile is that the coders actually start coding but they typically they'll start using tools such as git which is a distributed version control software it makes it easier for developers to all be working on the same code base rather than bits of the code that is rather than them working on bits of the code that they are responsible for so the goal with using tools such as git is that each developer always has the current and latest version of the code you then use tools such as maven and gradle as a way to consistently build out your environment and then we also use tools to actually automate our testing now what's interesting is when we use tools like selenium and junit is that we're moving into a world where our testing is scripted the same as our build environment and the same as using our get environment we can start scripting out these environments and so we actually have scripted production environments that we're moving towards jenkins is the integration phase that we use for our tools and another pointer here is that the tools that we're listing here these are all open source tools these are tools that any team can start using we want to have tools that control and manage the deployment of code into the production environments and then finally tools such as ansible and chef will actually operate and manage those production environments so that when code comes to them that that code is compliant with the production environment so that when the code is then deployed to the many different production servers that the expected results of those servers which is you want them to continue running is received and then finally you monitor the entire environment so you can actually zero in on spikes and issues that are relevant to either the code or changing consumer habits on the site so let's step through some of those tools that we have in the devops environment so here we have is a breakdown of the devops tools that we have and again one of the things i want to point out is that these tools are open source tools there are also many other tools this is just really a selection of some of the more popular tools that are being used but it's quite likely that you already using some of these tools today you may already be using jenkins you may already be using git but some of the other tools really help you create a fully scriptable environment so that you can actually start scripting out your entire devops tool set this really helps when it comes to speeding up your delivery because the more you can actually script out of the work that you're doing the more effective you can be at running automation against those scripts and the more effective you can be at having a consistent experience so let's step through this devops process so we go through and we have a continuous delivery which is our plan code build and test environment so what happens if you want to make a release well the first thing you want to do is send out your files to the build environment and you want to be able to test the code that you've been created because we're scripting everything in our code from the actual unit testing being done to the all the way through to the production environment because we're testing all of that we can very quickly identify whether or not there are any defects within the code if there are defects we can send that code right back to the developer with a message saying what the defect is and the developer can then fix that with information that is real on the either the code or the production environment if however your code passes the scripting test it can then be deployed and once it's out to deployment you can then start monitoring that environment what this provides you is the opportunity to speed up your delivery so you go from the waterfall model which is weeks months or even years between releases to agile which is two weeks or four weeks depending on your sprint cadence to where you are today with devops where you can actually be doing multiple releases every single day so there are some significant advantages and there are companies out there that are really zeroing in on those advantages if we take any one of these companies such as google google any given day will actually process 50 to 100 new releases on their website through their devops teams in fact they have some great videos on youtube that you can find out on how their devops teams work netflix is also a similar environment now what's interesting with netflix is that netflix have really fully embraced devops within their development team and so they have a devops team and netflix is a completely digital company so they have software on phones on smart tvs on computers and on websites interestingly though the devops team for netflix is only 70 people and when you consider that a third of all internet traffic on any given day is from netflix it's really a reflection on how effective devops can be when you can actually manage that entire business with just 70 people so there are some key advantages that devops has it's the actual time to create and deliver software is dramatically reduced particularly compared to waterfall complexity and maintenance is also reduced because you're automating and scripting out your entire environment and you're improving the communication between all your teams so teams don't feel like they're in separate silos but that are actually working cohesively together and that there is continuous integration and continuous delivery so that your consumer your customer is constantly being delighted a very warm welcome to all our viewers i'm anjali from simply learn and today i'll be showing you the installation for git on windows so we'll start by downloading and installing git on our system we'll then have a look at the git bash interface we'll type in some basic git commands next we'll create a local repository that is we'll create a repository on a local machine we'll then connect to a remote repository and finally we'll push the file onto github first things first we need to download and install git so download git for windows and i'll select the second link so 2.19.1 which is the most latest version of kit that's the one we want for windows system choose your version so mine is a 64-bit system and it's downloading so this will take a while so git is on our system click here run so go to next we don't have to change this path uh just so just click on in quick launch and on desktop next next again next nothing to change here either and install so now git is getting installed on our system so we don't need to view the release notes we just want to launch the git bash so let's just tick that and then click on finish and your git bash interface appears here so we are on the master branch the first thing we do is we'll check the version for our git so the command is git dash dash version and as you can see version 2.19.1 on our windows system which is exactly what we just downloaded we'll now explore the help command so let's just type git help config so config is another command and as i hit enter the manual page for the second command opened up which is config so what help command does is that it provides the manual or the help page for the command just following it so in case you have any doubts regarding how a command is used what a command is used for or the various syntax of the command you can always use the help command now there's another syntax for using the help command itself which is git config dash dash help enter this does the exact same thing as you can see now that we looked at the help command let's begin by creating a local directory so mkdir test now test is my new directory i'll move into this directory so cd test great so now that we are inside our test directory let's initialize this directory so git init is the command for initializing the directory and as you can see as you can see the path here this is the local path where a directory is created so i'll just show you the directory test and it's completely empty what we do now is we'll create a text file within this new directory that we created so new text document and i'll just name this demo i'll open this and just put in some dummy content so hello simply done save this file and go back to your bash interface let's just check the status now so git status and as you can see our file has appeared it's visible but nothing is committed yet so this means that we have not made any change to a file through the git tool itself so the next thing that we are going to do is we will be adding demo to our current directory the next command that we will be applying is the commit command and when you add certain files to the current directory the comment command is applied on all the above directories so git commit minus m and a message that will appear once the file is committed so as you can see one file is changed and one insertion just clear the screen next thing we need to do is we need to link our git to our github account so the command for doing that is git config global user dot username and this will be followed by our username so let me just show you my github account so this is my github profile and my username is simplylearn.github so guys before you begin this procedure just make a github account type in my username here simply learn github and enter and there you go a git is successfully linked with github next thing we do is we'll just open our github and we'll create a new repository give a repository name so i'll name it test underscore demo and create repository great so our repository is created this is our remote repository what we do next is just copy the link and then go back to your bash interface now we need to link our remote and our local repository so git remote origin and then paste the http link and now that our local repository and our remote repository are linked we can push our local file onto our remote repository so the command for doing that is git push origin master as we are on the master branch and that's done so now let's move back to github i'll just click on test demo and as you can see here our local file has been pushed to our remote repository with that we have successfully completed our demo thanks anjali now matthew and chadanan will take you through the remaining part of the course hello this is matthew from simply learn and today we're going to go through and explain how you can use get for your software development so what are we going to cover today so we're going to introduce the concept of version control that you'll use within your devops environment then we'll talk about the different tools are available in a distributed version control system we'll highlight a product called git which is typically used for version control today and you'll also go through what are the differences between get and github you may have used github in the past or other products like get lab and we'll explain what are the differences between git and git and services such as github and git lab we'll break out the architecture of what a get process looks like and how do you go through and create forks and clones how do you have collaborators being added into your projects how do you go through the process of branching merging and rebasing your project and what are the list of commands that are available to you in get finally i'll take you through a demo on how you can actually run git yourself and in this instance use the software of git against a public service such as github all right let's talk a little bit about version control systems so you may have already been using a virtual control system within your environment today you may have used tools such as microsoft's team foundation services but essentially the use of a virgin control system allows people to be able to have files that are all stored in a single repository so if you're working on developing a new program that such as a website or an application you would store all of your version control software in a single repository now what happens is that if somebody wants to make changes to the code they would check out all of the code in the repository to make the changes and then there will be an addendum added to that so um there will be the version one changes that you had then the person would then later on check out that code and then be a version two and add it to that code and so you keep adding on versions of that code the bottom line is that eventually you'll have people being able to use your code and that your code will be stored in a centralized location however the challenge you're running is that it's very difficult for large groups to work simultaneously within a project the benefits of a vcs system a version control system demonstrates that you're able to store multiple versions of a solution in a single repository now let's take a step at some of the challenges that you have with traditional version control systems and see how they can be addressed with distributed version control so in a distributed version control environment what we're looking at is being able to have the code shared across a team of developers so if there are two or more people working on a software package they need to be able to effectively share that code amongst themselves so that they constantly are working on the latest piece of code so a key part of a distributed version control system that's different to just a traditional version control system is that all developers have the entire code on their local systems and they try and keep it updated all the time it is the role of the distributed vcs server to ensure that each client and we have a developer here and developer here and developer here and each of those our clients have the latest version of the software and then that each person can then share the software in a peer-to-peer like approach so that as changes are being made into the server of changes to the code then those changes are then being redistributed to all of the development team the tool to be able to do an effective distributed vcs environment is get now you may remember that we actually covered git in a previous video and we'll reference that video for you so we start off with our remote git repository and people are making updates to the copy of their code into a local environment that local environment can be updated manually and then periodically pushed out to the git repository so you're always pushing out the latest code that you've code changes you made into the repository and then from the repository you're able to pull back the latest updates and so your git repository becomes the kind of the center of the universe for you and then updates are able to be pushed up and pulled back from there what this allows you to be able to accomplish is that each person will always have the latest version of the code so what is get git is a distributed version control tool used for source code management so github is the remote server for that source code management and your development team can connect their get client to that remote hub server git is used to track the changes of the source code and allows large teams to work simultaneously with each other it supports a non-linear development because of thousands of parallel branches and has the ability to handle large projects efficiently so let's talk a little bit about git versus github so get is a software tool whereas github is a service and i'll show you how those two look in the moment you install the software tool for git locally on your system whereas github because it is a service is actually hosted on a website get is actually the software that used to manage different versions of source code whereas github is used to have a copy of the local repository stored on the service on the website itself get provides command line tools that allow you to interact with your files whereas git help has a graphical interface that allows you to check in and check out files so let me just show you the two tools here so here i am at the git website and this is the website you would go to to download the latest version of git and again git is a software package that you install on your computer that allows you to be able to do version control in a peer-to-peer environment for that peer-to-peer environment to be successful however you need to be able to store your files in a server somewhere and typically a lot of companies will use a service such as git hub as a way to be able to store your files so git can communicate effectively with github there are actually many different companies that provide similar service to github git lab is another popular service but you also find that development tools such as microsoft visual studio are also incorporating git commands into their tools so the latest version of visual studio team services also provides this same ability but github it has to be remembered is a place where we actually store our files and can very easily create public and shareable is a place where we can store our files and create public shareable projects you can come to github and you can do a search on projects you can see at the moment i'm doing a lot of work on blockchain but you can actually search on the many hundreds of projects here in fact i think there's something like over a hundred thousand projects being managed on github at the moment that number is probably actually much larger than that and so if you are working on a project i would certainly encourage you to start at github to see if somebody's already maybe done a prototype that they're sharing or they have an open source project that they want to share that's already available and in github certainly if you're doing anything with azure you'll find that there are thousands 45 000 azure projects currently being worked on interestingly enough github was recently acquired by microsoft and microsoft is fully embracing open source technologies so that's essentially the difference between get and github one is a piece of software and that's git and one is a service that supports the ability of using the software and that's github so let's dig deeper into the actual git architecture itself so the working directory is the folder where you are currently working on your get project and we'll do a demo later on where you can actually see how we can actually simulate each of these steps so you start off with your working directory where you store your files and then you add your files to a staging area where you are getting ready to commit your files back to the main branch on your git project you will want to push out all your changes to a local repository after you've made your changes and these will commit those files and get them ready for synchronization with the service and will then push your services out to the remote repository an example of a remote repository would be github later when you want to update your code before you write any more code you would pull the latest changes from the remote repository so that your copy of your local software is always the latest version of the software that the rest of the team is working on one of the things that you can do is as you're working on new features within your project you can create branches you can merge your branches with the mainline code you can do lots of really creative things that ensure that that a the code remains at very high quality and b that you're able to seamlessly add in new features without breaking the core code so let's step through some of the concepts that we have available and get so let's talk about forking and cloning and get so both of these terms are quite old terms when it comes to development but forking is certainly a term that goes way way way back long before we had distributed cvs systems such as the ones that were using with git to fork a piece of software is a particular open source project you would take the project and create a copy of that project and but then you would then associate a new team and new people around that project so it becomes a separate project in entirety a clone and this is important when it comes to working with git a clone is identical with the same teams and same structuring as the main project itself so when you download the code you're downloading exact copy of that code with all the same security and access rights as the main code and then you can then check that code back in and potentially your code because it is identical could potentially become the main line code in the future now that typically doesn't happen your changes are the ones that merge into the main branch but also but you do have that potential where your code could become the main code with git you can also add collaborators that can work on the project which is essential for projects where particularly where you have large teams this works really well when you have product teams where the teams themselves are self-empowered you can do a concept what's called branching and get and so say for instance you are working on a new feature that new feature and the main version of the project have to still work simultaneously so what you can do is you can create a branch of your code so you can actually work on the new feature whereas the rest of the team continue to work on the main branch of the the project itself and then later you can merge the two together pull from remote is the concept of being able to pull in services software the team is working on from a remote server and get rebase is the concept of being able to take a project and re-establish a new start from the project so you may be working a project where there have been many branches and the team has been working for quite some time on different areas and maybe you kind of losing control of what the true main branch is you may choose to rebase your project and what that means though is that anybody that's working on a separate branch will not be able to branch their code back into the mainline branch so going through the process of a get rebase essentially allows you to create a new start for where you're working on your project so let's go through forks and clones so you want to go through the process so you want to go ahead and fork the code that you're working on so let's use this scenario that one of your team wants to go ahead and add a new change to the project the team member may say yeah go ahead and you know create a separate fork of the actual project so what does that look like so when you actually go ahead and create a fork of the repository you actually go and you can take the version of the mainline branch but then you take it completely offline into a local repository for you to be able to work from and you can take the mainline code and you can then work on a local version of the code separate from the mainland branch it's now a separate fork collaborators is the ability to have team members working on a project together so if you know someone is working on a piece of code and they see some errors in the code that you've created none of us are perfect at writing code i know i've certainly made errors in my code it's great to have other team members that have your back and can come in and check and see what they can do to improve the code so to do that you have to then add them as a collaborator now you do that in github you can give them permission within github itself it's really easy to super visual um interface that allows you to do the work quickly and easily and depending on the type of permissions you want to give them sometimes it can be very limited permissions it may be just to be able to read the files sometimes it's being able to go in and make all the changes you can go through all the different permission settings on github to actually see what you can do but you'll be able to make changes so that people can actually have access to your repository and then you as a team can then start working together on the same code let's step through branching and get so suppose you're working on an application but you want to add in a new feature and this is very typical within a devops environment so to do that you can create a new branch and build a new feature on that branch so here you have your main application on what's known as the master branch and then you can then create a sub branch that runs in parallel which has your feature you can then develop your feature and then merge it back into the master branch at a later point in time now the benefit you have here is that by default we're all working on the master branch so we always have the latest code the circles that we have here on the screen show various different commits that have been made so we can keep track of the master branch and then the branches that have come off which have the new features and there can be many branches and get so get keeps you the new features you're working on in separate branches until you're ready to merge them back in with the main branch so let's talk a little bit about that merge process so you're starting with the master branch which is the blue line here and then here we have a separate parallel branch which has the new features so if we're to look at this process the base commit of feature b is the branch f is what's going to merge back into the master branch and it has to be said there can be so many divergent branches but eventually you want to have everything merged back into the master branch let's step through git rebase so again we have a similar situation where we have a branch that's being worked in parallel to the master branch and we want to do a get rebase so we're at stage c and what we've decided is that we want to reset the project so that everything from here on out with along the master branch is the standard product however this means that any work that's been done in parallel as a separate branch we'll be adding in new features along this new re-based environment now the benefit you have by going through the rebase process is that you're reducing the amount of storage space that's required for when you have so many branches it's a great way to just reduce your total footprint for your entire project so get rebase is the process of combining a sequence of commits to form a new base commit and the prime reason for rebasing is to maintain a linear project history when you rebase you unplug a branch and re-plug it in on the tip of another branch and usually you do that on the master branch and that will then become the new master branch the goal of rebasing is to take all the commits from a feature branch and put it together in a single master branch it makes it the project itself much easier to manage let's talk a little bit about pull from remote suppose are two developers working together on an application the concept of having a remote repository allows the code to the two developers will be actually then checking in their code into a remote repository that becomes a centralized location for them to be able to store their code it enables them to stay updated on the recent changes to the repository because they'll be able to pull the latest changes from that remote repository so that they are ensuring that as developers they're always working on the latest code so you can pull any changes that you have made to your forked remote repository to your local repository the command to be able to do that is written here and we'll go through a demo of how to actually do that command in a little bit good news is if there are no changes you'll get a notification saying that you're already up to date and if there is a change it will merge those changes to your local repository and you get a list of the changes that have been made remotely so let's step through some of the commands that we have in git so git and it initializes a local git repository on your hard drive git add adds one or more files to your staging area get commit dash m commit message is a commit changes the get command commits changes to head up so the git command commits changes to your local staging area get status checks the status of your current repository and lists the files you have changed get locked provides a list of all the commits made on your current branch get diff use the changes that you've made to the file so you can actually have files next to each other you can actually see the differences between the two files a git push origin branch name so the name of your branch command will push the branch to the remote repository so that others can use it and this is what you would do at the end of your project get config global username or tell get who you are by configuring the author name and we'll go through that in a moment get config global user email will tell get the author of by the email id git clone creates a get repository copy from a remote source and get remote add origin server connects the local repository to the remote server and adds the server to be able to push to it get branch and then the branch name will create a new branch for you to create a new feature that you may be working on git check out and then the branch name will allow you to switch from one branch to another branch git merge branch name will merge a branch into the active branch so if you're working on a new feature you're going to merge that into the main branch a get rebase will reapply commits on top of another base tip and get rebase or reapply commits on top of another base tip and these are just some of the popular get commands there are some more but you can certainly dig into those as you're working through using get so let's go ahead and run a demo using get so now we are going to do a demo using get on our local machine and github as the remote repository for this to work i'm going to be using a couple of tools first i'll have the deck open as we've been using up to this point the second is i'm going to have my terminal window also available and let me bring that over so you can actually see this and the terminal window is actually running get bash as the software in the background which you'll need to download and install you can also run git batch locally on your windows computer as well and in addition i'll also have the github repository that we're using for simply learn and already set up and ready to go all right so let's get started so the first thing we want to do is create a local repository so let's go ahead and do exactly that so the local repository is going to reside in my development folder that i have on my local computer and for me to be able to do that i need to create a drive in that folder so i'm going to go ahead and change the directory so i'm actually going to be in that folder before i actually create and make the new folder so i'm going to go ahead and change directory and now i'm in the development directory i'm going to go ahead and create a new folder and that's going ahead and created a new folder called hello world i'm going to move my cursor so that i'm actually in the hello world folder and now that i'm in the hello world folder i can now initialize this folder as a get repository so i'm going to use the get command init to initialize and let's go ahead and initialize that folder so let's see what's happened so here i have my hello world folder that i've created and you'll now see that we have a hidden folder in there which is called dot get and we expand that we can actually see all of the different subfolders that git repository will create so let's just move that over a little bit so we can see the rest of the work and now if we check on our folder here we actually see this is users matthew development hello world dot get and that matches up with hidden folder here so we're going to go ahead and create a file called readme.txt in our folder so here is our hello world folder and i'm going to go ahead and using my text editor which happens to be sublime i'm going to create a file and it's going to have in there text hello world and i'm going to call this one readme.txt if i go to my hello world folder you'll see that we have the readme.txt file actually in the folder what's interesting is if i select the get status command what it'll actually show me is that this file has not yet been added to the commits yet for this project so even though the file is actually in the folder it doesn't mean that it's actually part of the project for us to do that we actually have to go and select for us to actually commit the file we have to go into our terminal window and we can use the git status to actually read the files that we have there so let's go ahead and use the git status command and it's going to tell us that this file has not been committed you can use this with any folder to see which files and subfolders have been committed and what we can now do is we can go and actually add the readme file so let's go ahead we're just going to select at git add so the git command is add readme dot txt so that then adds that file into our main project and we want to then commit those files into the main repositories history and so to that do that we'll hit the get command commit and we'll do a message in that commit and this one will be first commit and it has committed that project what's interesting is we can now go back into readme file and i can change this so we can go hello get git is a very popular version control solution and we'll we'll save that now what we can do is we can actually go and see if we have made differences to the readme text so to do that we'll use the diff command for get so we do get deaf and it gives us two releases the first is what the original text was which is hello world and then what we have afterwards is what is now the new text in green which has replaced the original text so what we're going to do now is you want to go ahead and create an account on github we already have one and so what we're going to do is we're going to match the account from github with our local account so to do that we're going to go ahead and say get config and we're going to do dash and it's going to be a global user dot name and we'll put in our username that we use for github in this instance we're using the simply learn dash github account name and under the github account you can go ahead and create a new repository name in this instance we call the repository hello dash world and what we want to do is connect the local github account with the remote helloworld.getaccount and we do that by using this command from get which is our remote connection and so let's go ahead and type that in open this up so we can see the whole thing so we can type in git remote add origin https backslash backslash github dot com slash simply learn dash github and you have to get this typed in correctly when you're typing in the location hello dash world dot get that creates the connection to your hello world account and now we want to do is we want to push the files to the remote location using the get push command commit get push origin master so we're going to go ahead and connect to our local remote github so i'm just going to bring up my terminal window again and so let's select git remote add origin and we'll connect to the remote location github github.com simply learn dash github slash hello world dot get oh we actually have already connected so we're connected to that successfully and now we're going to push the master gish so git push origin master and everything is connected and successful and if we go out to github now we can actually see that our file was updated just a few minutes ago so what we can actually do now is we can go and fork a project from github and clone it locally so we're going to use the fork tool that's actually available on github let me show you where that is located and here is our branching tool it's actually changed more recently with a new ui interface and once complete we'll be able to then pull a copy of that to our account using the fork's new http url address so let's go ahead and do that so we're going to go ahead and create a fork of our project now to do that you would normally go in when you go into your project you'll see that there are fork options in the top right hand corner of the screen now right now i'm actually logged in with the default primary account for this project so i can't actually fault the project as i'm working on the main branch however if i come in with a separate id and here i am i have a different id and so i'm actually pretending i'm somebody else i can actually come in and select the fork option and create a fork of this project and this will take just a few seconds to actually create the fork and there we are we have gone ahead and uh created the fork so you want to set clone or download with this and so this is the i select they'll actually give me the web address i can actually show you what that looks like i'll open up my text editor which doesn't correct i guess that is correct so i'm going to copy that and i can fork the project locally and clone it locally i can change the directory so i can create a new directory that i'm going to put my files in and then post in that content into that file so i can now actually have multiple versions of the same code running on my computer i can then go into default content and use the patchwork command to actually so i can create a copy of that code that we've just created and we call it that's a clone and we can create a new folder that we're actually putting the work in and we could for whatever reason we wanted to we could call this where folder patchwork and that would be uh maybe a new feature and then we can then paste in the url of the new uh directory that has the forked work in it and now at this point we've now pulled in and created a clone of the original content and so this allows us to go ahead and fork out all of the work for our project onto our computer so we can then develop our work separately so now what we can actually do is we can actually create a branch of the fork that we've actually pulled in onto our computer so we can actually then create our own code that runs in that separate branch and so we want to check out um the the branch and then push the origin branch down to our computer this will give us the opportunity to then add our collaborators so we can actually then go over to github and we can actually come in and add in our collaborators and we will do that under settings and select collaborators and here we can actually see we have different collaborators that have been added into the project and you can actually then request people to be added via their github name or by email address or by their full name one of the things that you want to be able to do is ensure that you're always keeping the code that you're working on fully up to date by pulling in all the changes from your collaborators you can create a new branch and then make changes emerged into the master branch now to do that you would create a folder and then that folder in this instance would be called test we would then move our cursor into the folder called test and then initialize that folder so let's go ahead and do that so let's call create a new folder and we're going to first of all change our root folder and we're going to go to development and we're going to create a new folder we'll call it test and we're going to move into the test folder and we will initialize that folder and we're going to move some files into that test folder call this one test one and then we're gonna do file save as this one's gonna be test two and now we're gonna commit those files okay add that kit add and then we'll use the dot to pull in all files and then git commit dash m files committed make sure i'm in the right folder here i don't think i was and now that i'm in the correct folder let's go ahead and and commit and it's gone ahead and edit those files and so we can see the two files that were created have been added into the master and we can now go ahead and create a new branch and call this one get branch test underscore branch and let's go ahead and create a third file to go into that folder this is file three into file save as we'll call this one test3.txt and we'll go ahead and add that file and to get add test3.txt and we're going to move from the master branch to the test branch git check out test underscore branch i switched to the test branch and we'll be able to list out all of the files that are in the tech in that branch now and we want to go through and merge the files into one area so let's go ahead and we'll do git merge test underscore branch as well we've already updated everything so that's good so otherwise it would tell us what we would be merging and now all the files are merged successfully into the master branch there we go all merged together fantastic and so what we're going to do now is move from master branch to test branch so git check out test underscore branch and we can modify the files the test3 file that we took out and pull that file up and we can now modify and we can then commit that file back in and we've actually been able to then commit the file with one changes and obviously it's the text free change that was made now we can now go through a process of checking the file back in switching back to the master branch and ensuring that everything is in sync correctly we may at one point want to rebase all of the work is kind of a hard thing you want to do but it will allow you to allow for managing for changes in the future so let's switch to it back to our test branch which i think we're actually on we're going to create two more files let's go to our folder here and let's go copy those and that's created we'll rename those tests four and five and so we now have additional files and we're going to add those into our branch that we're working on so we're going to go and select get add dash a and we're going to commit those files get commit dash a dash m adding two new files and it's added in the two new files so we have all of our files now we can actually list them out and we have all the files that are in the branch and we'll switch them to our master branch we want to rebase the master so we do get rebase master and that will then give us the command that everything is now completely up to date we can go [Music] get checkout master to switch to the master account this will allow us to then continue through and rebase the test branch and then list all the files that are all in the same area so let's go get rebase test underscore branch and now we can list and there we have all of our files listed incorrectly hello and welcome to git commands my name is chidanand and i am with the simply learn team today i am going to cover few essential git commands and walk you through a demonstration for most of it i begin by installing a git bash on the git client and configuring it for its first time usage once i'm done with configuring the git client i'm going to spin up few repositories locally and work on these repositories using git commands once i'm happy with my local changes i'm going to commit few of them and then push these changes to a git server of my choice for today's demonstration i plan to use github.com which is a cloud hosted free service and it provides free account registrations for anyone eventually i will cover few topics regarding git branching and merging because these two in my opinion are the killer features of the git distributed version control tool kit is one of the most popular distributed version control tools of recent times and like any other distributed version control tool it allows us to perform various version control tasks without really needing a network connection before i jump into my demonstration let me spend some time explaining to you the git client server working model what does it take for a couple of users to collaborate and start working with git the bare minimal thing that any user would need in order to start working with kit is something called as git bash or the git client this comes as an installer for all popular operating systems like windows all flavors of windows linux mac os and other operating systems so once you install git bash you get a command line utility using which you can fire up your git commands and ensure that you can bring up repositories you can work with these repositories by adding files to it committing changes to it and all those git commands would work perfectly well so you can bring up a small repository and you can work with your repository using this git bash but what does it take for you to share this repository with another user who has also got git bash installed on his system this is where you would need something called as a gig server so for user one or one user to share his repository with another user he would need to collaborate using something called as a git server so in the present market there are a bunch of git servers which are popularly available some of them are free some of them come with cost because they are licensed a bunch of these servers that i can think of are github which is one of the most popular kit servers that is around in the market for a long time now it comes with two variants one is the cloud hosted one which is the github.com and then the other one is an enterprise server which comes as a black box that can be installed into your data centers so typically organizations who don't want their source code to be put up on the cloud but go for this github enterprise servers wherein they buy these servers and these servers are hosted on their data centers other popular variant of the git server is bitbucket this is from the famous atlassian products and it integrates very well with all other atlassian products like jira in recent times there's one other variant of the git server called gitlab which is getting very very popular these days because gitlab not just provides a git server it also provides something called as a runner this is a part of the continuous integration strategy where as soon as you have a source code checking that is going in you have this runner that kicks up and builds your particular project so all these are bundled together in a gitlab and gitlab also provides you with a community edition which is almost free of cost having said that there are a bunch of these tools available for my demonstration what i'm going to do is i've already registered myself on the github.com by providing my username and my password and i'm going to use a free account so whatever repositories that i'm going to put up on this server is all public by default so i will register myself by using my username or my email id and my password so this set of authentication is also referred to as https authentication so in order for me to share my repository with say user2 the first thing that i got to do was have an access to a git server where both user 1 and user 2 are set up so once i have that i can push my repository onto this server and provide right kind of access so that user 2 or the other developer who wishes to collaborate with me on this repository can use his credentials as long as he's registered properly on the git server he can use his credentials and pull down this repository and work with this repository so this is the https way of authenticating and sharing repositories another popular way of sharing repositories or working with each other is called as ssh authentication as many of you would know ssh means nothing but creation of a private and a public key a bunch of keys public and private are created on the client machine while i create a keys i will be asked for a passphrase i need to type in a passphrase and remember my passphrase and then i take this public key and kind of post it onto my account within my git server so that whenever i'm going to connect next time using my command prompt or any of the tools it's going to challenge me for my passphrase as long as i remember my phrase correctly i can authenticate myself and get into the server so both https and ssh are popular ways of communicating with the git server having said this let me just get started by installing the git bash on my local system and then fire up some of these git commands and build up some repositories make some changes in my repository commit these changes and later use my github credential to connect to my github server and push my repositories out there let me now go ahead and find my git bash for my windows so that i can go ahead and install it so i just type in get client for windows open up a browser and type in for this the first one that i find as a gui tool i don't need this gui client i need this one so this is the git client or the git bash that i was referring to it's about 40 megs and let me just download this pause this video for some time so that the download happens and come back to you once i have the download the download seems to be complete when i'm recording this video the version that i have the latest version that is there is 2.19.0 and this is a 64-bit version for windows so i've got that downloaded let me go ahead and find it and install this exe all right i go ahead choosing all the default options i want it to be installed in c colon program files under the git folder everything looks good for me i'm not going to make any changes here get bash from windows yes i may want to use git from windows command prompt that sounds good for me open ssh that's fine looks good looks good now i don't want any of these new features so what i'm doing is installing the git bash or the git client so that i can start using my git commands the version that i am using is 2.19.0 which as i record this video this happens to be the latest version all right so looks good i don't want to see the release notes i'll say launch git bash so what i get is this window so this is the git command prompt so let me check if everything is looking good say get version it sounds good say get help all right a lot of git help commands everything looks good so i ended up installing the git bash i can go ahead with all other commands now that i have git bash installed on my system let me open up a git bash prompt and start using it but before i do anything i just need to configure it for the first time stating what would be the username and the email id with which i want the git bash to be configured with the command to do that is get config hyphen iphone global which means that this setting that i'm going to set would be a global setting user dot name and this the name of the user would need to be specified within a double quote in my case i am using a simply learn account which is the official account and this is the particular user name of this account the other setting is nothing but git config hyphen f1 global this is also a global setting and this would be the email id the email id need not contain a double quote so in my case this is the email id that i would be setting up git to work with all right let me check if the configurations were set correctly or not all right if you see this the username and the email id has been picked up the way i wanted it to be picked up looks good so far all right so what do we do next by default whenever you open up the git prompt you are placed into your home directory let me create a folder here which would be a repository folder from where i'm going to create all my multiple git repositories so i will create a folder called let's say git underscore demo all right there's already a folder called git demo so let me just delete that in the first place and let me create this git demo again all right let me navigate to this particular folder so this is the folder let me open up that folder using a command prompt this is my git demo so this would be a folder so i would use this folder as a base folder for creating all other repositories as a part of this demo so let me create my first repository a repository is nothing but a folder so let me create my first folder called first repo and let me get into this ripple all right as of now this is just an empty folder if at all i navigate to this you could see that this folder is empty there's nothing much in this folder at all at this moment okay so let me create a repository out of this how do i create a repository of this it's pretty simple i just run and get init command when i say git init it means it's going to initialize a repository as of now the folder contained nothing on it was an empty repository so it initialized an empty repository in this particular folder if at all i look at my content you'd see that there's a hidden folder this would be a hidden folder since i've configured my explorer to view all hidden folders i see it otherwise you wouldn't see it so a folder by name dot get is created within this first repo and this was created because we ran the command git init if i get into this folder i see a bunch of directories and other configurations if you see there's something called as a hooks directory a couple of configuration files that are here in this directory all these relate to one or the other type of a client hook that can be enabled and there's info this objects this references a bunch of stuff that is there so what is this folder this is a magical git folder and this is created whenever a repository is initialized so by default if at all you get into any git folder or any git repository you would see this dot get folder a word of caution do not get into this folder and try to modify anything because if you fiddle around with this and corrupt this particular folder your whole repository will go for a toss so let me come out of this and let me show you another thing now if you notice here when i was in the folder the repository did not have anything after this now when i say in it there is something called as a master that shows up so typically what happens is whenever a git repository is created for the first time it creates a default branch the name of the branch is called master and that is why you would see master within braces this doesn't mean anything else other than we have a repository out in this particular folder and by default there is a branch called master and we are currently on that branch if at all i create multiple branches in this folder whenever i navigate to different branches the branch name within this brackets would change all right so far so good so we created a folder we made it a git repository by initializing that with the git init and then i have not put anything yet in the repository so let me try to do that let me create a few files within this particular repository so i use a touch command touch is a linux command it kind of creates this particular folder if not you're not comfortable with the file creation by this way you can always use a file explorer and create some files in this so i create a file called touchmaster.txt i'll open up in the notepad okay so this is my first file that's what i'm going to write inside this i save this and i come out of this all right now let me do a git status command so what does this tell you there's no commit set you're on the master branch there is an untracked file and this shows up in red it also gives you all the commands that is supposed to be run so if at all you want to add this file you would say git add and the name of the file and then you're going to commit it so what happened is that when the repository was initialized it was an empty repository now that a git notices that there is a new file in this repository called master.txt if at all you want you want us to let git know that you want to track this file we need to add this to the repository so command to add that is called git add i can specify either the name of the file or i can give a while character saying git add dot now if i run the git status command again it says there's no commits yet added a file and the file which was earlier in red now shows up in green and it says it's a new file master dot text all right it also gives you something called git remove command if at all you want to unstage or undo your changes so what has happened is the file which was earlier untracked has been added into the git index or it is ready for staging or it's in a staged state now i can go ahead and commit this file if at all that's what i want to do so let me do that i'll do a kit commit minus m and i give a message that i want i would say this is my first get commit right i run the gate status command again and see it says on the branch master nothing to commit it's all nice and clean so typically what happened was i created an empty directory made into a git folder by doing a git init i put a file into it i made some changes to the file i added that file into the repository and i committed to the repository if it error i run the git log command it says this was the commit id or the commit number or the commit hashtag and this is the author if you notice this is the email id and the username that we set earlier this was done on this date and this is the commit message that is there all of it looks so good let me do one more commit into this repository so i will touch master one dot text i'll put in another file i'll open up this particular file write something into this all right i save this change let me also open up the older file make some more change into this all right so what i did i modified the older file that existed i also added a new file to my repository now if i run the git status command it says modified this file which is showing in red and it says you have one more file which is an untracked file all right now let me add both these files because these are the changes that i want to incorporate i would say git add dot i run the git status command again and it says now this is a modified file this is a new file looks good to me let me go ahead and commit this so git commit with the message that says this is my second commit all right if i do a git log now it shows both the commits the first command is the one that that shows up at the bottom and the topmost one is the latest commit that is there so far so good if at all you notice i'm doing all these git commands without really connecting it to any git server i don't need any network connection for doing any of these things i can keep on doing more and more git commands and run git commands the way i want to without really having any connections to the git server now that i have my changes the way i wanted and i've checked in all the code changes into my local repository i may want to share my repository with somebody else and work in a collaborative way so the way to do that is as i mentioned earlier to host this repository or to push this repository onto a kit server i already have a github account wherein i've registered myself using my username and password so i log into my github account using my username and credentials and let me go ahead and create a new repository my repository is already there on the local system so what i essentially need is just a placeholder for my repository so that i can link them together and push the content from my local repository to the server if you notice the name of my repository is this first repo so let me copy this i will create a folder structure or a repository structure on the server with the same name so that when i pushed my repository from my local box it will go ahead and get created in the server so the repository name should be the same as what i created on my local box description i would say this is my first git hub repository on the server this is just an optional description that will show up and if you notice i can't have any private repositories so whatever repositories that i'm going to create here would be a public repository so i choose public and i don't want to initialize this with a readme file or anything of that because i already have a repository with some content with some code checked in all that i need is a placeholder for my repository on the server so with this option with just the name and with an optional description and the repository being public i go and click on the create repository button so if you notice it gives me a bunch of commands that i would need to run in order to create a repository and pushed onto the server and also important point it shows me two urls one is the https url and the other one is the ssh url as of now we do not have any ssh keys that is set up so let me use this https url to connect my local repository with the repository on the server so i've done all these things i already have the repository set up all that i need is a placeholder for me to connect my local repository with the server url so this would be the command that i need to copy which essentially links or adds an origin with the url of the git server i just copy this command and run it here git remote add the name of the remote is called origin and it points to an https url which is nothing but the placeholder that i've created on the server all right so let me check if it's added correctly okay the url looks good i'm all set i've got two comments in my repository i'm done with all the changes so let me try to push my repository onto the server the command for doing it for the first time is git push hyphen u this is for linking the upstream origin master so git push hyphen u upstream linking origin master link the master uh branch of my local repository with the master branch on the server repository and where is it going to push it is going to push by default to an origin and the origin points to this url all right let me try doing this so it essentially opens up an url asking me the credentials with which i need to log into my github server so let me just copy and paste the username and the password with which i have registered i would say login okay success so i'm able to create a repository on the server and the content seems to be pushed and it created a new branch called master it has linked the master branch from the local repository with the master branch on the server so let me go back to my server and refresh this page and see if all my commits have come in master master1.txt if at all i see the commits there seem to be two comments this is my first comment this is my second comment and if at all you notice each of this commit comes with a hash key or a hash id which contains the details of my commits in this commit the files were added this was a line that was added if you look at the second commit i added these lines i made this you know one line chain that was there i also added this new file which was not existing these are the two commits that existed and if you see the timestamp of the commit this is the commit which was committed for 15 minutes ago so what does this mean is that the commit is the timestamp when we actually commented the code in the client the pushing is just after we committed the change we pushed it so there's no timestamp of the push that happened this completes the first part of the tutorial wherein i create a repository on the client and then i pushed it to the server after creating a repository skeleton on the server now let me demonstrate to you the ssh configuration that is required for creating a pair of ssh keys and ensuring that i connect to the github server using the ssh mechanism so for doing that i need to first create a pair of private and public key the command to do that is ssh keygen hyphen t i will use the rsa mechanism for creating my keys capital c and i need to specify the email id using which i'm going to connect to or i need to create the keys for all right so generating a private and public key pair it says let me know what folder should i create the keys in so by default the keys are created in the user home directory under a dot ssh folder so this is pretty good for me so let me go ahead and say that that's fine enter a passphrase so let me enter some passphrase keep in mind that i would need to remember my passphrase because when i'm going to authenticate myself using ssh mechanism it will challenge me for this particular passphrase i need to enter the right passphrase so that i get authenticated so the key is generated and by default it is created in this particular folder so let me go over to that particular folder and this is the folder where the keys is generated you would see a public key and a private key that is created let me open up the public key with a notepad and then copy the contents of the whole file and now i would need to stick it in on the server so that i can get myself authenticated using ssh keys so i've logged in to github using my credentials and go to my settings and you will see an ssh and gpg keys let me click on that as of now there are no ssh key or gpg keys so i'll click on new ssh key and let me paste my public key that i've copied and i would say add ssh key it prompts me for my password just to make sure that i've got the right kind of authentication to be adding a new key so this one looks good that means the keys were added successfully let me check if my ssh keys are working well so i do ssh hyphen capital t i would say git at github.com this is just to double check if my ssh keys are authenticated correctly are you sure you want the connection yes you want to continue yes all right now a check for my passphrase let me enter my passphrase i hope i remember it correctly all right pretty good i have been successfully authenticated so that means now my ssh keys are good and just by using my passphrase i can connect to my github server using ssh alright so for the second use case what i would do is create a repository on the server with some content in it and then clone that repository cloning is nothing but making a copy of that repository linking the origin so that i create an exact replica of the repository which is there on the server on my local drive so let me first create a repository on the server i will create a new repository let me give a name for my repository and let me kind of initialize that with the readme file i would call this my second repo give some description this is my second repo that is created straight on the git server on the github server to be precise it's a public repository this time i'm going to choose this option the reason being that i don't want to create just a skeleton as of now i want to create a repository with some readme file in it a random readme file with no content in it but then i want a repository to be created on the server all right i say create a repository so a random readme file with some content in it gets created and i can see there are two urls to this repository https and ssh so this time let me copy the ssh url so that i will use this url to clone this repository on the server rather from the server to my local box all right so let me navigate to my root folder the folder that i created for making all my repositories git repositories if you see my first deposit is already here so let me clone my repository the second repo that i created on the server the command to do that is get clone the url of the repository that i want to clone oops looks like i didn't copy the url correctly let me copy that again copy this url all right so if you notice there's no https to it i'm just cloning the url that is ssh url of the repository it asks for my passphrase again okay so i just had first repository earlier now i have something called a second repo this is an exact replica of whatever was the repository content that was created on the server so if i get into the repository there's just a simple readme file that is there let me try to add a new file out here i would say notepad second.txt some text file i'm going to put in here yes definitely you will not find that file there this is my second file that i will push onto the server i'm going to put in some content out there i'm going to save this and if i do get status it says this this is a new untracked file do you want to add this file onto the repository do you want to track this file yes i definitely want to track this file let me also commit my change to the file with the message this is my first commit for the second repo i'm giving some particular message and this time around since i cloned my repository from a server the origin is already set for me i don't really need to add the origin and all that stuff because what happened in the first case was that i was creating a repository from my local box and then pushing another server which is why i had to set up the upstream branches and stuff like that in this case i have cloned an existing repository from the server so when i clone i get the replica along with all references to my repository so i don't need to do anything more other than push the content i will just say git push to be on the safer side i would say origin and master it ask me for the passphrase okay it pushed the content onto the server let me see if the contents have come in here okay wonderful i had only one comment now i see my second comment so what was added as a part of the second commit it says a new file was added all this line that never existed got added this is the second file the content exactly what i pushed onto the server now this completes the tutorial when i create a repository on the server cloned it made some changes to the repository and pushed it to the server using ssh a quick recap of all the git commands that we ran till now git init is used to initialize a repository so if at all you are creating a local repository you can get into any of the folder which is not a git repository and run this command whenever you run this command a magical folder by name dot kit gets created in the folder and from that time onwards git will start tracking any changes that happen to that particular folder it also creates a new repository and a branch by default the branch that is created is called the master branch git add dot is a wildcard character for adding any number of new files into your git repository if you have a bunch of files one file two file ten files a folder containing multiple files in it all of them could be added into the repository using this command so once you add any number of files into the repository you can keep on adding more and more files into the repository once you're done with that you want to commit the changes that has happened in the repository you use the git commit command git commit hyphen m with a meaningful message this will commit all the work items that you done in terms of the files that got changed the files that got added all this with one particular message get status will give you the status of all the files and folders any file that got added any file that got created any file that was deleted all the status of the files will be obtained using the git status command git log will show you all the commit history with the latest history or the latest commit showing up on top git add remote origin this command is used whenever you want to link any local repository to a server and this is when you want to really push a repository from the local to the server for the first time git push hyphen u origin and master this is the command that you would use whenever you want to push contents from the local repository to the server git clone and the url of the server this is the command that you would run whenever you have an existing repository on the server and you want to make a fresh clone or a fresh copy of that particular repository onto your local box more than often many developers or git users find themselves in situations where they would want to go back in history and modify or change some of the files that they recently committed or checked into the repository now from a perspective of git this is a very very simple activity however from a perspective version control tool you will have to be really cautious about the chain that you're gonna do in case the changes are pretty local none of the developers or none of the other users who are collaborating with you would be affected by this change you're good to go ahead and make this change however if your repository has been pushed to the server and you're trying to modify something from history this would have a very bad or adverse effect on all the other users who will be using it you need to be really cautious while running these commands in case your repository contents are local you have not pushed your changes to your repository and if at all you made some changes to the repository maybe to fix a defect or to include a new feature and you missed few things you can go ahead and modify this with some of these commands let's say i have this simple repository called undo undo repo and if at all i look at the logs of these i have the commit histories which are like this i have about five commits that is there in my repository and in c1 i've added a file in c2 i've added another file in c3 i've also added some file in c4 i made some changes to m1 and m3 and in c5 i've added another file called m4 now this is my commit history now let's say i made these two comments but possibly i want to club them together along with some other changes because whatever changes i did did not really fix things the way i expected it to fix so there is a git command called git reset this would allow me to go back in history to go back to any of the snapshot get rid of all this commit history on the commit messages but retain the changes that were made as a part of these comments so if at all i need to go back in history and go back to this particular commit and however i just want to get rid of these commits that are there in terms of the commit messages but i want the changes that were there as a part of these commits to still exist i would do command called get reset hyphen iphone soft now this would go back or unwind my changes back to this particular snapshot getting rid of all the commit messages that were there as a part of these two commits but a soft would ensure that the file changes that were done as a part of these two comments would still exist a full remain so the command for that is git reset hyphen f and soft which is the snapshot that i want to get back to if this is the commit that i want to get back to i just copy this and i paste this all right it doesn't give me any message but if at all i do a good log pretty one line what it says is c1 c2 now the head which is nothing but the current pointer which is earlier pointing to c5 it's gone back to c3 now what happened to the changes that were there as a part of c4 and c5 if at all you do a git status if you see the changes that were part of c4 were some changes to m1 and m2 and c5 a new file got m4 got added if what i do a git status you'll see those changes still exist however from the git history or the git commit history c3 is the topmost commit that exists as of now c4 and c5 have been gotten rid of but the changes that were there still exist all right now i can make some more changes that i want if as a part of my commit and possibly go ahead and commit this whole thing as a new commit altogether maybe i'll open up a notepad and possibly create a new file called m6.txt and say one new file that was missed out i make this change i go to a get status all these files are still there i would say git add dot also get commit hyphen m c4 rewriting history and i do a get log pretty there you go so c1 c2 c3 still existed and whatever was a part of c4 and c5 the older ones i've gotten rid of this along with those old changes i also added a new file i made some changes and it was able to commit it now if you look at my gate history it's all nice and clean however do this activity only when you don't push your git repository to the server one other most powerful and useful command while undoing or resetting the history is something called as revert this is a safe way of undoing some things remember what i talked about when exactly you can do a reset it's only when your changes are local and is not being pushed to the repository take this scenario wherein let's say i have a git server and i have the changes which are c1 c2 c3 and possibly c4 all right so this is a bunch of comments that has happened this was the oldest comment and this is the newest comment and if at all developer one is the one who's responsible for pushing these contents in the first place so he has got c1 to c4 on his history i'm going to jump all these c2 and c3 and get into c4 and has pushed all these contents out here and there are a bunch of people from the server who has pulled this repository and all of them are at the same level now imagine a use case where developer 1 then figures out that by mistake he put in the c3 which was the wrong thing to do and possibly you know he could have done a better job of fixing this in a different way so he wants to get rid of the c3 or c4 or any one of these comments if he's going to do that and push to the server first of all the git server would not allow him to go back in history but assuming that he forcibly pushed something what would happen to all these people who are looking to work ahead from c4 all of them would be affected in adverse way so in short never go back in history if at all you need to undo something go ahead and put in a new commit to undo something let's take this example maybe if at all developer 1 wants to get rid of c3 better way to do that is by adding a new commit which is ahead say let's say c5 and what does this commit do this commit possibly gets rid of c3 instead of going back in history it actually is taking your git history ahead but as a part of this commit you're undoing some part of the work that was done as a part of c3 in short you're removing a commit but to remove a commit you're adding a new commit and this is why you're doing it because there are a bunch of other people who are you're using this repository collaboratively and were planning to get ahead in terms of history they cannot afford to go back in history and modify something in history all right the command to do this is called the get revert command so let me take up this repository which has got a bunch of commits in it and if at all i do a git log pretty one line i have these bunch of comments and c1 i've added a file in c2 over another file in c3 i've added one more file and in c4 i made some changes and this is the latest commit and this is the oldest command so if at all i see the contents of the git folder it contains m1 m2 m3 and m4 and let's say looking at the previous use case that i mentioned i may want to get rid of this particular commit i want to get rid of the c3 which is nothing but i've added a file called m3.txt but i added it by mistake i need a better way of fixing it possibly i look at it later on but for now i don't want this particular commit and the changes that were made as a part of that commit to exist so the better way to do that is using the git revert command so git revert and the commit id whatever i want to revert so if i want to revert this i copy this particular commit id and bear in mind that considering reset revert works on one commit at a time so i can do one commit revert at a time i can club bunch of them together and do one commit but otherwise the revert always works on one commit at a time so get revert paste this essentially what i'm trying to do i'm trying to undo this particular commit while i'm undoing this commit what does git do it safely adds a new commit and a part of this new commit it unders whatever changes that were done as a part of this comment so if i say git revert this all right i forgot to mention the message which is why this window is showing up but that message is good for me i say write and quit all right and if i do a git log all right if you see this c1 still is there in history and our history is going ahead from c5 i see another commit but what was it as a part of this commit actually is getting rid of this commit c3 so if at all i see my files here there's only m1 m2 and m4 m3 was added as a part of c3 now i've gotten rid of m3 by getting rid of the revert command for c3 so this is a better way of undoing things now that we have acquainted ourselves with few of those basic git commands let me get into a very very interesting topic regarding git which is about branching the topic is interesting because kit deals with branches in a completely different way compared to any other version control tool in most of the version control tools whenever you create a new branch a complete copy or a subset of the whole repository is created which in my opinion is very very cost inefficient because it is a very very expensive operation both in times of time and the disk space however in git you just create a very very lightweight mobile pointer whenever you create a branch there is no folder or no copy of the repository that is created in short the branches are very very inexpensive and great so go ahead and create any number of branches that you need however once you create your branches do all the work relating to that branch in that specific branch and once you've completed your task go ahead and merge your branches back into your base branch and then delete the branch let me go ahead and create a repository with some branches in this and give you a demonstration of that so let me go back to my base directory which is git demo and then i'm going to create a new folder here i'll create a folder called arc details all right this is my folder here all right i navigate into this folder and let me create a repository here i initialize a repository a dot git folder is created without any files in it so let me create a few files on it say notepad name dot text organization name i create a name.text within this maybe i'll create one more file in this notepad employees.txt simply learn has over 3k employees i just put in some content in this save this content all right so if i try to go get status i just see two files in it so let me go ahead and add these two files and let me commit my changes so okay commit iphone am or iphone m first command so if i do a gig log i see only one commit and there are two files so you can see those two files that is here all right so at this stage let me go ahead and create a new branch and the command for creating new branch is get branch name of the branch so let me name my branch as bangalore branch blr branch and this created a branch but i don't see any changes in the folder structure or in the file that exists here i've just created this branch so if at all i do a git branch i see these two branches and whatever star means i'm currently in this master branch so let me go ahead and get into the blr branch at this moment the number of files that exist in this repository is same there's absolutely no change in any of the branches both master and blr branch point to the same snapshot so let me get into the other branch and the command to do that is gate checkout plr branch i've actually switched into bangalore branch if you see the content of the files there's exactly two files the way it was earlier because i've created a branch from my base branch which is a master branch and when i created a branch there were two files existing here if you see the content of these files they would all be the same all right so let me create a file that is specific to panel branch so i would just say notepad maybe i want to put on the address of this organization specific to bangalore in this address.txt so i see notepad address dot text we'll say simply learn angular office say located in some address i would say bangalore that is the content of this file and i'm creating all this new file in this blr branch so i would say git add dot and i would say git commit hyphen m bangalore branch commit all right if you see this now i have an address.text which is there only in the bangalore branch that i'm presently on now let me switch back to my master branch the command to do that is get check out master branch if you remember when i forked out or when i created a new branch from the master branch i didn't have this address.text in it so let me go back to git git checkout master branch and keep noticing this particular what happened to the address.txt it kind of disappeared how did that happen it's all because of this magical git folder so what happened is now git knows that when you are in the master branch the file that was there in the bangalore branch which was called the address dot text didn't exist so git is playing around with the file system snapshot so if i come back let's say i come back to get check out bangalore branch if you see this the address.txt file appears so this particular file is only present in the branch and git is playing around with the file system snapshot by moving the pointers across and ensuring that whenever i am switching to bangalore branch i see this address dot text when i go back to my master branch i don't see this bangalore branch contents which is the address.txt right now that i created this branch maybe i may need to push this onto the server so let me go ahead and create a particular repository structure on the server similar to whatever we did in our previous exercise so the name of the folder needs to be exactly this one so let me copy this and let me create a repository very quickly go here and say new repository all right this is the name of the repository get branch demo this is what i'm going to put in as a description i would say public i don't want any readme files in it because i already have a repository i will just say create a repository all right let me use the ssh url yeah this is the ssh url this is good for me so let me copy this url let me add it here i'm linking my local repository with the server and i would let me come back to my master branch let me first push my master branch so i would say git push hyphen u origin master it asks for my passphrase all right i pushed my master branch it looks good it says new branch master is created so if i go back to my server and i do a refresh if you see it says only one branch and it says what is the name of the branch master if you see the contents there is employees.txt and name dot text now let's try to push the other branch that is there the command to do that is a little different compared to the way we created it on the client because on the client we created the branch with a git branch command now this branch doesn't exist on the server and i need to push this new branch onto the server so the command would be gate push hyphen u origin and the name of the branch that i want to specifically push all right now it created a branch out here let me do a refresh of this all right it says two branches if i get into this bankrupt branch i would see the address dot text if at all i go back into master branch i would not see this address.text wonderful now let me do one thing let me try to get the changes from the branch that i created back into the master branch that is what i would mean by merging i would want the changes that is there as a part of the bangalore branch to come back into my master branch so if i go to get checkout bankrupt branch i see this attach.tech this is there only as a part of bangalore branch now assuming that i'm done with the changes from the banner branch i want to get those changes back into my master so let me come back to my master branch and the way to merge in the changes is get merge so what is that you want to merge i want to merge the bangalore branch changes and where will it merge it'll merge it into the current branch that i'm on presently i'm in the master branch so this command would merge the changes from the bangalore branch into your master branch so with this if you'll keep looking here the content that was there that was added as a part of bangalore branch which is nothing but the address.txt this file should appear as soon as we do a merge all right so it happened quickly and you would see this address.txt that is here so if you see let's check the contents in terms of the git log all right so there was only one branch that we had earlier which had only one commit right now we are at this bangalore branch commit which was there as a part of the bangalore branch commit so with this change let me push this change onto the server as well gate push origin master okay those changes were pushed to the server and now if at all i do a refresh out here okay i mean two branches the branches master even if i'm in master brands i should see this address dot text that is there even if i'm in the bangalore branch the content is all same because we immerse these two branches now that i've merged this branch i don't need this branch anymore so i would go ahead and delete this locally as well as push it onto the server so that it gets deleted from the server as well so delete branch locally the command is get branch hyphen d and the name of the branch only thing you need to ensure is that you should not be currently on that branch which you're trying to delete so i would say git branch and i'm presently in the master branch so i can go ahead and delete the other branch so get branch hyphen d bangle branch all right this got rid of our bandwidth branch locally if at all i don't get branch hyphen we it shows there's only one branch as of now however this branch would still exist on the server because server doesn't know anything about the changes that we've done we need to specifically push our changes onto the server okay there are two branches still here so let me go ahead and push these changes so that the branch gets deleted from the server as well the command to do that is get push origin hyphen iphone delete and the name of the branch all right so this went ahead and deleted the branch from the server let's check that okay if you see now there's only one branch these are the basic steps that you would need to know before you work with any of the branches within git and as i said before the branches are very very inexpensive and good go ahead and create any number of branches that you want but just ensure that once you create your branch finish off your task and once you finish off your task go ahead and delete the branch and before you delete the brands please ensure that you merge in the branches so that you get in or take in the work that was done as a part of that particular branch to summarize the gate branching commands are as follows to create a new branch you issue a command git branch the name of the branch and this would automatically spin off or create a new branch from the existing branch from where you issue this command if you are on the master branch and if you execute this command git branch new underscored branch it creates a new underscore branch from the master branch once you create a new branch in order to get into this branch you would type in git check out the name of the branch this would automatically switch the file system snapshot between different branches now the branches that are created are by default created on the local repository in case you want to push any specific branch onto the server you would have to issue a command git push hyphen u origin and the name of the branch so this will ensure that the contents of the new branch are pushed onto the server at any time if you want to list out all existing branches on your local repositories the command would be git branch hyphen av assuming that you did all your work in a new branch and you're happy with the changes you would want to bring back those changes into the master branch the command to merge the contents of any new branch into a master branch would be git merge and the name of the branch this would automatically merge the branch contents that you specified into the branch on which you currently reside if at all you are on the master branch and if you issue a command git merge new underscore branch this would merge the contents of the new branch onto the master branch so once you merge the contents of any branch you would possibly want to go ahead and delete this branch so the command to delete those branch locally is get branch hyphen capital d and the name of the branch this again would delete the branch locally in order to make this change on the server which is in order to remove this branch from the server the command to do that is get push origin hyphen iphone delete and the name of the branch this would also delete the branch that you created on the server hello this is matthew from simply learn and today we're going to deep dive into what jenkins is and how it improves your devops continuous integration environments so this is what we're going to be covering in this session we're going to cover what life is like before using jenkins and the issues that jenkins specifically addresses then we'll get into what jenkins is about and how it applies to continuous integration and the other continuous integration tools that you need in your devops team then specifically we'll deep dive into features of jenkins and the jenkins architecture and we'll give you a case study of a company that's using jenkins today to actually transform how their it organization is operating so let's talk a little bit about life before jenkins let's see this scenario i think it's something that maybe all of you can relate to as developers we all write code and we all submit that code into a code repository and we all keep working away writing our unit tests and hopefully we're running our unit tests but the problem is that the actual commits that actually gets sent to the code repository aren't consistent you as a developer may be based in india you may have another developer that's based in the philippines and you may have another team lead that's based in the uk and another development team that's based in north america so you're all working at different times and you have different amounts of code going into the code repository there's issues with the integration and you're kind of running into a situation that we like to call development hell where things just aren't working out and there's just lots of delays being added into the project and the bugs just keep mounting up the bottom line is the project is delayed and in the past what we would have to do is we'd have to wait until the entire software code was built and tested before we could even begin checking for errors and this just really kind of increased the amount of problems that you'd have in your project the actual process of delivering software was slow there was no way that you could actually iterate on your software and you just ended up with just a big headache with teams pointing fingers at each other and blaming each other so let's jump into jenkins and see what jenkins is and how it can address these problems so jenkins is a product that comes out of the concept of continuous integration that you may have heard of as power developers where you'd have two developers sitting next to each other coding against the same piece of information what they were able to do is to continuously develop their code and test their code and move on to new sections of code jenkins is a product that allows you to expand on that capacity to your entire team so you're able to submit your codes consistently into a source code environment so there are two ways in which you can do continuous delivery one is through 90 builds and one is through continuous so the approach that you can look at continuous delivery is modifying the legacy approach to building out solutions so what we used to do is we would wait for nightly builds and the way that our nightly builds would work and operate is that as co-developers we would all run and have a cut-off time at the end of the day and that was consistent around the world that we would put our codes into a single repository and at night all of that code would be run and operated and tested to see if there were any changes and a new build would be created that would be referred to as the nightly build with continuous integration we're able to go one step further we're able to not only commit our changes into our source code but we can actually do this continuously there's no need to race and have a team get all of their code in an arbitrary time you can actually do a continuous release because what you're doing is you're putting your tests and your verification services into the build environment so you're always running cycles to test against your code this is the power that jenkins provides in continuous integration so let's dig deeper into continuous integration so the concept of continuous integration is that as a developer you're able to pull from a repository the code that you're working on and then you'll be able to then at any time submit the code that you're working on into a continuous integration server and the goal of that continuous integration server is that it actually goes ahead and validates and passes any tests that a tester may have created now if on the continuous integration server a test isn't passed then that code gets sent back to the developer and the developer can then make the changes it allows the developer to actually do a couple of things it allows the developer not to break the build and we all don't want to break the builds that are being created but it also allows the developer not to actually have to run all the tests locally on their computer write tests particularly if you have a large number of tests can take up a lot of time so if you can push that service off to another environment like a continuous integration server it really improves the productivity of your developer what's also good is that if there are any code errors that have come up that may be beyond just the standard ci test so maybe there's a code the way that you write your code isn't consistent those errors can then be passed on easily from the tester back to the developer too the goal from doing all this testing is that you're able to release and deploy and your customer is able to get new code faster and when they get that code it simply just works so let's talk a little bit about some of the tools that you may have in your continuous integration environment so the cool thing with working with continuous integration tools is that they are all open source at least the ones that we have listed here are open source there are some that are private but typically you'll get started with open source tools and it gives you the opportunity to understand how you can accelerate your environment quickly so bamboo is a continuous integration tool that specifically runs multiple builds in parallel for faster compilation so if you have multiple versions of your software that runs on multiple platforms this is a tool that really allows you to get that up and running super fast so that your teams can actually test how those different builds would work for different environments and this has integration with and maven and other similar tools so one of the tools you're going to need is a tool that allows you to automate the software build test and release process and buildbot is that open source product for you again it's an open source tool so there's no license associated with this so you can actually go in and you can actually get the environment up and running and you can then test for and build your environment and create your releases very quickly so buildbots also written in python and it does support parallel execution jobs across multiple platforms if you're working specifically on java projects that need to be built and test then apache gump is the tool for you it makes all of those projects really easy it makes all the java projects easier for you to be able to test with api level and functionality level testing so one of the popular places to actually store code and create a versioning of your code is github and it's a service that's available on the web just recently acquired by microsoft if you are storing your projects in github then you'll be able to use travis continuous integration or travis ci and it's a tool designed specifically for hosted github projects and so finally we're covering jenkins and jenkins is a central tool for automation for all of your projects now when you're working with jenkins sometimes you'll find there's documentation that refers to a product called hudson hudson is actually the original version of the product that finally became jenkins and it was acquired by oracle when that acquisition happened the team behind um hudson was a little concerned about the direction that oracle may potentially go with hudson and so they created a hard fork of hudson that they renamed jenkins and jenkins has now become that open source project it is one of the most popular and continuously contributed projects that's available as open source so you're always getting new features being added to it it's a tool that really becomes the center for your ci environment so let's jump into some of those really great features are available in jenkins so jenkins itself is really comprised of five key areas around easy installation easy configuration plugins extensibility and distribution so as i mentioned for the easy installation jenkins is a self-contained java program and that allows it to run on most popular operating systems including windows mac os and unix you even run it on linux it really isn't too bad to set up it used to be much harder than it is today the setup process has really improved the web interface makes it really easy for you to check for any errors in addition you have great built-in help one of the things that makes tools like jenkins really powerful for developers and continuous integration teams and your devops teams as a whole when you have plugins that you can then add in to extend the base functionality of the product jenkins has hundreds of plugins and you can go and visit the update center and see which other plugins that would be good for your devops environment certainly check it out there's just lots of stuff out there in addition to the plug-in architecture jenkins is also extremely extensible the opportunity for you to be able to configure jenkins to fit in your environment it's almost endless now it's really important to remember that you are extending jenkins not creating a custom version of jenkins and that's a great differentiation because the core foundation remains as the core jenkins product the extensibility can then be continued with newer releases of jenkins so you're always having the latest version of jenkins and your extensions mature with those core foundation the distribution and the nature of jenkins makes it really easy for you to be able to have it available across your entire network it really will become the center of your ci environment and it's certainly one of the easier tools and more effective tools for devops so let's jump into the standard jenkins pipeline so when you're doing development you start off and you're coding away on your computer the first thing you have to do when you're working in the jenkins pipeline is to actually commit your code now as a developer this is something that you're already doing at least you should be doing you're committing your code to a git server um or to an svn server or a similar type of service so in this instance you'll be using jenkins as the place for you to commit your code jenkins will then create a build of your code and part of that build process is actually going through and running through tests and again as a developer you're already comfortable with running unit tests and writing those tests to validate your code but there may be additional tests that jenkins is running so for instance as a team you may have a standard set of tests for how you actually write out your code so that each team member can understand the code that's been written and those tests can also be included in the testing process within the jenkins environment assuming everything past the the tests you can then get everything placed in a stage and release ready environment within jenkins and finally again ready to deploy or deliver your code to a production environment jenkins is going to be the tool that helps you with your server environment to be able to deploy your code to the production environment and the result is that you're able to move from a developer to production code really quickly this whole process can be automated rather than having to wait for people to actually test your codes or go through a nightly build you're looking at being able to commit your code and go through this testing process and release process continuously as an example companies etsy will release up to 50 different versions of their website every single day so let's talk about the architecture within jenkins that allows you to be so effective at applying a continuous delivery devops environment so the server architecture really is broken up into two sections on the left hand side of section you have the code the developers are doing and submitting that code to a source code repository and then from then jenkins is your continuous integration server and it will then pull any code that's been sent to the source code repository and will run tests against it it will use a build server such as maven to actually then build the code and every single stage that we have that jenkins manages there are constant tests so for instance if a build fails that feedback is sent right back to the developers so that they can then change their code so that the build environment can run effectively the final stage is to actually execute specific test scripts and these test scripts can be written in selenium so it's probably good to mention here that both mavin and selenium are plugins that run in the jenkins environment so before we were talking about how jenkins can be extended with plugins mavin and selenium are just two very popular examples of how you can extend the jenkins environment the goal to go through this whole process again is an automated process is to get your code from the developer to the production server as quickly as possible have it fully tested and have no errors so it's probably important at this point to mention uh one piece of information around the jenkins environment that if you have different code builds that need to be managed and distributed this will require that you need to have multiple builds being managed jenkins itself doesn't allow for multiple files and builds to be executed on a single server you need to have a multiple server environment with running different versions of jenkins for that to be able to happen so let's talk a little bit about the master slave architecture within jenkins so what we have here is an overview of the master slave architecture within jenkins on the left hand side is the remote source code repository and that remote source code repository could be github or it could be team foundation services or the new azure devops code repository or it could be your own git repository the jenkins server acts as the master environment on the left hand side and that master environment can then push out to multiple other jenkins slave environments to distribute the workload so it allows you to run multiple builds and tests and production environments simultaneously across your entire architecture so jenkins slaves can be running the different build versions of the code for different operating systems and the server master is controlling how each of those builds operate so let's step into a quick story of a company that has used jenkins very successfully so here's a use case scenario over the last 10 or 15 years there has been a significant shift within the automotive industry where manufacturers have shifted from creating complex hardware to actually creating software we've seen that with companies such as tesla where they are creating software to manage their cars we see the same thing with companies such as general motors with their onstar program and ford just recently have rebranded themselves as a technology company rather than just a automotive company what this means though is that the software within these cars is becoming more complex and requires more testing to allow more capabilities enhancements to be added to the core software so bosch is a company that specifically ran into this problem and their challenge was that they wanted to be able to streamline the increasingly complex automotive software by adopting continuous integration and continuous delivery best practices with the goal of being able to delight and exceed the customer expectations of the end user so bosch has actually used cloudbees which is the enterprise jenkins environment so to be able to reduce them the number of manual steps such as building deploying and testing bosch has introduced the use of cloud b's from jenkins and this is part of the enterprise jenkins platform has significantly helped improve the efficiencies throughout the whole software development cycle from automation stability and transparency because jenkins becomes a self-auditing environment now the results have been tangible previously it took three days before a build process could be done and now it's taken that same three-day process and reduced it to less than three hours that is significant large-scale deployments are now kept on track and have expert support and there is clear visibility and transparency across the whole operations through using the jenkins tools hello and welcome to continuous integration tutorial using jenkins my name is chidanand and i am with the simply learn team today let's get started with jenkins jenkins in my opinion is one of the most popular continuous integration servers of recent times what began as a hobby project by a developer working for sun microsystems way back in early or mid 2000s has gradually and eventually evolved into a very very powerful and robust automation servers it has a wide adoption since it is released under mit license and is almost free to use jenkins has a vast developer community that supports it by writing all kinds of plugins plugins is the heart and soul of jenkins because using plugins one can connect jenkins to anything and everything under the sun with that introduction let's get into what all will be covered as a part of this tutorial i will get into some of the prerequisites required for installing jenkins post which i will go ahead and install jenkins on a windows box there are few first time configuration that needs to be done and i will be covering those as well so once i have jenkins installed and configured properly i will get into the user administrative part i create few users and i will use some plugins for setting up various kinds of access permissions for these users i will also put in some freestyle jobs freestyle job is nothing but a very very simple job and i will also show you the powerfulness of jenkins by scheduling this particular job to run based upon time schedule i will also connect jenkins with github github is our source code where source code repository where i've got some repositories put up there so using jenkins i will connect to github pull up a repository that is existing on github onto the jenkins box and run few commands to build this particular repository that is pulled from github sending out emails is a very very important configurations of jenkins or any other continuous integration server for that matter whenever there is any notifications that has to be sent out as a part of either a build going bad or built being good or built being propagated to some environment and all these scenarios you would need the contains integration servers to be sending out notifications so i will get into a little bit details of how to configure jenkins for sending out emails i will also get into a scenario where i will have a web application a maven based java web application which will be pulled from a github repository and i will deploy it onto a tomcat server the tomcat server will be locally running on my system eventually i will get into one other very very important topic which is the master slave configuration it's a very very important and pretty interesting topic where distributed builds is achieved using a master slave configuration so i will bring up a slave i will connect the slave with the master and i will also put in a job and kind of delegate that particular job to the slave configuration finally i will let you know how to use some plugins to backup your jenkins so jenkins has got a lot of useful information set up on it in terms of the build environments in terms of workspace all this can be very very easily backed up using a plugin so this is what i'm going to be covering as a part of this tutorial jenkins is a web application that is written in java and there are various ways in which you can use and install jenkins i have listed popular three mechanisms in which jenkins is usually installed on in any system the topmost one is as a windows or a linux based services so if at all you have windows like the way i have and i'm going to use this mechanism for this demo so i would download a msi installer that is specific to jenkins and install this service so whenever i install it as a service it goes ahead and nicely installs all that is required for my jenkins and i have a service that can be started or stopped based upon my need any flavor of linux as well one other way of running jenkins is downloading this generic warf file and as long as you have jdk installed you can launch this war file by the command opening up a command prompt or shell prompt if it layer on linux box specifying java hyphen jar and the name of this war file it typically brings up your web application and you know you can continue with your installation the only thing being if at all you want to stop using jenkins you just go ahead and close this prompt you either do a control c and then bring down this prompt and your jenkins server would be down other older versions of jenkins were run popularly using this way in which you already have a java based web server running up and running so you kind of drop in this war file into the root folder or the httpd root folder of your web server so jenkins would explode and kind of bring up your application all user credentials or user administration is all taken care of by the apache or the tomcat server or the web server on which jenkins is running this was a very older way of running but still some people use it because if they don't want to maintain two servers if they already have a java web server which it's being nicely maintained and backed up jenkins can run attached to it all right so either ways it doesn't matter however you're going to bring up your jenkins instance the way we're going to operate jenkins is all going to be very very same or similar one with subtle changes in terms of user administration if at all you're launching it through any other web server which will take care of the user administration otherwise all the commands or all the configuration all the way in which i'm going to run this demo it is going to be same across any of these installations all right so the prerequisites for running jenkins as i mentioned earlier jenkins is nothing but a simple web application that is written in java so all that it needs is java preferably jdk 1.7 or 1.8 2gb ram is the recommended ram for running jenkins and also like any other open source tool sets when you install jdk ensure that you set in the environment variable java home to point to the right directory this is something very specific to jdk but for any other open source tools that you've installed there's always a preferred environment variable that you got to set in which is specific to that particular tool that you're going to use this is a generic thing that is there for you know for any other open source projects because the way open source projects discover themselves is using this environment variables so as a general practice or a good practice always set these environment variables accordingly so i already have jdk 1.8 installed on my system but in case you do not what i would recommend is just navigate on your browser to the oracle homepage and just type in or search for install jdk 1.8 and navigate to the oracle home page you will have to accept the license agreement and there are a bunch of installers that is okay that you can pick up based upon the operating system on which you're running so i have this windows 64 installer that is already installed and running on my system so i will not get into the details of downloading this or installing it let me show you once i install this what i've done with regard to my path so if you get into this environment variables all right so i have just set in a java home variable if you see this c colon program files java jdk 1.8 this is where my my java is located c program files c program files java okay so this is the home directory of my jdk so that is what i have been i've set it up here in my environment variable so if you see here this is my java home all right one other thing to do is ensure that in case you want to run java or java c from a command prompt ensure that you also add that path into this path variable so if you see this somewhere i will see yes there you go c colon program files java jdk 1.8 pin so with these two i'll ensure that my java installation is nice and you know good enough so to check that to double check that or to verify that let me just open up a simple command prompt and if i type in java iphone version all right and java c iphone version so the compiler is on the path java is on the path and if you're told i do this even the environment variable specific to my java is installed correctly so i'm good to go ahead with my jenkins installation now that i have my prerequisites all set for installing jenkins let me just go ahead and download jenkins so let me open up a browser and say download jenkins all right lts is nothing but the long term support these are all stable versions weeklies i would not recommend that you try these unless until you have a real need for that long term support is good enough and as i mentioned there are so many flavors of jenkins that is available for download you also have a docker container wherein you know you can launch jenkins as a container but i'll not get into details of that in this all right so what i want is yes this is the war file which is a generic war file that i was talking to you earlier and this is the windows msi installer so go ahead and download this msi installer i already have that downloaded so let me just open that up all right so this is my downloaded jenkins instance or other installer this is a pretty maybe a few months old but this is good enough for me before you start jenkins installation just be aware of one fact that there is a variable called jenkins home this is where jenkins would store all this configuration data jobs project workspace and all that specific to jenkins so by default if at all you don't set this to any particular directory if at all you install an msi installer all your installation gets into c colon program files 86 and jenkins folder if at all you run a war file depending upon the user id with which you're running a war file the jenkins folder there's a dot jenkins folder that gets created inside the user home directory so in case you have any need wherein you want to backup your jenkins or you want jenkins installations to get into some specific directories go ahead and set this jenkins home variable accordingly before you even begin your installation for now i don't need to do any of these things so i've already downloaded the installer let me just go ahead with the default installation all right so this is my jenkins msi installer i would just i don't want to make any changes into the jenkins configuration c colon program files is good for me yeah this is where all my destination folder and all the configuration specific to it goes i'm happy with this i don't want to change this i would just say go ahead and click installation okay so what typically happens once the jenkins installation gets through is it'll start installing itself and there are some small checks that needs to be done so and by default jenkins launches on the port 8080 so let me just open up localhost 8080. there's a small checking that will be done as a part of the installation process wherein i need to type in the hash key all right so there's a very very simple hash key that gets stored out here so i'll have to just copy this path if at all you're running as a war file you would see that in your logs all right so this is a simple hash key that gets created every time when you do a jenkins station so as a part of the installation it just asks you to do this so if that is not great it will about it but this looks good so it's going ahead all right one important part during the installation so you would need to install some recommended plugins what happens is the plugins are all related to each other so it's like the typical rpm kind of a problem where you try to install some plugin and it's got a dependency which is not installed and you get into all those issues in order to get rid of that what jenkins recommends there's a bunch of plugins that is already recommended so just go ahead and blindly click that install recommended plugin so if you see there is a whole lot of plugins which are bare essential plugins that is required for jenkins in order to run properly so jenkins as a part of the installation would get all these plugins and then install it for you this is a good combination to kind of begin with and mind you at this moment jenkins needs lots of bandwidth in terms of network so in case your you know your network is not so good few of these plugins would kind of fail and these plugins are all you know on available on openly or mirrored sites and sometimes some of them may be down so do not worry in case some of these plugins kind of fail to install you'll get an option to kind of retry installing them but just ensure that you know at least most or 1995 of all these plugins are installed without any problems let me pause the video here for a minute and then get back once all these plugins are installed my plugin installation is all good there was no failures in any of my plugins so after that i get to create this first admin user again this is one important point that you got to remember keen given any username and password but ensure that you kind of remember that because it's very hard to get back your username and password in case you forget it all right so i'm going to create a very very simple username and password something that i can remember i will that's my name and an email id is kind of optional but it doesn't allow me to go ahead in case i don't so i just given an admin and i got a password i've got i remember my password this is my full name all right i say save and finish all right that kind of completed my jenkins installation it was not that tough was it now that i have my jenkins installed correctly let me quickly walk you through some bare minimal configurations that is required these are kind of a first time configuration that is required so and also let me warn you the ui is little hard for many people to wrap their head around it specifically the windows guys but if at all you're a java guy you know how painful it is to write ui in java you will kind of appreciate you know all the effort that has gone into the ui bottom line you are a little hard to you know wrap your head around it but once you start using it possibly you'll start liking it all right so let me get into something called as managed in cans this can be viewed like a main menu for all jenkins configuration so i'll get into some of those important ones something called as configure system configure system this is where you kind of put in the configuration for your complete jenkins instance few things to kind of look out for this is a home directory this is a java home where all the configurations all the workspace anything and everything regarding jenkins is stored out here system message you want to put in some message on the system you just type in whatever you want and is possibly show up somewhere up here on the menu number of executors very very important configuration this just lets jenkin know at any point in time how many jobs or how many threads can be run you can kind of visualize it like a thread that can be run on this particular instance as a thumb rule if at all you're on a single core system number of executors two should be good enough in case at any point in time if there are multiple jobs that kind of get triggered the same time in case the number of executors are less compared to the number of jobs that have that woken up no need to panic because they will all get queued up and eventually jenkins will get to running those jobs just bear in mind that whenever a new job kind of you know gets triggered the cpu usage and the memory usage in terms of the disk right is very high on the jenkins instance so that's something that you got to kind of keep in mind all right but number of executors 2 for my system is kind of good label for my jenkins i don't want any of these things usage how do you want to use your jenkins this is good for me because i only have a primary uh server that is running so i want to use this node as much as possible quite pretty each of these options have got some pair minimal help kind of a thing that is that is out here by clicking on these question marks you will get to know as to what are these particular configurations all right so this all look good what i want to show you here is there's something regarding the docker timestamps git plugin svn email notifications i don't want that what i want the yes i want this smtp server configuration remember i mentioned earlier that i would want jenkins to be sending out some emails and what i've done here is i've just configured the smtp details of my personal email id in case you are in a in organization you would have some sort of an email id that is set up for a jenkins server so you can specify the smtp server details of your company so that you know you can authorize jenkins to kind of send out emails but in case you want to try it out like me i have configured my personal email id which is on my gmail for sending out notifications so the smtp server would be smtp.gmail.com i'm using the smtp authentication i have provided my email id and my password i'm using the smtp port which is 465 and i'm you know reply to address is the same as mine i can just send out an email and see if at all this configuration works again gmail would not allow you to allow anybody to send out notifications on your behalf so you'll have to lower the security level of your gmail id so that you can allow a programmatically somebody to send out email notifications on your behalf so i've done already that i'm just trying to see if i can send a test email with the configurations that i've set in yes all right so the email configuration looks good so this is how you configure your you know your gmail account in case you want to do that if not put in your organization smtp server details which are for the valid username and password and it should all be set all right so no other configurations that i'm going to change here all of these look good all right so i come back to managing kens okay one other thing that i want to kind of go over is the global tool configuration imagine this scenario look at it this way jenkins is a is a continuous integration server it doesn't know what kind of a code base it's going to pull in what kind of a tool set that is required or what is the code that is going to pull in and how is it going to build so you would have to put in all the tools that is required for building the appropriate kind of code that you're going to pull in from you know your source code repositories so just to give you an example in case your source code is a java source code and assuming that you know because in this demo this is my laptop and i've put in all the configurations jdk everything on my laptop because i'm a developer i am working on the laptop but my continuous integration server would be you know a separate server without anything being installed on it so in case i want jenkins to you know run a java code i would need to install jdk on it i need to specify the jdk location of this out here this way okay since i already have the jdk installed and i've already put in the java home directory or rather the environment variable correctly i don't need to do it get if at all i want the jenkins server to use git git is a you know command bash or the command prompt for for running git and connecting to any other git server so you would need git to be you know installed on that particular system and set the path accordingly gradle and maven if at all you have some mavens as well you want to do this any other tool that you're going to install on your system which is your continuous integration server you will have to come in here and configure something in case you don't configure it when jenkins runs it will not be able to find these tools for building your task and it will crib about it that's good i don't want to save anything managing let me see what else is required yes configure global security all right the security is enabled and if you see by default it's the security access control is set to jenkins own user database so what does this mean you know jenkins by default it uses file system where it stores all the user names which hashes up these usernames and kind of stores them so as of now it jenkins is configured to use its own database assuming that you are running in an organization you would probably want to have a you know some sort of an ad or an ldap server using which you would want to control access to your jenkins repository rather jenkins tool so you would specify your ldap server details the root dn password or the manager dn and the manager password and all these details in case you want to connect your jenkins instance with your ldap or ad or any of the authentication servers that you have in your organization but for now since i don't have any of these things i'm going to use this own database that's good enough all right so i will set up some authorization methods and stuff like that once i put in few jobs so for now let me not get into any of these details of this just be aware that jenkins can be connected for authorization to an ldap server or you can have jenkins managing its own servers which is happening as of now so i'm going to save all this stuff that's good for me so enough of all these configurations let me put in a very very simple job all right so job new item you know a little difficult to kind of figure out but then that's the new item so i'll just say you know first job this is good for me i just give a name for my job i would say it's a freestyle project that's good enough for me i don't want to choose any of that so unless until you choose any of this this particular button would not become active so choose the freestyle project and say okay at a very high level you would see general source code management build triggers build environment build and post build in case you install more and more plugins you will see a lot more options but for now this is what you would see so what am i doing at the moment i'm just putting up a very very simple job and the job could be anything and everything so i don't want to put in a very complicated job for now for the demo let me just put in a very very simple job i'll give a description this is an optional thing this is my first jenkins job all right i don't want to choose any of these again there are some helps available here i don't want to choose any of this i don't want to connect it into any source code for now i don't want any triggers for now i'll come back to this in a while build the environment i don't want any build environment as a part of this build step you know i just want to you know run few things so that i kind of complete this particular job so since i'm on a windows box i would say execute windows batch command all right so what do you want to do i would let me just echo something echo uh hello this is my first jenkins job and possibly i would want the date and the time stamp pertaining to the job i mean the date and time in which this job was run all right very very simple command that says you know this is my first job it just puts out something along with the date and the time all right i don't want to do anything else i want to keep this job as simple as this so let me save this job all right so once you save this job you know the job names comes up here and then i need to build this job and you would see some building stray out here nothing is there as of now because i've just put in a job i've not run it yet all right so let me try to build it now you see a build number you will see a date and a timestamp so if i click on this you will see a console output if i go here okay as simple as that and where is all the job details that is getting into if you see this if i navigate to this particular directory all right so this is the directory what i was mentioning earlier regarding jenkins home so all the job related stuff that is specific to this particular jenkins installation is all here all the plugins that is installed the details of each of those plugins can be found here all right so the workspace is where all the jobs that i've created whichever i'm running you know there will be individual folders specific to the jobs that has been put up here all right so one job one quick run that's what it looks like pretty simple okay let me do one thing let me put up a second job i would say second job i would say freestyle project all right this is my second job i just want to demonstrate the powerfulness of the automation server and how simple it is to automate a job that is put up on jenkins which will be triggered automatically remember what i said earlier about jenkins because at the core of jenkins is a very very powerful automation server all right so what i'm going to do i will just keep everything else the same i'm going to put in a build script pretty much similar to second job that gets triggered automatically every minute all right let me do that percentage date and i'll put in the time all right so i just put in another job called second job and it pretty much does the same thing as what i was doing earlier in terms of printing the date and the time but this time i'm just going to demonstrate the powerfulness of the automation server that is there if you see here there's a build trigger so a build can be triggered using various triggers that is there so we'll get into this github triggering or hook or a web hook kind of a triggering later on but for now what i want to do i want to ensure that this job that i'm going to put in would be automatically triggered on its own let's say every minute i want this job to be run on its own so build periodically is my setting if you see here there's a bunch of help that is available for me so for those of you who have written cron jobs on linux boxes you'll find it very very simple but for others don't panic let me just put up a very very simple regular expression for scheduling this job every minute all right so that's one two three four five all right come up come up come up all right so five stars is all that i'm going to put in and jenkins got a little worried and is asking me do you really mean every minute oh yeah i want to do this every minute let me save this and how do i check whether it gets triggered every minute or not i just don't do anything i'll just wait for a minute and if at all everything goes well jenkins would automatically trigger my second job in a minutes time from now this time around i'm not going to trigger anything look there you see it's automatically got triggered if i go in here yep second job that gets triggered automatically you know it was triggered at 42 1642 which is 442 my time that looks good and if everything goes well every one minute onwards this jog would be automatically triggered now that i have my jenkins up and running a few jobs that has been put up here on my jenkins instance i would need a way of controlling access to my jenkins server this is wherein i would use a plugin called role based access plugin and create few rules the rules are something like a global rule and a project rule project specific growth i can have different roles and i can have users who have signed up or the users who might create kind of assigned to these roles so that each of these users fall into some category this is my way of kind of controlling access to my jenkins instance and ensuring that people don't do something unwarranted all right so first things first let me go ahead and install a plugin for doing that so i get into manage jenkins and manage plugin a little bit of a confusing screen in my opinion there's updates available installed in advance as of now we don't have the role-based plugin so let me go to available it'll take some time for it to get refreshed all right now these are the available plugins these are the install plugins all right so let me come back to available and i would want to search for my role based access plugin so i would just search for role and hit enter okay role-based authorization strategy enables user authorization using a role-based strategy roles can be defined globally or for particular jobs or nodes and stuff like that so exactly this is the plugin that i want i would want to install it without a restart all right looks good so far yes go back to the top of the page yes remember jenkins is running on a java using a java instance so typically many things would work the same way unless and until you want to restart jenkins once in a while but as a good practice whenever you do some sort of a big installations or big patches on your jenkins instance just ensure that you kind of restart it otherwise there would be a difference in terms of what is installed on the system and what is there on the file system you will need to flush out few of those settings later on but for now these are all very small plugins so this would run without any problems but otherwise if at all there are some plugins which would need a restart you know kindly go ahead and restart your jenkins instance but for now i don't need that it looks good i've installed the plugin so where do i see my plugin i installed the plugin that is specific to the user control or the access control so let me go into yes global security and i would see this role-based strategy showing up now all right so this comes in because of my installation of my role based plugin so this is what i would want to enable because i already have my own database setup and for the authorization part in the sense that who can do what i'm going to install i mean i've already installed a role based strategy plugin and i'm going to enable that strategy all right i would say save okay now i've installed the role based access plugin i would need to just set it up and check that you know i would go ahead and create some rules and ensure that i assign users as per these roles all right so let me go to manage jenkins configure all right let me see where is this configure configure global security is that where i create my roles nope not here yes manage and assign roles okay again you would see these options only after you install these plugins so for now i've just enabled the plugin i have enabled role-based access control and i would go ahead and create some rules for this particular jenkins instance so i would say first manage rules so i would need to create some roles here and the rules are at a very high level these are global rules and there are some project rules and there are some slave rules i'll not get into details of all of this at a very very high level which is a global role let me just create a role a role can be kind of visualized like a group so i would create a role called developer typically the jenkins instance or the cia instance are kind of owned up or controlled by qa guys so qa guys would need to provide some sort of you know limited access to developers so that's why i'm creating a role called developer and i'm adding this role at a global role level so i would say add this here and you would see this developer role that is there and each of these options if we hover over it you would see some sort of a help on what what are these you know permissions specific to so what i want is like you know it sounds a little you know different but i would want to give very very little permissions for the developer so from an administration perspective i would just want him to have a read um kind of a role credentials again i would just want to view kind of a rule i don't want him to create any agents and all that stuff that looks good for me for a job i would want him to just possibly read i don't want him to build i don't want him to cancel any jobs i don't want him to configure any job i don't even want him to create any job i would just want him to read through things i would not give him possibly a role to the workspace as well i mean i don't want him to have access to the workspace i would just want him to read a job now check you know i have read only access to the job run um no i don't want him to give him any any particular access which will allow him to run any jobs view configure yeah possibly create yeah delete i don't want to read yes definitely and this is the specific role so what i'm doing i'm just creating a global role called developer and i'm giving him very very limited roles in the sense that i don't want this developer to be able to run any agents nor create jobs or build jobs or cancel jobs or configure jobs at the max i would just want him to read a job that is already put up there okay so i would save now i created a rule i still don't have any users that is there on the system so let me go ahead and create some user on the system that's not here house configure manage and cans manage users okay let me create a new user i would call this user as yeah developer one sounds good some password some password that i can remember okay his name is developer1 d at t com or something like that okay so this is the admin with with which i kind of configured or brought up the system and developer one is a user that i have configured so still have not set any rules for this particular user yet so i would go to manage and gains i would say manage and assign roles i would say assign roles okay so if you see what i'm going to do now is assign a role that is specific to that particular i will find the particular user and assign him the developer role that i have already configured the role shows up here i would need to find my user whoever created and then assign him to that particular role so if you remember the user that i created was developer one i would add this particular user and now this particular user what kind of a role i want him to have because this is the global rule that had created so developer i would assign this developer 1 to this particular global rule and i would go ahead and save my changes now let me check the permissions of this particular user by logging out of my admin account and logging back as developer one if you remember this role was created with very less privileges so there you go i have jenkins but i don't see a new item i can't trigger a new job i can't do anything i see these jobs however i don't think so i'll be able to start this job i don't have the permission set for that the maximum i can do is look at the job see what was there as a part of the console output and stuff like that so this is a limited role that was created and i added this developer to that particular role which was a developer role so that the developers don't get to configure any of the jobs because the jenkins instance is owned by a qa person he doesn't want to give developer any administrative rights so the rights that he set out by creating a developer role and anybody who is tagged any user who's tagged as a part of this developer role would get the same kind of permissions and these permissions can be you know fine grain it can be a project specific permissions as well but for now i just demonstrated the high level permission that i had set in let me quickly log out of this user and get back as the admin user because i need to continue with my demo with the developer role that was created i have very very less privileges one of the reasons for jenkins being so popular as i mentioned earlier is the bunch of plugins that is provided by users or community users who don't charge any money for these plugins but it's got plugins for connecting anything and everything so if you can navigate to or if you can find jenkins plugins you would see index of over so many plugins that is there all of these are wonderful plugins whatever connectors that you would need if you want to connect jenkins to an aws instance or you want to connect jenkins to a docker instance or any of those containers you would have a plugin you can go and search up if i want to connect jenkins to bitbucket bitbucket is one of the git servers there's so many plugins that is available okay so bottom line jenkins without plugins is nothing so plugins is the heart of jenkins for you to connect or for in order to connect jenkins with any of the containers or any of the other tool sets you would need the plugins if you want to connect or you want to build a repository which has got java and maven you would need to install maven and jdk on your jenkins instance if at all you are looking for a net build or a microsoft build you would need to have ms build installed on your on your jenkins instance and the plugins that will trigger ms build if at all you want to listen to some server side web hooks from github you will need github specific plugins if you want to connect jenkins to aws you need those plugins if you want to connect to a docker instance that is running anywhere in the world as long as you have the url which is publicly reachable you just have a docker plugin that is installed on your jenkins instance sonar cube is one of the popular static code analyzers so you can connect jenkins build you can build a job on jenkins and push it to sonar cube and get sonar cube to run analysis on that and get back the results in jenkins all of these works very well because of the plugins now with that let me connect our jenkins instance to github i already have very very simple java repository up on my github instance so let me connect jenkins to this particular github instance and pull out a job that is put up there all right so this is my very very simple uh you know repository that is there called hello java and this is what is there in the repositories hello hello.java application that is here or a simple class file that is there it's got just one line of system dot out so this is already present on github.com at this place and this would be the url for this repository if i pick up the https url this is my steepest url so what i would do is i would connect my jenkins instance to go to github provide my credentials and pull out this repository which is on the cloudhostedgithub.com and get it to my jenkins instance and then build this particular java file i'm keeping the source code very very simple it's just a java file how do i build my java file how do i compile my java file i just say java c and the name of my class file which is java and however run my java file i would say java and hello okay so remember i don't need to install any plugins now because what it needs is a git plugin so if you remember when we were doing the installation there was a bunch of recommended plugins so git is already installed on my system so i don't need to install it again so let me put up a new job here it says git job it will be a freestyle project that's good for me i would say okay all right so the source code management remember in the earlier examples we did not use any source code because we were just putting up some echo kind of uh jobs we did not need any integration with any of the source code systems so now let me connect this so i'm going to put up a source code and git would show up because the plugin is already there svn perforce any of those additional source code management tools if at all you would need just install those plugins and jenkins connects wonderfully well to all these particular source control tools okay so i would copy the https url from here i would say this is the url that i'm supposed to go and grab my source code from but all right that sounds good but what is the username and password so i'll have to specify a username and password all right so i would say the username this is my username and this is my https credential for my job okay so this is my username and this is my password i just save this i say add and then i would say you know use this credentials to go to github and then on my behalf pull out a repository all right if at all at this stage if there's any error in terms of not able to jenkins not able to find git or the git dot exe or if my credentials are wrong somewhere down here you would see a red message saying that you know something is not right you can just go ahead and kind of fix that for now this looks good for me i'm going to grab this url what am i going to do this step would pull the source code from the github and then what would be there as a part of my build step because this repository just has a java file correct hello.java so in order to for me to build this i would just say execute windows batch command and i would say java c hello dot java that is the way i would build my java code and if i have to run it i would just say java hello pretty simple two steps and this would run after the repository contents are fetched from github so java c java that sounds good i would say save this and let me try to run this okay if you see there's a lot of you know it executes git on your behalf it goes out here it provides my credentials and says you know it pulls all my repository and by default it will pull up the master branch that is there on my repository and it kind of builds this whole thing java c hello java and it runs this project java hello and there you see this is the output that is there and if at all you want to look at the contents of the repository if you can go here this is my workspace of my system hang on this is not right okay good job if you see here this is my hello.java this is the same program that was there on my github repository okay so this is a program that was there on the github repository all right so this was the same program that was here and jenkins on our behalf went over all the way to github pulled this repository from there and then you know it brought it down to my local system on my jenkins instance it compiled it and it ran this particular application okay now that i've integrated jenkins successfully with github for a simple java application let me build a little bit on top of it what i will do is i have a maven based web application that is up there as a repository in my github so this is the repository that i'm talking about it's called amv and web app it's got it's a maven based repository as you would know maven is a very very simple java based uh build tool that will allow you to run various targets and it will compile it will based upon the goals that you specify it can compile it can run some tests and you can you can build a war file and even deploy it into some other server for now what we're going to use maven is just for building and creating a package out of this particular web application it contains a bunch of things and what is important is just the index.jsp it just contains an html file that is there as a part of this web application so from a perspective of requirements now since i'm going to connect jenkins with this particular repository git we already have that set we only need two other things one is maven because jenkins will use maven so in order to use maven jenkins would have to have a maven installation that is there on the jenkins box and in this case the jenkins box is this laptop and after i have my maven installed i also need a tomcat server tomcat is a very very simple uh web server that you can freely download i'll let you know how to quickly uh download and install the tomcat all right so download maven first and the various ways in which you can kind of download this maven there is zip files binary zip files and archive files so what i've done is i've just already downloaded maven and if you see i've unzipped it here so this is the folder with which i have unzipped my maven so as you know maven again is is one open source build tool so you'll have to set in a few configurations and set up the path so mvn hyphen iphone version if i specify this after i set in my path my one should work and if at all i echo m2 home which is nothing but the variable environment variable specific to my onenote it is already set here so once you unzip mav1 just set this m2 home variable to the directory where you unzip your i1 also just set the path to this particular directory slash bin because that is where your maven executables are all found all right so that's with maven and you know since i've set the path and the environment variable maven is running perfectly fine on my system i just verified it okay next one is a tomcat server download apache tomcat server 8.5 is what i have on my system so i'm just going to show you where to download this from this is where you download tomcat server and um i already have the server downloaded again this doesn't need any installation i just unzip it here and it kind of has a bin and configuration i have made some subtle changes in the configuration first and foremost tomcat server also by default runs on port 8080 since we already have our jenkins server running on port 8080 we cannot let tomcat run on the same port there will be a port clash so what i've done i have configured tomcat to use a different port so if i go to this configuration file here there is a server.xml let me open this up here all right okay so this is the port by default it will be 8080 i've just modified it to 8081. so i've changed the port on which my tomcat server would run all right so that's one chain second change when jenkins kind of tries to get into my tomcat and deploy something for someone he would need some authentications so that he'll be alloyed deployment by tomcat so for that i need to create a user on tomcat and provide this user credentials to my jenkins instance so i would go to tomcatusers.xml file here i've already created a username called deployer and the password is deployer and i've added a role called manager hyphen script manager hyphen script will allow programmatic access to the tomcat server so this is the role that is there so using this credentials i will enable or i'll empower jenkins to get into my tomcat server and deploy my application all right only these two things that is required let me just start my tomcat server first so i get into my bin folder i open a command prompt here and there's a startup.bat it's pretty fast it just takes a few seconds yes there you go tomcat server is up and running now this is running on port 8081 so let me just check if that looks good so localhost 8081 okay my tomcat server is operating that sounds good the user is already configured on this that's also fine so what i'll do as a part of my first job my one is also installed on my system so i'm good to use maven as a part of my jenkins so i will put up a simple job now i will say job mbn web app i call this freestyle job that's good okay so this will be a git repository what is the url of my git repository is this guy https url okay that's this url i will use the credentials the old credentials that are set up will work well because it's the same git user that i'm kind of connecting into all right so now the change happens here where after i get this since i said this is a simple maven repository i will have some maven targets to run so the simple target first is let me run maven package this creates a war file okay so mvn package is the target package is the target so when whenever i run this package it kind of creates it it builds it it tests it and then creates a package so this is all that is required maybe let me try to save this and uh let me first run this and see if it connects well if there's any problem with my war file or the war file gets created properly okay wonderful so it built a war file and if you see it all shows you what is the location where this war file was generated so this will be the workspace if you see this this war file was successfully built now i need to grab this particular war file and then i would need to deploy it into tomcat server again i would need a small plugin to do this because i need to connect tomcat with my jenkins server let me go ahead and install the plugin for the container deployment so i will go to manage plugins available type in container container container deploy to container okay so let's put this the plugin that i would need i will install it without a restart right seems to be very fast nope sorry it's still installing okay it installed the plugin so if at all you see this if you go to my workspace okay in the target folder i would see this web application war file that is already built so i would need to configure this plugin to pull up this war file and deploy it onto the tomcat server for deploying onto the tomcat server i will use the credentials of the user that i've created okay so let me go to configure this particular project again and um okay all this is good so the package is good i'm gonna just create a package that's all fine now add a post build step so after the war file is built as a part of this package directive let me use this deployment to container now this will show up after you install the plugin so deploy this one to the container now what is that you're supposed to specify you're supposed to specify the what is the location okay so this is a global uh you know configuration that is there that will allow you to from the root folder it will pick up the war file that is there so star star forward slash star dot war that's good for me okay what is the context path context path is nothing but just the name of an application that you know under which it will get deployed into the tomcat server i will just say mvn web app that's the name of my thing now i need to specify what kind of a container that i'm talking about all right so the deployment would be for this tomcat 8.5 is what i need okay because the server that we have is a tomcat 8.5 server that i have so this would be the url so the credentials yes i need to add a credential for this particular server so if you remember i had created a credential for my web application so let me just find that my tomcat server yes configuration of this okay so deployer and deployer username is deployer password is deployer okay so let me use that credential i would say i would say add a new credential jenkins credential the username is deployer and the password is deployer so i would use this deployer credentials for that and what is the url of my tomcat instance so this is the url of my tomcat instance so take the war file that is fine found in this particular folder and then you know context path is mn web use the deployer deployment credentials and get into this local host which is there 8081 this is the tomcat server that is running on my system and then go ahead and deploy it okay so that is all that is required so i would say just save this and let me run it now okay it builds successfully built the war file it is trying to deploy it and looks like the deployment went ahead perfectly well so the context path was mvn web app so if i type in this all right if at all i go ahead into my tomcat server there would be a web apps folder you would see the you know the date timestamp so this is the file that get got recently copied and this is the explorer version of our application so the application was built the source code of this application was pulled from the github server it was built locally on the jenkins instance and then it was pushed into a tomcat server which is running on a different port which is 808.1 now for this demo i'm running everything locally on my system but assuming that you know this particular tomcat instance was running on some other server with some other different ip address all that you've got to go and change is the url of the server so this would be the server in case you you already have that uh you know if you have a tomcat which is running on some other machine that's all fine with a different ipa that's all good enough the whole bundle or the war file that was built as a part of this jenkins job gets transferred onto the other server and gets deployed that's the beauty of jenkins and automatic deployments or other deployments using jenkins and maven the four components are as follows the docker client and server which we've already done a deeper dive on docker images docker containers and the docker registry so if we look at the structure that we have here on the left hand side you see the relationship between the docker client and the docker server and then we have the rest api in between now if we start digging into that rest api particularly the relationship with the docker daemon on the server we actually have our other elements that form the different components of the docker ecosystem so the docker client is accessed from your terminal window so if you are using windows this can be powershell on mac it's going to be your terminal window and it allows you to run the docker daemon and the registry service when you have your terminal window open so you can actually use your terminal window to create instructions on how to build and run your docker images and containers if we look at the images part of our registry here we actually see that the image is really just a template with the instructions used for creating the containers which you use within docker the docker image is built using a file called the docker file and then once you've created that docker file you'll store that image in the docker hub or registry and that allows other people to be able to access the same structure of a docker environment that you've created the syntax of creating the image is fairly simple it's something that you'll be able to get your arms around very quickly and essentially what you're doing is you're creating the option of a new container you're identifying what the image will look like what are the commands that are needed and the arguments for within those commands and once you've done that you have a definition for what your image will look like so if we look here at what the container itself looks like is that the container is a standalone executable package which includes applications and their dependencies it's the instructions for what your environment will look like so you can be consistent in how that environment is shared between multiple developers testing units and other people within your devops team now the thing that's great about working with docker is that it's so lightweight that you can actually run multiple docker containers in the same infrastructure and share the same operating system this is its strength it allows you to be able to create those multiple environments that you need for multiple projects that you're working on interestingly though within each container that container creates an isolated area for the applications to run so while you can run multiple containers in an infrastructure each of those containers are completely isolated they're protected so that you can actually control how your solutions work there now as a team you may start off with one or two developers on your team but when a project starts becoming more important and you start adding in more people to your team you may have 15 people that are offshore you may have 10 people that are local you may have 15 consultants that are working on your project you have a need for each of those developers or each person on your team to have access to that docker image and to get access to that image we use the docker registry which is an open source server-side service for hosting and distributing the images that you have defined you can also use docker itself as its own default registry and docker hub now something it has to be very mind though is that for publicly shared images you may want to have your own private images in which case you would do that through your own registry so once again public repositories can be used to host the docker images which can be accessed by anyone and i really encourage you to go out to docker and see the other docket images that have been created because there may be tools there that you can use to speed up your own development environments now you will also get to a point where you start creating environments that are very specific to the solutions that you are building and when you get to that point you'll likely want to create a private repository so you're not sharing that knowledge with the world in general now the way in which you connect with the docker registry is through simple pull and push commands that you run through terminal window to be able to get the latest information so if you want to be able to build your own container what you'll start doing is using the pull command to actually pull the image from the docker repository and the command line for that is fairly simple in terminal window you would write docker pull and then you put in the image name and any tags associated with that image and use the command pulls so in your terminal window you would actually use a simple line of command once you've actually connected to your docker environment and that command will be docker pull with the image name and any associated tags around that image what that will then do is pull the image from the docker repository whether that's a public repository or a private one now in reverse if you want to be able to update the docker image with a new information you do a push command where you take the script that you've written about the docker container that you defined and push it to the repository and as you can imagine the commands for that are also fairly simple in terminal window you would write docker push the image name any associated tags and then that would then push that image to the docker repository again either a public or a private repository so if we recap the docker file creates a docker image that's using the build commands a docker image then contains all the information necessary for you to be able to execute the project using the docker image any user can run the code in order to create a docker container and once a docker image is built it's uploaded to a registry or to a docker hub where it can be shared across your entire team and from the docker hub users can get access to the docker image and build their own new containers welcome to all our viewers i'm anjali from simply learn and today i'll be taking you through the installation process for docker so before we begin let me give you an intro to docker so what is docker docker is a tool which is used to automate the deployment of applications in lightweight containers so when i say containers i mean a software package that consists of all the dependencies required to run the particular application and when you deploy containers you're basically providing the capability for the application to run in any kind of environment so dockers have multiple features some of which are that multiple containers can run on the same hardware it has very high productivity when compared to virtual machines it maintains isolated applications and has a very quick and easy configuration process so now let's begin with the demo we'll be installing docker on an ubuntu system so this is my system i'll just open the terminal so the first thing you can start with is removing any docker installation that you probably already have present in your system if you want to start from scratch so this is the command to do so sudo apt get remove docker docker engine docker dot io enter your password and docker is removed so now we'll start from scratch and we'll install docker once again before that i'll just clear my screen okay so before i install docker let me just ensure that all these software is on my system currently is in its latest state so sudo apt get update great so that's done next thing we'll actually install our docker so type in sudo apt get install docker now as you can see here there's an error that's occurred so sometimes it's possible that due to the environment of the machine that you're working in this particular command does not work in which case there's always another command that you can start with just type docker install and that by itself will give you the command you can use to install docker so as it says here sudo apt install docker dot io is a command that we will need to execute to install docker and after that we'll execute the suros snap install docker so sudo apt install docker dot io first and this will install your docker after that's done we will have sudo snap install docker so snap install docker installs a newly created snap package there are basically some other dependencies for docker that you'll have to install of course since this is the installation process for the entire docker io it will take some time great so our docker is installed the next thing we do as i mentioned earlier is that we need to install all the dependency packages so the command for that is sudo snap install docker enter your password so with that we have completed the installation process for docker but we'll perform a few more stages where we will test if the installation has been done right so before we move on with the testing for docker let's once again just check the version that we have installed so for that the command is docker version and as you can see docker version 17.12.1 ce has been installed next thing we do is we pull an image from the docker hub so docker run hello world now hello world is a docker image which is present on the docker hub docker hub is basically a repository that you can find online so with this command the docker image hello world has been pulled onto your system so let's see if it's actually present on your system now the command to check this is sudo docker images and as you can see here hello world repository this is present on our system currently so the image has been successfully pulled onto the system and this means that our docker is working now we'll try out another command sudo docker ps minus a this displays all the containers that you have pulled so far so as you can see here there are three hello world images displayed and all of them are in exited state so i did this demo previously too which is why the two hello worlds which is created two minutes ago is also displayed here and the first hello world which has been created a minute ago is the one we just did for this demo now as you have probably noticed that all the hello world images over here all these containers are in the exited state so when you give the option for docker ps minus a where minus a stands for all it displays all the containers whether they are in exited or running state if you want to see only those containers which are in their running state you can simply execute sudo docker ps suro docker yes and as you can see no container is visible here because none of them are in running state and with that we come to an end of our docker installation welcome everyone to our one another demo on which we are going to see that how exactly we can do the installation of docker on the windows platform specifically on windows 10. now docker is something which is available for most of the operating systems different different platforms so it supports both the unix and the windows platform as such so um linux through various commands we can do the installation but in the case of windows you have to download the exe file and a particular installer from the docker hub websites you can simply google it and you will get a kind of a link from where you will be able to download the package so let's go to the chrome and try to search on for the windows string uh particular installer you will get a link from docker hub you download it you get the stable version you get the edge version whichever version you want you wish to download you can download it so let's go back to the chrome so here you have the docker desktop for windows so you can go for this table or you can go for the edge right so you also have the comparison that what is the difference between these two versions right so um the particular edge version is something which is getting releases every month and uh the um stable version is getting the releases every quarter so they are not doing much of the changes to the stable version as compared to the edge there so you just have to double click on the installer and that will help you to do the installation of the process so let's get started so you just click on the get instable version so when you do that the particular installer is going to install now it's going to take like around 300 mb there so that's the kind of installed which is available so once the installer is downloaded so what you can do is that you can actually go ahead and you can uh proceed with the doing the double click on this installer when you double click on that you have to proceed with some of the steps like you know from the gui itself you are going to proceed with these steps so we'll wait for 10 to 20 seconds more and then the installer will be done and then we can do the double click and the installation will proceed so another thing is that uh there is a huge difference between the installer like for example in case of unix the installer is a little bit less but in case of windows it's a gui is also involved and there are a lot of binaries which is available there so that's the reason why you know the hue size is there now it's available for free that's for sure and it also requires the windows 10 professional or enterprise 64-bit there so um if you are working on some previous version of operating systems like windows 7 and all you have the older version called docker toolbox so they used to call it as like docker toolbox earlier but now they are calling it as a docker desktop with the new docker windows 10 support as such here so another couple of seconds and then the installer will be done and then we will be able to proceed with the installation so let's see that how much progress is there to the download so we'll click on the downloads and here still we have some particular installations or some download going on so we'll wait for some time and uh once the installation is done then we'll go back and we'll proceed with installation so couple of seconds so it's almost done so i'll just click on this one you can go to the directory to the downloads and you can double click on that also but if you want to do the installation you can click on this one also and it will ask for the approval yes or no you have to provide now once that is done so um a desktop a kind of a gui component will open there so it will start proceeding with installation so it's asking whether you want to add the desktop the shortcut to desktop so you can say okay i'm going to click on ok so it will unpack the files all the files which is required for docker to successfully install that is getting unpacked over here so it will take some time to do the installation because it's doing a lot of work here so you can just wait for till the execution of the installer to be completed and once the installer is done you can open your command line and start working on the docker so taking some time to extract the files now it's asking us to you know do the close and do the restart so once that is done you will be able to proceed further and you can just you know run the command line and any docker command if you can run so that will give you the response whether the docker is installed or not so you can see here the docker is you know something which is installed so you can run like docker version you will be able to get a version of the client when you do the restart of the machine then at that moment of time the docker server will also be started and then this particular error message will go off right now the docker daemon is not up and running because the installation requires a restart and when you close on this one and go for the restart the machine will be restarted here so this is the way that how exactly we can go for a docker installation and we can go on that part welcome everyone to this topic on docker tutorial for the beginners here now in this one we are going to talk about the various components for example what is the devops here why the devops is important what is docker the advantages of docker installation of docker architecture of docker components of docker game so these are the different things which we are going to talk about over here now let's start with what exactly isn't devops all about here now devops is nothing but a collaboration between the development and the operations teams over here which enables continuous delivery of the applications and services to the end user so it's a combination or collaboration between the two teams development and operations which helps us to go for a further implementation of various processes like continuous integration continuous deployment continuous delivery all these things are possible with this collaboration over here now why exactly docker is important for us as such in a city far away from this one so a developer decided to develop a tons of software on a single vm now he can put up applications he is having a big server and he wants to host the the multiple application on a single machine but for him these virtual machine got a crash because definitely if you are putting up more uh different different applications as such on the on a single vm uh it's not able to take up that much kind of load over there so in order to avoid this situation how exactly he can do the various options or what he can implement to have this uh issue resolved so ultimately docker is something which is the solution of him so if he really wants to host multiple applications uh on a single virtual machine or he want to have different different applications then docker is the ultimate solution for him to do that and by doing so uh there will be no issues as such when we talk about that uh the errors or the uh virtual machines can not be crashed and but there will be no impact on the virtual machine as such so why exactly docker is important so let's talk about the comparison between the docker and virtual machine there are very few reasons why exactly uh the docker is being chosen open a virtual machine because virtual machine is a full-fledged component which can take up any kind of load but some of the reasons are still that in which case we will be preferring the docker instead of the virtual machine here the very first one is the memory users so docker is having a very less memory uses because there is no concept of guest operating system whereas on the other hand the virtual machines are going to take a huge amount of ram uh resources in order to keep that particular operating system up and running second one performance because uh if definitely if you have less utilization and the users is less the performance will be better but for some reason if you are taking up more resources and you will not be able to deliver that amount of performance as such over there and then again we have portability so portability is also quite easy when we talk about the virtual machines in case of docker it it's quite portable as compared to the virtual machine and then the last time is that the boot up time so virtual machine boot up time is probably in minutes but uh in case of docker it's in seconds so if we summarize it on this one so the memory utilization the boot up time the performance everything is better in case of docker as compared to the virtual machines right for a classic example let's say that what about the memory user 9gb of memory uh you know is something if you can see that one f2 and f3 is there so green and red color represents the occupied memory and the wasted memory now there is a particular utilization of let's say that um you know a server which is available there and uh you know uh 3gb of ram is something which is wasted and 4gb of ram is something which is being occupied there so app 2 is using like 2gb and then you can see that the specific 2gb of ram is occupied and 3gb is occupied and the wasted one is 1gb so vm can see over there that you know 9gb of ram is used and the remaining 6 gb of uh unused ram can not be used here because if for example i allocate like 4 gb of ram to the virtual machine it's there itself it it gets locked on so it's not like okay i'm using only 4gb ram out of this 9gb of ram then the next 5 gb of ram will be totally wasted so you cannot reuse it for any purpose so this is a problem with the virtual machine that if you allocate some memory to the uh specific virtual machine if you are not using that much uh resources or that much memory the remaining memory will be simply wasted because this is hardcorely allocated to the virtual machine and you cannot utilize this unused memory as such over there so this is the limitation from the memory perspective but in case of docker definitely if uh you know out of the 9gb of ram if we are using uh only 3gb of ram so whatever the unused memory is there so we can actually use it to for the new containers so it's not dedicatedly allocated to the virtual machines if you are using out of 9gb if you're using 1gb 2gb 3gb or 4gb whatever the remaining memory is available that you can simply use it for the other containers also this ensures that you always go for an optimal utilization of your resources and unnecessarily there is no wastage of the different different resources or the different components over here in this one so that's the reason why the docker containers are quite efficient in terms of memory when we compare it with the virtual machine next one is a performance now um in a virtual machine we know that you know on the infrastructure on the physical infrastructure on top of that you put up a kind of operating system a base operating system is there and then on top of that you have the hypervisor and this hypervisor then is used to create the virtual machines which is having a kind of guest operating system inside it now the issue over here is that you have two operating systems over here so the guest operating system is also going to take some particular resources and for example memory or a space processor so in every terms in every source terms it is going to go ahead and say that okay i'm going to use the uh particular some of the resources now problem is that if you're hosting a very small application you will end up in having a more consumption because of this guest operating system will take more more of the resources and then you're just using some small very small resources for your application in an uh real world so docker container on the other hand does not have a concept of the guest operating system and that's where what happens that the docker engine is more than sufficient to handle the docker containers and inside the docker containers there is the application which is being hosted over there now with this one definitely the performance is quite better as compared to any virtual machine because of the terms that there is no unnecessarily wasted of the resources because in the previous case virtual machines a full-fledged guest operating system is up and running you need some resources to be allocated to that but in case of containers you don't need to have that particular functionality implemented then portability so um you know having a portability or moving an application which works on a system a does not means that it will work on the system b also this is a similar problem which we face that you know the usually the application works perfectly fine in a lower environment and when it goes to the production environment it sometimes fails and uh it does not comes up properly so this isn't portability issue which we have when we talk about the virtual machines but when in case of containers when we got a container up and running in a lower environment and once the container is prepared you don't have to do the changes on modification to that depending on the environment the same container can be deployed in production environment and it's very easy so whatever container prepared in the lower environment in a dev environment the same will work in production environment without any issues because it contains all the dependencies and executables everything inside it so you don't have to do any kind of modifications or changes to these containers as such over here and the last one is a boot up time so uh the boot up time of the virtual machines is in minutes and whereas the boot up time of the docker containers is in seconds or probably in milliseconds so these are the reason why we consider the virtual machine as an resources based environment but on the other hand docker is something which is quite light and fast as compared to virtual machines so let's see that what exactly is in docker all about here so in this one you can see uh we have the physical infrastructure on which we have the host operating system now we don't have the hypervisor and the guest operating system instead of that we have the docker engine which is taking care of managing the docker containers and inside these docker containers you have the applications and all the required binaries and the libraries here so there is no concept of the gift operating system these docker engine is the uh person or is the process which is taking care of all the requests which is going on and if this docker engine is up and running then all the containers will also be accessible and will be available now docker is nothing but a os level virtualization software platform that enables developers and id administrators to create deploy and run applications in a docker container with all its dependencies so docker is a very important tool nowadays if you want to look forward to the host your application because of its portability uh benefit that you don't require much of the changes uh depending on the platform or depending on the environment so whatever the developers are able to build up the same thing will go till the production environment now let's talk about what exactly is in docker container here now docker container is a lightweight software package that uh includes all the dependencies whether it's a base operating system uh some framework some for libraries it's it's something which is a part of this one so that if you will be if you want to host your application you will be able to do that so it's a complete package so both the source code and the dependencies is what you put up inside the docker containers now within this docker containers you can have multiple applications there multiple programming languages multiple frameworks can be hosted there in fact the databases also can be hosted within this docker containers you can host them as such over there now one of the different advantages of docker let's talk about that now the first one is the rapid deployment so rapid deployment is the very first advantages which we are getting over here second quick and easy configuration better efficiency portability scalability and security over here now there are different ways in which we can understand the benefits or we can have the understanding about the docker here so uh these are some couple of benefits now let's take up uh take them one by one the for the first one is the deployment part so it's it supports the rapid deployment now just like a web application for instance i want to deploy to website so there are some couple of steps which needs to be done across different different environments like for example if i have five environments i have to do the deployment the same steps in all the five environments at the different period of time now when we talk about docker containers we don't have to run those steps multiple times because once those steps are executed in one docker container the same docker container goes till the production environment so you don't have to actually do a rebuild of the docker containers and you can simply say that okay i have got the docker container here so you can just run one or two couple of commands and that will help you to host your docker containers so you don't have to redo or rework on any kind of the deployment work into the production environment or into the higher environments and then we goes for the uh quick and easy configurations so the configurations of docker containers is quite easy you can have your own custom docker images built out the docker files so these processes can be integrated within the jenkins also so cicd is quite possible with the help of docker if you want to really do that and then we have better efficiency yes they have a better efficiency and reduce resources management as compared to the virtual machines portability uh we have already thought about that that it's easy to move from one environment from one system to another in fact it's easy from the developer also so from one machine from one developer to another developer it's very easy to move these uh different docker containers then we have scalability scalability is also possible with this docker containers just like multiple virtual machines you can also create multiple containers and you can share the traffic out of that so but that scalability and all is possible with the orchestration tools then security is there so you can have your best practices of the security implemented because um it's just a kind of a virtualization just like a virtual machine so you have all the uh different uh possibilities which is available which you can explore when you talk when you want to make this stock container secure right so these were some of the benefits which was there for the docker here now let's talk about the installation of docker here now installation of docker is pretty common and uh whatever platform you want it's available over there so it's available for ubuntu windows it's available for mac operating system so for ubuntu you have to run the commands on the ubuntu part operating system to download it and for windows it's actually having a a particular exe file so you can go for the installation of exe file and then we have the mac operating system so for mac operating system you have the dmg files and when you do the dmg file so it will uh help you to install the component over there so mac you can also use the home view wants to do the installation of that particular component now next one is the architecture of docker now the architecture of docker is pretty straightforward there are key components which is available there in the architecture of docker here the very first one is the as the client facing is the docker client now docker client or you can say the docker cli is available there now this is the component it's a command line interface which takes up the request from the end user like for example i'm running the command so it will take all the requests from me as an end user and it will send those particular things or the requirements to the docker daemon to the server using the rest api interface so this docker daemon is something which keeps on running on the server and it's a background process it gets deployed as in service there in case this docker demon is not available or it's not up and running you will not get the proper response from the docker command line here so in order to get all your activity or your procedures implemented you need to have the docker server or docker daemon up and running in fact this docker cli can be configured to connect to any other servers or any other different server docker daemon also so it's a connectivity which is happening over the rest api interface where all the requests are processed are sent to the docker demon and whatever response is coming back that is given back to the end user so what are the different components of docker here let's talk about that so docker client consists of the cli company uh like we have already discussed so that is something which is uh going to take up all the requests from the end user then um it's using the rest api interface which is there to have the connectivity with the docker demon so docker daemon is something which is always up and running so if you run couple of commands like docker pull push and also that activity you are actually sending the request to the docker demon and docker daemon is actually doing all the work so it's not a docker client which is doing it just taking its kind of interface which will take the things up and send it back or send it forward to the docker day right so it's a background process which keeps on running on the server on the operating system and uh it's it's the same process across all the infrastructure all the different platforms whether you use mac operating system you use windows or you use linux everywhere this is the basic functionality and it keeps on listening to the rest api interface so uh whatever the request new request comes in it immediately process it and whatever the response whether you are stopping a container whether you are running a container according to that the action will be taken and the response will be sent back to the rest api to the docker client so a docker is a process or it's a command line which is used to start the docker daemon when you run the command like service docker status on the server you will be able to see the command which is getting executed in the back end so there you can see that docker is the process or is the command line which is used to initialize the docker daemon and of course there is a kind of configurations which you have to pass it on as a parameters so even if you run psiphony f command you will be able to know that what process is running from the docker perspective and the third component is a particular docker registry so we have a ready-made uh cloud service which is available known as the docker hub so you can actually store the docker images uh on these docker registry so there are a lot of ready-made uh docker images which is available there but of course you can have your own uh docker image also over there or you want you can have your own custom uh registry docker registries also for example in jfrog you can have your private hosted registries also so all these are different repositories you can use with your docker container so docker containers is having the flexibility that whichever registries or repositories you want to connect or you want to establish the connectivity you can always do that now let's talk about the docker image over here in detail now a docker image is nothing but a template of instructions it's having a couple of instructions in which you will be using these instructions to create a final dock container so docker image is uh nothing but uh for example i want to have a tomcat apache docker image so what are the instructions i have to do so the very first thing is that i have to pick up a base operating system then on top of that i will be installing the jdk or some prerequisite packages and then on top of that we will be configuring on to the tomcat apache there so these are the steps these are the different instructions step by step procedure which we are implementing so that at the end of the uh activity we will be able to have a docker container ready which we can deploy to host tomcat apache so it's a template which is having all the uh particular instructions in order to build a docker image you require a file called dockerfile which is uh of course having all these different uh instructions embedded in that so when you run the dockerfile it acts like a build tool so whatever the instructions you are putting up over there all those instructions are getting uh processed and you are able to get a end result uh then uh these docker images are actually comprised of multiple layers so multiple layers are available there so step by step so you will be first of all having a base layer from which the image starts on top of that another layer for example you did the installation of jdk so another layer will be there and then you did the configuration of of the tomcat apache and then another layer will be there so it's a multi-layered in uh component which is available and these layers are in incremental order so definitely the oldest one will be known as the base layer and then on top of that all the latest changes will be done in the latest layers there so this is an example now here you can see that there are four instructions and these four instructions are also known as the four layered docker image so whatever the final docker image you will get in there you will be able to see four layers of uh the docker image here now the very first is ubuntu so ubuntu 18.04 we are pulling up and then we are pulling like uh the file over here and uh then we are running the make utility and uh once that is done then we are running that file dot python file over here in this one so these are four instructions which is available right so uh which will definitely help us to do that how exactly we can make or we can perform the different activities right from the base so in this case the base layer will be like uh the one two one so one to one will be the base image and the moment the final cmd attribute or cmd uh attributions are processed and executed after that you will be able to get a final full-fledged docker image over there so this will be a full complete docker image or docker container which you will get once the whole build process is completed now in this ways you have you can see that the previous docker file which we processed so the base layer is ubuntu 18.04 we are not building that actually it's automatically so it's by default getting downloaded as in full base layer after that only we have like layer 1 layer 2 layer 3 added up because those modifications we are doing it after the because from does not means that you are doing any kind of layer modifications it's just like you are importing some base operating system or base image so the modification or adding up the layers is only done after the base layer there so whenever a user creates a container a new layer is something which is stopped on top of the image these layers are known as the containers layers so one two one the very first instruction does not comprises it it is it's not a kind of a container layer we can call it as an base image layer there so every container has a separate read write container layer and any modification in a container is reflected upon the container layers only so layer one may be having some different of instructions layer two maybe having different instructions so like that it's a step-by-step processor in case you know whenever a container is deleted the top layer also gets deleted over there in that one so if i delete some particular containers and that is something which is gone so it was suppose having like three layers so the top layer will be simply deleted because the container is also deleted for that particular part now docker uses a copy and write strategy with both docker images and docker containers so whenever we uh perform any activity or whenever copy on uh right strategies and strategy to share and copy files for better efficiency so it's something which is being used to increase the overall performance of the docker containers it also makes docker efficient by reducing the uses of disk space because whenever uh required okay whenever is needed only at that moment of time it's actually performing the copy and it definitely reduces the overall disk utilization and this is very important also because when you have so many docker containers up and running so you have to worry or you have to make the utilization of a disk or your disk users or resources also minimum so that is also you need to make sure that it does not exceed a particular value so once the utilization is also less then definitely the overall performance will also be better and improved over here in this case so uh let's talk about some of the facts here so the base layers are something which is purely read-only format so you cannot do the modifications union file system saves the memory uh you know by avoiding the duplications of the files uh the layers can be combined in a union file system to create a single image so all these layers are combined and got a final product or called a you know you can call it as docker image or docker containers and this allows a file system to appear as a write table which is known as copy on right so copy and write strategy is being used so to do the changes or to do the perform the modifications of this union file system by creating the different different layers so now let's talk about the docker registry very important component because even if you prepare the docker image locally and you don't have the collaborative tool or you don't have the tool where you can share these images definitely it's not easy for you to port these docker images from one system to another so docker register is very important for the portability feature of docker so docker registry is a service to host and distribute docker images among users so uh it can be built up by one user and multiple users can use it for example i'm using a jenkins one so um jenkins uh image is prepared only once but it's being used by different people as such docker registry uh again it's a repository is a collection of docker images so in a repository you will be having like different different uh docker images as such so in registry a user can distinguish between the docker images with their tag names so uh for example jenkins is an a kind of a single image they might be different type of images there let's say that okay let's talk about uh jenkins uh jdk so there will be image called jenkins the name of the image will be same but the tags will be different so one tag could be like jdk8 one tag would be like jtk9 and one time would be dedicated so the difference between all these three docker images is that that jdk 81 is prepared on jdk8 it's running on jdk8 and if you want to use jdk9 in that case you have to pull down the the darker image of jenkins with the jdk 9 installed so depending on the frameworks or depending on the runtimes so with these tags different tags are processed so you can have the distinguished between the docker images and whichever docker image you feel you want to use you can implement according to your requirement and docker by default is having its own uh cloud-based registry called as docker hub which gets automatically connected when the moment you do the docker installation but in case you are working into your private infrastructure so in that case uh you may use the repositories uh docker repositories on the uh specific jfrog artifactory or the private repositories also can be hosted but docker hub is something which is having a public and private repositories both so you can create either the public and private repositories you can use uh push and pull activities to interact with these docker images you want to download a docker image so you can run pull if you want to push your docker image to the docker registry in that case you can run the push activity so in both the cases you are doing the interaction with the uh docker hub so when you run some build process or when you run the containers there may be some docker containers which is getting pulled down from the specific registry from the docker hub and uh if that specific image and tag is available definitely pull command will be successful but in case of push command you are pushing your custom docker images uh you cannot push the ready-made docker images which is available you can publish your own docker image only to the docker hub and then you can further on do the poll and push activity accordingly to those repositories so whatever the ready-made and third-party images which is available you can only do the pull activities to them you cannot do the push activities push command scenario will only come when you are looking when you are interacting with your custom docker image next one is the docker container now docker container is an executable package of an application and its dependencies together so when we prepare a docker container we actually combine both the dependencies and the application source code inside it so the docker container don't have to look for the dependencies on the host machine on the server so this is one of the benefit where we see that the portability is very easy because whatever i'm building up on one system everything is inside the docker container so i don't really need to do any further activity on other machines so whatever works on one system can be easily worked on in the system so it's very lightweight also so it can be easily deployed and executed on you know on their particular uh servers because it does not take that much utilization like a virtual machine so if you are looking forward for having your application hosted into your local system you can easily do it within the docker container so the utilization is very less as compared to the virtual machine so you don't have to worry about your operating host operating system and the qualifications so docker containers can run in totally isolation just like a virtual machine so there is no interaction or there is no interference so they share the os called uh kernel but uh the thing is they don't have the interface as such over there so it works totally in isolation uh two different containers will not have any interaction unless and until we do we set up that the containers to have the interactions now there is uh data volumes where which can be shared and uh be used among the multiple containers so um you have the uh capability over here that you know if the there is some unused uh space that can be easily shared across the multiple containers here so uh it is uh built using the docker images so using the docker images the docker containers are built up and docker run is a specific command which is used to build a docker container so if you got a docker image let's say that i have prepared a docker image a custom docker image using the docker file now if i need to create a docker container out of a docker image i have to run a docker turn command so docker run command is nothing but it initializes or creates an instance of a docker image now next component is the docker compose now typically when we work with the the docker containers so we have to run like if i want to run like five six containers i have to initialize like five six times or i have to run docker and commands five six times there so um and if you want to create a relationship between the containers then definitely that is something which you need to manage using the docker commands but sometimes it's not an easy task for you to run those commands manually or remember those commands so docker compose is a utility over here using which you can actually run multiple containers with a single service you just got off a kind of a single file you just run it and all the containers definitions and relationships everything is inside this docker file here so you want to run two containers that's totally fine instead of running two commands now you just have to run one file and that will do take care of that now you may be having more than two containers five six seven whatever number of containers you have that can be easily deployed without any um manual activity or without any maintenance onto the server so uh these containers totally run in isolation but yes you can establish the interaction that how they want to interact with each other and that interaction you can put up in these uh docker compose files so docker compose file is something which is written in the vml format and in this you define that okay i need two containers and this in this image should be there these are the parameters which you need to run and if you want to have the interaction or the linkage between the images that also you can establish within this yml file or this in within this docker compose file here right so let's say for example i want to have an application which requires apache web server and mysql database now uh i can create a docker compose file which can run both containers and uh without you know you don't require to restart or start them separately or initialize them separately so i will be having a final docker compose file which is having like uh you know containers like apache containers and mysql containers both inside a single docker compose file here so that's the benefit you're going to get because now you don't have to run them separately separate apache and separate via mysql so you can get it done with the help of a single docker compose file here now what exactly is the docker swamp now docker swamp is a inbuilt or it's a default uh orchestration tool which is being provided by the docker itself it's a service for containers which allows the id administrators and the developers to create and manage a cluster of smart nodes so um in a typical world when you do the installation of docker there is a limitation that you some of the features like high ability multiple uh instances or scalability all these things are not possible as it is with the core docker now on top of that you require the organization tools like talker swamp where you have a swam of node machines five six known machines are there on which you can have your highly available application infrastructure implemented now this one means that if i want to run my applications five containers six containers and seven containers for that purpose we are using the docker spam so each node of docker swim is having a docker daemon which is uh used to for performing the interaction so there will be a master node which is available so uh it will be interacting with all the known machine on the all the client machines and this interaction will be in such a way that it will let them know okay i want to deploy the container or onto the node 3 so it will perform that activity and the container will be up and running on the node 3 there so a swamp consists of two type of nodes one is the manager node and one is the worker node so manager is the master node which you can refer and the worker node is the one which is actually doing most of the work so uh the manager node is not something which is expected to host uh anything apart from the main component but yes you can always configure if you feel that you want to only deploy the container on the worker node so that also you can uh do during the deployment of the containers so manager will will keep on connecting to the different worker nodes there and whenever new request comes in so it will basically try to connect and see that okay i just want to go ahead and deploy a container so it will do the deployment of the container on the worker node so the main work will be done by the worker nodes and manager is just doing the management of all these resources the worker nodes will be definitely receiving the request from the manager and uh it will be proceeding and performing those activities and uh if for some reason worker node is not active or online so the master node will not be able to deploy the containers on that server so right now let's talk about a demo in which we have to create a docker image using a docker file on the open220 so uh you want to use a base operating system as an ubuntu 20 there and using that we will we are going to create a custom docker image so let's connect to our virtual machine and see that how we can do that okay so as part of this demo here we are going to prepare a docker image and uh in that we are going to use a base image of ubuntu so uh let's take an operating system of ubuntu 20 20 is the latest version of the one two operating system so i have got a server so let's see so first of all we have to install the docker.io package which will install the docker package and then we can enable the service we can make sure that the docker service is up and running because if the service itself is down then we will not be able to perform any activity and then our sample if you run docker run hello hyphen world so that is going to pull a darker image from the docker hub and a particular activity will be done to you so once that is done so we can say that okay the container is up and running and everything is got uh you know over there up and running and then you can also try some of the ubuntu uh docker image also if you feel that you want to try you can do that but the main uh demo which we need to do over here is that we need to prepare a custom docker image and in this one we will be having like we will be using an ubuntu base operating system on top of that we will be installing php engine x all these different components so we are going to have a php based website hosted onto the apache web server as an nginx and then we will be able to access that application so the very first thing which we need to do is that we need to create a folder uh let's say that i just create a particular folder called nginx iphone image and then i create a docker file over there so docker file uh content we are going to take from this one so let's go to the server and see that how we can go ahead with the activity now uh what we can do is that uh what uh we have got a server over here now in this server what i'm going to do is that i'm going to proceed with the installation of docker first now when i do the docker installation first so what will happen that i will be able to have my docker installation completed and then further on i can take the further steps of having the accessibility of uh going through the understanding that what is this uh particular platform or how to prepare different different kind of containers so i have got a particular container over here so what i can really do is that i can simply say that okay i have got a server so this is a one-two server so i'm going to run the command called app update first now this will do the update locally so all the links and everything will be updated now the whole idea about doing the updates is that so that i will be able to establish the fact that i'm trying to connect i'm trying to uh connect with the 0.2 url so that at any moment of time i should be able to fetch on the latest packages now once that is done so i'm going to run the command called apt install docker dot io now the moment i do this what will happen that it will try to download some packages it will ask for the confirmation you can give your confirmation and i can say yes the moment you give the confirmations the download of the packages will proceed over here step by step it will download the packages and the installation will go on now once the installation is done so what you can do is that you can see that docker is deployed into your system or not you can run docker version command or you can also use the service command to check to make sure to ensure that the docker service is always up and running if the service is not up and running you have to make sure you have to enable it you have to start that service so these are some couple of steps which you need to do in order to make sure that your connectivity or your environment is ready for with the docker installation and once the docker installation is done then only you will be able to proceed further with the further activities like preparing the containers preparing the images and all that stuff but if you don't have the docker installed prior to that then definitely it's going to be a problem so just make sure that before even doing any kind of activity before even doing any kind of uh steps we ensure that the docker is fully up and running so it's going to take some time to download the packages and we will wait for that and once the installation is done we will continue further on that part okay now once the installation is done so you can run the command call service docker status and when you do that it will let you know that whether the docker service is available there or not so uh you can start the service in case you don't get uh the proper response and uh now you can see that it's showing up that a process is running so earlier it was not uh up and running it was inactive but now it's active and you can see here that dark ride is the process which is running over here as an uh process to initialize the docker daemon so docker demon is up and running now and we are good to go here so now uh if we go and say like docker run hello hyphen vault so we are doing a validation to our docker instance that whether our docker instance is working fine as expected or not because uh if there is any kind of problem with the docker demon and docker demand is not up and running properly we would be able to get an error message over there but right now there is no error message which represents that uh yes you are pretty much running the docker image and it's giving you the proper response so you will be able to see the uh docker images when you run so you will be able to see that the docker image is pulled down to your system and definitely you have executed that and it got you the response now the next thing which we need to do is that we need to run a particular ubuntu one so for example if i go for docker run iphone it open to bash so what exactly we are doing in this one is that we are pulling down a ubuntu docker image and we are running the bash executable on top of that the whole purpose of doing that activity is that so that if i want to do some uh process or i want to perform some activity i should be able to connect to that docker image so the moment i do this first of all if this docker image is available locally that's great it will immediately create the container if the docker image is not available locally the very first thing which it will perform over here is to pull down the docker image the correct one and as you can see here that it uh definitely pulls down the docker image and once it is done so it got a terminal for you now you can see it shows that root at the id uh root and the rate one id is there this id is nothing but it's a container id which we are getting here so definitely you are inside the docker container and when you run the docker when you run the exit command you will see that you have come out of the docker container over here and the moment you run docker ps hyphen a command you can actually see also that how many container you deployed so far so you ran hello world you ran the one to one both of them are stopped there you can see the exit 0 in the status over here because both of them are something which you ran in the interactive mode so they are done so if you want to keep your container up and running you need to have a container which runs in the dash mode and always up and running but for that you have to go ahead to the level where we need to create a custom docker image and do the setup accordingly so let's see that how we can do that now i'm going to create a particular folder called nginx hyphen image you can pretty much give any name whatever name you feel you want to perform you can do that now in this one i'm going to put up a file called docker file so let's see that what content we need to put as such when you talk about the docker file because whatever content you put over here that is what according which the docker image will be prepared now here we have to talk about so first we have to see a base image so we are saying that okay 0.220.04 is what i want to use then we are giving some labels now these labels are used to give some information like uh what is the version who is the maintainer what is the description uh you can do the customizations to this one so uh these labels are actually defined and utilized so that i will be able to give an information that okay so and so developer is something which we can contact so if there is any kind of problem with the uh docker file or will this build script we have to reach out to that specific person so this is what we typically do with the labels here now um we also have to uh set up a parameter like deviant underscore printed as in non-interactive because whenever we talk about any build process in dockerfile it it cannot be there in the interactive session it should be there in the non-interactive and definitely a typical uh scenarios like you can run app update command and then we are installing the php engine x super supervisor all these things we are doing we are doing the installation uh we are using the run attribute run a particular step and in there whatever the commands are executed these statements will be executed when we will be doing the build process and then we are using some environment variables okay so nginx we are virtual host php underscore conf engineering centers for conf and supervisor underscore conf these are the different environment variables which we configuring over here now next thing which we have is that uh we have to transfer some files like probably nginx and the virtual host file we may have to transfer from our linux server to the destination to the docker container so for that we are going to run some copy commands copy statement so that we can do that stuff and uh then definitely we are transferring the supervisor configuration file also to the server so that we can have that configurations uh setup over there and uh these are config uh supervisor underscore conf and all these different variables we have already established where we are saying that what exactly configuration we are looking forward over here and then we are uh you know creating some folder structure we adw html folder structure we are creating so that we will be able to push some content over there and then we have the volume configurations so whatever the volume configurations are there we are putting up so that um you know we will be able to have all the specific direct directories mounted on the host machine on the whole server so that we can even talk about locally also and then the star.search is something which we need to run because that's uh that will go into the command statement which needs to be executed because when you want to run your process when you want to run your engine x process or your docker container some process should be up and running that's where uh if any interactive process will run then only the docker container will be hold up so that's what you can do or you can configure over there so i'm going to copy this whole content now this whole content represents the complete docker file so uh we have already gone through this content one by one but uh if we combine all these content together till the exposed part that's where we come up into these one so expose we are doing 80 and 443 over here right after this one we have to create the uh host file virtual host file supervisor d file and the star.search file so let's go back to the server here now in this one you have to uh you can use whatever editor you feel you want to use you can do that uh so this is what i will my master file will look like you can see i'm running some run commands uh these are the environment variables i'm using then further on these variables i'm using just to define the paths okay so you can see that this default file we are copying from the host machine from the server to the docker container and then second file is super wiser d not con file and then we have star dot sh file so these are the different files which we want to configure here so once this file is prepared we have to uh type in or we have to create a default file over here in this current directory so in this current directory we have to put up a file called default now in this one we will be having the default virtual host configuration config so we need to put our server attribute saying that we have to look forward for index.html or index.php and these kind of files from the www.html directory there so i'm just going to copy this content and put it up into this one so this is how my final file look like so i'll save it and uh then we have a supervisor dot com file so so i'll just copy the content of this one i'll copy it and we have to create a file called supervisor file now names we can double check because we are referring these names into a docker file so we can double check that so once this is done save it and we can check the docker file you can grab so you can see okay this one so super wiser d so the name is correct that's cool so we got default we got supervisor d now we have to create the star dot sh file because in this one we will be configuring that how we want to start our process of this nginx or this web server so in this one again we have to run the uh supervised process over here and a configuration file so we are going to put this one and we'll go here and put this content right so we got the supervised config also available so that's also there and the moment is done so you are going to give the execute permission to this one because uh you can see that into our docker file that we are having the attribute where we want to run this into the dots slash there so if you want to refer in that mode you have to change the permissions in case so that uh change mode you have to give the execute permission to the start dot sh here now once this is done so what you got is that you got a particular uh components like you got uh some couple of commands you have uh you know got docker file super isd.conf default all these files are there now the only thing which we need to do next is that we need to build to the build process so uh if we look forward for the build process so what happens the docker build is the command which we need to use and uh nginx iphone image is the name which you want to refer now you can totally use this uh custom name you whatever name you feel that you want to use you can always use that there's no restriction on that part and once that is on so you will be able to list down those images that you got a new custom docker image here so you can see that you got the ubuntu as a base image which is built down and the nginx iphone image is also prepared over there and once that is done so you are going to create a web root folder and you can run the docker run command to see that if you really got connected with that particular one so let's go back and uh see that how we can create the docker custom docker image here so the moment i do this so it will pick up the docker file over here and start performing the activities uh the package installations all those configurations ones will be performed over here in a non-interactive mode there is no confirmation which will be asked from the user because you cannot put that confirmations so the command should be in such a way that it should not take any kind of confirmations at all from the end user so because tomorrow you may be putting up these things in the jenkins so that will not be a feasible thing for you to put up the commands and put up the confirmations during the execution so so it's trying to run do the package installation for the php so fbm packages are getting installed and um here we go so we caught the successful message now you can see it says successfully tag this one this means that there is no issues at all with the docker image and when you run the docker images or docker images ls you can run these two couple of commands so when you do that you got a 203 37 mb of the size of this image that represents that this docker image is fully ready to run to execute so now what we are going to do is that we are going to map a volume over here with the wwe html directory during the execution of this container because i really want to deploy some pages some web pages there if you have the web pages now as part of the build process as part of the docker file you can do a git clone also of some repository but i don't want to do that uh during the preparation of docker image i just now got an environment where the nginx is set up properly and it's up and running there now it should be upon me that how i want to put up my content and all so that's the reason why i'm putting up a kind of a folder structure over here and sharing it with the docker container so that the moment i put the html files or the php files into that local folder i should be able to get some web page up and running there so that's what i'm going to do over here in this one so i'm going to create a folder structure saying slash web root here okay so there is a folder structure which is created now in this web root i will be putting up some files and then same files will be redirected inside the docker container with the help of docker volume so let's go back to the document now here you have to run uh the uh once like i said that uh you are going to connect on that part okay so uh you can use this port as an at also but here 880 is used so whatever port you feel you want to use you can do that and we are giving the container name as test an iphone container that also you can change nginx hyphen image is the one which we are using so if you use some different uh particular image you can also use that and uh you can see that the webroot is something which we are connecting with the wear www html directory so the moment you put some file in the web route the same file will be reflected into the ww html directory there so i'm going to run this command and once this is done so you can see that for the testing purpose we are putting up some content in the html one so i'll come to that point also and we'll validate as of now we should be having our container up and running over here in this one now i'm going to copy that command okay and hit on so if the at port is used so you have to make sure that photo gets uh you know gets stopped automatically or it should not be up and running so just double check whatever port you're using that should be free now when you run the docker ps command so uh you will be able to see a docker container up and running but if you see like docker psyphony that gives also the stopped and running both the containers are there but the main thing which we want to run is that we want to run the docker ps command and you can see that this is a container which is running so 8080 is the container which is running and test hyphen container is the name now you got a container up and running so you can actually do a call on localhost colon 8080 so um this is a way to get the content you can get the public ip also and uh in the browser you can put up and you can have the content there but right now uh we are getting the response at least uh it's showing his 403 error forwarding error but that's fine we will be putting up some web pages there in the rwb html directory or we can say in local web root directory so that we will be able to have some content so what we can do is that we can have some echo commands executed so that we can have some added some content added up into the uh particular uh files there so let's see that what different contents we can do so uh you can use the vi mode also to do a uh you know couple of files so uh i'm just using this echo command so um the moment i go for this execution so what happens that a file will be created into that uh slash web route there so um i'm expecting that i have put up a index.html page so if you feel that you want to put some different page you can always do and the moment you do the curl now you will be able to see the same response so whatever the things you put up in the index.html file the same content is going to pop up when you run the curl command now um you want to use some php files you want to put up some php content all that stuff you can do it now what i'm going to do is that rather than just a html static content i'm going to put some php content that is going to help me to get a file so my web root will have now two files so um if if you feel uh that index.html file should have some meaningful content you can always do that you can pull some of the websites also you can put that into the you can do a git clone in the web root and the whole application will be started and uh now if you feel that i got a info.php file so i can use the in that name into the url i have to modify the url because index.html file is the what as in default name page you do if you want to put up some different name you can change the server configuration of nginx and according to that it will work so um when i do this so you will start getting the output so this is the php content output which is there it's a complete information now it may look like a very bulky content to you but when you put up this one into the browser into the web browser that will be a meaningful information probably in terms of some tables or something like that right so um let me scroll to the top here so you can see that it's showing an error sorry it's showing in the output and this output represents that you got a particular php content here so this is how we will be able to have a container and uh you know this container is up and running you can modify to at any level you feel that you want to prepare a darker image with the code embedded also uh you can have the clone or you can have this configuration into some git repository have the source code also there and then prepare a complete bundle altogether so this is what we have in this demo how we can deploy a container and a container up and running in case of the docker here today we're going to cover what is a docker container why you would want to consider using it in your devops environment so let's have a look at what we have in our current environment so today when you actually have your standard machine you have the infrastructure you have the host operating system and you have your applications and then when you create a virtual environment what you're actually doing is you're actually creating virtual machines but those virtual machines actually are now sitting within a hypervisor solution that sits still on top of your host operating system and infrastructure and with a docker engine what we're able to do is we're able to actually reduce significantly the different elements that you would normally have within a virtualized environment so we're able to get rid of the the bins and the so we're able to get rid of the guest os and we're able to eliminate the hypervisor environment and this is really important as we actually start working and creating environments that are consistent because we want to be able to make it so it's really easy and stable for the environment that you have within your dev and ops environment now critical is getting rid of that hypervisor element it's just a lot of overhead so let's have a look at a container as an example so here we actually have a couple of examples on the right hand side we have different containers we have one containers running apache tomcat in a with java a second container is running sql server and microsoft.net environment the third container is running python with mysql these are all running just fine within the docker engine and sitting on top of a host os which could be linux it really could be any host os within a consistent infrastructure and you're able to have a solution that can be shared easily amongst your teams so let's have a look at an example that you'd have today if a company is doing a traditional java application so you have your developers working in jboss on his system and he's coding away and he has to get that code over to a tester now what will happen is that tester will then typically in your traditional environment then have to install jboss on their machine and get everything running and cool and hopefully set up identically to the developer chances are they probably won't have it exactly the same but they're trying to get as close as possible and then at some point you want to be able to test this within your production environment so you send it over to a system administrator who would then also have to install jboss on their environment as well yeah this just seems to be a whole lot of duplication so why go through the problem of installing jboss three times and this is where things get really interesting because the challenge you have today is that it's very difficult to almost impossible to have identical environments if you're just installing software locally on devices the developers probably got a whole bunch of development software that could be conflicting with the jboss environment the tester has similar testing software but probably doesn't have all the development software and certainly the system administrator won't have all the tools that the developer and tester have their own tools and so what you want to be able to do is kind of get away from the challenge you have of having to do local installations on three different computers and in addition what you see is that this uses up a lot of effort because when you're having to install software over and over again you just keep repeating doing really basic foundational challenges so this is where docker comes in and docker is the tool that allows you to be able to share environments from one group to another group without having to install software locally on a device you install all of the code into your docker container and simply share the container so in this presentation we're going to go through a few things we're going to cover what docker actually is and then we're going to dig into the actual architecture of docker and kind of go through what docker container is and how to create a docker container and then we'll go at through the benefits of using docker containers and then the commands and finalize everything out with a brief demo so what is docker so docker is as you'd expect because all the software that we cover in this series is an open source solution and it is a container solution that allows you to be able to containerize all of the necessary files and applications needed to run the solution you're building so you can share it from different people in your team whether it's a developer a tester or system administrator and this allows you to have a consistent environment from one group to the next so let's kind of dig into that architecture so you understand why docker runs effectively so the docker architecture itself is built up of two key elements there is the docker client and then there is a rest api connection to a docker daemon which actually hosts the entire environment within the docker host and the docker daemon you have your different containers and each one has a link to a docker registry the docker client itself is a rest service so as you'd expect a rest api and that sends command line to the docker daemon through a terminal window or command line interface window and we'll go through some of these demos later on so you can actually see how you can actually interact with darker the docker demo then checks the request against and the docker components and then performs the service that you're requesting now the docker image itself all it really is a collection of instructions used to create a container and again this is consistent with all the devops tools that we have the devops tools that we're looking to use throughout this series of videos are all environments that can be scripted and this is really important because it allows you to be able to duplicate and scale the environments that you want to be able to build very quickly and effectively the actual container itself has all of the applications and the dependencies of those applications in one package you can kind of think of it as a really effective and efficient zip file it's a little bit more than that but it's one file that actually has everything you need to be able to run all of your solutions the actual docker registry itself is an environment for being able to host and distribute different docker images among your team so say for instance you had a team of developers that were working on multiple different solutions so say you have a team of developers and you have 50 developers and they're working on five different applications you can actually have the applications themselves the containers shared in the docker registry so each of those teams at any time check out and have the latest container of that latest image of the code that you're working on so let's dig into what actually is in the container so the important part of a docking container is that it has everything you need to be able to run the application it's like a virtualized environment it has all your frameworks and your libraries and it allows the teams to be able to build out and run exactly the right environment that the developer intended what's interesting though is the actual applications then will run in isolation so they're not impacting other applications that's using dependencies on other libraries or files outside of the container because of the architecture it really uses a lot less space and because it's using less space it's a much more lightweight architecture so the files the actual folder itself is much smaller it's very secure highly portable and the boot up time is incredibly fast so let's actually get into how you would actually create a docker container so the docker container itself is actually built through command line and it's built of a file and docker image so the actual docker file is a text file that contains all the instructions that you would need to be able to create that docker image and then we'll actually then create all of the project code with inside of that image then the image becomes the item that you would share through the docker registry you would then use the command line and we'll do this later on select docker run and then the name of the image to be able to easily and effectively run that image locally and again once you've created the docker image you can store that in the docker registry making it available to anybody within your network so something to bear in mind is that docker itself has its own registry called docker hub and that is a public registry so you can actually go out and see other doc images that have been created and access those images as your own company you may want to have your own private um repository so you want to be able to go ahead and either do that locally through your own repository or you can actually get a licensed version of docker hub where you can actually then share those files now something that's also very interesting to know is that you can have multiple versions of a docker image so if you have a different version control different release versions and you want to be able to test and write code for those different release versions because you may have different setups you can certainly do that within your docker registry environment okay so let's go ahead and we're going to create a docker image using some of our basic docker commands and so there are essentially really you know kind of just two commands that you're going to be looking for one is the build command another one is to actually put it into your registry which is a push command so if you want to get a image from a docker registry then you want to use the pull command and a pull command simply pulls the image from the registry and in this example using nginx as our registry and we can actually then pull the image down to our test environment on our local machine so we're actually running the container within our docker application on our local machine we're able to then have the image run exactly as it would in production and then you can actually use the run command to actually use the docker image on your local machine so just a you know a few uh interesting tidbits about the docking container once the container is created a new layer is formed on top of the docker image layers called the container layer each container has a separate read write container layer and any changes made in that docking container is then reflected upon the particular container layer and if you want to delete the container layer the container layer also gets deleted as well so you know why would using docker and containers be of benefit to you well you know some of the things that are useful is that containers have no external dependencies for the applications they run once you actually have the container running locally it has everything it needs to be able to run the application so there's no having to install additional pieces of software such as the example we gave with jboss at the beginning of the presentation now the containers are really lightweight so it makes it very easy to share the containers amongst your teams whether it's a developer whether it's a tester whether it's somebody on your operations environment it's really easy to share those containers amongst your entire team different data volumes can be easily reused and shared among multiple containers and again this is another big benefit and this is a reflection of the lightweight nature of your containers the container itself also runs in isolation which means that it is not impacted by any dependencies you may have on your own local environment so it's a completely sandboxed environment so some of the questions you might ask is you know can you run multiple containers together without the need to start each one individually and you know what yes you can with docker compose docking compose allows you to run multiple containers in a single service and again this is a reflection on the lightweight nature of containers within the docker environment so we'll end our presentation by looking at some of the basic commands that you'd have within docker so we have here on the left hand side we have a docker container and then the command for each item we're actually going to go ahead and use some of these commands in the demo that we're going to do after this presentation you'll see that in a moment but just you know some of the basic commands we have are committing the docket image into the container kill is a you know standard kill command to you know terminate one or more of the running containers so they stop working then restart those containers but suddenly you can look at all the image all of the commands here and try them out for yourself so we're going to go ahead and start a demo of how to use the basic commands to run docker so to do this we're going to open up terminal window or command line deploying whether you're running linux pc or mac and we're going to go ahead and the first thing we want to do is see what our docker image lists are so we can go sudo docker images and this will give us well first we'll enter in our password so let's go enter that in and this will now give us a list of our docker images and here are the docker images that have already been created in the system and we can actually go ahead and actually see the processes that are actually running so i'm going to go ahead and open up this window a little bit more but this will show you the actual processors and the containers that we actually have and so on the far left hand side you see under names we have learn simply learn the underscore cool these are all just different ones that we've been working on so let's go ahead and create a docker image so i'm gonna do sudo docker run dash d dash p 0.0.0.0 go on 80 colon 80. ubuntu and this will allow us to go ahead and run an ubuntu image and this will run the latest image and what we have here is a hash number and this hash number is a unique name that defines the container that we've just created and we can go ahead and we can check to make sure that the container actually is present so we're going to do sudo docker.ps and this actually shows some there so it's not in a running state right now but that doesn't mean that we don't have it so let's list out all the containers that are both running and in the exit state so let's do sucker ps dash a and this lists all the containers that i have running on my machine and this shows all the ones that have been in the running state and in the exit state and here we see one that we just created about a minute ago and it's called learn and these are all running ubuntu and this is the one that we had created just a few seconds ago let's open it up and there we go so let's change that to that new dock container to a running state so scroll down and we're going to type sudo docker run dash i t dash dash name my um so this is gonna be the new container name it's gonna be my docker so this is how we name our docker environment and we'll put in the image name which is ubuntu and dashbin dash bash and it's now in our route and we'll exit out of that so now we're going to go ahead and start the new my docker container so sudo docker start my and we'll get the container image which will be my docker my docker return and that started that docker image and let's go ahead and check against the other running docker images to make sure it's running correctly so sudo docker ps and there we are underneath name on the right hand side you actually see my docker along with the other docker images that we created that's been running for 13 seconds quite fast so we want to rename the container let's use the command sudo docker rename we can take another docker image this says grab this one and we'll put it in rename and we'll rename and put in the old container name which is image and then we'll put in the new container name and let's call it purple so now the container image that had proved something called image is now called purple so we do sudo docker ps to list all of our docker images and if we scroll up and there there we go purple how easy is that to rename an image and we can go ahead and use this command if we want to stop containers so we're going to write sudo docker stop and then we have to put in the container name and we'll put in my docker the container that we originally created and that image has now stopped and let's go ahead and prove that we're going to list out all the docker images and what you see is that it's not listed in the active images it's uh not on the list on the far right hand side but if we go ahead and we can list out all of the docker images so you actually see it's still there as an image it's just not in an active stage it's what's known as in an exit state so here we go and there's my ducker it's in an exit state so that happened 27 seconds ago so if we want to to remove a container we can use the following command so sudo docker rm will remove my docker and that will remove it from the exit state and we're going to go ahead and we're going to double check that and yep yep it's not not listed there under exit state anymore it's gone and there we go there that's where it used to be all right let's go back so if we want to exit a container in the running state so we do sudo kill and then the name of the container i think one of them is called yellow let's just check and see if that's going to kill it oops no i guess we don't have one called yellow so let's find out name of the container that we actually have so pseudo docker kill oh we're gonna list out the ones they're running oh okay there we go now yellow isn't in that list so let's take i know let's take simply learn and so we can actually go ahead and let's write sudo docker kill simply learn and that will actually kill an active docker container boom there we go and we list out all the active containers you can actually see now that they simply learn container is not active anymore and these are all the basic commands for docker container hey everyone welcome to this simply learn video on cubanity's full course in this full course we will learn everything about communities in detail we will begin this full course video with understanding what exactly is kubernetes is next we will see how to install kubernetes and the differences between kubernetes and docker then we will look into the services and components of kubernetes in detail after that we will be discussing everything about the architecture of kubernetes then we will see how communities can be implemented on aws finally we will conclude this video by discussing the important kubernetes interview questions and answers for beginners and experienced candidates by the end of this video i can assure you that all your communities career related queries will have been answered so let's start with an exciting video on the configuration management tool before we begin make sure to subscribe to our youtube channel and hit that bell icon to never miss an update from simplylearn hello and welcome to simplyline in this session we're going to cover what kubernetes is and why you would want to be using it within your devops team but before we get started remember to hit the like button if you like this video and the subscribe button if you want to get notified about more of these videos as they come out and as always if you have any comments about content that we're covering or questions that you have on the subject please post them in the comments below so let's get started we're gonna break up this presentation into four key areas we're gonna talk about life before kubernetes which some of you are probably experiencing right now what is kubernetes the benefits that kubernetes brings to you particularly if you are using containers in a devops environment and then finally we're going to break down the architecture and working infrastructure for kubernetes so you understand what's happening and why the actions are happening the way that they are so let's jump into our first section of life before kubernetes so the way that you have done work in the past where you may be doing work right now it's really building out and deploying solutions into two distinct areas one is a traditional deployment where you're pushing out code to physical servers in a data center and you're managing the operating system and the code that's actually running on each of those servers another environment that you may potentially be using is to buying code out to virtual machines so let's go through and look at the two different types of deployment that you may be experiencing when you have applications running on multiple machines you run into the potential risk that the setup and configuration of each of those machines isn't going to be consistent and your code isn't going to work effectively and there may be issues with uptime and errors within the infrastructure of your entire environment there's going to be problems with resource allocation and you're going to have area issues where applications may be running effectively and not not effectively and not load balance um effectively across the environment the problem that you have with this kind of infrastructure is that it gets very expensive uh you can only install one piece of software one service on one piece of hardware so your hardware is being massively under utilized this is where virtual machines have become really popular with a virtual machine you're able to have better resource utilization and scalability at much less cost and this allows you to be able to run multiple virtual machines on a single piece of hardware the problem is is that vms or for virtual machines are not perfect either some of the challenges you run with vms is that the actual hardware and software need needed to manage the vm environment can be expensive there are security risks with virtual with vms there are security risks with vms there have been data breaches recorded about solutions that run in virtualized environments you also run into an issue of availability and this is largely because you can only have a finite number of virtual machines running on a piece of hardware and this results in limitations and restrictions in the types of environment you want to be running and then finally setting up and managing a virtualized environment is time consuming it can take a lot of time and it can also get very expensive so how about kubernetes well kubernetes is a tool that allows you to manage containerized deployment of solutions and inherently kubernetes is a tool that is really a next level maturity of deployment so if you can think of your maturity curve as deploying code in directly to hardware in a data center and then deploying your solutions to virtual machines the next evolution of that deployment is to use containers and kubernetes so let's kind of go through and look at the differences between a virtual machine and kubernetes we've got a few here that we want to highlight and you'll get an understanding of what the differences are between the two so first of all with virtual machines there is inherently security risks and what you'll find as we get dig through the architecture later in the presentation is that kubernetes is inherently secure um and this is largely because of the legacy code uh the legacy of kubernetes and where it came from we will talk about that in just a moment but kubernetes is inherently secure virtual machines are not easily portable now with that said they they are technically portable they're just not very easily portable whereas with kubernetes it's working with docker container solutions it is extremely portable that means that you can actually spin up and spin down and manage your infrastructure exactly the way that you want it to be managed and scale it on the demands of the customers as they're coming in to use the solution from a time consuming point of view kubernetes is much less time consuming than with a virtual machine a few other areas that we want to kind of highlight from differences virtual machines use much less isolation when building out the encapsulated environment than kubernetes does for instance with a virtual machine you have to run hypervisor on top of the os and hardware and then inside of the virtual machine you also have to have the operating system as well whereas in contrast on a kubernetes environment because it's leveraging a darker container and or container-like technologies it only has to have the os and the hardware and then inside of each container it doesn't need to have that additional os layer it's able to inherit what it needs to be able to run the application this makes the whole solution much more flexible and allows you to run many more containers on a piece of hardware than versus running virtual machines on a single piece of hierarchy so as we highlighted here vms are not as portable as kubernetes and kubernetes is portable directly related to the use of containerization and because kubernetes is built on top of containers it is much less time consuming because you can actually script and automatically allocate resource to nodes within your kubernetes environment because it allows the infrastructure to run much more effectively and much more efficiently so this is why if we look at our evolution of the land of time before kubernetes why we are running into a solution where kubernetes had to come about because the demand for having more highly scalable solutions that are more efficient was just really a natural evolution of this software deployment model that started with pushing out code to physical hardware and then pushing code out to virtual machines and then needing to have a solution much more sophisticated kubernetes would have come about at some point in time i'm just really glad it came back when it did so what is kubernetes let's let's dig into the history of kubernetes and how it came about so in essence kubernetes is an open source platform that is allows you to manage and deploy and maintain groups of containers and the container is something like docker and if you're developing code you're probably already using docker today consider kubernetes as the tool that manages multiple docker environments together now we talk a lot about docker and as a container solution with kubernetes the reality is is that kubernetes can actually use other container tools out there but docker just simply is the most popular container out there both these tools are open source that's why they're so popular and they just allow you to be able to have flexibility in being able to scale up your solutions and they were designed for the post-digital world that we live and exist in today so a little bit of background a little bit of trivia around uh kubernetes so kubernetes was originally a successor to a project at google and the original project was google board um google borg it does exactly what kubernetes and does today but kubernetes was rewritten from the ground up and then released as an open source project in 2014 so that people outside of google could take advantage of the power of kubernetes containerization management tools and today it is managed by the cloud native computing foundation and there are many many companies that support and manage kubernetes so for instance if you're signing up for microsoft azure aws google cloud all of them will leverage kubernetes and it's just become the the de facto tool for managing large groups of containers so let's kind of step through some of the key benefits that you'd experience from kubernetes and so we have nine key benefits and the first it is highly portable and is 100 open source code and this means that you can actually go ahead and contribute to this code project if you want to through github uh the ability to scale up the solution is incredible um what's um the the history of kubernetes being part of a google project for managing the google network and infrastructure kind of really sets the groundwork for having a solution that is highly scalable the out of the high scalability also comes the need for high availability and this is the desire to be able to have a highly efficient and highly energized environment that also you can really rely on so if you're building out a kubernetes management environment you know that it's going to be available for the solutions that you're maintaining and it's really designed for deployment so you can script out the environment and actually have it as part of your devops model so you can scale up and meet the demands of your customer then what you'll find is that the load balancing is extremely efficient and it allows you to distribute the load efficiently across your entire network so your network remains stable and then also the tool allows you to manage the orchestration of your storage so you can have local storage such as an ssd on the hardware that the kubernetes is maintaining or if the kubernetes environment is pulling storage from a public cloud such as azure or aws you can actually go ahead and make that available to your entire system and you can inherit the security that goes back and forth between the cloud environments and one of the things you'll find consistent with kubernetes is that it is designed for a cloud first environment um kubernetes as well is that it's it's really a self-healing environment so if something happens or something fails uh kubernetes will detect that failure and then either restart the process kill the process or replace it and then because of that you also have automated rollouts and drawbacks in case you need to be able to manage the state of the environment and then finally you have automatic bin packaging so you can actually specify the compute power that's being used from cpu and ram for each container so let's dig into the final area which is the actual kubernetes architecture i'm going to cover this at a high level there's actually another video that you can that simply learn has developed which digs deeper into the kubernetes architecture and so the kubernetes architecture is a clustered based architecture and it's really about two key areas you have the kubernetes master which actually controls um and all of the activities within your entire kubernetes infrastructure and then you have nodes um that actually are running on linux machines um outs that are controlled by the master so let's kind of go through some of these um areas so i would look at the kubernetes master and to begin with um then we'll start with uh etc this is a tool that allows for the configuration of information and the management of nodes within your cluster and one of the key features that you'll find with all of the tools that are managed within either a the master environment or within a node is that they are all accessible via the api server and what's interesting about the api server is that it's a restful based infrastructure which means that you can actually secure each connection with ssl and other security models to ensure that your entire infrastructure and the communication going back and forth across your infrastructure is tightly secured scheduler goes ahead and actually as you'd expect it actually um manages the schedule of activities within the actual cluster and then you have the control and the controller is a daemon server that actually manages and pushes out the instructions to all of your nodes so the other tools really are the uh the infrastructure and you can consider them the administration site um of the master whereas controller is the management actually pushes out all of the controls via the api server so let's actually dig into um one of the actual nodes themselves and there are three key areas of the nodes one is the dark environment which actually helps and manage and maintain the container that's actually inside of the node and then you have the kubelet which is responsible for information that goes back and forth and it's going to do most of the conversation with the api server on the actual health of that node and then you have the actual kubernetes proxy which actually runs the services actually inside of the node so as you see all of these infrastructures are extremely lightweight and designed to be very efficient and very available for your infrastructure and so here's a quick recap of the different tools that are available and it really breaks down into two key areas you have your kubernetes master and the kubernetes node now the kubernetes master has the instructions of what's going to happen within your kubernetes infrastructure and then it's going to push out those in structure to an indefinite number of nodes that will allow you to be able to scale up and scale down your solution in a dynamic way hi guys in today's video i am going to show you how to install ubuntu on kubernetes kubernetes is an open source platform used for deployment and management of containers so let's get started now the first step is to install the necessary dependencies using two of the commands first is sudo apt get update which gets all the updates and then you have to enter a password so this will take a few seconds that's done the next step is to install app transport https this is basically used to make repositories via https so let's go ahead with that sudo okay that's done going ahead the next thing we have to do is to install the docker dependency using the command sudo apt install docker dot io you have to choose y and this will take a few minutes okay that's done so after installing the docker we have to start and enable the docker using the command sudo system ctl start docker and sudo system cto enable dock so now the docker is enabled so now we are done with the first step of installation the next step is we have to install the necessary components for kubernetes before that we have to install the curl command because current command is used to send data using a url syntax let's install the curl command using the command sudo apt-get install code so you have to select y and this will take few minutes [Music] so moving on the next step we have to do is download and add key for kubernetes installation from a url so let's go ahead with that sudo curl so this is where we get the key and then we have to add it using sudo apt key add and it's done so the next step is we have to add a repository in a certain location so before doing that we have to change the permission of that file so we do that using c or sudo chmod command okay so the permission is changed now let's go ahead and save this file in that location first we have to enter this command into the file this is a url save as so we have to save it in this location let's rename this to kubernetes dot list now let's see okay so here it is now moving forward now we have to check for any updates available in the next command we are going to install the kubernetes components that is cubelet cubedium cubectel and kubernetes cmi so let's go ahead with that so this is going to take a few minutes now that's done so the next step we have to initialize the master node and to do this we have to first use a swap off command to disable the swapping on other devices so let's do that sudo swap of hyperlink now let's go ahead with the initialization sudo cube adm in it this is going to take a few minutes okay so before going to the next step to start the cluster we have to use these three commands let's just copy paste it okay that's done so moving forward the next step is to deploy paths using the following command [Music] now the pods have been deployed to our network to see all the pods you have to use the command sudo cubic till get bots there you can see the paws that have been deployed hello and in this video we're going to cover a common conversation which is kubernetes versus daca but before we jump into that i want you to hit the subscribe button so you get notified about new content as it gets made available and if you hit the notification button that notification will then pop up on your desktop as a video is published from simply learn in addition if you have any questions on the topic please post them in the comments below we read them and we do reply to them as often as we can so with that said let's jump into kubernetes versus darker so let's go through a couple of scenarios let's do one for kubernetes and then one for docker and we can actually go through and understand what the problem specific companies have actually had and how they're able to use the two different tools to solve them so our first one is with bose and bose had a large catalog of products that kept growing and their infrastructure had to change so the way that they looked at that was actually establishing two primary goals to be able to allow their product groups to be able to easier more easily catch up to the scale of their business so after going through um a number of solutions they ended up coming up with a solution of having kubernetes running their iot platform as a service inside of amazon's aws cloud service and what you'll see with both these products is they're very cloud friendly but here we have um bose and kubernetes working together with aws to be able to scale up and meet the demands of their product catalog and so the result is that we were able to increase the number of non-production deployments significantly by taking the number of services from being large bulky surfaces down to small microservices being able to handle as many as 1250 plus deployments every year an incredible amount of time and value has been opened through the use of kubernetes now let's have a look at daca and see what a similar problem that people would have so uh the problem is with paypal and paypal and processes something in the region of over 200 payments per second across all of their products and paypal doesn't just have paypal they have braintree and venmo so the challenge that paypal was really being given is that they had different architectures which resulted in different maintenance cycles and different deployment times and an overall complexity from having a decades-old architecture with paypal through to a modern architecture with venmo through the use of docker paper was able to unify the application delivery and be able to centralize the management of all of the containers with one existing group the net net result is that paypal was able to migrate over 700 applications into docker enterprise which consists of over 200 000 containers this ultimately opened up a fifty percent increase in availability for being able to add in additional time for building testing and deploying of applications just a huge win for paypal now let's dig into kubernetes and docker so kubernetes is an open source uh platform and it's designed for being able to maintain a large number of containers and what you're gonna find is that your argument for kubernetes versus docker isn't a real argument it's kubernetes and docker working together so kubernetes is able to manage the infrastructure of a containerized environment and docker is the number one container management solution and so with docker you're able to automate the deployment of your applications being able to keep them in a very lightweight environment and being able to create a nice consistent experience so that your developers are working in the same containers that are then also pushed out to production so with docker you're able to manage multiple containers running on the same hardware much more efficiently than you are with a vm environment the productivity around docker is extremely high you're able to keep your applications very isolated uh the configuration for docker is really quick and easy you can be up and running in minutes with docker once you have it installed and running on your development machine or inside of your devops environment so we look at the deployment between the two and the differences kubernetes is really designed for a combination of pods and services in its deployment whereas with docker it's around about deploying services in containers uh so the the difference here is that kubernetes is going to manage the entire environment and then and that environment consisting of pods and inside of a pod you're going to have all of your containers that you're working on and those containers that can control the services that actually power the applications that are being deployed kubernetes is by default an auto scaling solution has it turned on and is always available whereas a docker does not and that's not surprising because docker is a tool for building out solutions whereas kubernetes is about managing your infrastructure kubernetes is going to run health checks on the liveness and readiness of your entire environment so not just one container but tens of thousands of containers whereas docker is going to limit the health check to the services that it's managing within its own containers now i'm not going to kid you kubernetes is quite hard to set up it's it's if of the tools that you're going to be using in your devops environment it's it's not an easy setup for you to use and for this reason you want to really take advantage of the services within azure and other similar cloud environments where they actually will do the setup for you docker in contrast is really easy to set up you can as i mentioned earlier you can be up and running in a few minutes as you would expect the fault tolerance within kubernetes is very high and this is by design because the architecture of kubernetes is built on the same architecture that google uses for managing its entire cloud infrastructure in contrast docker has lower fault tolerance but that's because it's just managing the the services within its own containers what you'll find is that most public cloud providers will provide support for both kubernetes and docker here we've highlighted microsoft azure because they were very quick to jump on and support kubernetes but the reality is is that today google amazon and many other providers are having first level support for kubernetes it's just become extremely popular in a very very short time frame the company's using both kubernetes and docker is fast and every single day there are more and more companies using it and you should be able to look and see whether or not you can add your own company to this list hello and welcome to simply learn today we're going to be giving you a kubernetes tutorial will provide the overview of how and why you should be using kubernetes but before we get started don't forget to hit the subscribe button and the like button if you want to get notified about additional content from simply learn and as always if you have any comments about the material that you're about to see and you have any questions please post them in the comments below we do read them and we do reply to them so with that said let's jump in so we're going to go through a bunch of things today around kubernetes we're going to cover what kubernetes is and why you should be using it along with the features of kubernetes we're also going to take a little bit of a look at kubernetes versus docker swarm as the two technologies do tend to get referred to quite frequently and then we'll do a deep dive into the actual kubernetes architecture itself and how kubernetes runs and operates within your network and then finally we're going to look at some kubernetes use cases of how you can actually apply best learnings that other companies have used with kubernetes we'll finish everything off with a demo where we actually step through and use some of the actual codes that you can actually use to run uh from your terminal window to actually activate kubernetes itself so with that said let's jump in so why kubernetes well kubernetes is really being built from allowing you to be able to have a more reliable infrastructure the tool itself is designed to manage the containers you have within your network it's very modular in architecture which allows it to be very easy to be able to maintain and to be able to scale and meet the demands that your current customers are offering you today the actual ability for you to be able to deploy and update software at scale is really at the heart and center of kubernetes it's really one of the reasons why kubernetes was created was to be able to scale up the deployment of software into tens of thousands of networks and servers uh within a data center and then finally uh when you start working with kubernetes really what you're doing is laying the foundation for building cloud native applications and part of this comes from the architecture and history of kubernetes which we'll get to in a little bit but it's good to know that as you're using kubernetes and containerized solutions that you're really building out an infrastructure that's designed for cloud-centric solution delivery so let's dig into why what kubernetes is so the history of kubernetes is that it was originally developed by google google actually uh used the precursor to kubernetes to manage and maintain the data structure and data infrastructure that they had within their network they really liked it and one of the big benefits that kubernete that google provided is that they converted their code into an open source program that they released as kubernetes in 2014 and made an open source this means that any company can use kubernetes there's no charge for it all you have to do is know how to use the command lines to actually get up and running the benefit of kubernetes being open source is that most cloud providers now support kubernetes so whether you're using microsoft azure you're using google cloud or aws you'll find that you can actually use kubernetes and kubernetes and edit core really helped eliminate manual processes so it allowed for deploying and scaling your container-based solutions much more effortless within a cloud environment so let's jump into the features of kubernetes and why you'd want to be using it and we have a number of features that we want to kind of step through again the first thing that kubernetes is a strong benefit of is that it eliminates those manual processes and allows you to automate them the benefit you have from this is that you can then scale up your automation across your entire network so whether you are managing 10 nodes within your network or whether you're managing 10 000 doesn't matter you have the ability to scale that something you simply didn't have before kubernetes also will manage the actual containers themselves um including the security and the storage and the networking within those containers the great uh benefit of working with kubernetes is that uh not only is it a cloud-centric solution but it was really built from a core need of a secure solution so security is at the heart and center of all the work you're doing which really helps benefit any teams scaling out containers your kubernetes environment will be constantly monitored and so that when you have nodes that have your containers the health of those nodes is being constantly communicated back to the kubernetes environment so that the entire environment is always being managed and checked so if you have any um additional hosting and launching that needs to be done uh kubernetes will help you automate those manual processes uh if something does go wrong you can actually set up your kubernetes environment to do an automatic rollback again this is just allowing you to have that confidence that when you're deploying a solution that everything is going to work within that environment and then finally kubernetes will allow you to mount and add the storage needed to run your apps so whether you have storage such as a local ssd storage device physically on the hardware or if you're connecting to storage via the cloud kubernetes will allow you to access the storage you need how you're accessing it so let's look a little bit into the differences between kubernetes and docker swarm so if we just step back a little bit kubernetes and docker often get connected with each other largely because kubernetes manages containers and docker is really famous for creating the most popular uh container management solution which is called docker if you're a developer you're probably already using docker today so one of the things that kubernetes is really really good at is that it manages containers at scale just massive volume of containers docker swarm is docker's alternative to kubernetes it's a tool that allows you to also manage large number large numbers of containers um there are differences between the two of them and right now without doubt the most popular um open source solution out there is kubernetes from google it just has a massive community behind it and uh the community uh really have embraced um kubernetes um yes the original product was by developed by google but google has been great steward and they've allowed the community to drive the future evolution of the product uh docker swarm is actually managed and developed by docker um there is a smaller community but it is also a very passionate community and the reason being is that both products are actually pretty good and you just really kind of have to choose which one you want to go with without doubt there is more customization and extensions built for kubernetes simply because the community is just so much bigger than the docker swarm community but with that said docker swarm is really easy to set up so if you are used to working within the docker environment you will be able to set up docker swarm very very quickly whereas kubernetes does require some heavy lifting to get up and running now with that said if you are using microsoft azure or google cloud very often those environments will actually set up kubernetes for you so you don't have to do it and you can actually then focus on writing the instructions to maintain your kubernetes environment kubernetes um has extremely high fault tolerance again it's because kubernetes is has been road tested the foundation for kubernetes is based on managing sites such as gmail google.com youtube i mean just incredibly robust environment compared to docker swamp which has lower fault tolerance and and this really kind of leads into the final areas where kubernetes does guarantee strong clusters and management just a really reliable environment dr swarm however is able to take containers and deploy them into large casters very very quickly two interesting differences is that with kubernetes the load balancing is still manual and be interesting to watch how kubernetes matures that skill set uh in contrast load balancing is automatic within docker swarm so let's dig into the kubernetes architecture and just kind of expose some of those key elements um if you're interested in actually having a very deep dive into kubernetes architecture we have already got another video and we should have a link in the comments that you can select that will take you to that video as well but this will provide you an overview as well um so the way that kubernetes runs is that it uses nodes to actually manage all the individual hardware within a specific unit and so a node is really a physical machine in a data center or a virtual machine and when you're setting up an environment on a cloud environment such as microsoft azure or google cloud or aws kubernetes will be managing the containers within that node so it's important to know that kubernetes itself doesn't work with the individual nodes but actually works with the cluster as a whole and and why that's important is that it's a tool to manage your entire environment it's not going to come down and manage the individual node itself it's going to match the entire environment the actual control of data within a cluster um it's very very flexible within kubernetes you can have local storage such as an ssd card or a traditional local storage or you can actually connect to the cloud and use cloud-based storage and the benefit that cloud-based storage offers you is that you are increasingly moving your solution into more of a hardware free environment cloud-based architecture where you're not reliant on a physical piece of hardware within your data center and why that's important is that it gives you the opportunity to scale up your solutions much more effectively so if you need to increase the hard drive space you can do that very easily from the cloud whereas local hardware is a little bit harder to manage great news kubernetes gives you both options to be able to choose from and so the the next layer down is the actual management of the application that's within a container and as mentioned earlier docker is the most popular container solution out there and again if you're a developer you're probably already using docker as it was a natural maturity from using a virtual machine but kubernetes itself will actually manage the containers that you have within your nodes and so you have a collection of containers working together and then you pull those together into what's called a pod and what's great is kubernetes will actually manage the health of that part so if a container or a pod fails kubernetes will automatically deploy out a replica so your entire environment can stay running and being efficient and so the pods themselves will actually be managed by kubernetes and you really want to let kubernetes do this because it's allowing you to step away from the manual management of parts your environment is just going to be healthier relying on kubernetes to do the actual management of all the pods you have out there so you are able to allow access to from services outside of your kubernetes and this is done through a process called ingress and going back earlier we talked about how security is a key part of all the work that goes into kubernetes and this is a great example of where you can use an external service but still have that layer of security that is vital for your solution because to be able to provide the ingress connection kubernetes requires an ssl connection which automatically allows for a secure data transfer absolutely fantastic so here we have an overview of the actual architecture on the right hand side we have our nodes and the pods within our nodes and within those pods are the containers on the left hand side we have the master architecture and the master architecture is the actual brains behind the work that you're doing it's actually managing the schedules the controllers for all of the nodes and then at the bottom left hand side we have commands which are written on a local workstation and those commands are pushed up to the master so let's kind of dig a little bit further into the architecture of the master um so the master itself is comprised of a number of key components and again this is the tool that's going to manage your uh entire kubernetes infrastructure the key components that you have are the etcd control manager scheduler api server and the key four components are etcd or the cluster store the controller manager the scheduler and the api server the cluster store is the tool where all the details of the work that you're doing is stored and it's going to manage the rules going to manage all the instructions and this is where you're going to post that information the controller is actually going to then perform the tasks that are stored um in the cluster store and as you'd expect the schedule is the tool that actually controls when instructions and tasks are pushed out to all the pods and nodes within your network and then finally you have an api server which allows you to be able to control all the operations the actual operations themselves are all instructions written um as rest instructions and which allows you to take advantage of the security that has been built into rest and so you end up with a worker slave architecture and slaves are all the different nodes and pods within your network and so a part itself will have multiple one or more multiple containers and then each container and then each pod will have a docker container a kubelet and a kubernetes proxy and the these are tools to allow you to communicate back up to the master layer so the docker is the actual container itself and this is going to actually run your application and uh the instructions here are to pull down the correct container from the docker images that you have within your library the cubelet is going to manage your containers and your container instructions that will be deployed to the cubelet are going to be written in either yaml or json format yaml is just great if you haven't had a chance to start working with yaml i certainly encourage you to do so it's very easy to pick up and if you know any xml you'll pick up yammer very quickly and finally the kubernetes proxy is the tool that interacts with the master environment within the kubernetes network and allows all the instructions from that are pushed to the kubelet and to docker to be able to perform effectively so let's have a look at some of the companies using kubernetes there are a lot and if you're doing any work today in the cloud and or even if you're just starting off and doing some simple work in the cloud there's a really good chance that you are using kubernetes but some of the companies that you would are probably not surprised at using kubernetes are spotify sap ebay and of course some of the the big companies out there google with their entire network of solutions and microsoft with office and their tools so let's take a a step into a youtube now let's have a look at a kubernetes use case and in this example we're going to look at the new york times and how they use kubernetes to solve a problem that their iit department was struggling with so when new york times started working after their data centers their their actual deployments were small and the applications were managed across vms but the problem is is that as new york times became more of a digital company they started building more digital solutions and at some point they realized that what they needed to do was to embrace the cloud and move all their solutions to the cloud however their first step of using the cloud and this is i think typical for a lot of companies is that they treated the cloud hosted provider in the same way as they would with a data center and they at some point they realized they had to take a change and this is where the development team came in and they said look how about we stop thinking of our solutions as a data center even though they're being run in amazon and start thinking them as cloud first solutions and so what they proposed and what they ended up doing was using google cloud platform with kubernetes as a service um to provide greater efficiency and so by using kubernetes they had the following advantages they're able to deliver solutions at a much faster speed and in the world of digital content speed is everything they were able to reduce the time for deployment from minutes to seconds and by doing this they were able to increase the amount of time that uptime for the actual uh infrastructure um updates began to be deployed independently and when required so you didn't have to schedule an event when you did the deployment it just happened and there was more unified approach to deployment from the engineering staff which just made the whole solution more portable so the fundamental end line for the new york times is that by leveraging kubernetes they were able to automatically manage their deployments and scheduling of solutions instead of having to do solutions through a more traditional ticket-based system it just really speed up the efficiency of the new york times website so let's finally kind of look into some of the important terminologies before we take our time to do a demo so we have six terms that we're going to look through um and uh we'll let you uh uh read through these but uh we have a cluster which is a set of machines physical or vertical in which applications are managed and run we have a node that are the working machines that run containerized applications and other workloads there is the pod which is within a node and is a group of containers that are deployed together on the same host we have replication controllers and a replication controller is used to define a pod life cycle rather than to create the pods directly we have a selector which is an expression matched label to filter specific resources labels are key value pairs that are attached to objects such as parts and the key value pair or the label can be filter organize and perform operations on resources replication sets define how many replicas of each pod will be running and managed when you want to either replace a pod when it dies annotation is a label with much larger data capacity a name is a resource as it is identified volume is the directory which data is accessible to a container namespaces provide additional qualification to resources name and then finally service is an abstraction of a top tier pods which provide a single ip address and dns name by which the pods can be accessed so this time what we want to do is take you through kubernetes demo actually runs through some of these basic controls so i'm going to take you through a video of how to actually install kubernetes um and to use some of the basic instructions so kubernetes is the most widely used platform for container management the first step is to actually install the necessary dependencies so let's see if there's any updates first so the first so to check if there's any updates we use the command sudo app dash get update we're going to enter in our password and this looks like it's going to take a few minutes because we've got a few updates that need to be running all right we're done all right next we're going to write sudo apt-get installed y apt transport https and this will install all the dependencies we need to actually start installing the solutions needed for kubernetes all right that was quick now we're going to put in the docker dependency and the way we do that is write the command sudo apt install docker dot io i'll go typo correct that quickly and put in y so vs and we continue and it's going to take a while because we're going to download and install docker off the internet and it's quite a large installation now would be a good time to go get a cup of coffee because it'll take us about a minute to get this all done once we actually have docker installed we'll let you go ahead and we'll start docker and enable docker using our sudo commands all right so that's all done so let's start to enable docker we're using so we're going to use sudo system ctl start docker and sudo system ctl enable docker this will be quite quick so the next step is to install all the components for kubernetes and we're going to install the curl command and then kubelet cube c and then cube adm and we're going to install all the necessary tools we'll do the curl command cubelet cubic c2 and cube adm and so let's go ahead and put in the sudo apt-get install curl we're going to get a prompt we can say yes for that and then after this we'll go ahead and install kubernetes so the first what we're going to do is we're going to actually put in the kubernetes key so that's going to be pseudo curl dash s and then we have long url which is https packages.cloudgoogle.com apt doc apt key dot gpg sudo apt dash key there we go and create my typo there okay now we have to go ahead and change permissions of like folder so we do that with sudo chmod 777 slash etc apt apt sources.list.d all right we're good okay now you want to go ahead and open up your text editor and we're going to make um an actual edit and we're going to create a simple file and so what we're going to type in is then the following command db http apt dot kubernetes dot io slash kubernetes xenial main and we're going to save this file in the etc folder and then look for and then apt which is going to be down a little bit and there we have it and we're going to save it in the apt-get updates and we're going to name this file as kubernetes kubernetes.list and we'll save that and then we can close out the notepad and go back to the terminal window and let's look and see if there are any updates again so we're going to go ahead and run apt-get update and it's going to run any updates from what we've created and we want to check what's in the folder and so we do cat slash accept etc ept sources dot d slash kubernetes dot list and there we are it shows what we have in that text file and so we're gonna go ahead and we're going to uh install uh cubelet cubic cubic 2 and cube adm but first we're going to have to initialize the master i'm going to show you what an error you would get if you were to install cubelet um and the other files too quickly you get the following error all right so we have installed cubelet qbadm cubex tool and kubernetes cni and so we're going to go ahead and use sudo swap dash hyphen a and now we're going to initialize the master node and we're going to do that with sudo cube adm in it and this will take a few minutes for it to initialize all right so that's all done all right so we're going to deploy the pod to a network and we have two commands that we're going to use we use sudo kubu cube ctl apply and then we have a long pathway to type out which is dash f https raw githubs content.com choreos final master documentation cube dash final dot yml what we we have to do is we have to actually go ahead and deploy the nodes first so we're going to do that by now creating mktr p and then we have dollar sign home dash dot cube i'll create that new directory and sudo cp dash i and then the path will be etc slash kubernetes admin dot conf and then dollar sign home slash dot cube dot slash config and then final one will be sudo chow n dollar sign open bran id dash u close paren colon dollar sign a brand id slash dash g close paren dollar sign home slash dot cube config all right now we're going to go ahead and deploy and uh let's see uh oh yeah i think i have a typo in my command here yep oh yeah i did have a typo and so um i've got that incorrect now so documentation is actually spelled with a t not an s and then we're going to go ahead and we're going to put in the next line which is pseudo cubic uh cube ctl apply uh dash f and then we're just going to change a little bit of the actual link here and that runs great and now we want to be able to see all of the parts that we have in our network and so we're going to do that by uh writing the command sudo tcl uh get pods dash all dash dash name spaces and it should list all of our pods we only have one there we are there's all of our content now we're going to so there's only one node so let's see what we have here so we're going to use the command sudo you know cubic tl get nodes there we are just one node so our final step is to actually go ahead and deploy a service and so what we're going to go ahead is deploy the nginx service and we're going to do that with the command sudo cubex ctl run dash dash image equals nginx and we're going to put this into port 80. so it's going to be nginx nash app dash dash port equals 80 environment emd equals uh ubun cone open quotes so domain equals cluster close quotes and we're going to write one more command which is pseudo cube ctl expose deployment nginx dash app dash dash port equals 80 name equals nginx http and once we've done this we will be able to then run the command to actually see our services and there'll be quite a few of them and our final command is sudo docker ps dash a and we have quite a lot of lists quite a lot of services listed so there you are that's the instructions on how to get docker up and running and uh with some basic commands um it it's worth going through a few times to get right um but once you actually have um the kubernetes environment up and running and managing your docker environment in containers it's extremely powerful if getting your learning started is half the battle what if you could do that for free visit scale up by simply learn click on the link in the description to know more welcome to kubernetes architecture from simply land we're going to take a deep dive into the kubernetes architecture if you're new to kubernetes we already have an introduction that goes and provides an overview of kubernetes and we have the link below so you can go straight to that if you want to have a refresher otherwise hang around and join us in this presentation and if you want to see more of these videos hit the subscribe button and if you want to get notified hit the notification bell and every time we post up a new video you'll get the notification right onto your phone or on your desktop that the new video has been uploaded and as always if you have any questions please post them in the comments below we do read the questions and we try to respond to all of them as quickly as we can with that said let's jump into kubernetes architecture so we're going to break out the presentation into the following areas we're going to cover what kubernetes is and why you want to be using it we're going to introduce the actual architecture and provide an overview of how it contains the containers and the other components that you'd have within the architecture we'll also compare kubernetes with docker swarm and one of the things that you hear with kubernetes almost in the same um breath as darker and we have to be careful not to confuse daca from a container point of view and docker swarm which is a container management tool and kubernetes is also a container management tool so compare the two of those and then we'll look at the hardware and software components and then finally do a deep dive into the architecture of kubernetes and provide a case study of where kubernetes has been used successfully in the past so let's jump into this so what actually is kubernetes so kubernetes is a tool that was designed to actually help manage and contain large groups of containers it was developed originally by google out of the google borg solution and then open source by google the actual environment allows you to manage large groups of containers so if you're a developer in devops and you're working with docker then you're used to the concept of the container and you'll also know that containers and cloud technologies tend to be almost breathed in the same breath so the work that you're doing with kubernetes is to be able to manage large groups of containers inside of the cloud the thing that's great about working with kubernetes is that it is incredibly flexible and allows you to have incredibly complex applications run efficient efficiently so let's step into why you'd want to use kubernetes so a couple of key points uh kubernetes itself is an open source solution originally developed by google but it is available on most cloud platforms today so aws google cloud microsoft azure all of them support kubernetes and what you'll find is as you're setting up your infrastructure particularly if you're using containers you'll see that the support for kubernetes is floated up very very efficiently and effectively by all three vendors it's extremely um useful for being able to manage large modular environments and that's really kind of these i think one of the big benefits of kubernetes is its modularity is that it really breaks down containers into their smaller parts and once you do that it becomes much more efficient for you as an administrator to be able to manage that entire entire environment the reproductibility of kubernetes is extremely high you can build up infrastructures very quickly and um have them being able to manage and have containers coming on and off and killed and uh created to be able to help low balance your entire environment and so again you know we keep talking about containers but that's really what kubernetes is all about is being able to manage your applications that are managed through containers and being able to do that in a virtualized infrastructure and the thing that's also really good with kubernetes is it's really easy to deploy solutions you just have to use a simple curl call and you can actually push out your kubernetes infrastructure so let's have an overview of the kubernetes architecture so kubernetes is really broken up into three key areas you have your workstation where you develop your commands and you push out those commands to your master and the master is comprised of four key areas um which essentially control all of your nodes and and the node contains multiple pods and each pod has your docker container built into it so consider that you could have a really almost an infinite number of pods sorry infinite number of nodes are being managed by the master environment so you have your cluster which is a collection of servers that um maintain the availability and the compute power such as ram cpu and disk utilization and you have the master which is really components that control and schedule the activities of your network and then you have the node which actually hosts the actual docker virtual machine itself um and be able to actually control and communicate back to the master the health of that pod and we'll get into more detail in the architecture later in the presentation so you know you keep hearing me talk about containers but they really are the center of the work that you're doing with kubernetes and can the concept around kubernetes and containers is really just a natural evolution of where we've been with internet and digital technologies over the last uh 10-15 years so before kubernetes you had tools where you're either running virtual machines or you're running uh data centers that had to maintain and manage and notify you of any interruptions in your network kubernetes is the tool that actually comes in and helps address those interruptions and manages them for you so the solution to this is the use of containers so uh you can think of containers as that natural evolution from you know uh 15 20 years ago you would have written your code and posted it to a data center and more recently you probably post your code to a virtual machine and then move the virtual machine and now you actually just work directly into a container and everything is self-contained and can be pushed out to your environment and the thing that's great about containers they're they're isolated environments very easy for developers to work in them but it's also really easy for operations teams to be able to move a container into production so let's kind of step and back and look at a competing product to kubernetes which is docker swarm now one of the things that we have to remember is that uh docker um containers which are extremely popular um built by the company docker and made open source and docker actually has other products one of those other products is dr swann and docker swarm is a tool that allows you to be able to manage multiple containers so if we look at some of the uh the benefits of using docker swarm versus kubernetes and one of the things that you'll find is that both tools have strengths and weaknesses but it's really good that they're both out there because it helps keep it really kind of justifies uh the importance of having these kind of tools so a kubernetes was designed originally from the ground up to be auto scaling whereas docker swarm isn't the load balancing is automatic on docker swan whereas with kubernetes you have to manually configure load balancing across your nodes the installation for docker swarm is really fast and easy i mean you can be up and running within minutes a kubernetes takes a little bit more time is a little bit more complicated eventually you'll get there um i mean it's not like it's going to take you days and weeks but it's it is a tool that's when you compare the two docker swarms much easier to get up and running now what's interesting is that kubernetes is incredibly scalable and it's you know that's its real strength is its ability to have strong clusters whereas with docker swarms cluster stream isn't um as strong when compared to kubernetes now you compare it to anything else on the market it's really good um so this is kind of a splitting hairs kind of comparison um but kubernetes really does have the advantage here if you're looking at the two compared to each other for scalability i mean kubernetes was designed for by google to scale up and support uh google cloud network infrastructure they both allow you to be able to share um storage volumes with docker you can actually do it with any container with that is managed by the docker swamp whereas with kubernetes it manages the storage with the podge and a product can have multiple containers within it but you can't take it down to the level of the container interestingly uh kubernetes does have a graphical user interface um for being able to control and manage uh the environment the reality however is that you're likely to be using terminal uh to actually make the controls and commands to control your either docker swarm or kubernetes environment and it's great that it has a gui and to get you started but once you're up and running you're going to be using terminal window for those fast quick administrative controls that you need to make so let's look at the hardware components for kubernetes so and what's interesting is that kubernetes is extremely light of all the systems that we're looking at is extremely lightweight it allows you to have if you compare it to like a virtual machine which is very heavy and you know kubernetes is extremely lightweight and harder uses any resources at all interesting enough though is that if you are looking at the usage of cpu it's better to actually take it for the cluster as a whole rather than individual nodes because the nodes will actually combine together to give you that whole compute power again this is why kubernetes works really well in the cloud where you can do that kind of activity rather than if you're running in your own data center um so you can have persistent volumes such as a local ssd or you can actually attach to a cloud data storage again kubernetes is really designed for the cloud i would encourage you to use cloud storage wherever possible rather than relying on physical storage uh the reason being is that if you connect to cloud storage and you need to flex your your storage the cloud will do that for you i mean that's just an inherent part of why you'd have cloud storage whereas if you're connecting to physical storage you're always restricted to the limitations of the physical hardware so let's kind of pivot and look at the software components as compared to the hardware components so the main part of the components is the actual container and all of the software running in the container runs on linux so if you have docker installed as a developer on your machine it's actually running inside of linux and that's what makes it so lightweight and really one of the things that you'll find is that most data centers and cloud providers now are running predominantly on linux inside of the um the the container itself is then managed inside of a pod and a pod is really just a group of containers bundled together and the kubernetes scheduler and proxy server then actually manage what how the pods are actually pushed out into your kubernetes environment the the pods themselves can actually then share resources both networking and storage so parts aren't um pushed out manually they're actually managed through a layer of abstraction and part of their deployment and and this is the strength of kubernetes you use um to find your infrastructure and then kubernetes will then manage it for you and there isn't that problem of um manual management of pods if you have to manage the deployment of them and you that's simply taken away and it's completely automated and the the final area of software services is on ingress and this is really the secure way of being able to have communication from outside of the cluster and passing of information into that cluster and again this is done securely through ssl layers and allows you to ensure that security is at the center of the work that you have within your kubernetes environment so let's dive now into the actual architecture before we start looking at a use case of how kubernetes is being employed so kubernetes again is um we looked at this uh diagram at the beginning of the presentation and there are really three key areas there's the workstation where you develop your commands and then you have your master environment which uh controls the scheduling the communication um and the actual commands that you have created and pushes those out and manages the health of your entire node network and each node has various pods so we like break this down so the master node is the most vital component and with the master you have four key controls you have etc controller manager scheduler and api server the cluster store etc this actually measures the details and values that you've developed on your local workstation and then we'll work with the output control schedule and api server to communicate that out those instructions of how your infrastructure should look like to your entire network uh the control manager is really an api server and again um this is all about security so we use restful apis which can be packaged in ssl to communicate back and forth um across your uh pods and the master and indeed the services within each of them as well so um at every single layer of abstraction uh the communication is secure uh the schedule as the name would imply really schedules um when tasks get sent out to the actual nodes themselves uh the nodes themselves are are dumb nodes they just have uh the applications um running on them the master and the scam is really doing all of the work uh to make sure that your entire network is running um efficiently and then you have the api server which has your rest commands and the communication and back and forth across your networks that is secure and efficient so your node environment is where all the work that you do with your containers gets pushed out too so a work is really a it's a combination of containers and each container will then logically run together on that node so you'd have a collection of containers on a node that all make logical sense to have together within each node you have docker and this is your isolated environment for running your container uh you have your kubelet which is a service for conveying information back and forth um to the service about the actual health of the kubernetes node itself and then finally you have the proxy server and the proxy server is able to manage the nodes the volumes the the creation of new containers um and actually helps pass the communicate the health of the container back up to the master to see whether or not the container should be either killed stop started or um updated so finally let's look and see where kubernetes is being used by other companies so you know kubernetes is being used by a lot of companies and they're really using it to help manage complex existing systems so that they can have a greater performance and with the end goal of being able to delight the customer increase value to the customer and enhance increased value and revenue into the organization so example of this is a company called blackrock where they actually went through the process of implementing kubernetes uh so they so blackrock had a challenge where they needed to be able to have much more dynamic access to their resources uh they were running complex installations on people's desktops and it was just really really difficult to be able to manage their entire infrastructure and so they actually went and pivoted to using kubernetes and this allowed them to be able to be much more scalable and expansive in the in the management of their infrastructure and as you can imagine kubernetes was then hooked into their entire existing system and has really become a key part of the success that blackrock is now experiencing of a very stable infrastructure and the bottom line is that blackrock is now able to have confidence in their infrastructure and be able to give that confidence as back to their customers through the implementation and more rapid deployment of additional features and services kubernetes specific to aws cloud platform so what's a part of this tutorial and demo what's in store for you at offset i would like to cover the basics of orchestration tools as most of you would know kubernetes is one of the most popular orchestration tools in recent times specifically for applications which are cloud native and deployed on some or the other types of containers but what are these organization tools why would one need to use orchestration tools what are the facilities or features provided by these orchestration tools that's the first thing that i'm going to cover after that i will pick two orchestration tools specific to container management docker swarm versus kubuntis i'm going to compare them with regard to the features that they provide the use cases that they cover what facilities that they provide and when to use what after that i will get into a little bit details of the kuvurnis architecture what exactly is required for setting up a kubernetes cluster what runs in a control plane or a master node what runs as a part of the worker node after that i will end the tutorial by running you through a demo by setting up a three node kubernetes cluster on aws platform i will use something called as cops which is one of the admin tools for setting up production grade kubernetes cluster so i will use this to set up a three node cluster on aws all right now that we set the context right let me get started so what are orchestration tools the application development and the lifecycle management from the time the application is developed connecting the source code to your continuous integration to testing them all along your development process and eventually moving it down to production managing your production servers the dependencies that your software has the requirement that it has in terms of the hardware the features of fault tolerance self-healing capabilities auto scaling all this has complicated over the last few years as a part of devops one thing that everyone is interested in is managing all this application dependency or lifecycle management using some or the other kind of a tool so that's where these orchestration tools have become very very popular so what kind of features that they provide is you'll have to just let them know what are the kinds of tool sets that is required what are the fault tolerance mechanisms that it has to be adopted to what kind of a self-healing capabilities that application will have to need and if at all there is any auto scaling features that is required if at all you can bake in all these specific parameters into your tool your organization tool becomes a one-stop shop for all your deployment and configuration management needs that's where this orchestration tools have become a very very popular and relevant specifically these days when people are more and more interested in adopting devops practices all right so having said that about orchestration tools let's concentrate on two of these orchestration tools specifically for container management docker swamp and kubernetes many of the applications that are written for these kind of containers are called or kind of fall into a category of cloud native where the application need not know much about the underlying infrastructure or the platform where these applications are running so these two along with apache missions there are many other container management systems but i'm going to pick docker swarm and kubernetes for comparing the features that is provided by these two frameworks for the sake of comparison i picked up docker swarm and kubuntes just because of the reason that both of them operate in the space of container management they do something very very similar to containers that's the similarity that exists between these two orchestration tools however there's a big difference that exists when it comes to the types of containers that both of them cater to and also the capacity of workloads they can be used for docker swamp is a cluster management solution that is provided by docker container docker is one of the very very popular containers of recent times and in case you have your applications that is totally powered by only docker containers if you have a need where you want to run a cluster of servers which are running only docker containers docker swarm should be your choice for your orchestration tool kubernetes on the other hand can cater to any other types of containers other than docker and including docker as well so in case you have your applications which have got docker in them which have got rkt in them which they have got alex in them or any other type of container kubernetes should be your choice of orchestration too a docker swarm can manage up to 40 50 or 60 max nodes so in case your application is totally written in docker containers and the load or the expected cluster size is over 50 60 nodes docker swarm should be your choice of orchestration tool on the other hand kubernetes is something that was open sourced by google and the kind of the scale of operations that kubernetes can cater to is google scale so if in case your applications are more than a thousand nodes that is where kubernetes comes into play that's the big difference that exists between docker swarm and kubernetes putting those differences aside let me compare docker swarm and kubernetes based upon other features which are similar in nature the first and important feature that kubernetes provides is something called as auto scaling if at all the load on your cluster is too high and your application is experiencing more load so kubernetes can add new nodes to your cluster of course you'll have to configure kubernetes in order to have those capabilities where it can spin up new vms or new nodes if at all you do that configuration correctly kubernetes has got the capacity or it has got the feature where it can bring up a new node and add it to the cluster on the fly based upon the load that exist at that moment on similar lines if at all the load is not too high it can identify a few of those nodes which have got less number of replicas or less number of application containers running on it it can move them to some other node and also scale down by deleting few nodes from your cluster so that is the powerfulness of auto scaling which is provided by kubernetes and this unfortunately does not exist in docker swamp the other feature of load balancing specifically application node balancing so docker swarm gives you an application level auto load balancing however kubernetes gives you the flexibility of manually configuring any other type of load balancer of your choice for application load balancing installation as i mentioned earlier docker swarm is a loose cluster of containers which are running on nodes it is very easy to spin up a new node and then connect them to a swarm and this can be done in a very very loosely coupled way where you can create swarms of your choice nodes are allowed to connect to swarms and quit or leave the swamps on the fly so in and all the installation is pretty easy and fast kubernetes on the other hand the configuration of kubernetes is the way to spin up a big cluster specifying the size of the node how many masternodes how many config planes that you want how many nodes that you want it's a pretty tough thing to bring up a kubernetes scalability kubernetes strength is very very strong they are very tightly coupled and even they have the capability where on the cluster size things or the nodes can be increased on the fly based upon the requirement on a darkest form the cluster strength is weak as compared to cobra days worker nodes can be added to a swarm worker nodes can be asked to leave us form or can be taken out of a swarm based upon the requirement so this is kind of a loosely coupled architecture for docker swarm while kubernetes the cluster is very tightly coupled since docker swarm runs only docker containers containers find it very easy to share data and lot of other things with each other because they all have the same signature they are all from the same family so it's very easy for them to share not just volumes but lot of other things however for kubernetes since it manages containers of different types if your application has to share some data across different containers there's a little bit of a complexity in how you would want your containers to share data also uh when it comes to kubernetes kubernetes groups containers in something called as pods while a pod can have either one container that's the preferred choice or multiple containers and the idea of pod is like each part can run in any other node it's not guaranteed that you would want to have two or three parts to be running on the same node that makes data sharing a little bit of a different thing compared to docker's form uh gui so there's not a good enough ui tool in docker's form at least the ce edition of it which will allow you to get to know what is running on your containers what is the size of the containers what is the volume of the containers and all that stuff there are some free and open source tools like pertainer which gives you a good visibility into your running docker containers however there's nothing at a swarm level that is provided by docker kubernetes gives you an out of the box dashboard which is easy to configure and set up you can also club it with some metrics services and you can also get information about the size of the cluster that is running what is the load on the nodes and stuff like that all this across your cluster in a very beautiful dashboard perspective so that way um this is little bit of a good ui for kubernetes while for docker's form there isn't anything that is provided out of the box let me spend some time in explaining a little bit about the kubernetes architecture what comprises of the kubernetes cluster what resides in a master or a control plane and what resides in a worker node this is a very high level depiction of a kubernetes cluster so this is the master node which has got something called as a cluster store a controller a scheduler component and an api server and these are the bunch of nodes that are connected or administered by the master node the master node can be one three five typically an odd number this is as per the typical cluster management thing where you would need the master nodes to be in odd numbers so that whenever there's any contention in terms of what needs to be deployed where or where to give the job to whom and stuff like that all the masters would cast their vote and they would decide on the outcome so that's the master node and there are a bunch of nodes which can be connected and administered by the master node cuba ctl or the command from using which anybody can assign some workloads to the cluster can also be run either on the same node or on a separate node so cubectl is the command line tool that we will be installing and using in order to fire up our commands to our cluster so if at all i have to put up a job on our cluster if i have to give out any specific commands to my cluster all this is done using cubicctl there's a bunch of rest apis which is used by cuba ctl and cubectl will talk to the api server and fire up the commands specific to my cluster what runs in the master node master node without saying it's the most important component of any of the cluster if at all you're running um cluster with only one master node if your master node goes down there's no other way that you know you can or any user can talk to the different nodes in the cluster so master node is the most vital component responsible for the complete kubernetes cluster there's always one node that should be running as a master node that's the bare minimum requirement for your cluster so what are the other components in the master node the most important one is called etcd so this is nothing but a data store a value of key value pair which is stored which contains all information about your cluster so all the configuration details which node is up which worker node is down all this information is stored in the cluster store so all the managers would access this cluster store before they go ahead and decide any other work item or anything else that has to be done specific to your cluster what's in the controller as the name says controller is something it's like a daemon server that is running in a continuous loop all the time it is responsible for controlling the set of replicas the kind of workloads that is running based upon the configuration that the user has set in so if at all if you are aware of something called as replication controllers endpoint controllers namespace controllers all these controllers are managed by the controller component if any user asks for some three replicas of a particular pod or a service to be running and if at all any of the nodes goes down or the part goes down for whatever reason the controller is the one who wakes up and assigns this particular job or the part to some other available node by looking up for the details using in the cluster store that's the importance of the control manager or the controller the scheduler the scheduler assigns the tasks based upon whoever is asked for any job to be scheduled based upon a time frame or based upon some criteria it also tracks the working load as to what what exactly is the load who is running what and in the cluster and places the workload on whoever is the available resource at that time all right epa server this is one other important component in in our kubernetes cluster where how would the end user deploy or give out any sort of a workload onto your cluster all the requests come to the api server so this is an entry point for all your requests that come into your cluster anybody wants to deploy something anybody want to scale up a controller anybody who wants to bring down a few services anybody wants to put in a service all this will have to come in as a part of a rest api endpoint and api server is the entry point in case you don't want uh you know somebody to access your cluster in case you want only specific people or specific users to be running some specific workloads you can set all those role based access control for this apa server so this is the entry point for anyone who wants to submit any job to your cluster so that's a quick overview about the master node what what are the important components of a master node now let's go over to what runs in a worker or a slave node as i mentioned earlier slave nodes are something where the job that is submitted to your cluster is eventually run as a pod as a service as a container so in kubernetes world there's something called as a pod a pod is nothing but a combination of a container it is a wrapper around your running containers so sleeve nodes or the worker nodes typically run these parts so that is where the whole workload typically gets run but there are a bunch of other components in the in the node which also manages what runs in the pod who has to have access to the pod what is the state of the pod is it in a running state is it going down for some reason and all the stuff so let's quickly go over those components that is there in the slave node the most important component in my opinion that should be on the on the node should be the container runtime as i mentioned earlier kubernetes can run any different types of containers not just docker so in case you want to have a node which wants to have docker running on it rkt running or lkc running on it or any other container environment that is running on it you have to ensure that the specific container runtime environment is available on that specific node so that whenever a job is submitted a description of what needs to be run as a part of the pod what should be the image with it should be powered and what kind of a container to spin up so that's what you know when the job is finally assigned to this particular node it would definitely need a container runtime to be up and running so that exists only if at all the container runtime is installed and running on your kubernetes worker node okay now that we know what our kubernetes node can run how would somebody assign a job to our node that is using something called as a cubelet as the name says cubelet or cubelet is a small subset which talks to the cube api server so any node which has to run any kind of a pod all these instructions are passed around by the qbps server to the cubelet and cubelet is capable of processing whatever job that is assigned to it and ensuring that so many parts and so many services are spun up based upon the requirement the last component that exists in the worker node is the cube proxy or kubernetes proxy this plays a very very important role acting as a load balancer and a network proxy so what typically happens is whenever the pods are running in nodes the parts can be typically running in any node there is no um affinity towards any node on which these parts are running because pods or containers are something called as ephemeral they can run anywhere so how would somebody reach out to these applications that is running in some pod which is running in one container now and running in probably another container another node altogether tomorrow so that is where your kubernetes proxy contains a picture and this component will ensure that any container that is spun up or any pod that is spun up it keeps track of all these parts and kind of connects the end points or acts like a dns server so that it knows when somebody is trying to reach out to this particular service which is the part on which the service is typically running so that plays a very very important role by acting as a load balancer and a network proxy now that at a very high level we know what are all the components that make up of our kubuntu's cluster let's go ahead and create a small cluster and spin up this cluster on aws platform finally we get to the demo section of my tutorial now that you guys are aware as to what is kubernetes what is the use of kubernetes you also know at a very very high level what are the components of a cluster what is a master node or a control plane what are the components of a master node and what should be existing in a worker node you're probably thinking that it's maybe pretty hard to kind of set this up so let me demystify that by running you with a quick demo of how to set up a simple free node kubernetes cluster using a tool called cops and i will be doing this whole setup on my aws cloud okay so what is cops cops is nothing but a simple tool this is an admin tool that allows you to bring up production grade kubernetes environments as i said setting up a kubernetes cluster on a bare metal is little challenging so that is why i would use something called as a cops and i will use this on my cluster which i'll spin up on my aws instance so i have an aws account and what i'll do first is i will first spin up an ec2 server i will power this with one of the linux amis and i will install cops on it now this will be my starting point from where i'm going to trigger running up a big cluster in our case i'll try to keep it little small i don't want too many nodes in my cluster i'm going to have one master node and two worker nodes this whole operation would be done programmatically by installing this cops admin tool so as you may know aws is little hard for me to programmatically run other components or rather bring up servers it's not pretty simple so what i would do is i will create an iam role which will attach to my ec2 server so that my ec2 server gets powered with all the required role so that it can go ahead and spin up cluster based upon my configuration that i'm going to specify all right so once i have my cluster set up i will also install cubicctl which you would probably know by now is nothing but a command line utility using which i can kind of connect to my cluster give it some jobs put some parts and stuff like that so i will use cube ctl i will also have a pair of ssh keys so that from this machine i would be able to successfully get onto the master node and submit jobs on my behalf so let me begin by first logging into my aws account all the steps that i will be following to bring up my aws cluster kubernetes cluster will be documented and i'll be sharing you this document in a github repository so that in case anybody wants to try you'll be more than happy to find all the required instructions in one place okay now the first thing that i would need is to launch an aws ec2 instance now this is an instance where i'm going to install cops and i will use it as a base server from where i'm going to find some commands which will bring up my kubernetes cluster so let me log into my aws account and um let me stick to one particular region and let me bring up some a one ec2 instance all right so let me launch my instance let me choose um doesn't matter in the ways but in case you want to try it on your free tire i would recommend choose free tire only and choose an ami of your choice let me choose this instance i will make it t2 micro that is good for me configuration instance details all right i don't want to change anything here that looks good maybe i will change this 30 gig that's good for me let me add a tag to my instance i will name this as my cops server all right i would need a bunch of posts to be opened up so what i'll do is i'll create a new security group called open to the word i would not recommend that you do this since i don't want to it will take a long time for me to specifically to pick and choose the ports that has to be open for running my cluster i don't want to do that i will open up all the http and https ports so that it's quicker so i will say all tcp i would say from anywhere just to be on the safer side also open http and https and i will make this anywhere and anywhere this is good for me i will say review and launch all right so i chose a t2 micro instance just for the sake of easiness i've opened up all the ports and um the instant details are you've seen what is that i've chosen all right now important part is like i would need a key pair i need to create a new keypad i will call this as simply learn keys i will download this keypair and then i will launch the instance it will take some time for my ec2 instance to come up in the meanwhile what i'll do is i will convert i have this pen file which is the pair of keys ssh keys with which i need to log into my ec2 instance so i'll need to convert this because i'm trying to connect from my windows box i would need to convert this into a ppk file so i have puttygen i'm going to load my set of keys here and convert that key into a ppk key all right open all right i would say save hang on save private key yes i would say simply learn or simply learn private private key i will save this here done that that's all so this is my public key which is the pem file and this is my ppk or my private key now using the private key i will log into my ec2 instance okay now my ec2 instance is up and running it is running in this ohio region and this is the ip address of my public eye pedals of my machine so let me connect to it so i use a moba x term which is nothing but an ssh emulator you can use putty or any of these sessions which allow you to do an ssh this is my ip address of my machine and since i've spun up an amazon ami the user this is the default user i would need to specify the private keys for my session and this is the private key for my session say okay i am in my amazon ec2 instance so let me look at what are the other things that i need to do all right so as i said i just brought up my ec2 instance but i would need my ec2 instance to run few things on my behalf so that it can spin up easy to other ec2 instances it can also talk to an s3 bucket where i'm going to store the instant state of my kubernetes cluster also some sort of an auto scaling group because i want to spin up more and more instances i also want to have a private hosted zone using my route 53 so i would need my ec2 instance to have all these sort of permissions for my server so what i would do is i will go and create a permission a rule in the ec2 rather in the aws im role and i will attach this im role to my ec2 instance all right so let me go to my iam which is nothing but the identity and access management i will create a rule called as possibly corpse roll i will create a role this role will be used by my ec2 so i'm going to click on ec2 and say next permissions they would need a lot of permissions uh specific permissions to be very honest and the permissions are all listed out here s3 ec2 and all this stuff just to keep it pretty simple what i would do is i'll create a role with the administrative access all right so i don't want any tags i will review i'll create a role name i will say cops fold for easy two all right so i'm going to create a role which has got administrative access rules i'm going to create a role so this would ensure that my ec2 instance from which i'm going to run my cluster would have all the prerequisite permissions so that it can go and spin up instances talk to s3 buckets and all that stuff all right now let us have our running instance get powered by this role so i will connect this role to my running instance all right so this is my easy to instant that is running i'll say action attach attach attach attach replace im rules so there is no rule as of now i would want my corpse roll for this role that i created so i want this new room to be assigned to my ec2 all right great now my ec2 instance which is my cop server has got all the required permissions so that he can create a cluster on my behalf great now before this let me do a sudo m update iphone y so that any of the because this is a newly provisioned vm i just want to ensure that all my library system libraries are all updated so that when i do install cop and cube ctl and all those things none of them will fail because of some dependency or package issues so i'm just running a sudo update iphone y so that all the libraries are updated okay looks like it is done so let me go ahead and uh install cops cups is a pretty simple installation it is available in a particular location i have to just copy this and ensure that i do a curl and i install this particular stuff all right so it is fetching uh the corpse tool it's installing for me once it is copied down let me change the mode so that i can execute it and then also let me move it to my user local bin so that it is available for me in my path okay that's pretty much the corpse installation let me also do one other thing let me install something called as cuba ctl so this is what would be you know a tool using which i'm going to be firing my commands to my kubernetes cluster once it comes up right this is a pretty smaller executable so ctl all right so i have cube ctl as well as cops installed on my ec2 server now okay now what i'll do next is i'm going to create an s3 bucket in aws so that the kubernetes cluster state is persisted in this bucket so let me create a simple bucket with this name s3 bucket with this name so let me go to s3 and let me create a simple bucket with this name i will just say create a bucket okay so the simply learn kubernetes is the bucket that i created so this bucket will store all information about my cluster let me now go and create a dns entry or a dns zone or aws route 53 hosted zone for my cluster so this is required so that you know i can give a cluster name for my kubernetes cluster that i'm coming up so i will create a private hosted zone and possibility possibly i will call this as simply learn dot in so this will be my name of my private hosted zone in case you already have any other public hosted zones in your name you can always put a route 53 hosted zone specific for your domain name since i don't have anyone i'm just going to create a simple private hosted zone so let me head down to route 53 click on any of these and you'll get these hosted zones out here so i'm going to create a hosted zone here i'm going to create a hosted zone click on create hosted zone um this would be simply learn lot in i want to create a public no i don't want public i want a private hosted zone and i'm going to associate my vpc id for this hosted zone since i'm going to try out all my exercises in the ohio region i will associate the vpc of the ohio region for this all right so this is the one that is specific to the ohio region so i will say this one i'll say create a hosted song great now let me come back to my ec2 box my ec2 is all powered with whatever it needs in order to go ahead with the installation of my cluster only few things that i need to take care of is that you know now that i put a name for my cluster i also have an s3 bucket that i've configured for my cluster i will have to ensure that i need to put in these two configurations as a part of my cluster building activity so i will open my bash rc file and i'm going to export these two variables if at all you remember well these two variables are nothing but the cluster name which is nothing but the public uh sorry the private hosted zone that i created and the s3 bucket where i'm going to create or rather i'm going to store my cluster state so let me copy these two and open up my bash rc file all right so i will just add these two and copy these and export them out as my variable i'm going to save this here now let me ensure that this gets picked up all right i've got these two that have configured and i also ensure that these environment variables are set now let me create a pair of ssh keys this is to ensure that i can log into the box that i'm going to be provisioning as a part of my cluster ssh hyphen keygen i don't want to give any parameters let it go to the home directory with the default name and without any passphrase all right so i created a bunch of my key pair now i'm all set to go ahead and run my cops now cops will ensure that this will take some time for the cluster to come up but this is exactly how i would define my corpse command how do i have copied here so corps create cluster the state of the cluster will be stored in this particular variable which is what i have exported out the number of node count is two node count is the worker node count if at all i don't specify configuration for specifying the master or the control pane as well if i don't specify that the default one is one so i'm going to create one primary or the master node and two worker nodes uh size of my master node size of the worker nodes are what are the zones where i want this to be created and where do i store the cluster information and uh okay i've already added this master account this is actually not required but this is the command that i'm going to fire off so that i bring up my cluster all right that went off uh very fast but um this actually did not create a cluster it is just the definition of a cluster and uh and that's why this came out very fast now that everything looks good with whatever configuration that i specified now let me go ahead and create the cluster by saying corpse update cluster iphone iphone 8 yes now this will take some time a good five to ten minutes for it to because this will actually start provisioning the servers and as i mentioned earlier based upon my configuration i'm going to come up with one master server and uh two nodes or the worker nodes all right so this is the command for me to validate the cluster and i'm gonna try it out first i'm pretty sure that it will fail because all my servers are not up yet it is taking some time the validation everything failed but let me try to look at my ec2 instances and see how many servers do i have running as of now if you see i had only one server that i had started which was my cop server and automatically the other three instances one is called not start simply not in this these two are the nodes and this is the master node so these three got automatically provisioned by the cops command that i ran all right so this may take a while for it to get validated a good five to ten minutes so what i'll do is i'll pause the video now and i'll come back once uh the server the cluster is up and running it's been a good eight to nine minutes uh since i started my cluster so let me validate now okay seems good so there's a master node minimum one max one some nodes which are nothing but the slave nodes or the worker nodes there are two of them due to micro subnet so my clusters seem to be up and running and my cluster name is simply learn.n so what i would want to do is now let me just log into the master and see if i can run some parts so let me get back to my insta installation steps here the validation cluster is done so let me login to my since my cluster name is simplyland.and this is the way to log into my box so let me get into my box so if you see here uh this is the host name of the machine that i'm currently in if you see this this is nothing but our the scop server this is the server now from here i'm trying to get into the master box all right so if you see the ip address has changed i was in a different box i'm in a different box now if i see host name you'll find a different host name so this is 153 which is nothing but the master node yep it's 153 this is the particular house so i'm getting into this machine now so i started the cluster from my cop server here it ran and brought up three nodes so i'm actually getting into my master node and see if i can run some parts on it all right so let me try cuba ctl get cluster info try cube ctl get nodes all right so there are three nodes here master node and um one master node and two worker nodes so let me see if i have some pods here cube ctl get pods so there's no pod as of now so what i'll try to do is let me just spin up a very very simple part just to check if my connections is everything is correct or not correct i have a simple uh node.js application that i've built uh i've got a container for that this is already pushed to the docker i've registered is called simply learn docker hub and the image name that i have here is called my node app and i will use this image to power one of my ports that i will run so this is the way in which i'm going to launch my pod let me show you these commands cube ctl run the name for my deployment that i'm going to run hyphen iphone image and this is the image that is going to be powering my container so this is simply learn docker hub forward slash my node app hyphen f1 replica is equal to i want two replicas to be running and the port that i want to expose this particular part is on 8 0 8 0. so let me run this as soon as this is run it creates something called as a deployment all right now let me just say i'm not really interested in the deployment i'm interested in checking if all the parts are up and running so i will say cube ctl get but all right so that was pretty fast so i have two parts that are running here these are two replicas because i asked for two replicas these parts are running so i can run cube ctl describe pod pickup all right i can pick up any of the pod name and see what is the pod what is the information does it contain from what is the image that is pulling what is the image image name id this is actually running my pod which is actually spinning up a container great so far so good so in kubernetes the pods can are ephemeral so they can be there at any time cannot beat in time if i need to expose my pot outside i will have to create a service for my part so what i'll do is i'll create a very very simple service by exposing this particular deployment before that let me check cube cdl get deployment so there's a deployment that is created the deployment name is simply run app so i'll expose my deployment simply learn app as uh yeah i'll just expose this let me see all right i'm not specifying what type and all i don't want to get into the complications of uh what are the different types of exposing the service and all that stuff so if i don't specify anything uh it gives me something called a cluster ip so this is where my pod is actually exposed so let me just check if at all this part is up and running i'll just try a simple curl command curl http colon this is my cluster ip and uh the port is 8080 if at all i had this it's actually hitting my application and giving me whatever i put a simple uh sys dot out kind of a thing where i'm just printing um the container id of whatever pod is serving uh makes a lot of so i'm actually hitting my container and i'm getting this output from the container so everything looks good my container is up and running so let me just go and clean it up i'll say cube ctl delete deployment simply learn app this should get rid of all the parts that i created cube cdl get pod all right these spots are intimidating things let me just check cube ctl get services there's one service i don't want this service let me delete that cube ctl delete service i want this all right that sounds good so let me come back to my host my cop server from where i'm running so i managed to successfully verify this part see if everything is up and running so what i'll do is i'll just go ahead and complete this uh demo by going and uh getting rid of my cluster so corpse delete cluster hyphen f1 yes all right so this will ensure that it will clean up my complete cluster that i created so i had three or four running instances if you see that all the three are shutting down because you know the cop server which had cops installed on it uh has now got a mandate to go ahead and shut down and clean up the whole instances and all those things that are created as a part of my deployment hey everyone i welcome you all to this puppet fill course by simply learn in this complete course we will learn everything that we need to know to excel in puppet we have math and anjali to guide us through this journey we shall begin with having a basic understanding of puppet and what is its relevance today after that we shall dive deeper and understand the components of puppet followed by its architecture after checking its architecture we shall look at the process to install puppet and configure it in our local system moving forward we shall see companies that have adopted puppet and all about writing manifest and puppet after that we will check regarding the puppet community and puppet dsl post switch we shall see the hands-on demo that will explain the working of puppet from scratch after we have ample knowledge regarding the tool we shall understand the difference between chef puppet ansible and salt stack finally we shall take a look at some of the interview questions that might increase your chances of cracking any puppet interview so before we begin make sure that you have subscribed to our youtube channel and don't forget to click that bell icon to never miss an update from simply learn now without further ado let's begin with our introduction hello this is matthew from simply learn we're going to focus specifically on puppet so in this session what we're going to do is we're going to cover what and why you would use puppet what are the different elements and components of puppet and how does it actually work and then we'll look into the companies that are adopting puppet and what are the advantages that they have now received by having puppet within their organization and finally we'll wrap things up by reviewing how you can actually write a manifest in puppet so let's get started so why puppet so here is a scenario that as an administrator you may already be familiar with you as an administrator have multiple servers that you have to work with and manage so what happens when a server goes down it's not a problem you can jump onto that server and you can fix it but what if the scenario changes and you have multiple servers going down so here is where puppet shows its strap with puppet all you have to do is write a simple script that can be written with ruby and write out and deploy to the servers your settings for each of those servers the code gets pushed out out to the servers that are having problems and then you can choose to either roll back to those servers to their previous working states or set them to a new state and do all of this in a matter of seconds and it doesn't matter how large your server environment is you can reach to all of these servers your environment is secure you're able to deploy your software and you're able to do this all through infrastructure as code which is the advanced devops model for building out solutions so let's dig deeper into what puppet actually is so puppet is a configuration management tool maybe similar tools like chef that you may already be familiar with it ensures that all your systems are configured to a desired and predictable state public can also be used as a deployment tool for software automatically you can deploy your software to all of your systems or to specific systems and this is all done with code this means you can test the environment and you can have a guarantee that the environment you want is written and deployed accurately so let's go through those components of puppet so here we have a breakdown of the puppet environment and on the top we have the main server environment and then below that we have the client environment that would be installed on each of the servers that would be running within your network so if we look at the top part of the screen we have here our puppet master store which has and contains our main configuration files and those are comprised of manifests that are actual codes for configuring the clients we have templates that combine our codes together to render a final document and you have files that will be deployed as content that could be potentially downloaded by the clients wrapping this all together is a module of manifest templates and files you would apply a certificate authority to sign the actual documents so that the clients actually know that they're receiving the appropriate and authorized modules outside of the master server where you'd create your manifest templates and files you would have public client is a piece of software that is used to configure a specific machine there are two parts to the client one is the agent that constantly interacts with the master server to ensure that the certificates are being updated appropriately and then you have the fact of it the current state of the client that is used and communicated back to through the agent so now that we know what puppet is let's look into the puppet architecture so puppet has a master slave architecture as you can see here on the top we have the puppet master and below we have the puppet clients so the puppet master is the system where your system administrator sits the clients are those systems which require configuration you can have n number of clients connected to the puppet master so don't worry we look into each of these components in detail firstly we have the puppet master so the puppet master is where the system administrator sits and he or she types in the codes here so the codes are not only written in the system but they are also stored here and whenever the client requires these codes for their configuration they simply request it so manifest is the first subcomponent of the puppet master these are the actual files where the codes are written so in case of puppet these are ruby files modules tie manifests templates and files into a single unit now these codes dictate the instructions for performing various tasks on your client the next component is templates so if manifests are your code templates are the data so let me explain this to you with an example consider you want to create a file called index.html on one of your client machines so what you do is you write the code for creating this file in your manifest and the content of this file is written in your templates then you simply link them but here's the thing when you divide your code and data in this way it's a very clean structure so it's preferred to do it in this manner files are your static contents so the files are basically sent to your client and they are simply downloaded at the client there is no execution or compilation involved with your files now this is where files and templates are extremely different because with templates they are never sent over to your client the templates are simply read by the parser at the master itself modules ties the manifest templates and files into a single unit so modules is not a separate component as such it just provides a very clean hierarchy this is the most important component of the puppet master ca which stands for certificate authority now as we will see later in this video the puppet master and the puppet client communicate through an exchange of ssl certificates so when the client sends over the certificate to the master ca provides the master the authority to sign these certificates the minute the master signs a certificate the master is basically saying that yes this client is registered with me and i agree to exchange data with this client our next component puppet client of course these are the systems that require to be configured agent is a service installed on the client the agent is solely responsible for all the communication between the master and the client and finally we have factor so factor is responsible for collecting all the data about the present state of our client and sending this data over to the puppet master at the master this data is compared against the manifest which are your code files and then these manifests are sent over to the client depending upon the results of this comparison so now that we had a look at the various components of puppet let's see how these work together so we have a master and we have three clients now before any information can be transferred between the master and the client it's important that these components recognize each other also there needs to be an encrypted channel of communication between each pair of master and client now all the data which is transferred between the master and the client it flows through this channel so let's see how this entire procedure of recognizing each other works the core of this procedure are the ssl certificates ssl stands for secure socket layer secure socket layer is a security protocol and it forms an encrypted channel between two components what this encrypted channel ensures is that all the data that is passed through this channel is not accessible by any other system outside so it provides complete security okay so we will consider a very general scenario we have a master and a client here this is not the puppet master or the puppet client they're just two systems and we're just going forward with master slave architecture so the client so far is not a client it is just a system and we will refer to the system as system a so system a sends over a client certificate to the master associated with this client certificate is the host name of system a now master goes through the certificate it sees the host name and then it decides whether it wants to form a communication link with the system a if it does it signs the certificate and the signed certificate is sent back to system a now the system a is officially a client the minute the master signed the client certificate it said that yes i am willing to exchange information with this system and i do take the system as one of my clients so now that the system is officially a client it can request for data from the master and the master will simply send over the data now we are here to talk about puppet so if this was a puppet master and this was a puppet client then the first three stages will be the same so the puppet client sends over the certificate to the master the master signs the certificate sends it back and the client requests for data now by data here of course we mean the configuration data those are the manifest or even could be your files so what happens after this request after this request the factor service which is present on your clients collects all the information regarding the present state of your client so this information could include the operating system that's installed on them the various number of software present on them the versions of these software now all this information is collected it's called the fact and it's sent over to the master at the master this information that is the facts is compared against the manifest so why does the master do this the master basically compares the facts against manifest to know if what the manifest is about to perform is already performed on these systems so if the manifest has some additional functionality they are compiled into catalogs and then these catalogs are sent over to the client now remember those agent services which existed in your clients here's where they come in the agents take the catalog and execute them on the client once executed these catalogs make the required changes on the client and a report is generated this report is then sent back to the master so the master keeps record of all the changes that's made in the system and this comes very handy when one of the systems fail and it's required to be rolled back to its previous working state so this entire procedure can be repeated n number of times and can be repeated at regular intervals so if you have a scenario where a system needs to be configured every 30 minutes what you can do is you can set your interval and automatically every 30 minutes your systems will be configured if getting your learning started is half the battle what if you could do that for free visit scale up by simply learn click on the link in the description to know more so today i'll be showing you how you can use puppet to install apache on one of your client machines so we'll be using two machines here we'll have a server and we'll have a node machine the server is essentially your master the node is your client i'm just using the term server and node instead because i want to keep it very close to as they are used in the commands now both these machines are going to be sent to a seven machines and i have oracle vm virtualbox installed on my machine so on that i've simply created two virtual boxes both centos 7 of course so before we move on to the actual demo i'll just take you through the various steps so first we'll install the puppet server on our server or master machine we'll then start the server service we then shift over to our node or the client machine and we'll install the puppet agent service we will then start the service too and the minute you start the puppet agent service an ssl certificate is created so this certificate will have the host name of your node attached to it and this is how the master or the server will recognize the certificate and then it will sign this certificate so once your master or the server signs the certificate you can write your manifest so the manifesto is just the codes for configuration and once that tool is done you shift back to your client or the node system and you run the asian service so once you run the agent service your manifest or your catalogs because as you know the manifests are compiled into catalogs these are executed on your node and this will make the required changes so that's how this entire process is going to work now let's move on to the actual demo this is my oracle vm virtualbox manager the last two machines you see here are the ones we'll be using for this demo the puppet server is your puppet master puppet client of course is the client so let's just have a look at these two machines this is my puppet server so just so you can differentiate guys the terminals have been set accordingly puppet server has a black background with white text and puppet client here has a black background with green text so we'll move on to server machine or the master machine because that's the one we'll be working on first so before we begin we need to set the hostname of our master to puppet so this is a very essential step in the configuration of puppet if you miss this out there are quite a lot of other changes you'll have to make in certain files so let's do this first so that's the command to set the hostname the last word you see there that's the new hostname you want okay so now we'll just check if our host name is set so just type host name and yeah it's puppet so next we will check the ip of this machine so for that you just type if config enter so this is your ip guys under enp0s3 in its value that is 192.168.2.183 of course the ip will differ for you guys needless to say so note that down somewhere because we'll require it when we are configuring our client okay so now you want your puppet server and your client to communicate so for this we need to remove certain firewalls so the command for this is sudo because we want the root permission system ctl stop firewall d and put in the password okay so all your firewalls are removed so the next step now we start on with a puppet process so we download the puppet labs at first now guys ensure that your systems your virtual boxes in this case are connected to the net also you won't be getting an ip if it's not for the first time so here's a command to download the puppet lab enter the password again okay so the puppet labs is downloaded now you need to install this so installation guys we will be installing the puppet server on the master machine or the puppet server machine so this may take a while so that took a while so i just skipped to the last part where it's completed the installation process is over now we need to make a small change in our system config puppet server file so let's just move to the file first this file will open in vi editor so this line here it basically allocates memory for your puppet server now you do not need gb so we are going to reduce the size 512 mb should be enough so guys notice here i did not type the b okay so i just changed this to xms 512 m let's save that always remember to open this file using sudo otherwise saving the changes you made could be quite troublesome okay so now we can start a puppet server service we enable the service okay so that's done so basically our puppet server service has started it's running presently so for a while we are done with the server machine let's move on to our client machine okay first things first we open our etc host file with sudo because you need to make a few changes and save it open this in the vi terminal okay guys so here you have to add this one line at the end type in the ip for your puppet server machine puppet puppet dash master so let's save this file now we'll move on to downloading puppet labs on our puppet client so the command is the same that we use for puppet server so puppet labs is downloaded now we need to install the puppet agent service here again this will also take some time puppet server is installed let me just clear this window now that our puppet agent service is installed we'll start this service so i'll just type in the command for starting the puppet agent service as you can see here ensure changed stopped to running so basically the service was not running before and now it's changed to the running state and while we executed this command not only did the service start running but an ssl certificate was also created so this certificate should have the host name of this machine that's a puppet client machine attached to it so let's just check what the host name of our machine is that's localhost.localdomain so a certificate should have this attached to it so guys the first time your client sends a certificate the name of the certificate itself is usually the host name of the machine so our certificate's name should be localhost.localdomain we'll check this on our master system and that's where we'll sign the certificate to so first we list out all the certificates in our case since we have just one client and we just started the client we should have technically just one certificate and it will be so yeah so this is a certificate localhost dot local domain we'll now sign the certificate okay sorry about that just need to remove these brackets here as you can see here the certificate is signed sign certificate request here so once the certificate is signed the request for signing the certificate is removed that's this line now that a certificate is signed we can begin to write a manifest so let's move into a manifest folder and we'll create a placeholder file so we'll create a file called sample.pp this is a sample file we won't be writing anything in this file as of now so we are just checking here we are just creating this file and then we'll go to our client machine and just check the if we can access this file from there so now we will move to our client machine so let's test the connection so as you can see here it says applied catalog in 0.03 seconds so a catalog is executed on our client machine now a catalog was just a placeholder basically so the file that we created which is sample.pp had no content and we were just testing our client now let's go back to our master server put some content into this file and see how the execution takes place so we'll open our file sample.pp which we created earlier in the vi editor let's type our code here so node and then within the brackets we enter the hostname of our client machine so this ensures that our manifest is executed on this particular machine now the apache service package on centos is called the httpd so this line that i just typed in is the action it will ensure that the httpd package is installed on your client and that's it so this is our code we'll save it now let's go back to our client machine and test it so this is our client machine and our catalogs applied on the client so i just cleared the screen now we'll start the apache service okay so how do we see if it's running we just need to go to our browser and at localhost you must have the testing page for your apache so we type localhost enter and yeah this is the testing page for apache so as you can see we have successfully used puppet to install the apache service on our puppet client and this is the end of our demo so let's have a look at companies that are using puppet today there are a number of companies that have adopted puppet as a way to manage their infrastructure so companies that are using puppy today include spotify google att so why are these companies choosing to use puppet as their main configuration management tool the answer can be seen if we look at a specific company staples so staples chose to take and use puppet for their configuration management tool and use it within their own private cloud the results were dramatic the amount of time that the it organization was able to save in deploying and managing their infrastructure through using puppets enabled them to open up time to allow to experiment with other and new projects and assignments a real tangible benefit to a company so let's look at how you write a manifest in pocket so so manifests are designed for writing out in code how you would configure a specific node in your server environment the manifests are compiled into catalogs which are then executed on the client each of the manifests are written in the language of ruby with a dot pp extension now we step through the five key steps for writing a manifest they are one create your manifest and that is written by the system administrator two compile your manifest and is compiled into a catalog three deploy the catalog is then deployed onto the clients four execute the catalogs are run on the client by the agent and then five end clients are configured to a specific and desired state if we actually look into how manifest is written it's written with a very common syntax if you've done any work with ruby or really configuration of systems in the past this may look very familiar to you so we break out the work that we have here you start off with a package file or service as your resource type and then you give it a name and then you look at the features that need to be said such as ip address then you're actually looking to have a command written such as present or start the manifest can contain multiple resource types if we continue to write our manifest and puppet the default keyword applies a manifest to all clients so an example would be to create a file path that creates a folder called sample in a main folder called etc the specified content is written into a file that is then posted into that folder and then we're going to say we want to be able to trigger an apache service and then ensure that that apache service is installed on a node so we write the manifest and we deploy it to a client machine on that client machine a new folder will be created with a file in that folder and an apache server will be installed you can do this to any machine and you'll have exactly the same results on those machines now let's have a look at the puppet community the puppet community is a rich community of a lot of people officially there is no count but counting the various channels where people collaborate there are a little over 2 900 committers to github projects who are directly or indirectly associated with puppet labs there are 4 893 modules on the official puppet forge these include modules from the puppet labs and also from open source contributors to github so that's nearly 5 000 modules of ready-made code available on the puppet forge so before you write any code for puppet you can check to see if it's already been written by someone now there's a mailing list which has sixty five hundred and eighty eight participants apart from this on slack overflow you have a big tag of puppet labs and puppet which gives you a list of only puppet questions that's a good playground for you to interact with people who are already using puppet and who are contributing and helping others now let's have a look at puppet forge so puppet forge is a github client for a repository which is hosted by puppet labs that has almost five thousand modules of puppet code so these are both puppet labs and community contributed the best thing about this is that they're easy to customize so when you have a requirement for anything you're doing with your infrastructure you can just grab one of those five thousand modules and then you can customize it according to your needs one of the other benefits is they have built-in tests so they follow all of the best practices now if you write your own module you also have to write your own tests and you have to spend extra time writing best practice code all of that has been taken care of and you can just download any one of those and customize it according to your needs and you're good to go so that's the benefit you get from such a large community where people contribute to the modules and the code and share with the community the puppet dsl puppet dsl is a puppet domain specific language it's the puppet's primary programming language it's just like plain english and you don't really need to be a programmer to write code in it it's kind of a sub child of ruby because it approaches and uses is equal to and greater than marks to define things so it's very easy to learn you can specify packages that you want to install you can specify services to be managed you can do things based on the operating system that you have so we'll be writing a small puppet module which will be using this puppet dsl we will do that after we install the puppet server and the client and then we will write a simple puppet module to show how to use the puppet dsl let's have a look at the demo we are going to do so you will be setting up the machines using pseudo dns because we want to have a resolvable dns because only by having a resolvable dns can we use puppet because puppet uses the ssl certificates now we'll be installing the puppet server and the client will be signing the certificates of the client on the server side and then we'll proceed with writing an ntp module which we'll be using for installation of ntp on the client and then we'll look at the puppet programming language or dsl now we're going to look at a demo here so we will be setting up the machines we'll set up a pseudo dns locally because puppet requires a fully qualified domain name in order to run since there is a certificate communication taking place here we will be installing the puppet server the puppet client will be signing the certificate and finally we will write an ntp module and make it run [Music] so now we will be starting a puppet master and a client i have two machines here one is the puppet master and the second is the puppet client the puppet master has got almost four gig of ram and four v cpus which i can show you here because it's a ruby on rails application it requires a lot of cpu and ram for the client i don't need that much ram and cpu so i've allotted one gigabyte of ram and one vcpu i will start both of these machines and find out the ip addresses right click normal start both of these machines are starting up the puppet client has started up and the first thing i'll do is log in using root and red hat password and then i will change the host name so i'll say vim etc hostname this will open a hostname file in a text editor which is vim then we'll do a dd which will delete that existing host name and then i press the i for insert mode and finally i'll type the puppet hyphen client which will be the host name of this machine i'll save this file by doing escape semicolon wq exclamation mark and i'll reboot the machine similarly going to the puppet master i'll say root red hat i'll open the hostname file say dd then i'll say puppet hyphen master i'll save this file and reboot the machine for the hostname change to take effect the hostname change does not take effect unless you reboot the machine now both these machines are ready i'm going to find out the ips and use the mac terminal to log into these so this is 23 and 21 so 23 is my server and 21 is my client i'll ssh to 23 first now in order for the puppet master to be installed i need a fully qualified domain name here and a fake dns which means a fully resolvable dns name so if i do hostname here you see my host name is puppetmaster in order to set up a fake dns i need to go to the etc hosts file go to the end then i say ip address host name local then space host name now this is one of the techniques which makes the linux server believe that there's a fully qualified domain name on this machine i'll save this then i do a hostname hyphen f you see it has a fully qualified domain name now so now we will be downloading the puppet labs repository for ubuntu 16.04 which is xenial so i'll say curl hyphen 0 https apt.puppetlabs dot com slash puppet labs hyphen release hyphen pc1 hyphensenial.dab you can see there is a debian file which has been downloaded here i will import this using the dpkg command so i'll say dpkg hyphen i puppet labs hyphen release hyphen pc1 hyphensenial.dap and then i'll say apt-get update which will update my app package indexes on this machine and now i'll install the puppet server so i'll say apt-get install puppet hyphen puppet server hyphen wide so the puppet server is now installed and we will be configuring the memory allocation for the puppet server so i'll say etc default puppet server so by default it has only got two gigabyte allocated we'll give it three gigabyte because it's jvm i'll make this three and i'll change this to three as well and i'll save this file i'll make sure the unconfigured firewall allows port 8140 which is the puppet lab puppet server port the rules have been updated to allow that i'll now start the puppet server so i'll say a system ctl start puppet server it takes some time because it's jvm so it takes a couple of minutes to start the puppet server so the puppet server is now running i'll also enable it at boot time so i'll say systemctl enable puppet to server it will enable it boot time now i also need to configure a fully qualified domain name here so okay so i'll do that by going into the hosts file i'll go to the end and i'll say 192.168.1.23 and then puppet hyphen server dot local and then puppet hyphen server if i do a host name hyphen f put hyphen master.local because the current hostname is puppet hyphen master so the trick is to have the hostname.local in front of the ip address in the host file having a fully qualified domain name now now we will go to the client setup the client setup is almost the same with just a small difference the package which we need to install on the client is puppet agent so we will go to the client so you see this has got a hostname now we also need to set up the fake dns here so i'll copy this hostname open the host line go to the end write the ip address and then dot local and then host name again so this will give me a fully qualified domain name so when i do a host name it says puppet hive and client when i do a host name hyphen f it says puppet client local to install the puppet agent on this machine i will need to download the zenio packages so i'll say wget https apt.puppetlabs.com slash puppet labs hyphen release hyphen pc one hyphen xenial.net i'll download this and then i'll say dpkg hyphen i puppet labs iphone release hyphen pc1 hyphensenial.dap so this is the repository package which i'll be downloading here and then i'll say apt-get update hyphen y this will update my apt caches on this machine and now i'll finally install the puppet agent so i'll say apt-get install puppet agent hyphen agent hyphen y now the puppet agent is here i'll start the puppet agent and i'll enable it at boot time so i'll say system control start puppet then system control enable puppet now puppet is enabled now we have set up the fully qualified domain name on the client and the server but each of them should know each other so we'll have to do a cross certificate host mapping so we will be copying the host file entry of the client and adding it to the server so i'll go to the etc hosts on the server side go to the end paste it there okay and i'll be adding the entry of the master on the client side okay now both of the servers will recognize each other now because one of them has the host entry of the other so if i do from the master ping puppet hyphen agent dot local if i ping the agent from the server hyphen client.local it is able to ping it and if i ping the master from the client if i say ping puppet hyphenmaster.loco both of them will be able to ping each other now the next step would be to create a certificate signing request so i'll go to my to this directory which is opt puppet labs puppet bin and i'll say puppet agent hyphen t hyphen knife and server is equal to and i'll give the fully qualified domain name of the puppet master then i'll say hyphen f when i do this it will create a certificate signing request you see now if i go to the server and i do an opt i'll check and see how many certificates are pending to be signed you'll see there is a puppet client certificate which is pending to be signed which is denoted by the absence of a plus sign here i'll be signing the certificate so i'll say puppet search sign and then the name so puppet hyphen client dot local this will sign the certificates and when i go to the server on the client side if i run the puppet agent again you see it's actually pulling down the catalog as of now it has an empty catalog that says there is nothing now we have configured the puppet agent and the master now we will be taking a module from puppet forge puppet forge is a big collection of open source modules which are free to use as a first step we will be using a ready-made module and that would be a demonstration for how to actually get up and running with a module so search for apache module here this is the puppet labs apache module i'll be using the puppet command line puppet module install this is the command for that but i will make sure i run this command from the puppet home so i'll go to the master i'll go to etc puppet labs code environments production in this i'll do modules now i will run it from the binary which is present at the opt puppet labs bin puppet and then module install puppet labs hyphen apache and then this version so we are specifying the latest version and i'll hit enter when i press enter it starts downloading from the forge api so you see the module is downloaded in addition to the dependencies so there is concat stdlib and apache so now i'll go to a folder here which is one level up called manifest so this is the file here etc puppet labs code environments production manifest so in this there's a file which i'll be creating called site.pp the site.pp will specify and this is the main controller of how the infrastructure looks in the site pp i specify that i want puppet hyphen client dot local node to have the apache module included i'll save this file and the only thing i need to do now is since all is set on the server side i need to go to the client side and run puppet agent it will automatically download the apache module and compile it and it may then make sure the server is in a state so when i press enter we see a lot of output it tries to configure apache on this machine and done now when i go to the front end of this machine on this port i should be able to see apache's default page so this is apaches default page so the module which we downloaded from the forge was actually written in a way that apache would be installed and run at boot time so that is why it's running here and that is all we have from the puppet module and this concludes the demo today we're going to go through a real tough battle we're going to decide which is better for your operations environment is it chef puppet antipal or salt stank all four are going to go head to head so let's go through the scenario of why you'd want to use these tools so let's meet tim he's our system administrator and tim is a happy camper putting and working on all of the systems in his network but what happens if a system fails if there's a fire a server goes down well tim knows exactly what to do he can fix that fire really easily the problems become really difficult for tim however if multiple servers start failing particularly when you have large and expanding networks so this is why tim really needs to have a configuration management tool and we need to now decide what would be the best tool for him because configuration management tools can help make tim look like a superstar all he has to do is configure the right codes that allows him to push out the instructions on how to set up each of the servers quickly effectively and at scale all right let's go through the tools and see which ones we can use the tools that we're going to go through are chef puppet and support and salt stacks and we have videos on most of these software and services that you can go and view to get an overview or a deep dive in how those products work so let's go and get to know our contestants so our first contestant is chef and chef is a tool that allows you to configure very large environments it allows you to scale very effectively across the entire ecosystem and infrastructure chef is by default an open source code and one of the things that you find is a consistent metaphor for the tools that we recommend on simply learn is to use open source code the code itself is actually written in language of ruby and erlang and it's really designed for heterogeneous infrastructures that are looking for a mature solution the way that chef works is that you write recipes that are compiled into cookbooks and those cookbooks are the definition of how you would set up a node and a node is a selection of servers that you have configured in a specific way so for instance you may have apache linux servers running or you may have a my sql server running or you may have a python server running and chef is able to communicate back and forth between the nodes to understand what nodes are being impacted and need to have instructions sent out to them to correct that impact you can also send instructions from the server to the nodes to make a significant update or a minor update so there's great communication going back and forth if we look at the pros and cons the pros for chef is that there is a significant following for chef and that has resulted in a very large collection of recipes that allow you to be able to quickly stand up environments there's no need for you to have to learn complex recipes the first thing you should do is go out and find the recipes that are available it integrates with git really well and provides for really good strong version control some of the consoles are really around the learning speed it takes to go from beginner user with chef to being an expert there is a considerable amount of learning that has to take place and it's compounded by having to learn ruby as the programming language and the main server itself doesn't really have a whole lot of control it's really dependent on the communication throughout the whole network all right let's look at our second contender puppet and puppet is actually in many ways very similar to chef there are some differences but again puppet is designed to be able to support very large heterogeneous organization it is also built with ruby and uses dsl for writing manifests so there are some strong similarities here to chef as with a chef there is a master slave infrastructure with puppet and you have a master server that has the manifests that you put together in a single catalog and those catalogs are then pushed out to the clients of an ssl connection some of the pros with puppet is that as with chef there is a really strong community around puppies and there's just a great amount of information and support that you can get right out of the gate it is a very well developed reporting mechanism that makes it easier for you as an administrator to be able to understand your infrastructure one of the cons is that you have to really be good at learning ruby again as with shaft you know the more advanced tasks really need to have those ruby skills and as with chef the server also doesn't have much control so let's look at our third contender here ansible and so ansible is slightly different it is the way the ansible works is that it actually just pushes out the instructions to the server environment there isn't a client server or master slave environment where ansible would be communicating backwards and forwards with its infrastructure it is merely going to push that instructions out the good news is that the instructions are written in yaml and yowell stands for yaml 8 markup language yaml is actually pretty easy to learn if you know xml and xml is pretty easy if you know xml you're going to get yamo really well ansible does work very well on environments where the focus is getting servers up and running really fast it's very very responsive and can allow you to move quickly to get your infrastructure up quick very fast and we're talking seconds and minutes here really really quick so again the way that ansible works is that you put together a playbook and an inventory or you have a playbook so the way the ansible works is that you have a playbook and the playbook it then goes against the inventory of servers and will push out the instructions for that playbook to those servers so some of the pros that we have for ansible we don't need to have an agent installed on the remote nodes and servers it makes it easier for the configuration yaml is really easy to learn you can get up to speed and get very proficient with yamo quickly the actual performance once you actually have your infrastructure up and running is less than other tools that we have on our list now i do have to add a proviso this is a relative less it's still very fast it's going to be a lot faster than individuals manually standing up servers but it's just not as fast as some of the other tools that we have on this list and gamble itself as a language while it's easy to learn it's not as powerful as ruby ruby will allow you to do things that at an advanced level that you can't do easily with the ammo so let's look at our final contender here sort stack so source stack is a cli based tool it means that you will have to get your command line tools out or your terminal window out so you can actually manage the entire environment via salt sac the instructions themselves are based on python but you can actually write them in yammer or dsl which is really convenient and as a product it's really designed for environments that want to scale quickly and be very resilient now the way that sort snap works is that you have a master environment that pushes out the instructions to what they call grains which is your network and so let's step through some of the pros and cons that we have here with salt stack so substance is very easy to use once it's up and running it has a really good reporting mechanism that makes your job as an operator in your devops environment much much easier the actual setup though is a little bit tougher than some of the other tools and and it's getting easier with the newer releases but it's just a little bit tougher and related to that is that source tank is fairly late in the game when it comes to actually having a graphical user interface for being able to create and manage your environment other tools such as ansible have actually had a ui environment for quite some time all right so we've gone through all four tools let's see how they all stack up next to each other so let the race begin let's start with the first stage architecture so the architecture for most of our environments is a server client environment so for chef puppet and salt snack so very similar architecture there the one exception is ansible which is a client only solution so you're pushing out the instructions from a server and pushing them out into your network and there isn't a client environment that there isn't a two-way communication back to that main client for what's actually happening in your network so let's talk about the next stage ease of setup so we look at the four tools there is one tool that really stands out for ease of setup and that is ansible it is going to be the easiest tool for you to set up and if you are new to having these types of tools in your environment you may want to start with ansible just to try out and see how easy it is to create automated configuration before looking at other tools now and so with that said chef puppet and socksec aren't that hard to set up either and you'll find there's actually some great instructions on how to do that setup in the online community let's talk about the languages that you can use in your configuration so we have two different types of language with both chef and ansible being procedural in that they actually specify at how you actually supposed to do the task in your instructions with puppet and sort stack it's decorative where you specify only what to do in the instructions let's talk about scalability which tools scale the most effectively and as you can imagine all of these tools are designed for scalability that is the driver for these kind of tools you want them to be able to scale to massive organizations what do the management tools look like for our four contenders so again we have a two-way split with ansible and sort stack the management tools are really easy to use you're gonna love using them they're just fantastic to use with puppet and chef the management tools are much harder to learn and they do require that you learn some either the puppet dsl or the ruby dsl to be able to be a true master in that environment but what does interoperability look like again as you'd imagine with the similar to scalability interoperability with these products is very high in all four cases now let's talk about cloud availability this is increasingly becoming more important for organizations as they move rapidly onto cloud services well both ansible and softstack have a big fail here neither of them are available in the most popular cloud environments and puppet and chef are actually available in both amazon and azure now we've actually just haven't had a chance to update our chef link here but chef is now available on azure as well as amazon so what does communication look like with all of our four tools so the communication is slightly different with them chef has its own knife tool and whereas puppet uses ssl to secure sockets layer and ansible and saltsec use secure socket hashing and ssh as their communication tool bottom line all four tools are very secure in their communication so who wins well here's the reality all four tools are very good and it's really dependent on your capabilities and the type of environment that you're looking to manage that will determine which of these four tools you should use the tools themselves are open source so go out and experiment with them there's a lot of videos our team has done a ton of videos on these tools and so feel free to find out other tools that we have then covered so you can learn very quickly how to use them but consider the requirements that you have and consider the capabilities of your team if you have ruby developers or you have someone on your team that knows ruby your ability to choose a broader set of tools becomes much more interesting if however you're new to coding then you may want to consider yaml-based tools again the final answer is going to be up to you and we'll be really interested on what your decision is please post in the comments below which tool you have decided to go with and what are the challenges that you're coming up with because we'll help answer those challenges for you answer will provide you with one of the easiest ways to automate your application and i3 infrastructure on that note hi guys welcome to this complete ansible tutorial by simplylearn in this video we'll be covering some of the most important concepts relating to the devops tool ansible we'll take you through how you can install ansible and how you can use the tool to automate your processes first we have anjali to tell us how to install ansible on your system a very warm welcome to all our viewers i'm anjali from simply learn and today i'll be showing you how you can install the configuration management tool ansible so let's have a brief about why one would use ansible and what exactly is ansible so if you consider the case of an organization it has a very large infrastructure which means it has more than probably hundreds of systems and giving one or even a small team of people the responsibility to configure all these systems makes their work really tough repetitive and as you know manual work is always prone to errors so ansible is a tool which can automate the configuration of all these systems with ansible a small team of system administrators can write simple code in guiamil and these codes are deployed onto the hundreds and thousands of servers which configures them to the desired states so ansible automates configuration management that is configuring your systems it automates orchestration which means it brings together a number of applications and decides an order in which these are executed and it also automates deployment of the applications now that we know what ansible does let's move on to the installation of ansible so here is my oracle vm virtualbox manager i'll be using two systems there's the node system which is basically my client system and there's the server system or the master system so let's begin at our server system so this is my master system guys so the first thing we do is we download our ansible tool so one thing we must remember with ansible is that unlike chef or puppet ansible is a push type of configuration management tool so what this means is that the entire control here lies with your master or your server system this is where you write your configuration files and these are also responsible for pushing these configuration files onto your node or client system as and when required traits so ansible tool is installed now we need to open an ansible host file and there we'll specify the details of our node or client machine so this is a ansible host file as you can see here the entire file is commented but there's a certain syntax that you'd observe for example here we have a group name web servers under which we have the ip addresses or certain host name so this is about how we'll be adding the details for our client system first we need to give a group name under this group basically we add all the clients which require a certain type of configuration since we are using just one node we'll give only the details for that particular node first we need to add the ip address of our client machine so let's just go back to our client machine and this here is the ip address 192.168.2.11 once you have typed in your ip address give a space and then we'll specify the user for our client machine so all communications between the server or the master system and the client or the node system takes place through ssh ssh basically provides a secure channel for the transfer of information follow this up with your password in my case it's the roots password and that's it we are done so now we save this file and go back to our terminal so now that our hosts file is written the next thing we do is we write a playbook so playbook is the technical term used for all the configuration files that we write in ansible now playbooks are written in yaml yaml is extremely simple to both write and understand it's in fact very close to english so now we'll write our playbook the playbook or any code in yaml first starts with three dashes this indicates the beginning of your file next thing we need to give a name to our playbook so name and i'm going to name my playbook sample book we next need to specify our host systems which is basically the systems at which the configuration file or the playbook in our case will be executed so we'll be executing this at the client machines mentioned under the group ansible servers so we had just one client machine under it we'll still mention the group name we next need to specify the username with which we'll be logging into our client machine which is root in my case and become true specifies that you need to become the root to execute this playbook so becoming the roots called a privilege escalation next we need to specify our tasks so these are basically the actions that the playbook will be performing so you would have noticed everything so far is aligned that is name host remote user become and task because these are at one level now whatever comes under task will be shifted slightly towards the right although yaml is extremely simple to understand and read both it's a little tricky while writing because you need to be very careful about the indentations and the spacing so my first task is install httpd which is basically apache server so now my command yum and this will be installing the httpd package and the latest state of it will be installed so that's our first task now our second task would be running our apache service so name run httpd and the action which is service will be performed on httpd hence the name httpd and state must be started now we come to our third task so here we'll create a very simple web page that will be hosted so create content is the name of our task and the content that we are creating here will basically be copied to our note system at a particular file location that we'll provide our content will be congrats and then we'll provide the destination at which this file will be copied so this is the default location for all our html files and that's it we are done writing our playbook just save this and go back to your terminal before we execute the playbook or push the playbook onto our note system let's check the syntax of our playbook so the command for doing so is and if everything is fine with your playbook the output would be just your playbook name so our syntax is perfectly fine now we can push on the playbook to our node or the client machine and that's the syntax for doing so now as your playbook is being sent over to the client machine you can see that first the facts are gathered that is the current state of your client machine is first fetched to check what all is to be changed and what is already present so the first thing is installing httpd so our system already had httpd so it says okay because this does not need to be changed our next task was running httpd now although our system had the apache service it was not running so that is one thing that was changed the next was there was no content available so the content was also added so two tasks were changed and four things were okay now everything seems fine and before you move any forward it is very important that you check this one line of documentation provided by ansible you have all kind of information available here regarding which all tasks were executed if your client machine was reachable or unreachable and so on so now that everything's fine here we can move on to our note system and we'll just go to our browser so if our playbook has been executed here what should happen is that the httpd service must be in the running state and the web page that we created should be hosted so let's just type localhost and great everything is working fine so our webpage is displayed here so we come to an end for our installation and configuration video for the configuration management tool ansible if you have any doubts please post them in the comment section below and we'll definitely get back to you as soon as possible thanks actually now we have matthew and anjali to take us through how to work with ansible ansible today as one of the key tools that you would have within your devops environment so the things that we're going to go through today is we're going to cover why you would want to use a product like ansible what ansible really is and how it's of value to you and your organization the differences between ansible and other products that are similar to it on the market and what makes ansible a compelling product and then we're going to dig into the architecture for ansible we're going to look at how you would create a playbook how you would manage your inventory of your server environments and then what is the actual workings of ansible as a little extra we're going to also throw in ansible tower one of the secret source solutions that you can use for improving the speed and performance of how you create your ansible environments and finally we're going to go through a use case by looking at hootsuite social media management company and how they use ansible to really improve the efficiency within their organizations so let's jump into this so the big question is why ansible so you have to think of ansible as another tool that you have within your devops environment for helping manage the service and this definitely falls on the operations side of the dev ops equation so if we look here we have a picture of sam and like yourselves sam is a system administrator and he is responsible for maintaining the infrastructure for all the different servers within his company so some of the servers that he may have that he has to maintain could be web servers running apache they could be database servers running uh mysql and if you only have a few servers then that's fairly easy to maintain i mean if you have three web servers and two database servers and let's face it would we all love just to have one or two servers to manage it would be really easy to maintain the trick however is as we start increasing the number of servers and this is a reality of the environments that we live and operate in it becomes increasingly difficult to create consistent setup of different infrastructures such as web servers and databases for the simple reason that we're all human as if we had to update and maintain all of those servers by hand there's a good chance that we would not set up each server identically now this is where anspo really comes to the rescue and helps you become an efficient operations team ansible like other system solutions such as chef and puppet uses code that you can write and describe the installation and setup of your servers so you can actually repeat it and deploy those servers consistently into multiple areas so now you don't have to have one person redoing and re-following set up procedures you just write one script and then each script can be executed and have a consistent environment so we've gone through why you'd want to use ansible let's step through what ansible really is so you know this is all great but you know how do we actually use these tools in our environment so ansible hat is a tool that really allows you to create and control three key areas that you'd have within your operations environment first of all there's it automation so you can actually write instructions that automate the it setup that you would typically do manually in the past the second is the configuration and having consistent configuration imagine setting up hundreds of apache servers and being able to guarantee with precision that each of those apache servers is set up identically and then finally you want to be able to automate the deployment so that as you scale up your server environment you can just push out instructions that can deploy automatically different servers the bottom line is you want to be able to speed up and make your operations team more efficient so let's talk a little bit about pull configuration and how it works with ansible so there are two different ways of being able to set up a different environments for server farms one is to have a key server that has all the instructions on and then on each of the servers that connect to that main master server you would have a piece of software known as a client installed on each of those servers that would communicate to the main master server and then would periodically either update or change the configuration of the slave server this is known as a pull configuration an alternative is a push configuration and the push configuration is slightly different the main difference is as with a pull configuration you have a master server where you actually put up the instructions but unlike the pull configuration where you have a client installed on each of the services with a push configuration you actually have no client installed on the remote servers you simply are pushing out the configuration to those servers and forcing a restructure or a fresh clean installation in that environment so ansible is one of those second environments where it's a push configuration server and this contrasts with other popular products like chef and puppet which have a master slave architecture with a master server connecting with a client on a remote slave environment where you would then be pushing out the updates with ansible you're pushing out the service and the structure of the server to remote hardware and you are just putting it onto the hardware irrelevant of the structure that's out there and there are some significant advantages that you have in that in that you're not having to have the extra overhead weight of a client installed on those remote servers having to constantly communicate back to the master environment so let's step through the architecture that you would have for an ansible environment so when you're setting up an ansible environment the first thing you want to do is have a local machine and the local machine is where you're going to have all of your instruction and really the power of the control that you'd be pushing out to the remote server so the local machine is where you're going to be starting and doing all of your work connected from the local machine are all the different nodes pushing out the different configurations that you would set up on the local machine the configurations that you would write and you would write those in code within a module so you do this on your local machine for creating these modules and each of these modules is actually consistent playbooks the local machine also has a second job and that job is to manage the inventory of the nodes that you have in your environment the local machine is able to connect to each of the different nodes that you would have in your hardware network through ssh clients so a secure client let's dig into some of the different elements within that architecture and we're going to take a first look at playbooks that you would write and create for the ansible environments so the core of ansible is the playbook this is where you create the instructions that you write to define the architecture of your hardware so the playbook is really just a set of instructions that configure the different nodes that you have and each of those set of instructions is written in a language called yammer and this is a standard language used for configuration server environments did you know that yaml actually stands for yaml 8 markup language that's just a little tidbit to hide behind your ear so let's have a look on one of these playbooks it looks like and here we have a sample yaml script that we've written so you start off your yaml script with three dashes and that indicates the start of a script and then the script itself is actually consistent of two distinct plays at the top we have play one and below that we have play two within each of those plays we define which nodes are we targeting so here we have a web server in the top play and in the second play we have a database server that we're targeting and then within each of those server environments we have the specific tasks that we're looking to execute so let's step through some of these tasks we have an installed apache task we have a start apache task and we have an installed my sql task and when we do that we actually execute a specific set of instructions and those instructions can include installing apache and then setting the state of the apache environment or starting the apache environment and setting up and running the my sequel environment so this really isn't too complicated and that's the really good thing about working with yaml is it's really designed to make it easy for you as an operations lead to be able to configure the environments that you want to consistently create so let's take a step back though we have two hosts we have web server and database server where do these names come from well this takes us into our next stage and the second part of working with ansible which is the inventory management part of ansible so the inventory part of ansible is where we maintain the structure of our network environment so what we do here is part of the structure in creating different nodes is we've had to create two different nodes here we have a web server node and a database server node and under web server node we actually have the names that were actually pointing to specific machines within that environment so now when we actually write our scripts all we have to do is refer to either web server or database server and the different servers will have the instructions from the yammer script executed on them this makes it really easy for you to be able to just point to new services without having to write out complex instructions so let's have a look at how ansible actually works in real world so the real world environment with is that you'd have the ansible software installed on a local machine and then it connects to different nodes within your network on the local machine you'll have your first your playbook which is the set of instructions for how to set up the remote nodes and then to identify how you're going to connect to those nodes you'll have an inventory we use secure ssh connections to each of the servers so we are encrypting the communication to those servers we're able to grab some basic facts on each server so we understand how we can then push out the playbook to each server and configure that server remotely the end goal is to have an environment that is consistent so let's ask you a simple question what are the major opportunities the ansible has over chef and puppet really like to hear your answers in the comments below pop them in there and we'll get back to you and really want to hear how you feel that ansible is a stronger product or maybe you think it's a weaker product as it compares to other similar products in the market here's the bonus we're going to talk a little bit about ansible tower so ansible tower is an extra product that red hat created that really kind of puts the cherry on the top of the ice cream or is the icing on your cake ansible by itself is a command line tool however asphalt tower is a framework that was designed to access ansible and through the asphalt tower framework we now have an easy to use gui this really makes it easy for non-developers to be able to create the environment that they want to be able to manage in their devops plan without having to constantly work with the command prompt window so instead of opening up terminal window or a command window and writing out complex instructions only in text you can now use drag and drop and mouse click actions to be able to create your appropriate playbooks inventories and pushes for your nodes all right so we talked a lot about ansible let's take a look at a specific company that's using ansible today and in this example we're gonna look at hootsuite now hootsuite if you not already use their products and they have a great product hootsuite is a social media management system they are able to help with you managing your pushes of social media content across all of the popular social media platforms they're able to provide the analytics they're able to provide the tools that marketing and sales teams can use to be able to assess a sentiment analysis of the messages that are being pushed out really grateful and very popular but part of their popularity drove a specific problem straight to hootsuite the challenge they had at hootsuite is that they had to constantly go back and rebuild their server environment and they couldn't do this continuously and be consistent there was no standard documentation and they had to rely on your memory to be able to do this consistently imagine how complex this could get as you're scaling up with a popular product that now has tens of thousands to hundreds of thousands of users this is where ansible what came in and really helped the folks over at hootsuite today the devops team at hootsuite write out playbooks that have specific instructions that define the architecture and structure of their hardware nodes and environments and are able to do that as a standard product instead of it being a problem in scaling up their environment they now are able to rebuild and create new servers in a matter of seconds the bottom line is ansible has been able to provide hootsuite with it automation consistent configuration and free up time from the operations team so that instead of managing servers they're able to provide additional new value to the company a very warm welcome to all our viewers i'm anjali from simply learn and today i'll be taking you through a tutorial on ansible so ansible is currently the most trending and popular configuration management tool and it's used mostly under the devops approach so what will you be learning today you learn why you should use ansible what exactly is ansible the ansible architecture how ansible works the various benefits of ansible and finally we'll have a demo on the installation of apache or the http package on our client systems we'll also be hosting a very simple web page and during this demo i'll also show you how you can write a very simple playbook in yaman and your inventory file so let's begin why should you use ansible let's consider a scenario of an organization where sam is a system administrator sam is responsible for the company's infrastructure a company's infrastructure basically consists of all its systems this could include your web servers your database servers the various repositories and so on so as a system administrator sam needs to ensure that all the systems are running the updated versions of the software now when you consider a handful of systems this seems like a pretty simple task sam can simply go from system to system and perform the configurations required but that is not the case with an organization is it an organization has a very large infrastructure it could have hundreds and thousands of systems so here is where sam's work gets really difficult not only does it get tougher sam has to move from system to system performing the same task over and over again this makes sam bored not just that repeating the same task leaves no space for innovation and without any ideas or innovation how does the system grow and the worst of it all is manual labor is prone to errors so what does sam do well here is where ansible comes in use with ansible sam can write simple codes that are deployed onto all the systems and configure them to the correct states so now that we know why we should use ansible let's look at what exactly is ansible ansible is an i.t engine that automates the following tasks so first we have orchestration orchestration basically means bringing together of multiple applications and ensuring an order in which these are executed so for example if you consider a web page that you require to host this webpage stores all its values that it takes from the user into a database so the first thing you must do is ensure that the system has a database manager and only then do you host your web page so this kind of an order is very crucial to ensure that things work right next ansible automates configuration management so configuration management simply means that all the systems are maintained at a consistent desired state other tools that automate configuration management include puppet and share and finally ansible automates deployment deployment simply means the deploying of application onto your servers of different environments so if you have to deploy an application on 10 systems with different environments you don't have to manually do this anymore because ansible automates it for you in fact ansible can also ensure that these applications or the code are deployed at a certain time or after regular intervals now that we know what exactly ansible is let's look at ansible's architecture ansible has two main components you have the local machine and you have your node or the client machine so the local machine is where the system administrator sits he or she installs ansible here and on the other end you have your node or the client systems so in case of ansible there's no supporting software installed here these are just the systems that require to be configured and they are completely controlled by the local machine at your local machine you also have a module a module is a collection of your configuration files and in case of ansible these configuration files are called playbooks playbooks are written in yaml yaml stands for yaml ain't a markup language and it is honestly the easiest language to understand and learn since it's so close to english we also have the inventory the inventory is a file where you have all your nodes that require configuration mentioned and based on the kind of configuration they require they are also grouped together so later in the demo we'll have a look at how the playbook and the inventory is written and that will probably make it clearer so of course a local machine needs to communicate with the client and how is this done this is done through ssh ssh is your secure shell which basically provides a protected communication in an unprotected environment okay so we saw the various components of ansible now how does ansible exactly work you have your local machine on one end this is where you install ansible if you have gone through any previous material on ansible you would have come across the term agentless often being associated with the stool so this is what agentless means you're installing ansible only on your local machine and there's no supporting software or plugin being installed on your clients this means that you have no agent on the other end the local machine has complete control and hence the term agentless another term that you would come across with ansible is push configuration so since the local machine has complete control here it pushes the playbooks onto the notes and thus it's called a push configuration tool now the playbooks and the inventory are written at the local machine and the local machine connects with the notes through the ssh client this step here is optional but it's always recommended to do so it's where the facts are collected so facts are basically the current state of the node now all this is collected from the node and sent to the local machine so when the playbook is executed the task mentioned in the playbook is compared against the current status of the node and only the changes that are required to be made further are made and once the playbooks are executed your nodes are configured to the desired states so as i mentioned before ansible is currently the most trending tool in the market under the configuration management umbrella so let's have a look at the various benefits of ansible which gives it this position well ansible is agentless it's efficient it's flexible simple important and provides automated reporting how does it do all this let's have a look at that agentless as i already mentioned before you require no supporting software or plugin installed on your node or the client system so the master has complete control and automatically this means that ansible is more efficient because now we have more space in our client and note systems for other resources and we can get ansible up and running real quick ansible is also flexible so an infrastructure is prone to change very often and ansible takes no amount of time to adjust to these changes ansible cannot get any simpler with your playbooks written in a language such as yaml which is as close to english as you can possibly get important basically means that if you have a playbook which needs to be run n number of systems it would have the same effect on all of these systems without any side effect and finally we have automated reporting so in case of ansible your playbook has a number of tasks and all these stars are named so whenever you run or execute your playbook it gives a report on which tasks ran successfully which failed which clients were not reachable and so on all this information is very crucial when you're dealing with a very large infrastructure and finally we reach the most exciting part of our tutorial the hands-on before we move on to the actual hands-on let me just brief you through what exactly we'll be doing so i'll be hosting two virtual boxes both centos 7 operating systems one would be my local machine and other my node or the client machine so on my local machine first i'll install ansible we then write the inventory and the playbook and then simply deploy this playbook on the client machine there's just one thing that we need to do is that we need to check if the configurations that we mentioned in our playbook are made right so we'll now begin our demo this is my oracle virtualbox here i have my master system which is the local machine and this is the client machine so let's have a look at these two machines this is my client machine the terminals open right now so the client machine terminal has a black background with white text and the master machine terminal has a white background with black text just so you can differentiate between the two so we'll start at the master machine the first thing to do is we need to install our ansible so yum install ansible hyphen y is the command to do so so this might take some time yeah so ansible is installed the next step we go to our hosts file so a host file here is basically the inventory it's where you'll specify all your nodes in our case we just have one node that's the path to your host file as you'll see everything here is commented so just type in the group for your client notes so i'm going to name it ansible clients [Music] and here we need to type the ip address of a client machine so my client machine's ip address is 192.168.2 so before you come to this it's advised that you check the ip address on your client machine the simple command for that is ifconfig now once you type the ip address put a space and here we need to mention the username and the password for our client so i'll be logging in as the root user so this is the password and then the user which is root in my case that's it now you can save this file just clear the screen next we move on to our playbook we need to write the playbook so the extension for our playbook is yml which stands for yaman and as you can see here i have already written my playbook but i'll just explain to you how this is done so a yaml file always begins with three dashes this indicates the start of your yaml file now the first thing is you need to give a name to the entire playbook so i have named it sample book host is basically where this would be executed so as we saw earlier in our inventory i mentioned clients group name as ansible clients so we use the same name here the remote user is the user you'll be using at your client so in my case that's root and become true is basically to indicate that you need to set your privileges at root so that's called a privilege escalation now a playbook consists of tasks so we have here three tasks the first task i've named it to install httpd so what we're doing here is we are installing our httpd package which is basically the apache server and we are installing the most latest version of it hence the state value is latest the next task is running httpd so for the service the name is httpd because that's the service we need to start running and the state is started our next task is creating content so this is the part where we are creating our web page so copy because this is the file that will be created at the client the content will be welcome and the destination of the file will be www.html index.html as you know this is like a default path that we use to store all our html files now as you can see here there's quite a lot of indentation and when it comes to yaml although it's very simple to write and very easy to read the indentation is very crucial so the first dash here represents the highest stage that is the name of the playbook and all the dashes under tasks are slightly shifted towards the right so if you have two dashes at the same location they basically mean that they are siblings so the priority would be the same so to ensure that all your tasks are coming under the tasks label make sure they are not directly under name so yeah that's pretty much it so when you write your yaml file the language is pretty simple very readable indentations absolutely necessary make sure all your spaces are correctly placed we can now save this file next thing we need to check if the syntax of our yaml file is absolutely right because that's very crucial so the command to check the syntax of the yaml file is ansible playbook the name of your playbook syntax check so we have no syntax errors which is why the only output you receive is sample.yml which is the name of your playbook so our playbook is ready to be executed the command to execute the playbook is ansible playbook and the name of your playbook so our playbook is executed as you can see here gathering facts that's where all the facts of the note that the present state of the note is collected and sent to the local machine so it's basically to check that if the configuration changes that we are about to make is already made so it's not made we do not have the httpd package installed on our node so this is the first change that's made also if it's not installed of course it's not running that's the second change that's made so it's put into the running state and a final task which is create content is under the okay state this means that the contents already present in the client machine so i made it this way so that you can actually see the different states that's present so over here we have okay four so four things are all fine the facts are gathered two things are changed and one is already present two changes are made zero clients are unreachable and zero tasks have failed so this is the documentation that i was referring to previously that ansible provides automatically and is very useful as you can see so our next step we need to just check on our client machine if all the changes that we desired are made so let's move to our client so this is my client machine so to check this since we are installing the http package and hosting a web page the best way to do it is open your browser and type in localhost so there you go your apache server is installed and your web page is hosted hey there are you exploring test automation are you wondering why everyone is talking about selenium around you do you want to know what selenium is all about if yes this is the video for you simplylon brings to you a very simple tutorial on selenium what should you expect from this video before you start understanding any automation tool it's good to look back into what manual testing is all about what are its challenges and how automation tool overcomes these challenges challenges are always overcome by inventing something new so let's see how selenium came into existence and how did it evolve to become one of the most popular web application automation tool selenium suite of tools selenium is not a single tool it has multiple components so we will look into each of them and as you know every automation tool has its own advantages and limitations so we will be looking at what the advantages are and the limitations of selenium and how do we work around those limitations all right so let's get started manual testing a definition if you can say a manual testing involves the physical execution of test cases against various applications and to do what to detect bugs and errors in your product it is one of the primitive methods of testing a software this was the only method which we knew of earlier it is execution of test cases without using any automation tools it does not require the knowledge of a testing tool obviously because everything is done manually also you can practically test any application since you are doing a manual testing so let's take an example so say we have a use case you are testing say a facebook application and in facebook application let's let's open the facebook application and say create an account this is your web page which is under test now now as a tester what would you do you would write multiple test cases to test each of the functionalities on this page you will use multiple data sets to test each of these fields like the first name the surname mobile number or the new password and you will also test multiple links what are the different links on this page like say forgotten account or create a new page so these are the multiple links available on the web pages also you look at each and every element of the web page like your radio buttons like your drop down list apart from this you would do an accessibility testing you would do a performance testing for this page or say a response time after you say click on the login button literally you can do any type of tests manually once you have these test cases ready what do you do you start executing this test cases one by one you will find bugs your developers are going to fix them and you will need to rerun all these test cases one by one again until all the parks are fixed and your application is ready to ship now if one has to run test cases with hundreds of transactions or the data sets and repeat them can you imagine the amount of effort required in that now that brings us to the first demerit of the manual testing manual testing is a very time consuming process and it is very boring also it is very highly error prone why because it is done annually and human mistakes are bound to happen since it's a manual executions tester's presence is required all the time one is to keep doing manual steps step by step again all the time he also has to create manual reports group them format them so that we get good looking reports also send these reports manually to all stakeholders then collection of logs from various machines where you have run your tests consolidating all of them creating repositories and maintaining them and again since it's all as a manual process there is a high chance of creating manual errors there scope of manual testing is limited for example let's say regression testing ideally you would want to run all the test cases which you have written but since it's a manual process you would not have the luxury of time to execute all of them and hence you will pick and choose your test cases to execute that way you are limiting the scope of testing also working with large amount of data manually is impractical which could be the need of your application what about performance testing you want to collect metrics on various performance measures as a part of your performance testing you want to simulate multiple loads on application under test and hence manually performing these kind of tests is not feasible and to top it all up say if you are working in an agile model where code is being churned out by developers testers are building their test and they are executing them as and when the bills are available for testing and this happens iteratively and hence you will need to run this test multiple times during your development cycle and doing this manually definitely becomes very tedious and boring and is this the effective way of doing it not at all so what do we do we automate it so this tells us why we automate one for faster execution two to be less error prone and three the main reason is to help frequent execution of our test so there are many tools available in the market today for automation one such tool is selenium birth of selenium much before selenium there were various tools in the market like say rft and qtp just to name a few popular ones selenium was introduced by gentleman called jason huggins way back in 2004. he was an engineer at thoughtworks and he was working on a web application which needed frequent testing he realized the inefficiency in manually testing this web application repeatedly so what he did was he wrote a javascript program that automatically controlled the browser actions and he named it as javascript testrunner later he made this open source and this was renamed as the selenium core and this is how selenium came into existence and since then selenium has become one of the most powerful tool for testing web applications so how does selenium help so we saw all the demerits of manual testing so we can say by automation of test cases one selenium helps in speedy execution of test cases since manual execution is avoided the results are more accurate no human errors since your test cases are automated human resources require to execute automated test cases is far less than manual testing because of that there is a lesser investment in human resources it saves time and you know time is money it's cost effective as selenium is an open source it is available free of cost early time to market since you save effort and time on manual execution your clients will be merrier as you would be able to ship your product pretty fast lastly since your test cases are automated you can rerun them any point of time and as many times as required so if this tool offers so many benefits we definitely want to know more detail about what selenium is selenium enables us to test web applications on all kind of browsers like internet explorer chrome firefox safari edge opera and even the headless browser selenium is an open source and it is platform independent the biggest reason why people are preferring this tool is because it is free of cost and the qtp and the rft which we talked about are chargeable selenium is a set of tools and libraries to facilitate the automation of web application as i said it is not a single tool it has multiple components which we'll be seeing in detail in some time and all these tools together help us test the web application you can run selenium scripts on any platform it is platform independent why because it is primarily developed in javascript it's very common for manual testers not to have in-depth programming knowledge so selenium has this record and replay back tool called the selenium id which can be used to create a set of actions as a script and you can replay the script back however this is mainly used for demo purposes only because selenium is such a powerful tool that you should be able to take full advantage of all its features selenium provides support for different programming languages like java python c-sharp ruby so you can write your test scripts in any language you like one need not know in-depth or advanced knowledge of these languages also selenium supports different operating systems it has supports for windows macs linux even ubuntu as well so you can run your selenium test on any platform of your choice and hence selenium is the most popular and widely used automation tools for automating your web applications selenium set of tools so let's go a little more deeper into selenium as i said selenium is not a single tool it is a suite of tools so let's look at some of the major components or the tools in selenium and what they have to offer so selenium has four major components one selenium ide it's the most simplest tool in the suite of selenium it is integrated development environment earlier selenium ide was available only as a firefox plugin and it offered a simple record and playback functionality it is a very simple to use tool but it's mainly used for prototyping and not used for creating automation in the real-time projects because it has its own limitations like any other record and replay too selenium rc this is nothing but selenium remote control it is used to write web application test in different programming language what it does it basically interacts with the browser with the help of something called as rc server and how it interacts its uses a simple http postgate request for communication this was also called as selenium 1.0 version but it got deprecated in selenium 2.0 version and was completely removed in 3.0 and it was replaced by webdriver and we will see in detail as why this happened selenium webdriver this is the most important component in the selenium suite it is a programming interface to create and execute text test cases it is obviously the successor of the selenium rc which we talked about because of certain drawbacks which rc had so what webdriver does is it interacts with the browsers directly unlike rc where the rc required a server to interact with the browser and the last component is the selenium grid so selenium grid is used to run multiple test scripts on multiple machines at the same time so it helps you in achieving parallel execution since the selenium web driver with you can only do sequential execution grid is what comes into picture where you can do your parallel execution and why is parallel execution important because in real time environment you always have the need to run test cases in a distributed environment and that is what grid helps you to achieve so all this together helps us to create robust web application test automation and we will go in detail about each of these components so before that let's look at the history of selenium version so what did selenium version comprised of it had an ide rc and grid and as i said earlier there were some disadvantages of using rc so rc was on its path of deprecation and web driver was taking its path so if you look at selenium 2 version it had an earlier version of webdriver and also the rc so they coexisted from 3. onwards rc was completely removed and webdriver took its place there is also a four dot version around the corner and it has more features and enhancements some some of the features which are talked about are w3c webdriver standardization improved id and improved grid now let's look at each of the components in the selenium suite selenium ide is the most simplest tool in the suite of selenium it is nothing but an integrated development environment for creating your automation scripts it has a record and playback functionality and is a very simple and easy to use too it is available as a firefox plugin and a chrome extension so you can use either of this browser to record your test scripts it's a very simple user interface using which you can create your scripts that interact with your browser the commands created in the scripts are called selenis commands and they can be exported to the supported programming language and hence this code can be reused however this is mainly used for prototyping and not used for creating automation for your real-time projects why because of its own limitation which any other record and replay tool has so a bit history of selenium id so earlier selenium id was only a firefox extension so we saw that ide was available since the selenium version one selenium id died with the firefox version 55 that was id was stopped supporting from 55 version onwards and this was around 2017 time frame however very recently all new brand selenium id has been launched by apply tools and also they have made it a cross browser so you can install the extension on chrome as well as an add-on on firefox browser so they completely revamped this ide code and now they have made it available on the github under the apache 2.0 license and for the demos today we'll be looking at the new id now with this new ide also comes a good amount of features reusability of test cases better debugger and most importantly it supports parallel test case execution so they have introduced a utility called selenium side runner that allows you to run your test cases on any browser so you can create your automation using idec on chrome or firefox but through command prompt using your site runner you can execute this test cases on any browser thus by achieving your cross browser testing control flow statement so initially in the previous versions of idea there were control flow statements available however one had to install a plugin to use them but now it is made available out of box and what are these control flow statements these are nothing but your effects conditions the while loops the switch cases so on it also has an improved locator functionality that means it provides a failover mechanism for locating elements on your web page so let's look at how this id looks and how do we install it and start working on that so for that let me take you to my browser so say let's go to the firefox browser so on this browser i already have the id installed so when you already have an id installed you will see an icon here that says selenium id and how do you install this you simply need to go to your firefox add-ons here where it says find more extension so just type in selenium id and search for this extension so in the search results you see the selenium id just click on that and now since i've already installed here it says remove otherwise for you it is going to give you an add button here just click on the add button it will install this extension once it is installed you should be able to see this selenium ide icon here okay so now let's go ahead and launch this id so when i click on that it is going to show me a welcome page where it's going to give me few options the first option is it says record a new test case in a new project so straight away if you choose this option you can start recording a test case in which case it's going to just create a default project for you which you can save it later then open an existing project so you can open if you already have a saved project create a new project and close so i already have an existing project with me for the demo purpose so i'll go ahead and open that so i'll say open existing project and i have created a simple script what the script does is it logs me into the facebook using a dummy user mail your sorry username and password that's all it's very simple script with few lines and this is what it's going to do so what we will simply do is we'll just run the script and see how it works for that i'm just going to reduce the test execution speed so that you should be able to see every step of execution here all right so what i'll do now here is i'll just adjust this window and i'll just simply say run current test all right so i'll just get this side by side so that you should be able to see what exactly the script is doing okay so now you are able to see both the windows okay so now it's going to type in your user email here there you go and now it will enter the password and it has clicked on the login button so it's going to take a while to say login and since these are the dummy ids it is you are not able to log in here and you are going to see this error window fine that is what is the expected output here now on the id if you look here after i execute the test case every statement or every command which i have used here is colored coded in green so that means this particular step was executed successfully and then here in the log window it will give you a complete log of this test case right from the first step till the end and your end result is it says fb login which is my test case name completed successful let's look at few components of this id the first one is the menu bar so let's go to our id all right so the menu bar is right here on the top so here is your project name so either you can add a new project here or rename your project so since we already have this project which is named as facebook and then on the right you have options to create a new project open an existing project or save the current project and then comes on toolbar so using the options in this toolbar you can control the execution of your test cases so first one here is the recording button so this is what you use when you start recording your script and then on the left you have two options here to run your test cases the first one is run all tests so in case you have multiple test cases written here you can execute them one by one sequentially by using this run all test else what you can do is if you just want to run your current test this is what you would use then id has this debugger option which you can use to do a step execution so say for example now whenever i run the script it's going to execute each and every command here sequentially so instead if i just select the first command and say do step execution all right so what it does is the moment it finishes the first command which is opening of facebook right i think which is already done here yeah all right so once this is done it is going to wait immediately on the second command and it says positive debugger so from here you can do whatever you would like to do in case you want to change the command here you can do that you can pause your execution you can resume your execution here right you can even completely stop your test execution or you can just select this to run the rest of the test case so if we say run the test case what it is going to do is just going to simply go ahead and complete the com complete the test case now there is another option here which is you see the timer there it says test execution speed so to execute your test cases in the speed you want say whenever you're developing an automation script right and say you want to give a demo so you need to control the speed sometimes so that the viewer is able to exactly see all the steps which is being performed and this gives you an option to control that complete execution right so do you see the grading here so we have somewhere from fast to completely slow execution so the previous demo which i showed was i control the speed and then i executed it so that we could see every command how it is being executed all right so what's the next this is called as an address bar so whichever whenever whenever you enter an url here that is where you want to conduct your test and another thing what it does is it keeps a history of all the urls which have used for running your test then here is where your script is recorded so each and every instruction is displayed here in the order in which you have recorded the script and then if you look here you have something called as login reference so now log is an area where it records each and every step of your command as in when they get executed right so if you see here it says open https facebook.com and ok so that means this command was executed successfully and after the complete test case is done it gives you whether the test case passed or failed so in case there is a failure you will immediately see this test case is filled in red color also there is something called as reference here for example say if i click on any of this command the reference tab what it is going to show me is a details of this command which i have used in the script it gives you the details of the command as well as what the arguments have been used or how how is that you need to be using this particular command okay so now what we'll do is let's go ahead and write a simple script using this id so with this you'll get an idea how do we actually record scripts in id so for that i have a use case here a very very simple use case so what we will do is we will open amazon dot in then we'll search simply search for say product iphone and once we get that search page where all your iphones are displayed we will just do an asset on the title of the page simple all right so let's do that so first thing what i need is an url okay so first let me go to my firefox browser here and say amazon.com so why i'm doing this just to simply get the right url absolute url path here and so that i don't make any mistakes while typing in the url okay so i got this so let me close all this windows i don't need any of this let's minimize this all right so here what i'll do in the tests tab i'll say add a new test and name this test as amazon search done i'll say add now i'll enter this url which i just copied it from my browser okay and then i'll just say start recording so what a dot did was since i've entered the url in this address box it just opened the amazon dot in url now let's do our test case so in my test case what i said was i want to search for iphone once i have this i'm just going to click on my search button so now this gives me a list of all iphones and then i said i want to add an assertion on the title of this page so for me to do that what id gives me an option is i have to just right click anywhere on this page and you'll see the selenium id options here so in this i will select assert title and then i will close this browser so that kind of completes my test case so now take a look at all the steps which is created for me so it says open slash because i've already provided the url here so either you can replace it with your regular url or you can just leave it as it is so what i will do since this is going to be a proper script and i might be using this to run it from a command prompt also so i'll just replace this target with the actual url and then what it is doing it is setting a window size then there are whatever i did on that particular url on that website it has recorded all the steps for me so this is where it says type into this particular text box which is my search box and what did it type iphone this was the value which i entered now there was one more feature which i told you in this new id which had which i said it has a failover mechanism for your locating techniques now that is what this is now if you look here this id is equal to tab search textbook this is nothing but that search box where we entered the text iphone and it has certain identification through which this ide identifies that web element and that has multiple options to select that particular search box so right now what it has used is id is equal to two tab search box however if you know the different locating techniques you will be able to see here that it has other techniques also which it has identified like the name and the css and the xpath so how does this help in failovers say tomorrow if amazon dot in website changes the id of this element right you are not going to come and rewrite the scripts again instead by using the same script what it will do is if this particular id fails if it is unable to find the element using the first locator which is the id it simply moves to the next available ones and it tries to search for that element until one of these becomes true that is what was the failure mechanism which has got added it's a very brilliant feature because most of our test cases break because of element location techniques well let's come back to this so then we added an assert title right so what is asa title here it simply captures the title of that particular page and it checks this is all a very simple test case so what we will do now is we will stop the recording and then i have also given a closed browser so right now what i will do is i'll just comment this out why because if i just run this test case it's going to be very fast and you might not be able to catch the exact command execution what has happened all right so right now i'll just disable it so that it will just do all these test cases and it just stays there without closing the browser so now i'll just say run the current test day so your amazon n is launched okay it was typed in the iphone it's also clicked on the search so it is done so now if you look here since we are in the reference tab it is not able to show so let's go to the log and now let's see the log so it's going to be a running log so if you notice here the previous examples which we have run for facebook is also in the same lock so we will have to see the log from running amazon search because that's our test case so if you see here every command line right was executed successfully assert title was also done and your test case was executed successfully so it passed now what we will do is on this assert title i'll just modify this and let's say just add some text i just add double s here now this by intentionally i am going to fail this test case just to show you that whenever there is a test case failure how does the id behaves and how do you get to know the failures all right so i'll just run the disk test case again so before that let's close the previous window all right done and now here i'll also uncomment the close because anyway it's a failure which i'm going to see which i should be able to see it in the logs so i'll close the browser after the execution of test case okay so let's simply go and run the test case okay amazon.com is launched it should search for iphone now yeah there you go all right now it should also close the browser yes it has closed the browser and it has failed now see here now this is the line where our command filled why because the expected title was not there and if you look in the logs it says your assert title on amazon.in failed actual result was something different and it did not match with what we had asked it for so this is how simple it is to use your id to create your automation scripts so we saw all the components of id we saw the record button then i showed you the toolbar i showed you the editor box and also the test execution lock so now let's come to what are the limitations of this id with ide you cannot export your scripts your test scripts to web driver scripts this support is not yet added but it is in the works data driven testing like using your excel files or reading data from the csv files and passing it to the script this capability is still not available also you cannot connect to database for reading your test data or perform any kind of database testing with selenium webdriver yes you can also unlike selenium webdriver you do not have a good reporting mechanism with the id like say for example test ng or repotentie so that brings us to the next component of the suite which is selenium rc selenium remote control so selenium rc was developed by paul hammond he refactored the code which was developed by jason and was credited with jason as a co-creator of selenium selenium server is written in java it is used to write web application test in different programming languages as it supports multiple programming languages like your java c perl python and ruby it interacts with the browser with the help of an rc server so this rfsave server uses a simple http get and post request for communication and as i said earlier also selenium rc was called as selenium 1.0 version but it got dedicated in selenium 2.0 and was completely removed in 3.0 and it got replaced by what webdriver and we'll see why this happened and what was that issue which we had with the rc server so this is the architecture of selenium remote control at a very high level so when jason huggins introduced selenium you know the tool was called as javascript program and then that was also called as a selenium core so every html has a javascript statements which are executed by web browser and there is a javascript engine which helps in executing this command now this rca had one major issue now what was that issue say for example you have a test script say test.javascript here which you are trying to access elements from anywhere from the google.com domain so what used to happen is every element which is accessible e are the elements which can belong only to google.com domain like say for example mail the search or the drive so any elements from this can be accessible through your test scripts however nothing outside the domain of say google.com in this case was accessible say for example if your test scripts wanted to access something from yahoo.com this was not possible and this is due to the security reasons obviously now to overcome that the testers what they had to do was they had to install the selenium core and the web server which contained your web application which is under test on the same machine and imagine if you have to do this for every machine which is under test this is not going to be feasible or even effective all the time and this issue is called as a same origin policy now what the same origin policy issue says is it prohibits a java script from accessing elements or interacting with scripts from a domain different from where it is launched and this is purely for the security measure so if you have written a scripts which can access your google.com or anything related to google.com these scripts cannot access any elements outside the domain like as we said in the example yahoo.com this was the same origin policy now to overcome this what this gentleman did was he created something called a selenium remote control server to trick the browser in believing that your core your selenium core and your web application under test are from the same domain and this is what was the selenium remote control so if you look at again a high level architecture or how did this actually work first you write your test scripts which is here right in any of the supported language like your php or your java or python and before we start testing we need to launch this rc server which is a separate application so this selenium server is responsible for receiving the cellini's commands and these salinis commands are the ones which you have written in your script it interprets them and reports the result back to your test so all that is done to your rc server the browser interaction which happens through rc server right from here to your browser so this happens through a simple http post and get request and that is how your rc server and your browser communicate and how exactly this communication happens this rc server it acts like a proxy so say your test scripts ask to launch a browser so what happens is this commands goes to your server and then your rc server launches the browser it injects the javascript into the browser once this is done all the subsequent calls from your test script right from your test scripts to your browser goes through your rc and now upon upon receiving these instruction your selenium core executes this actual commands as javascript commands on the browser and then the test results are displayed back from your browser to your rc to your test scripts so the same cycle gets repeated right until the complete test case execution is over so for every command what you write in your javascript here or your test script here goes through a complete cycle of going through the rc server to the browser collecting the results again from the rc server back to your test scripts so this cycle gets repeated for every command until your complete test execution is done so rc had definitely lot of shortcomings and what are those so rc server needs to be installed before running any test scripts which we just saw so that was an additional setup since it acts as a mediator between your commands which is your selenius command and your browser the architecture of rc is complicated why because of its intermediate rc server which is required to communicate with the browser the execution of commands takes very long it is slower we know why because every command in this takes a full trip from the test script to your rc server to the core engine to the browser and then back to the same route which makes your overall test execution very slow lastly the apis supported by rc are very redundant and confusing so rc does have a good number of aps however it is less object oriented so they are redundant and confusing say for example say if you want to write into a text box how and when to use a type key command or just a type command is always confusing another example is some of the mouse commands using a click or a mouse down both kind of you know almost providing a similar functionality so that is a kind of confusion which developers used to create hence selenium rc got deprecated and is no more available in latest selenium versions it is obsolete now now to overcome these shortfalls webdriver was introduced so while rc was introduced in 2004 webdriver was introduced by simon stewart in 2006. it's a cross-platform testing platform so web driver can run on any platform like say linux windows mac or even if you have a ubuntu machine you can run your selenium scripts on this machine it is a programming interface to run test cases it is not an ide and how does this work actually so test cases are created and executed using web elements or objects using the object locator and the web driver method so when i do a demo you will understand what this webdriver methods are and how do we locate the web elements on the web page it does not require a core engine like rc so it is pretty fast why because web driver interacts directly with the browser and it does not have that intermediate server like the rc hat so each browser in this case what happens is each browser has its own driver on which the application runs and this driver is responsible to make the browser understand the commands which you will be passing from the script like say for example click of a button or you want to enter some text so through your script you tell which browser you want to work with say chrome and then the chrome driver is responsible for interpreting your instructions and to execute it on the web application launched on the chrome browser so like rc webdriver also supports multiple programming languages in which you can write your test scripts so another advantage of web driver is it supports various frameworks like test ng junit n unit and report and change so when we talk about the limitations of web driver you will appreciate how this support for various frameworks and tool help in making the selenium a complete automation solution for web application so let's look at the architecture of webdriver at a high level what is in webdriver so webdriver consists of four major components the first one is we have got client libraries right or what we also call it as language bindings so since selenium supports multiple language and you are free to use any of the supported languages to create your automation script these libraries are made available on your selenium website which you need to download and then write your scripts accordingly so let's go and see from where do we download this so if i go to my browser so seleniumhq.org right so if you're working with selenium this website is your bible so anything and everything you need to know about selenium right you need to come here and use all these tabs here in this website so right now what we are going to look at is what are those language binding so for that i'll have to go to this download tab here okay and if you scroll down here you will see something like selenium client and web driver language bindings and for each of the supported language of selenium you have a download link right so say for example if you are working with java here what you need to do is you need to download your java language binding so let us go back to the presentation so this is where your language bindings are available next so selenium provides lots of apis for us to interact with the browser and when we do the demo i'll be showing you some of this aps right and these are nothing but the rest apis and everything whatever we do through the script happens through the rest calls then we have a json wire protocol what is json javascript object notation it is nothing but a standard for exchanging data over the web so for example you want to say launch a web application through your script so what selenium does it it creates a json payload and posts the request to the browser driver that is here and then we have this browser drivers themselves and as i said there is a specific driver for each browser as you know every tool has its own limitation so does selenium so let's look at what these limitations are and if there are any workarounds for them cannot test mobile applications requires framework like apm selenium is for automating web application it cannot handle mobile applications mobile applications are little different and they need its own set of automation tool however what selenium provides is a support for integrating this apm tool which is nothing but a mobile application automation tool and using apm and selenium you can still achieve mobile application automation and when do you usually need this when your application under test is also supported on mobile devices you would want a mechanism to run the same test cases on web browser as well as your mobile process right so this is how you achieve it the next limitation so when we talked about the components of selenium i said that with webdriver we can achieve only sequential execution however in real time scenario we cannot just live with this we need to have a mechanism to run our test cases parallelly on multiple machines as well as on multiple browsers so though this is a limitation of webdriver but what selenium offers is something called as grid which helps us achieve this and we will see in shortly what the selenium grid is all about also if you want to know more details as how do we work with the grid how do you want to install that grid so do check out our video on simply learn website on grid third limitations so limited reporting capability so selenium webdriver has a limited reporting capability it can create basic reports but what we definitely need is a more so it does support some tools like say test ng report ng and even extent reports which you can integrate with selenium and generate beautiful reports powerful isn't it also there are other challenges uh with selenium like selenium is not very good with image testing especially for the ones which are designed for web application automation but then we have other tools which can be used along with selenium like auto it and securely so if you look at all this selenium still provides a complete solution for your automation needs and that's the beauty of selenium and that is why it makes the most popular tool of today for automation okay let's do a quick comparison between the selenium rc and the web driver so rc has a very complex architecture you know why because of the additional rc server whereas due to direct interaction with the browser webdriver architecture is pretty simple execution speed it is lower in rc and much faster than webdriver why because in web driver we have eliminated the complete layer of selenium server right that the rc server and we established a direct communication with the browser through browser drivers it requires an rc server to interact with the browsers we just talked about it and whereas webdriver can directly interact with the browser so rc again we talked about this as one of the limitations that we have lot of redundant abs which kept developers guessing as which api to use for what functionality however webdriver offers pretty clean apis to work with rc did not offer any support for headless browser whereas in web driver you do have a support for using headless browsers let's see the web driver in action now now for the demo we will use this particular use case and what this use case says is navigate to the official simply learn website then type the selenium in search bar and click on it and click on the selenium 3.0 training so we are basically searching for selenium 3.0 training on the simply learn website first let's do the steps manually and then we will go ahead and write the automation script so let's go to my browser on my browser what i'll do is let me first launch the simply learn website okay and here what my use case step says is i need to search for selenium and click on the search button so once i do that it is going to give me a complete list of all kind of selenium trainings which is available with simpler and what i am interested in is the selenium 3.0 training here once i find this on the web page i need to go and click on that all right so this is all the steps which we are going to perform in this use case okay now so for writing the test cases i'll be using an id which is eclipse i've already installed my eclipse and also i have installed selenium in this instance of my eclipse all right so if you can see the reference library folder here you will see all the jars which are required for the selenium to work next another prereq which is required for selenium and that is your driver files now every browser which you want to work with has its own driver file to execute your selenium scripts and since for this demo i'll be working with the firefox browser i will need a driver file for firefox now driver file for firefox is the gecko driver which i have already downloaded and placed in my folder called drivers now where did i download this from let's go ahead and see that so if i go back to my browser and if you go to your selenium hq dot website you have to go to this download tab here in the download tab when you scroll down you will see something like third party drivers bindings and plugins in this you will see the list of all the browsers which is supported by selenium and against each of this browser you will find a link which has the driver files now since we'll be using the gecko driver this is the link where you need to go to and depending on which operating system which you're working on you need to download that particular file now since i'm working on mac this is the file which i'm using if you're a windows user you need to download this zip file and unzip it so once you unzip that you would get a file called gecko driver for your firefox or a chrome driver for your chrome browser and then what you do is you just create a directory called drivers under your project and just place the driver files here so these are the two prereqs for your selenium one is importing your jar files like this and then having your drivers downloaded and keep them under a folder where you can reference to okay so now we'll go ahead and create a class i already have a package created in this project so i'll use this project and create a new class so i'll say create new java class and let's call this as search training i'll be using a public static void main and i'll click on finish so let's remove this auto generated lines as we do not need them all right now the first statement which you need to write before even you start writing the rest of your code is what you need to do is you need to define or declare your driver variable using your class webdriver so what i would do is i'll say web driver driver done all right now you will see that this id is going to flash some errors for you that means it is going to ask you to import certain libraries which is required by the web driver so simply just go ahead and say import web driver from org dot open sq dot selenium this is the package which we will need all right so you have a driver created which is of the class web driver and now after this i'm going to create three methods all right so first method i will have for launching the firefox browser okay and then i will write a simple method for searching selenium training and clicking on it this is the actual use case what we'll be doing and then third method i'm going to write is just to close the browser which i am going to be opening right so these are the different methods which i'll be creating and from the public static void main i will just call these methods one after the other okay so let's go ahead and write the first method now my first method is launching the firefox browser so i'll say public void since my return type is null or there is no return type for this let's call it as launch browser okay all right now in this for launching any browser i need to mention two steps now the first step is where i need to do a system.set property okay let's do that first and then i'll explain what this does i'll just say system dot set property so this accepts a key and a value pair so what is my key here my key here is webdriver dot gecko dot driver and i need to provide a value so value is nothing but the path to the gecko driver and we know that this gecko driver which i'm going to use here is right here in the same project path under the drivers folder correct and that is what the path which i'm going to provide here so here simply i need to say drivers slash gecko driver gec ko all right done and let me close this sentence all right now since i'm a mac user my gecko driver installable is just the name gecko driver if you're a windows user and if you're running your selenium scripts on the windows machine you need to provide a complete path to this including dot exe because driver executable on your machines is going to be geckodriver.exe so just make sure that your path which you mentioned here in the system.set property is the correct path okay then the next thing what we need to do is i need to just say driver is equal to new firefox driver okay so this command new firefox driver creates an instance of the firefox browser now this is also flagging me error why because again it's going to ask me to import the packages where the firefox driver classes present okay we did that now these two lines are responsible for launching the firefox browser form so this is done so what's my next step in the use case now i need to launch the website simply learn so for that we have a command called driver.get driver.get what it does is whatever url you're going to give it here in this double quotes as an argument it is going to launch that particular website and for us it's a simply learn website so what i do as a best practices instead of typing out the url i go to my browser launch that url which i want to test and i simply copy it come back to your eclipse and just simply paste it so this ensures that i do not make any mistakes in the url okay so done so our first method is ready where we are launching the browser which is our firefox browser and then launching the simply learn website now the next method what is my next method in my next method i need to give the search string to search selenium training on this particular website now for that we need to do few things what are those few things let's go to the website again all right so let me relaunch this let's close this okay let me remove all this and let's go to the home page first okay this is my home page so as you saw when i did a manual testing of this i entered the text here so now since i have to write a script for this first i need to identify what this element is for that what i'm going to do is i'm just going to say right click here and i'll say inspect element all right now this element let's see what attribute it has which i can use for finding this element so i i see that there is an id present so what i'm going to do is i'm just going to simply use this id and then i'll just copy this id from here go back to eclipse let's write a method first so i'll say public void and what do we give the method name say search training or just search all right now in this i need to use a command called driver dot find element by id is what i'm going to use as a locating technique and in double quotes the id which i copied from the website is what i'm going to paste here okay and then what am i going to do on this element is i need to send that text the text which i'm going to search for which is selenium so i'll just say send keys and whatever text i want to send i need to give it in double quotes so for that selenium so this is done so now i've entered the text here and after entering the text i need to click on this button so for that i need to first know what that button is so let's inspect that search button okay now if you look at the search button other than the tag which is span and the class name i do not have anything here all right so what i can do is i can either use the class name or i can write an x path since this is a demo which we have already used id locating technique i would go ahead and use the x path here so for me to construct an x path uh i will copy this class first okay and then i already have a crow path installed on my firefox so i'll use the crow path and first test my xbox so i'll just say double slash let's see what was that element it has a span tag okay so i'll have to use pan and at class equal to and i'll just copy the class name here and let's see if it can identify that element yeah so it is able to identify so i'll just use this x path in my code so i'll go back to eclipse and i'll say driver dot find element by dot x path and the x path which i just copied from crow path is what i'm going to paste here and what is the action i need to do here i need to say click done so i have reached a stage where i have entered this selenium okay and then i have clicked on the search button once i do this i know that expected result is i should be able to find this particular link here selenium 3.0 training okay and i should be able to click on that so for that again i need to inspect this so let's inspect this selenium 3.2 all right so now what are the elements this has now this particular element has attributes like it has a tag h2 then it has got some class name and some other attributes so i would again would like to use a xpath here now this time while using the x path i'm going to make use of a text functionality so that i can search for this particular text so i'll simply copy this i'll go to my crow path the tag is h2 so i'll say simply h2 okay and here i'll say text equal to and this is the text which i copied i missed out that yes there so i'm just going to add an s okay so let's first test here whether it is able to identify that element yeah so it is able to identify so can you see a blue dotted line it is able to show us which element it is identified so i'll copy this x path now and let's go to my ide eclipse so now here what i need to do is i'll have to again simply say driver dot find element by dot x path and paste the x path which we just did and then again i have to do a click operation all right so technically we have taken all the steps of the use case and we have written the commands for that now let's add an additional thing here say after coming to this page after finding this we want to say print the title of this page now what is the title of this page if you just hover your mouse on this it says online and classroom training for professional certification courses simply so what i will do is after doing all these operations i will just print out this page title on our console so for that i have to just do this driver dot so let's do a sysop so i'll say this out system.out.println okay and here i would say let's add a text here the page title is and then let's append it with driver dot get title so this is the command which we'll be using to fetch the page title done now what is the last method i need to add just to close the browser all right so let me add a method here i'll say public void close browser and it is one single command which i need to call i will say driver dot quit okay and then i need to call all these methods from my public static void main so i let me use my class name which is this so i'm going to create an object obj is equal to new class name and then using this object first is i need to call the method launch browser and then i'll call the method search right and then i'll call the method close process done so technically our script is ready with all the functionality which we wanted to cover from our use case now there are few other tweaks which i need to do this and i'll tell you why i need to do this now for example after we click here right after we click on the search if you observed on your website it took a little while before it listed out all the selenium trainings for us and visually when you're actually doing it you wait for the selenium 3.0 training to be available and then you click on that now same thing you also need to tell your scripts to do that you need to tell your scripts to wait for a while until you start seeing the selenium 3.0 training or it appears on your web page there are multiple ways to do that in your script and it is a part of overall synchronization what we call where we use kind of implicit and explicit kind of favorites now since this is a demo for demo purpose what i'm going to do is i'm going to use a command called thread.sleep and i'm just going to give an explicit weight of say three seconds so you can use this mainly for the demo purposes you can use a thread.sleep command now this thread.sleep command needs us to handle some exceptions so i'm just going to click on add throws declaration and say interrupted exception now same thing i'll have to do it in my main function also okay so let's do that and complete it all right so this is done so by doing this what am i doing i'm ensuring that before i click on the selenium three dot training we are giving enough time for the script to wait until the webpage shows this link to the selenium 3.0 training that's one thing i'm doing all right and also now since you're going to be seeing this demo through the video recording the script when it starts running it is going to be very fast so you might just miss out saying how it does the send keys and how did it click on the search button for us to enable us to see it properly i'll just add some explicit weights here just for a demo purpose so after entering the keys right so what i'll do is i'll just give a simple thread dot sleep here okay so probably a three seconds or two seconds wait should be good enough okay three seconds wait should be good enough here so that we should be able to see how exactly this works on your browser when we execute this okay now our complete script is ready so what i'll do is i'll just save the script and then we will simply run the script so to run the script i'll just say right click run as java application okay it says ask me to select and save i have saved the script now so let's observe how it runs okay the simplylearn.com the website is launched so the selenium text has been entered in the search box it has clicked on the search okay all right so now it did everything whatever we wanted it to do all right so since we are closing the browser you are unable to see whether the selenium three dot training was selected or not however what i have given here is to fetch the title after all these operations were complete and if you see here the complete operations was done and we were able to see the page title here okay so now what i'll do since we are unable to see whether it clicked on the selenium three dot or training or not i'll just comment out the close browser the command okay so we will not call the close browser so that the browser remains open and we get to see whether did it really find the training link or not okay so let me close this window we don't need this firefox window close all tabs and then i'll just exactly execute the script so i'll say run as java application so save the file okay simplylearn.com is launched so search text is entered now it's going to click on the search button yes all right so we've got the search results it should click on selenium theodore training and yes it is successfully able to click on that all right so now it's not going to close the browser because we have commented on that line however it did print us the title here all right so this is a simple way of using the selenium scripts selenium grid so grid is used to run multiple test scripts on multiple machines at the same time with webdriver you can only do sequential execution but in real time environment you always have the need to run test cases in distributed environment and that is where selenium grid comes into picture so grid was conceptualized and developed by patrick the main objective is to minimize test execution type and how by running your test parallely so design is in such a way that commands are distributed on multiple machines where you want to run test and all these are executed simultaneously what do you achieve by this methodology of course the parallel execution on different browsers and operating system grid is pretty flexible and can integrate with many tools like say you want a reporting tool integrated to pull all the reports from the multiple machines where you're running your test cases and you want to present that report in a good looking format so you have an option to integrate such report okay so how does this grid work so grid has a hub and node concept which helps in achieving the parallel execution let's take an example say your application supports all browsers and most of the operating systems like as in this picture you could say one of them is a windows machine one of them is a mac machine and another one is say a linux machine so your requirement is to run the test on all supported browsers and operating system like the one which is depicted in this picture so what you have to do is first thing is you configure a master machine or what you also call it as a hub by running something called a selenium standalone server and this talent standalone server can be downloaded from the selenium hq website using the server you create a hub configuration that is this node and then you create nodes specific to your machine requirement and how are these nodes created you again use the same server which is your standalone selenium server to create the node configuration so i'll show you where the selenium server can be downloaded so if we go back to our selenium hq website so you can see here right on the top it says selenium standalone server and this is the latest version available so you just need to download once you download this keep a version of this one on your master machine that is your hub and this should also be present on your node machines and there is a certain command which you can use to install the server as a node and as a master depending on your configuration so once that is done you have your master and your master or the hub and the nodes created your master can control on which nodes you want to execute test now say if this is a mac machine this is a windows machine through your master or your hub you control on which machine you want to execute what is then also you could have a multiple combination of tests like say for example smoke test to be run on all these nodes that's the kind of configuration you can create or a certain component test to be run on say only windows 10 with an edge browser or say regression test on mac machine so any such things whatever configurations you have for your node what happens is the hub picks the right node which confirms to your requirements or the configurations which you have set up and then sends a command to the node to execute those test cases on the node everything is controlled through your master machine only here what happens on the node so once the node gets the command from the master the node actually runs all the test cases which is it which it intends to run on the specific browser which you have mentioned in your configuration file so this is a very high level architecture and how a grid works so if you want to know more details of how the grid works you can refer to our video on simply learn all right again limitations so what limitation does this grid have now the first one is it has relatively poor scalability compared to many modern applications it is pretty static each server is configured in advance with a subset of required browser instances available so if you want to vary this you will need to reconfigure so any configuration tomorrow on any machines you want to change it cannot be done automatically you need to completely reconfigure that node and then add it back to the grid thirdly although it is available to run on virtual servers it is actually not designed as a cloud native application and because of as a result of this it isn't optimized to take advantage of things like say distributed storage or dynamic scaling or even automatic failover so these are some of the limitations of the grid advantages of selenium speed and accuracy of course since there is no manual execution your test can run faster and with grit it helps in parallel execution and you can run or execute large volumes of test within a very short time frame let's take an example say before just before the release of your product say during the last lap of your testing you find a bug and of course it has to be fixed so the developer fixes the bug and you want to run all the test cases pertaining to that area now can you imagine that if you have to do this testing manually how long it's going to take so by automation you can achieve this in a very short duration of time and it will also help you release your product in time with very minimal or absolutely no human errors the second advantage it's an open source so it's available free of cost so anyone can download it and start using it it supports wide spectrum of programming languages we have been seeing throughout this presentation it is not restricted to any particular programming language so whatever language you are comfortable with you have an option to use that and write your selenium scripts selenium has support for almost all browsers and operating system we also talked about headless browser support so this helps you create test cases once using any of the browser and run them across all browsers and all operating system thus saving you lot of time from manual execution and helping and achieving a very broad test coverage and we also saw that pretty easy to use tool right so you can check out all our videos and you can it's just a matte matter of time that you can master the skill of writing your automation scripts reusability of course like any programming language it provides you a mechanism to reuse your code and avoid redundancy well again let's see an overall what are the limitations of selenium itself now since selenium is open source that is one of the biggest advantage now there is a little disadvantage which comes with that and what is it you do not have much technical support because it's an open source code however there are loads and loads of documentations and forums available which you can definitely refer to and selenium as we said it is only for automating web applications it cannot handle the mobile applications or even the desktop application however it does provide support for integrating tools like apm for mobile testing selenium is not very good with image testing especially designed for your web application tools again we have tools like autoit and cqla which can be integrated with selenium selenium webdriver has limited limited reporting capability but again it does provide us support for integrating tools like test ng report ng and extent report it does not have test management capabilities it need not right not one tool need not have each and everything what is required but there is always a way to integrate it so selenium does provide a way to integrate any of the test management tools now since selenium supports multiple programming language the developer of test automation will require to have some basic knowledge of any of the programming language it supports to write effective automation scripts right so this you could look at it as an advantage or as a disadvantage so overall if you look at all this right selenium still provides a complete solution for our automation need and that is why we can still say that selenium is one of the most popular tool used in the industry today for your web application automation how to install selenium on windows system selenium installation is a three-step process so it has certain prereqs the first prerequisite you need to have java on your system so we will be installing java first and then we will be working with eclipse id so we will be installing eclipse and then we will install selenium for java we will install the version java 8 and for eclipse we have a version 4.10 this was the last stable version which was released in december last year so i'll be using that version and selenium we will download the latest 3.14 version okay so let's get started with our first step which is the java installation so to install java let's go to the browser and simply just search for java 8 download so now you will see that there is an oracle site which is listed there and that is where you would be downloading all your java package so go ahead and click on that and for you to download any jdk package from the oracle site you need to create an account so if you already have one you just need to login using that account and then you can download any of the jdks and if you do not have one please go ahead create a new account on the oracle login to that account and then you can just download the java it so since i already have an account and i have already downloaded the package but i'll show you how and where to download it from so in this page if you scroll down so you will see this java development kit 8211. so this is the version we'll be downloading it so click on the accept license agreement and then since we are working on the windows system today so we will be downloading this the windows package so just click on that and it will get downloaded in your downloaded folder and as i said i've already downloaded the packages so here it is what i've done is i've just created a directory called installers and i'm going to be keeping all my installables here so here i have a folder called java installer and this is where my installable is so now that we have this file so we will just go ahead double click on it and launch this installer the installer is launched and just click on run so this will take a few minutes to install java the installer is launched now just click on the next button here so here for the installation directory you can change the directory to the choice of whatever drive and the folder structure you want to i would like to leave it as default here and we just go and click on next and then the java installation is in progress so let's wait until this is completed it really shouldn't take too much time maybe just a few more minutes here okay accept the license term just click on next we leave the destination folder as it is so jdk8 is successfully installed on your system so close the installer now and let's go ahead and check whether the installation is done properly so for that what i'll do is i'll go to my command prompt and i'll send just say java minus version so it says java version 1.8 and this tells us that the java is installed successfully now after this installation there are couple of configurations which we need to do and what are those configuration one is you need to set the path variable and then you we are also going to set a java home directory so for that first let's go ahead and check where is the java installed actually let's figure out the directory first so if you remember the directory structure where the java got installed was in program files java i have there are certain previous versions which are installed and then uninstalled so that is why you see some residuals here sitting here let's not worry too much about that instead let me go to the latest one what i have installed which is this okay and there is a bin folder here and this is the path which we need to set in our path variable so what i will do is i'll just copy this path and then go to your control panel here go to your where is my system yeah so click on the system go to advanced system setting and here in the environment variables find the path variable okay and then say edit now what are we doing here in the path variable is we are going to add the java bin directory to the path be very careful whenever you are editing your path variable do not overwrite anything always go into the edit mode go towards the end here and then just say control v paste the path which you have just copied from the explorer window that's it now just say okay done so your path setting is done so what's the next one we need to do we need to add a new environment variable called the java now what i'll do for that is i just say new i just type java home here and what is the value of this we need to set we need to set the same path but without the bin directory so we just need to set the path till your java directory that is this so we'll just copy the path again and paste it here that is all just say okay click on ok click on ok here and we are done so again let's go to our command prompt and just say java minus version so everything seems to be fine so now successfully we have installed java on the system so what is our next installation step what we have now we need to install the eclipse so let's go back to the browser again so to download eclipse we will be downloading the package from the eclipse.org so when you go here to eclipse.r you can see the latest version which is available and the latest version available when this video was made was 2019 06. so especially with eclipse since it's an open source i prefer to work with the last stable version and so does most of the developers do and hence that is the reason why i have picked up the version which is like last year's version which is 4.10 which was released in last december so you can always choose to go with the latest version but then if there are any issues and if you're like first time working with the eclipse you're going to get confused as where these issues are coming from right so i would still recommend that you use the last stable version which is available with your eclipse so now to get the last table version what you need to do is go and click on this download packages and here if you scroll down this page you can see here more downloads so there is a list of all the previous releases of app eclipse which is available and this is what we need to download so just click on that 4.10 version and then click on the os on which you want to install eclipse for us it is windows so i'll just click here on the 64-bit windows and then click on the download and you will be downloading the complete package so once you download this is what it will look like so let's go back to our directory of installers so this is the installer for the eclipse which i got now what's the next step i need to do just launch this installer and install eclipse so i'll just say double click on this i'll say run so here you'll see multiple options here for eclipse installation so depending on your requirement you can go ahead and install any of these packages so for us we just need an eclipse id for java developer so i'll select this and i'll say install so again you'll have a choice of directory where you want to install so i have chosen d drive here this is the default directory name it takes which is ok we can leave it as it is and then also you have an option to create a start menu entry and desktop shortcut so just leave the default selection as it is and go ahead and click on install so this will take a while to install the eclipse this is select all you can close this window this is select all and accept it okay so the installation has been completed successfully so let's go and click on this launch and let's see the first window what opens when you launch the eclipse you need to specify a workspace directory now what is this workspace directory so this is a directory or a folder wherein all the java files or any programs or any artifacts which you are going to create through eclipse will be stored in this particular folder so this could be any location on your system so this is you can go ahead browse the location and change it so for in our case what we will do is i'll go to the d drive and i already have a directory so here i'll create i'll just clear select this folder and then create a folder called workspace i'll say my workspace and then i'll say launch so every time i open the eclipse right so this is going to take as my default workspace and all my programs all my java scripts or my automation scripts are getting are going to be stored in this particular location so we'll say launch so this is a welcome window which opens we can just close this and there we go the eclipse is open with a certain perspective so there are certain windows here which we do not need let's just close them so now the first thing what you do after launching the eclipse is go ahead and create a new project so i'll say file new and since i'm going to be using java with selenium i'll say create java project so give a project name let's say my first project now you have an option here to select the jre which you want to use so we just installed this jdk 1.8 okay so i'm going to click on use default jre otherwise you also have an option to use a project specific gr for example i could have two different projects where one project i'm going to be working with gre 1.8 and there is another project which i want to work with the latest java maybe java 12 and i can have more than one java installed on the machine so this gives me an option to select whichever java i want to work with so if you have another java installed here it will show up in this list and you can just go ahead and select that now since we have only one java installed on our machine which is java 1.8 i will say use default grd which is 1.8 and i will click on finish now if you observe this folder structure the project which is created see all the reference libraries to this particular java have been created here now we are ready to create any kind of java programs in this project so now we have successfully done the second step of our installation which is the eclipse installation after this we need to install the selenium so again let's go back to the browser and see what files we need to download to install selenium so let me go to my browser and here i will be going to the seleniumhq.org so if you are working with selenium this particular website the seleniumhq.org is going to be a bible everything and anything related to selenium is available in this website whether you want to download the files whether you want to refer to the documentation anything regarding to selenium is available here so what we want now is the installables for selenium so here go to the download tab now for you to install selenium and start working with selenium there are three things which are required for you to download one is a standalone selenium server so this is not required immediately when you get started with selenium however when you start working with remote selenium web driver you would be requiring this when you have a grid setup you will be requiring the standalone server so for that what you can do is you can just download the latest version available here so when you click on that it will download the file into your download folder so this is one particular file which you need to keep next selenium client and web driver language bindings now in today's demo we will be looking at selenium with java so that means my client package of java is what i need to download so whatever programming language selenium supports we have respective downloadables available with it say if you're working with python then you need to download your client library for python and since we are working with java you need to download this package so simply what you need to do click on this link and it will download the java package for you which are basically the jar files so we have client libraries now and then there is another component what we need now with selenium you are going to be automating your web browser applications correct and you also want your applications to run on multiple browsers so that means your scripts the automation scripts which you create should be able to run on any browser selenium works with multiple browsers like edge safari chrome firefox and other browsers even it has a support for headless browser now every browser which it supports comes with its own driver files now say for example we want to say work with firefox driver so that means for us to start working with firefox browser we need to download something called as a gecko driver here and if you want to work with chrome browser you need to install the chrome driver so depending on what browsers you'll be testing with go ahead click on each of this link and download the latest driver files now since we are going to be working with firefox in this demo what i need to do is i just need to click here on the latest link so when i click on the latest link it is going to take me to this driver files so driver files are specific to each of the operating system so if you go down here you will see there is a separate driver file available for linux for mac and for windows so depending on which operating system where you'll be running your test download that particular driver file and this is the driver file i need because we are working on windows machine so these are the three different packages which we need to download from the seleniumhq.org for us to install selling so let me show you the folder where i've already downloaded all this so if you see here selenium java 3.141.59 okay this is nothing but our client library which we saw here let's go back to the main page here that is this so once i download this this is a zip file after i unzip the file this is the folder structure i see and let's see what is there in this folder structure so there are two jar files here and then in the lips there are multiple jar files and we will need all this to work with selling it and then we also downloaded the driver files so what i did was after downloading those driver files for the browser i created a directory here called drivers and i've kept all my browser drivers here so i have a driver file downloaded for chrome i want to work with firefox so i have a gecko driver here and then for internet explorer that's it so this is all we need so once we have all this what you need to do is go to your eclipse in the eclipse right click on the project which you have created and then go to the build path and say configure build path go to the libraries tab here now do you see this jre libraries here this is what got installed first and now similarly we are going to add the selenium jars to this library and how do we add that on your right you can see this add external jar click on add external jars go to your folder where you have downloaded your selenium which is this select all the jar files which is available so i have two jar files here i'll just say click open again i will click on add external jar now from the libs folder i will select all this file so select all the five jars and click on open so you should see all the seven jar files here so once you have this just say apply and close now if you look into your project directory here you will see some a folder called referenced library and this is where you will see all the selenium charts here this is a very simple installation in eclipse when you want to install selenium you just need to export all the jars of the selenium into eclipse and now your system is ready to start working with selenium scripts so for us to install java let us go to our browser and simply what i'll do is i'll just google for java 8 download okay so we would select the oracle site for downloading the sd okay so once you go here you can see that we have something called as java software development kit 8 u211 so this is the version we'll be working with so in order for you to download you need to have an oracle account so since i already have one i have directly been logged into that so if you do not have an oracle account please go ahead create an account for yourself so that you can download any of the java installers now let's go for and download our package so i'll be downloading this package for that i need to click on accept license agreement and then since we are going to be working on mac this is the package which i am going to download so let's go ahead and click on that and now the download is going to start however we will not wait for the complete download to happen since i have already downloaded the package and i have kept it in one of my folders so let me show you where that is so let me go to my finder so what i have done is i have created a folders called installers and all the software which we are going to require today for us to work with selenium is what i have downloaded it and kept it here so we will go step by step so that you understand what each of these packages are so as you see here i have the jdk 8 which i've already downloaded so let's go ahead and install jtk so how do we install jdk so you have the dmg file here you just go ahead and double click on that okay and then i'll double click on this package and then we will say continue and install so it's going to ask me for my password okay i've given my password and i'll say install the software probably it will just take a couple of minutes to install java okay there you go so now the java has been installed so i let me close this window let me close this window tool and then now i'll go to my terminal window so let's open my terminal and here let's say java minus version bingo so there we have java 1.8 installed on our system now there are couple of other things which you need to do apart from installation of your jdk you also need to set some system variables right so in mac how do you set the system variables or the environment variables you do that in your bash profile so let's see how we do that all right so what i will do is i'll use an editor which is my vi okay and then i'll open my bash profile so what is bash profile it's a config file for your bash cell this is mainly used to export variables in your shell like in windows we are going to do the system variable setting from the advanced setting right so similarly we are going to do that here so i will say vi dot bash underscore profile so this is the file where i will be adding java home path now i have other parts here already written because i am working with other multiple software on this machines and now i'll go to the end of this file and i'll say export java home equal now what is the path we need to give for java home we need to give the path where the java has been installed on your system okay first let's go and find out that path so let's go back to our finder and then i have a folder called library here so here there is a java and then there is something called as java virtual machines so if you have a single java or multiple java versions installed so all those versions you are going to see under this directory which is java virtual machines and java one dot is is what we installed just now so let me go and open that folder there is something called as content and then home and in this you have your jre and bin folder so that means i need to give my java home path until here till this home directory so simply what i am going to do is i'll just go here say get info because i want to just copy the path and what you can do is you copy this path okay and then go back to your shell and here i'm going to paste this path plus i want to till the home directory so i'm just going to type out h o me home that is it so you have now set your java home environment variable so simply you do save the file escape colon wq and come out of it and now what you need to do for this batch profile to be activated you just exit out of this terminal okay close this terminal window and i will go ahead and open another terminal so let me close this info text so here let me again give java minus version okay so everything seems to be working fine so this completes our java installation now your system is ready and you have your java up and running other thing which you also can do is go to your system preferences okay here you will see this java control panel this is another way of finding out that your java has been installed successfully on the machine so you can just open this java control panel and if there is any settings which you want to do here you can do it through your control panel also all right let's close this let's minimize this window and this folder windows so we have successfully done a step one so what's our step two we need to go ahead and install the eclipse now same thing what i will do i'll go to my browser okay and now i'll open another tab here and i'll say eclipse download and we'll go to this eclipse.org now here whenever you're going to do this installation you will see the latest version appears here my recommendation to you is instead of if you are working first time instead of working on the latest version available i would suggest that you work with the last stable version and that's exactly what we are going to do today so for me to go to the previous versions i just need to go here on this download packages click on it okay so once you click on that on this page if you scroll down you will see something called as more downloads right in this you will see all the bills of eclipse which is available all your previous versions so since today i'll be working with 4.10 version i will go ahead and click on this photo tip this 4.10 was released last year this is 28 in december okay and now when you look at this installer you have multiple packages available here so depending on your need you can choose which package you want to work with so today for our demo what i would be using is eclipse id for java developers so since we are going to be working on mac i will go ahead and click on this downloadable and then i'll click on this download which is going to download this particular file for me i've already downloaded this file and kept it in my folder let's go to my folder okay it's not opening so let me click from here and then let's go to our selenium demo folder and in that installers okay so now do you see this eclipse java 2018 12 r blah blah blah dot dmg so this is the file which we are going to be using to install eclipse now as simple as what we did for jdk i am going to double click on this and install the eclipse let's see how the eclipse gets installed all right so now i'll just move this folder into applications done so my eclipse is ready now so simply let's go ahead and launch so what i'll do i'll just simply search for that on my system eclipse there you go and i'll just double click this is an application downloaded from the internet are you sure you want to open it yes please go ahead and open it okay it is launching it so did you see how simple the eclipse installation is it's just downloading the package and just installing it on your mac machine all right when you start the eclipse for the first time you need to set your workspace so workspace is place where you're going to keep all your folders all your java code or whatever you're going to do in eclipse all under that particular directory um so right now it gives me a default directory rather what i will do is since i have created a separate folder for our demos i will go ahead and select that folder which is my simply demo and then in this i will add a folder called workspace okay so that means in this particular location whatever files i'm going to create whatever code i'm going to write in selenium is going to be stored at this location and then i'll go ahead and say launch so it will take a minute to load your workbench yeah there it goes so when you launch your eclipse right so let me just make this window bigger the first screen what you are going to see is the welcome screen just go ahead and close the welcome script and this is the first view of your eclipse what you're going to get so let's close out all the windows which we are not going to be working today and i just need this main screen here now what is the first step you're going to do once you have the eclipse i would go ahead and create a project first so i'll simply say open file new java project let's give a nice name to this since this is a first project which is going to be created so i'll say selenium first project now if you observe here so we have a jdk already installed and eclipse needs a jdk to work with and what it is done while i create my new project it has already given me an option to use the execution jre which you already have in the system that's why it has already chosen java one dot it in case you have multiple javas installed in in that case what you can do is you can always go here select this and select the java of your choice so say if you have got the latest java which is java 12 installed on your system it will give you an option here along with java software 8 it will give you java 12 here so you can go ahead and select that for this particular project that option is also available now since we have only one jtk installed here i will go ahead and create this project so i'll just click on finish and it's going to create a project from so there are few default things which it grades created one is definitely the gre system libraries right this is what we told eclipse when we created project that my java is 1.8 so what it has done is it has imported all this jar file which it requires for our java programs to work in our eclipse id and then it creates a src which is your source directory wherein we will be creating all our java code or selenium descripts okay good so far so this is a simple way of installing your eclipse and getting your eclipse ready to work with your selenium all right so now let's go back so we have done the installation of java we have done the installation of eclipse now what's left just the selenium right so what is that we need for selenium let's go back to our browser okay and this time we will go to selenium hq dot org so this is going to be a bible for you if you're going to be working with selenium so in this website what i do is i go to downloads and there are two things which we need to download for us to work with selenium one is the selenium standalone server so when you are going to be working with remote selenium webdriver is when you would require this however i would suggest that we download whatever is required for us even though we might not use it immediately so what i just need to do is i need to click on this and it's going to download this server for me okay so i have already downloaded it and i'll show you that and then what we need is the selenium client and web driver language bindings okay we need a client so what is the client which you are going to work in we are going to working with java client with the selenium and for that i need to go ahead and download this if you're working on say c sharp or ruby or python then you have the respective packages which you can go and download now since we are going to be working with java here i would go ahead and click on this and this is going to download the java client for me now as i said since i've already downloaded it so let's go ahead and look at what are those downloadables what we have so let's close out this window now if you see here one is the selenium server standalone which we just downloaded and then when you download the client libraries what it does it it creates a zip file it downloads a zip file and it unzips it and what is contained in that particular folder so if you look here selenium java 3 in this folder you have multiple jar files with which is what we'll be working with selenium so here you have two jar files here and then under the folder libs you have other jar files now these are the files which we will need and now let's see how do we utilize this file in our eclipse so for that let's go back to our eclipse okay so let me maximize this window so in eclipse first thing what you need to do is click on your project okay right click on that and then you have something called as build path go to the build path and say configure build path now here in this configure build path what you need to do is go to your libraries okay go to add external jars now when you click on that this is where you need to select all your jar files from what you have downloaded from the selenium site so that is present in our selenium java 3. so what i'll do i'll simply select these two first okay then again you click on the external jar files what we have is an ellipse so select all the jar files in the list say open you have all the java files what we need identified here go ahead and click on apply and close once you do that how do you know whether all those jars have been imported correctly go to the reference libraries here you should be able to see all the selenium jars which we have downloaded to be listed out here and so it lists here so that means we have successfully imported selling hello this is matthew from simply learn and today we're going to go through part one of a two-part series on interview questions that you are likely to be asked if you are interviewing for a job that focuses on devops so there are approximately seven sections that we cover in devops we go from general devops questions source code management with tools such as git continuous integration and here we'll focus on jenkins continuous testing with tools such as selenium and then you also have the operation side of devops which is your configuration management with tools such as chef puppet and ansible containerization with docker and then continuous marching with tools such as nagios so let's just get into those general devops questions so one of the questions that you're going to be asked is how is devops different from agile and the reality is is that devops is a cultural way of being able to deliver solutions that's different from agile if we look at the evolution of delivery over the last five to ten years we've gone from waterfall based delivery to agile delivery which is on sprints to where we are with continuous integration and continuous delivery around dev ops the whole concept of devops is culturally very very different from agile and the difference is is that you're looking at being able to do continuous releases what does that mean the difference is is that you want to be able to send out code continuously to your production environment that means the operations team the development team have to be working together that means that any code that gets created has to be able to go to production very quickly which means you need to be testing your code continuously and then that production environment must also be able to be tested continuously and any changes or any errors that come up have to be communicated effectively and efficiently back to the dev and op team another area in which i see that devops is different is really the response that we have for how we engage with the customer so the customer is coming to your website to your mobile app to your chat bot or any digital solution that you have and has an expectation when you're going through and actually doing a devops paradigm the old model would be that you would capture requirements from the customer then you do your development then you do your testing and there would be these barriers between each of those as we move faster through from waterfall to agile what we saw is that with agile we're able to respond much faster to customer demands so instead of it being weeks or months sometimes in some cases years between releases of software what we saw it would was a transition to weeks and months for releases on software now we see with devops is that the release cycle has shrunk even further with the goal of continuously delighting the customer how further has that release cycle shrunk to there are companies that have gone from having releases of once a week or once every two weeks or once a month to now having multiple releases a day indeed some companies have up to 50 releases a day this isn't something to also bear in mind is that each of those releases are tested and verified against test records so that you can guarantee that the code that's going to production is going to be good continuous code so what are the differences between the different phases of devops so effectively there are two main phases of devops there's the planning and coding phase and then there's the deploying phase and you have a tool such as jenkins that allows you to integrate between both environments some of the core benefits that you may have to devops are going to be some technical benefits and some business benefits so when somebody asks you what are the benefits of devops you can reply that from a technical point of view you're able to use continuous software delivery to constantly push out code that has been tested and verified against scripts that have been written and approved you can be able to push out smaller chunks of code so that when you have an issue you're not having to go through massive blocks of code or massive projects you're going through just very small micro services or small sections of code and you're able to detect and correct problems faster on the business side the benefits are absolutely fantastic from a customer that's coming to your website and or to your mobile app they're going to see responses happening continuously so that the customer is always aware that you as a company are listening to their demands and responding appropriately you're able to provide a more stable environment and you're able to scale that environment to the demands of the number of customers that are using your services so how you approach a project that needs to implement devops so this is really an exciting area for you to be in so there are effectively three stages when it comes to actually working in a dev ops the first stage is an assessment stage and think of this as the back of the napkin ideation stage this is where you are sitting with a business leader and they're giving you ideas of what they would like to see from feedback that they've had from their customers this is blue sky opportunity this is thinking of big ideas that second stage and this often comes as a fast follow to stage one is actually proving out that concept so developing a proof of concept and a proof of concept can actually be a multiple different things so it could be something as simple as a wireframe or it could be something that is as complex as a mini version of the final application depending on the scope of the work that you're delivering will really depend on how complicated you want the poc to be but with that in mind whatever choice you make you have to be able to deliver enough in the poc so that when you present this to a customer they're able to respond to that creation that you've developed and able to give you feedback to be able to validate that you are going with the right solution and able to provide the right product to your customers that third stage is where you get into your dev ops stage and this is just the exciting part this is where the rubber hits the road and you start releasing code based on a backlog of features that are being requested for the solution in contrast to doing agile delivery where you just continuously work through a backlog with devops what you're also looking at is putting in analytics and sensors to be able to validate that you are being successful with the solution that being delivered so that once you actually start delivering out code that customers can interact with you want to be able to see what are the pieces of the solution that they are using what do they like what is complicated where are the failure points and you want to use that data and feed that back into your continuous integration and have that as a means to be able to backfill the demand of work that gets completed in the bank log so what is the difference between continuous delivery and continuous deployment so continuous delivery is based on putting out code that can be deployed safely to production it ensures that your businesses and functions are running as you would expect them to be so it's going through and completing the code that you'd actually see continuous deployment in contrast is all about ensuring that you're automating the deployment of a production environment so you're able to go through and scale up your environment to meet the demands of both the solution and the customer this makes software development and release processes much more faster and more robust so if we look here we can actually see where continuous integration and continuous deployment come hand in hand so when you actually start out with the initial pushes of your code that's where you're doing your continuous integration and your continuous delivery and then at some point you want to get very comfortable with deploying the code that you're creating so it's being pushed out to your production environment one of the things that's great about working with the tools that you use in a devops continuous integration and continuous delivery model is that the development tools that you use the containerization tools the testing tools should always reflect the production environment what this means is that when you actually come to deploying solutions to production there are no surprises because your development team have been working against that exact same environment all the way through so a question that you'll also be asked is you know what is the role of the configuration management in dev ops and so the role of configuration management really has three distinct areas and the first and this is really obvious one and this is the one where you probably already have significant experiences is the ability to manage and handle large changes to multiple systems in seconds rather than days hours or weeks as that may have happened before the second area is that you want to also demonstrate the business reasons for having configuration management and the business reason here is that it allows it and infrastructure to standardize on resource configurations and this has a benefit in that you're able to do more with fewer people so instead of having a large configuration team you can actually have a smaller more highly skilled team that's able to actually manage an even larger operational environment and thirdly you want to be able to highlight the ability to scale so if you have configuration management tools you're able to manage a significant number of servers and domains that may have multiple servers in it allows you to effectively manage servers that are deployed on cloud or private cloud and allow you to do this with high accuracy so how does continuous monitoring help and maintain the entire architecture of the system so when this question comes up you want to dig in and show your knowledge on how configuration and continuous monitoring is able to control an entire environment so the number one topic that you want to bring up when it comes to continuous monitoring is that with being able to effectively monitor your entire network 24 7 for any changes as they happen you're able to identify and report those thoughts or threats immediately and respond immediately for your entire network instead of having to wait as it happens sometimes for a customer to email or call you and say hey your website's down nobody wants that that's an embarrassing thing the other three areas that you want to be able to highlight are the ability to be able to ensure that the right software and the right services are running on the right resources that's your number one takeaway that you want to be able to give of continuous monitoring the second is to be able to monitor the status of those servers continuously this is not requiring manually monitoring but having a agent that's monitoring those servers continuously and then the third is that by scripting out and continuously monitoring your entire environment you're creating a self-audit trail that you can take back and demonstrate the effectiveness of the operations environment that you are providing so one of the cloud companies that is a strong advocate for devops is amazon's web services aws and they have really five distinct areas them that you can zero in on board services so when the question comes up what is the role of aws in devops you want to really hold out your hand and list of five areas of focus using your thumb and finger so you want to have flexible services built for scale automation secure and a large partner ecosystem and having those five areas will really be able to help demonstrate why you believe that aws and other cloud providers but aws is certainly the leader in this space are great for being able to provide support for the role of devops so one of the things that we want to be able to do effectively when we're releasing any kind of solution is to be able to measure that solution and so kpis are very important so you will be asked for three important dev of kpis and so three that really come to mind that are very effective the first one is mean time to failure recovery and what this talks about is what is the average time does it take to recover from a failure and if you have experience doing this then look at the experience you have and use a specific example where you are able to demonstrate that mean time to failure recovery the second is deployment frequency and with deployment frequency you want to be able to discuss how often do you actually deploy solutions and what actually happens when you're actually doing those deployments and what does the impacts to your network look like when you're doing those deployments and then the third one is really tied to that deployment frequency which is around what is the percentage of failed deployments and so and how many times did you deploy to a server and something happened where the server itself failed what we're looking for when you're going through and being asked for these kpis is experience with actually doing a devops deployment and being able to understand what devops looks like when you're pushing out your infrastructure and then the second is being able to validate that self-auditing ability and one word of caution is don't go in there and say that you have a hundred percent success uh the reality is that servers do degrade over time and you maybe want to talk about a time when a server did degrade in your environment and use that as a story for how you were able to successfully get over and solve that degradation so one of the terms that is very popular at the moment is infrastructure as code and so you're going to be asked to explain infrastructure as code and really it's it's something that actually becomes a byproduct of the work you have to do when you're actually putting together your devops environment and infrastructure's code really refers to the writing of code to actually manage your environment and you can go through many of the other tools that we've covered in this series but you'll see that xml or ruby or yaml are used as languages to describe the configuration for your environment this allows you to then create the rules and instructions that can be read by the machines that are actually setting up the physical hardware versus a traditional model which is having software and installing that software directly onto the machine this is really important when it comes to cloud computing there really is a strong emphasis of being able to explain infrastructure as a service and infrastructure as code is fundamental to the foundation to infrastructure as service and then finally allows you to be able to talk about how you can use scripted languages such as yaml to be able to create a consistent experience for your entire network all right so let's now get into the next section which is source code management and we're going to focus specifically on git the reason being is that get is really the most popular source code management solution right now there are other technologies out there but for the types of distributed environments that we have uh source code management with git is probably the most effective so the first question you'll be asked when it comes to git is to talk about the difference between centralized and distributed version control and if we look at the way that the two are set up older technologies such as older versions of team foundation server though the current version does actually have git in it but older versions required a centralized server for you to check in and check out of code the developer in the centralized system does not have all the files uh for the application and if the centralized server crashes then you actually lose all of the history of your code now in contrast a distributed model actually we do check in our code to a server however for you to be effective and building out your solution you actually check out all of the code for the solution directly onto your local development machine so you can actually have a copy of the entire solution running on your local machine this allows you to be able to work effectively offline it really allows for scalability when it comes to building out your team so if you have a team that may be in europe you can actually then scale that team with people from asia from north america or south america very easily and not have to worry about whether or not they have the right code or the wrong code and in addition to that if the actual main server where you're checking in your code does crash it's not a big deal because you actually have each person has a copy of the code so as soon as the server comes back up you have to check back in and everybody's running back as if there was nothing that happened at all so one of the questions you'll be asked is to give the answer to some of the commands you use for working with git so if you were to be asked the question is what is the git command that downloads any repository from github to your computer on the screen we have four options we have get push get fork get clone and get commit the answer in this instance would be git clone now if you want to be able to push code from your local system to a github repository using git then first of all you want to be able to do is connect the local repository to a remote repository and in the example you may want to talk about using the command get remote add origin and then the actual path to a github repository you could if you want to actually at this point also talk about other repositories such as git lab that you can also work with or a private git repository that would be used just for the development team once you've actually then added the local repository into your local computer then the second action you want to use is a push which is to actually push your local files out to the master environment so you use the command git push origin master so one question you may be asked is what is the difference between a bare repository and a standard way of initializing a get repository so let's look through what is the standard way so the standard way using git init allows you to create a working directory using the command git in it and then the folder that creates is the folder that creates all the revision history related to the work that you're doing in contrast using the bear way you have a different commands for setting that up so it would be git init dash dash bear and it does not contain any working or checked out source files locally on your machine in addition the revision history is actually stored in the root folder versus a subfolder that you would have with the normal git in it initialization so which of the following cli commands would you be used to rename a file so we have git rm git mv get rm-r or none of the above well in this instance it would be get mv a question that you'll be asked around commit is going to be what is the process to revert a commit that has already been pushed and made public and there are two ways you can address this the first is to actually address the bad file in the new commit and you can use the command git commit dash m and then put in a comment for why that file is being removed the second is to actually create a new commit that actually undoes all the changes that were made with the bad commit and then to do that you would use git revert and then the commit id and the commit id and it could be something such as 560e 0938f but you'd have to find that from the the commit that you had made but that would allow you to revert any bad files that you had submitted so the two ways of being able to get files from a get repository and you're going to be asked to explain the difference between git fetch and get paul so get fetch allows you to fetch and download only new data from a new repository it does not integrate any of the new data into your working files and it can be undone anytime if you wanted to break out the remote tracking branches in contrast git pull updates the current head branch with the latest changes from the remote server so you get all of the files and downloaded it downloads new data and integrates it with the current working files you have on your system and it tries to merge remote changes with your local ones so one of the questions you'll get asked about get is what is a gate stash so as a developer you will be working on the current branch within a solution but what happens if you come up with an idea where it's something that will take a different amount of time for you to be able to complete but you don't want to interrupt the mainline branch so what you can actually do is you can actually create a branch that allows you to start working on your own work outside of the mainline branch and this is called git stash allows you to be able to modify your files without interrupting the mainline branch so you once you start talking about branching and git be prepared to answer and explain the concept of branching so essentially what it allows you to do is have a mainline master branch that has all the code that the team is checking in and checking out against but allows you to have an indefinite number of branches that allows for new features to be built in parallel to the mainline branch and then at some point be reintroduced to the mainline branch to allow the team to add in new features and so if we look through the merge and get rebase these are the two features that you'd be using continuously to be able to talk about how you take a branch and merge it back into the mainline branch so on the left hand side we have git merge which allows you to take the code that you're creating and merge it back into the master on the right hand side what you have is a slightly different approach this is for projects where you reach a point in a project where you go okay we're going to effectively restart the project at this point in time and we want to ignore the complete history that's happened before that and that's called get rebase and that would allow you to rewrite the project history by creating a brand new mainline branch that ignores all other previous branches that have happened before it you can if you want to very quickly and easily find out all the files that have been used to make a particular commit so when somebody asks you the question how do you find a list of files has been changed in a particular commit you can actually say that all you have to go is find the command get diff dash tree dash r and then the hash that you would use for the commit and that would actually then give you a breakdown of all the files that have been made with that particular commit a question you'll be asked when you're talking about merging files is what is a merge conflict in git and how can it be resolved so essentially a merge conflict is when you have uh two or more branches that are competing with commits in git and you have to be able to determine which is the appropriate files that need to be submitted and this is where you would go in and to actually help resolve this issue you'd actually go in and manually edit the conflicted files to select the changes you want to keep in the final merge so let me go through the steps that you would take to be able to illustrate this when you're talking about this particular question in your interview now there are essentially four stages the first would be under the repository name you want to select a pull request and you want to be able to show how that pull request would be demonstrated inside of github so within the pull request there's going to be a highlight of conflict markers and you'll be able to select which conflicts you want to keep and which you want to merge and which ones you want to change so it's a step through how you would actually resolve a merge conflict and the first step would be under github you want to be able to pull the repository name and then the pull request around that repository in the pull request list click the pull request with a merge conflict and that you'd like to be able to resolve they'll pull up a file that will list out all of the conflicts for you near the bottom of that file will be a list of the requests that need to be resolved and then if you need to make a decision on which branches you want to keep or which ones you want to change that will have to be something you have to put in instructions inside of the file you'll actually see that there are conflict markers within the instructions which are going to ask you which files you want to change which ones you want to keep if you have more than one merge conflict in your file scroll down to the next set of conflict markers and repeat steps four and five until you resolve all of the conflicts you will want to mark your file as resolved in github so that the repository knows that you are having everything resolved if you have more than one file with a conflict then you want to go then onto the next file and start working on those files and just keep repeating the steps we've done up to this point until you have all of the conflicts resolved and then once you have all of the resolutions created then you want to select the button which is commit merge and then merge all your files back into github and this will take care and manage the resolution of the merge conflict within github so you can also do this through command line and with the command line you want to use get bash and so you want to as a first step open up get bash and then navigate to the local get repository and command line by using the cd change directory and then list out the actual folder where you actually are putting all of your code and then you want to be able to generate a list of the files that are affected with the merge conflict and in this instance here you can actually see the file styleguy.md has a merge conflict in it and as before with working with github you actually go through and use a text address and use any text editor but as you go through and edit out what you want to keep and what you want to manage in your conflict so you actually have a resolution that's been created so that you'll be able to then once you're using the conflict markers you can actually merge your files together so that the solution itself will allow you to incorporate your commits effectively into the resolution once you've gone through and applied your changes you're able to then merge the conflicted commits into a single commit and able to push that up to your remote repository all right let's talk about the next section which is continuous integration with jenkins so the first question you'll be asked about with jenkins is explain a master slave architecture of jenkins so the way that jenkins is set up is that the jenkins master will pull code from your remote git repository such as github and will check that repository every time there is a code commit it will distribute the workload of that code and the tests that need to be applied to that code to all of the jenkins slaves and then on request the jenkins master and the slaves will then carry out all the builds and tests to be able to produce test reports the next question you'll be asked is what is a jenkins file and simply put a jenkins file is a text file that has a definition of the jenkins pipeline and is checked into a source code repository and this really allows for three distinct things to happen one it allows for a code review and iteration of the pipeline it permits an audit trail for that pipeline and also provides a single source of truth for the pipeline which can be viewed and edited so which of the following commands runs jenkins from the command line is it java jar jenkins dot war java dash war jenkins.jar java.jar jenkins jar java dash war jenkins.org and the answer is a java jar jenkins dot war so when working with jenkins you're going to be asked what are the key concepts and aspects of working with the jenkins pipeline and you want to really hold out your fingers here and go through four key areas and that is pipeline node step and stay so pipeline refers to the user-defined model of a cd continuous delivery pipeline node are the machines which is which are part of that jenkins environment within the pipeline step is a single task that tells jenkins what to do at that particular point in time and then finally stage defines a conceptually distinct subset of tasks performed through the entire pipeline and tasks could be build test and deploy so which of the following file is used to define dependency in maven and do we have a build.xml b palm.xml c dependency.xml or d version.xml and the answer is palm.xml working with jenkins you're going to be asked to explain the two types of pipeline used in jenkins along with the syntax and so a scripted pipeline is based on groovyscript as their domain specific language for jenkins and there are one or more noteblocks that are used throughout the entire pipeline on the left hand side you can actually see what the actual script would look like and the right hand side shows what the actual declaration for each section of that script would be the second type of jenkins pipeline is a declarative pipeline and a declarative pipeline provides a simple and a friendly syntax to define what the pipeline should look like and then you can actually at this point use an example to actually break out how blocks are used to define the work completed in a decorative pipeline so how do you create a copy and backup of jenkins well to create a backup periodically back up jenkins to your jenkins home directory and then create a copy of that directory it's really as simple as that a question you'll be asked as well is how can you copy jenkins from one server to another well there essentially there are three ways to do that one is you can move a job from one installation of jenkins to another by copying the corresponding job directory the second would be to create a copy of an existing job directory and making a clone of that job directory but with a different name and the third is to rename an existing job by renaming a directory so security is fundamental to all the work that we do within devops and jenkins provides the center core to all the work that gets completed within a devops environment there are three ways in which you can apply security to authenticate users effectively and when you're asked about this question of security within jenkins the three responses you want to be able to provide is a jenkins has its own internal database that uses secured user data from end user credentials b is you can use a ldap or lightweight directory access protocol server to be able to authenticate jenkins users or see you can actually configure jenkins to authenticate by using such as oauth which is a more modern method of being able to authenticate users you're going to be asked how to deploy a custom build of a core plug-in within jenkins and essentially the four steps you want to go through are first of all copying the hpi plugin file into the jenkins home plugins subdirectory you want to remove the plugins development directory if there is one you want to create an empty file called plugin.hpi.pinned and once you've completed these three steps restart jenkins and your custom build plugin should be available how can you temporarily turn off jenkins security if the administrative user has locked themselves out of the admin console this doesn't happen very often but when it does it's good to know how you can actually get into jenkins and be able to resolve the problems of authenticating effectively into the system as an administrator so when you want to be able to get into a jenkins environment what you want to be able to do is locate the config file you should see that it's set to true which allows for security to be enabled if you then change the user security setting to false security will disable allow you to make your administrative changes and will not be re-enabled until the next time jenkins is restarted so what are the ways in which a build can be scheduled and run in jenkins well there are four ways in which you can identify the way a build can be scheduled on running jenkins the first is when source code management commits new code into the repository you can run jenkins at that point the second can be the after the completion of other builds so maybe you have multiple builds in your project that are dependent to each other and when so many other builds have been executed then you can have jenkins run you can schedule build to run at a specified time so you may have nightly builds of your code that illustrates the changes in the solution you're building and then finally you also can manually build a environment on request occasionally you will want to also restart jenkins and so it's good that when a question around how do you restart jenkins manually comes up that you have the answers and there are two ways in which you can do it one is you can force a restart without waiting for bills to complete by using the jenkins url that you have for your environment slash restart or you can allow all running bills to complete before restart are required in which case you would use the command of the url for your jenkins environment slash safe restart so let's go into the fourth and final section of this first video which talks about continuous testing with selenium so the first question you will be asked most likely around selenium are what are the four different selenium components and again you will want to hold open your fingers because there are four distinct environments you have selenium integrated development environment or selenium ide you have selenium remote control or selenium rc you have selenium web driver and then selenium grid you'll be asked to explain each of those areas in more detail but let's start off with by looking at selenium driver what are the different exceptions in selenium webdriver so it's useful to remember that an exception is an event that occurs during the execution of a program that disrupts the normal flow of that program's instructions and so we have four we have a timeout exception an element not visible exception no such element exception and a session not found exception and each of those if we step through them are the four different types of exceptions that can be thrown up when using the selenium web driver so as we evolve in our digital world with the different types of products that are available for us to be able to build solutions onto multiple platforms you're going to be asked can selenium and other devops tools run in other environments and so a good question around this is cancelium test an application in an android web browser and the short answer is absolutely yes it can we have to use the android driver for it to be able to work so you want to be able to talk about the three different types of supported tests within selenium so when the question comes up what are the different test types supported by cylinder you can answer that and there are three different types of tests first is a functional test second is a regression test and third is a load testing test the functional test is a kind of black box testing in which test case are based on a specific area of feature within the software a regression test helps you find any specific areas that functional tests or non-functional areas of the code wouldn't be able to detect the load testing test allows you to monitor the response of a solution as you increase the volume of hits in how you're using the code are put onto it an additional question you'll be asked is how can you get a text of a web element using selenium well the get command is due to achieve text of a specific web element it's important to remember however that the command does not return any parameters but just returns a string value so you want to be able to capture that stream value and discuss about it a question you'll be asked around selenium is can you handle keyboard and mouse actions using selenium and the answer is yes you can but you have to make sure that you're using the advanced user interaction api and the advanced user interaction api is something that can be scripted into your tests and it allows you for capturing methods such as a click and hold and drag and drop mass events and then keyboard down or keyboard up key release events so that if you want to to capture say the use of control shift or a specific function button off the keyboard you'd be able to capture those of the following four elements which of these elements is not a web element method a get text b size c get tag name d send keys and it's b size you're going to be asked to explain what is the difference for when we use find element or find elements and so if we look at find element find element finds the first element in the current web page that matches the specified located value in contrast find elements finds all of the elements on the web page that matches the specified value when using webdriver what are the driver close and driver quit commands and these are two different methods used to close a web browser session in selenium so driver close will close the current web browser which your focus is set and driver quick closes all the browser windows and ends the web driver session completely the final question that you are likely to be asked in using selenium is how can you submit a form using selenium well in this instance that's relatively easy the following lines of code will let you submit a form in selenium which would be web element el equals driver dot find element and then you put in the id and the element id and then l submit open close parentheses semicolon so let's just get into the first section which is configuration management so one of the questions that you'll get asked right away is why do you have ssl certificates used for chef really fundamentally your immediate answer should be security ssl provides for a very high level of private security and private and public key pairing this really is essential to ensure that you have a secure environment throughout your entire network the second part should be that if you're using ssl and you're using the private public key model within ssl you're able to guarantee the systems on your network that the chef that you'll be able to validate that the nodes within your network that chef is validating against actually are the real nodes themselves not imposters so you will also be asked some questions such as the following which is the following commands would you use to stop or disable the http service when the system boots and you'll typically get four responses and there'll be system ctl disable httpd.service or is it system disable http dot service system disable httpd or the final option which is system ctl disable dot service your answer should be the first one which is hashtag system ctl disable http dot service so chef comes with a series of tools that allow it to function effectively and one of the tools that you're going to be asked about is what is test kitchen and test kitchen is essentially a command line tool that allows you to be able to test out your cookbook before you actually deploy it to a real note so some of the commands that you would use are for instance if you want to create an instance of test kitchen you would do kitchen create if you want to destroy an instance after you created it you do kitchen destroy and if you want to be able to combine multiple instances you would do kitchen converge so a question you'll get is around chef is how does chef apply differ from chef client so fundamentally the difference between them is that chef apply will validate the recipe that you're working on whereas chef client looks to apply and validate the entire cookbook that's run in your server environment so one is focused on the recipe and the other is focused on the entire cookbook so there are some differences when you're working with different command lines so for instance when you're working with puppets and you're working with one version of puppet and you want to do what is the command to sign a requested certificate the top example here is for public version 2.7 whereas the lower option here is for puppet version 3. now something to bear in mind when you're going through your interview process is that the tools that are used within a continuous integration continuous delivery devops model do vary and so you want to be able to talk knowledgeably about the different versions of the tools so that when you're talking to your interviewer you're able to show the deep knowledge that you have which open source or community tools do you use to make puppet more powerful and essentially this question is going to be asking you to look beyond the core foundation of puppet itself and so the three options you have is being able to track configurations with jira which you should be doing anyway but it's a great way to be able to clearly communicate the work that's being done with puppet our version control can be extended with get and then the changes should be passed through jenkins so the three tools you want to be looking at integration with jira git and jenkins so what are the resources in puppet well fundamentally there are four the resources are basic unit of any configuration management tool they are the features of the nodes they are the written catalog and the execution of the catalog on a node so as we dig deeper into puppet one of the things that you are likely to be asked regarding puppet is what is a class in puppets and so a class in puppet is really the name blocks in your manifest that contain the various configurations and this can include services files and packages and we have on the screen here an example of what a class would look like when you write it out and you may want to memorize just one class don't memorize just a whole set of classes just memorize one the person that's interviewing you is just really looking for someone who has a working knowledge they're not looking for you to have memorized complete massive classes but having one small class to be able to illustrate the experience you have is extremely valuable to the interviewer particularly if it's a technical interview so as we move into ansible one of the things that you're going to be asked around ansible is what is ansible role so a role is an independent block of tasks and variable files and templates embedded with inside of the playbook so example we have here on the screen actually shows you one role within a playbook and in this role it is to install tomcat on a node again as with a previous question within puppet of a class it's probably good to have memorized just one or two roles so you can talk knowledgeably about ansible when you're having your interview so when you're working with ansible when should you be using the curly brackets and so just as a frame of reference uh there's often two different ways that these kind of brackets are referred to now they're either referred to as french brackets or curly brackets either way and what you'll be wanting to ask is when would you use these specific types of brackets within ansible and really the answer comes down to two things one is that it makes it easier to distinguish strings and undefined variables and the second is for putting together conditional statements when you are actually using variables and the example we have here is this prints the value of and we have foo and we have to then put in the variable conditional statement of foo is defined as something so what is the best way to make content reusable and redistributable with ansible and there's really essentially three the first is to include a sub module or another file in your playbook the second is to import an improvement of an include which ensures that a file is added only once and then the third is roles to manage the tasks within the playbook so a question you will be asked is provide a differences between ansible and puppets so if we look at ansible it's a very easy agent-less installation it's based on python you can configure it with yaml and there are no support for windows in contrast puppet is an agent-based installation it's written in ruby the configuration files are written in dsl and it has support on all popular operating systems so we dig deeper into the actual architecture ansible it has a much more simple architecture and it's definitely a push only architecture in contrast to puppet it's a more complicated but more sophisticated architecture where you're able to have a complete environment managed by the puppet architecture so let's get on to our next section which is containerization so let's go through and you're going to be asked to explain what the architecture of docker is and docker really is the most popular containerization environment so docker uses a client server architecture and the docker client is a service which runs in a command line and and then the docker daemon which is run as a rest api within the command line will accept the requests and interacts with the operating system in order to build the docker images and run the docker containers and then the docker image is a template of instructions which is used to create containers the docker container is an executable package of applications and its dependencies together and then finally the docker registry is a service to host and distribute docker images among other users so you'll also be asked to provide what are the advantages of docker over virtual machine and and this is something that comes up very consistently in fact and you may want to even extend it as having what are the differences between having a dedicated machine a virtual machine and a docker or docker-like environment and really the the arguments for docker are just absolutely fantastic you know first of all docker does contain and occupy docket containers occupy significantly less space than a virtual machine or a dedicated machine the boot up time on docker is significantly faster than a vm containers have a much better performance as they are hosted in a single docker image docker is highly efficient and very easy to scale particularly when you start working with kubernetes easily portable across multiple platforms and then finally for space allocation uh docker data volumes can be shared and reused among multiple containers the argument against virtual machines is significant and particularly if you're going into an older environment where a company is still using actual dedicated hardware and haven't moved to a cloud or cloud-like environment your arguments for docker are going to be very very persuasive be very clear on what the advantages are for docker over a virtual machine because you want to be able to succinctly share them with your team and this is something that's important when you're going through the interview process but also equally important particularly if you're working with a company that's transitioning or going through a digital transformation where they aren't used to working with the tools like docker you need to be able to effectively share with that team what the benefits are so how do we share docker containers with different nodes and in this instance what you want to be able to do is leverage the power of docker swarm so docker swarm is a tool which allows the it administrators and developers to create and manage clusters of swarm nodes within the darker platform and there are two elements to the node there's the manager node and then there's the the worker node the manager node as you'd assume matches the entire infrastructure and the working node is actually the work of the agent as it gets executed so what are the commands to create a docker swarm and so here we have an example of what a manager node would look like and once you've created a swarm on your magic node you can now add worker nodes to that swarm and again when you're stepping through this process be very precise in the execution part that needs to be taken to be able to effectively create a swarm so start with the manager node and then you create a worker node and then finally when a node is initialized as a manager node it can immediately create a token and that token is used for the worker nodes and associating the ip address with the worker nodes question 17 how to run multiple containers using a single service it is possible to run multiple containers a single service by using docker compose and docker compose will actually run each of the services in isolation so that they can interact with each other the language used to write out the compose files that allow you to run the service is called yaml and yamos stands for yet another markup language so what is the use of a docker file so dockerfile actually is used for creating docker images using the build command so let's go through and show on the screen what that would look like and this would be an opportunity where if you're actually in a technical interview you could potentially even ask hey can i draw on a whiteboard and show you what the architecture for using the build command would look like and what the process would look like um again when you're going through an interview process as someone who interviews a lot of people one of the things i really like is when an interview candidate does something that's slightly different and in this instance this is a great example of where you can stand up to the whiteboard and actually show what can actually be done through actually creating images on the whiteboard very quickly little square boxes where you can actually show the flow for creating a build environment as an architect this should be something that you are comfortable doing and by doing it in the interview and suddenly you want to ask permission before you actually do it but doing this in the interview really helps demonstrate your comfortable feelings of working with these kind of architecture drawings so back to the question of creating a docker file so we go through and we have a docker file that actually then goes ahead and creates the docker image which then in turns creates the docker container and then we are able to push that out up to a docker hub and then share that docker hub with everybody else as part of the docker registry with the whole network so what are the differences between docker image and docker containers so let's go through the docker image so the docker images are templates of a docker container an image is built using a docker file and it stores that docker file in a docker repository or a docker hub and you can use docker hub as an example and the image layer is a read only file system the docker container is a collection of the runtime instances of a docker image and the containers are created using docker images and they are stored in the docker daemon and every container is a layer is a read write file system so you can't replace the information you can only append to it so while you can actually use yaml or writing your so a question you can be asked is instead of yaml what can be an alternate file to build docker compose so yaml is the one that is the default but you can also use json so if you are comfortable working with jason and my that is something that you should get comfortable with is you want to be able to use that to name your files and as a frame reference json is a logical way of being able to do value paired matching using a javascript like syntax so you're going to be asked to how to create a docker container so let's go through what that would look and we'll break it down task by task so the task is going to be create a mysql docker container so to do that you want to be able to build a docker image or pull from an existing docker image from a docker repository or hub and then you want to be able to then use docker to create a new container which has my sequel from the existing docker image simultaneously the layer of read write file system is also created on top of that image and below at the bottom of the screen we have what the commands lines look for that so what is the difference between a registry and a repository so let's go through that so for the docker registry and repository for the registry we have docker registry is an open source server-side service used for hosting and distributing docker images whereas in contrast for repositories a collection of multiple versions of a docker image in a registry a user can distinguish between docker images with their tag names and then finally on the registry docker also has its own default retreat called docker hub for the repository it is a collection of multiple versions of docker images it is stored in a docker registry and it has two types of public and private registry so you can actually create your own enterprise registry so you're gonna be asked you know what are the cloud platforms that uh support docker really you know list them all and we have listed here amazon web services microsoft azure google cloud rackspace but you could add in their ibm bluemix could put in red hat really any of the cloud service providers out there today do support docker it's just become an industry standard so what is the purpose of expose and publish commands in docker so if we go through expose is an instruction used in dockerfile whereas publish is used in docker run command for expose it is used to expose ports within a docker network whereas with publish it can be used outside of a docker environment for expose it is a documenting instruction used at the time of building an image and running a container whereas with publish is used as to map a host port to a running container port for expose is it's the command used in docker whereas for publish we use the command-p for when we're doing our command line used in docker and examples of these are expose 8080 or with docker we would put in or for publish would do the example docker run d dash p and then a 0.0.0.80 80 as our command line so let's look at continuous monitoring so with continuous monitoring how does nagios help in continuous monitoring of systems applications and servers and so this is really just a high level question of using nagios within your environment and you should be able to just come back very quickly and say nagios allows you to help manage the servers and check if they've been sufficiently utilized and if there are any task failures that need to be addressed and so there are three areas of utilization and risk that you want to be able to manage this is being able to verify the status and services of the entire network the health of your infrastructure as a whole and if applications are working properly together with web services and apis that are reachable so the second question you'll be asked is how does negatives help in continuous monitoring of systems applications and services so it's able to do this by having the initial negative process and scheduler and the additional plugins that you would use for your network connect with the remote resources and the negatives web interface to be able to run status checks on a predefined schedule so what do you mean by nagios remote plugin executor or the mpre of nagios so mpre allows you to execute plugins on links unix machines that allow you to do additional monitoring and machine metrics such as disk usage cpu load etc what are the ports used by nagios for monitoring purposes in this example there are three and they're easy to remember so i would memorize these three but they're essentially ports five six six six five six six seven and five six six eight so there are two types of checks within nine years so you will be asked for what is an active and passive check in nagios so an active check and is initiated by the nagios process and is run on a regular schedule a passive check is initiated and formed by an external application or process so this may be where a system is failing and checks our results are submitted to the new gears for processing and to continue with this for what is an active and passive check active checks are initiated by the check logic within the nagios daemon negas will execute a plug-in and pass information about what needs to be checked plug-in will then check the operational state of the host oil service and then process the results of the host or service check and send out notifications in contrast with the passive check it is an external application that initiates the check it writes the results of the check to an external command line file and i guess reads the external command file and places the results of all passive checks in a queue for later processing so you can go back and revalidate and then negotiate may send out notifications log alerts etc depending on the results that they get from checking the information so you're going to be asked to explain the main configuration file and its location in nagios so the main configuration file consists of a number of directives that affect how nagios operates so consider this as the configuration file that's read by both nagios processor and the cgis so this will allow you to be able to manage the main configuration file that's placed into your settings directory so what is the nagios network analyzer and again hold out your four fingers because there are four options here so the nagios network analyzer are one provides an in-depth look at all your network traffic source and security threats two allows system admins to gather high-level information on the health of your network three provides a central view of your network traffic and bandwidth data and then 4 allows you to proactive in resolving outages abnormal behavior and threats before they affect critical business processes so what are the benefits of http and ssl certificate monitoring with nagios so with http certificate monitoring it allows you to have increased server and services and application availability obviously very important fast detection of network outages and protocol failures and allows web transaction and web service performance monitoring the ssl certificate monitoring allows you for increased website availability frequent application availability and provides increased security so explain virtualization with nagios so in responses the first thing you should be able to talk about is how nagios itself can run on many different virtualization platforms including microsoft visual pc vmware zen amazon ec2 et cetera et cetera so just make sure you get that right off the bat now yes it was able to provide capabilities tomorrow and so much of metrics on different platforms it allows for ensure for quick detection of service and application failures and has the ability to be able to monitor against many metrics including cpu usage memory networking and vm status so name the three variables that affect recursion inheritance in nagios and it is name use and register so name is a template name that can be referenced in other object definitions use specifies the name of the template object that you want to inherit its properties and variables from and register indicates whether or not the object definition should be registered to dagios and on the right hand side of the screen we have an example of what that script would look like again you may want to be able to memorize this as it's something that you can actually write down and show someone if you're going through a technical interview so why is nagios said to be object oriented and fundamentally comes down to the object configuration format that you can use in your object definitions it allows you to inherit properties from other object definitions and this is typical object oriented development and is now applied for the nagios environment so some of the objects that you can inherit are services hosts commands and time periods so finally explain what is state talking in nagios and so there are really four options here when you're talking about state stalking so state stalking is used for logging purposes in nagios it allows you to enable for a particular host or service that nagios will watch over very carefully it will log any changes it sees in the output of the check results and then finally helps the analysis of log files and so with that we come to the end of this devops tools full course if you have any doubts or queries let us know in the comments section below so watch out for more interesting videos like these until then keep learning and stay tuned to simply learn hi there if you like this video subscribe to the simply learn youtube channel and click here to watch similar videos to nerd up and get certified click here you

Channel: Simplilearn

Views: 170,675

Keywords: devops tools, devops tools explained, devops tools 2021, devops tools tutorial, devops tools and architecture clear explanation, devops tools to learn, devops tools overview, devops tools list 2021, jenkins tutorial, maven tutorial, docker tutorial, gradle tutorial, selenium tutorial, git tutorial, puppet tutorial, kubernetes tutorial, ansible tutorial, learn devops tools, devops, devops tutorial, devops tools full course, simplilearn devops, simplilearn, 2022

Id: 12HwsqHoaE8

Length: 679min 22sec (40762 seconds)

Published: Wed Jan 19 2022