Complete Jenkins Tutorial | Learn Jenkins From Scratch In 3 Hours | LambdaTest

hey what's up everybody and welcome  to this tutorial series on jenkins   my name is moss and i'm an experienced devops  engineer with six plus years of experience in the   devops industry now that you know who i am let's  go ahead and take a look at the roadmap of topics   that we're going to cover in this series we'll  start off with the basics and then progressively   get into more advanced topics related to jenkins  and in our first topic we're going to answer   the question what is jenkins after we have a high  level understanding of jenkins we'll then install   and configure jenkins on our local machine  once we've done that we'll then install and   configure the maven plugin and after we install  the maven plugin we'll then learn how to integrate   jenkins and github i'll then show you how to  schedule jenkins builds using build triggers once you learn how to schedule builds i'll then  walk you through the creation of a jenkins job   for a maven project i'll then walk you through  the configuration of a jenkins controller and   jenkins agent nodes to facilitate distributed  builds this will lead us to an introduction   of jenkins pipelines and we'll create our  very own pipeline in our jenkins instance   and finally we'll add automated cross browser  testing into our jenkins pipeline using lambda   test now that you know what we're going to cover  in this series let's get started with the first   topic what is jenkins jenkins is an open source  automation server and we use jenkins to automate   certain steps in the software development process  jenkins helps software teams facilitate continuous   integration and continuous deployment or delivery  in addition jenkins is a highly extensible tool   there are hundreds of plugins for jenkins and  these plugins add functionality to jenkins   or they enable jenkins to integrate with other  tools of the software development process   but why do we need jenkins what problem is jenkins  solving in the software development lifecycle   to understand this let's take a look at  a couple of key objectives for developers   the first objective is that developers who are  working in parallel want to make sure that the   features that they're developing integrate into  the latest version of the code base without errors   without jenkins manual integration  of changes happens infrequently   and because manual integration doesn't happen  frequently there's often a large backlog of   changes pending to be merged and this causes the  manual integration process to be time consuming   and we often refer to it as integration hell the  second objective is that developers also want to   make sure that their changes are built and tested  in a standardized environment when a developer is   testing a new feature locally on their machine  their machine configuration might not match the   same configuration of another developer's machine  for instance the developers machines might not   have the same version of the software framework  that they're using to develop their product   and because of this the new feature might work  on one developer's machine but not a different   machine jenkins provides software teams a  stance might not have the same environment   allowing them to avoid this issue now that we know  why we need jenkins let's understand where jenkins   fits into the software development process the  software development life cycle begins with the   requirements definition or planning followed by  the design of the product we then implement the   product based on the design and once we have an  implementation we then test that implementation   to ensure that it meets the requirements that were  first defined at the beginning of the life cycle   once the product is successfully tested we deploy  it to the customer and perform maintenance or   operations on it jenkins fits into the testing  and deployment steps of the software development   lifecycle and more specifically jenkins helps us  automate the testing and deployment of our product   now that we know where jenkins fits into the  software development process let's take a look   at the architecture of the software development  workflow with jenkins this sequence diagram   outlines the software development workflow and  it primarily focuses on the role of jenkins in   the software development workflow to begin the  sequence the developer writes code and when they   feel like their code is ready to be reviewed and  tested they then check that code into the source   code repository and this might be a tool like  gitlab or github or bitbucket when the developer   checks in their code to the repository this should  trigger a build in jenkins in this diagram i show   the communication going from the source code  repository to jenkins but this isn't always the   case sometimes jenkins is configured to pull the  source code repository for new changes and anytime   it detects new changes in a particular repository  that will also trigger a build but it's important   to note that it's always best if possible to push  new data to jenkins rather than having jenkins   consistently pull the source code repository  because that might put an unnecessary load on   the source code repository system once a build  is triggered in jenkins jenkins will then pull   the developer's code changes down to its local  workspace where it will build and test the code   automatically using the latest version of the  code base if the automated build and test steps   complete successfully jenkins will automatically  deploy those changes to the production environment   if the build and test steps fail the automated  deployment will not happen regardless of whether   the build passes or fails jenkins will still  publish results to the source code repository   it's important to note that jenkins itself is  not performing the build tests and deployment   operations it's simply facilitating the automation  of these steps for instance in the testing step   jenkins may trigger j unit test cases to run  automatically and although jenkins may not deploy   changes directly to the production environment  itself it may utilize a tool like chef or ansible   in order to update the configuration  of the production servers to use the   latest version of the software now that  we have a better understanding of jenkins   architecture we can begin with our next  topic install and configuration of jenkins hey what's up everybody my name is moss and  welcome back to our tutorial series on jenkins   in this video we're going to be covering how to  install and configure jenkins there are multiple   installation methods for installing jenkins you  can run jenkins from a web application resource   file also known as a war file you can also run  jenkins as a docker container from a docker image   you can install jenkins on windows using a windows  executable file you can also install it on mac   os and finally you can install it on many linux  distributions such as red hat or ubuntu in this   video we're going to be running jenkins from a  war file and the reason that we're going to be   running jenkins from a war file is because the war  file is platform and operating system independent   whether you're following this tutorial  from a mac or a windows or a linux machine   you'll be able to run jenkins from a war file  using the same steps that i'll be taking the only   requirement is that you have a supported version  of the java runtime environment installed on your   machine jenkins supports java versions 8 and  11. java versions 9 10 and 12 are not supported   now that we've covered the necessary details let's  go ahead and get started with the installation okay so the first thing that i want  to do is just verify that i have   java installed on my machine and to do that  i'm going to open up a terminal session   and in my terminal window i'm  going to type in java version   and you can see in the output of this command  that i have the open jdk version 11 installed on   my machine so if you don't have java installed  on your machine you'll want to make sure that   you go ahead and install either version 8 or  version 11 of either the jdk or the java runtime   environment before proceeding with the tutorial  so now that i've verified the version of java   that's installed on my machine i'm going to open  up my browser and i'm going to search for jenkins in in the first link jenkins.io i'm  going to select the download link   and in the download page i'm going to  scroll down to where i see the generic   java package which is the war file and i'm  going to go ahead and download this package   and click save file and i'm going to open  the containing folder and i'm going to move   this jenkins.war file into my home directory if  you would like to run jenkins from a different   directory that's fine i'm just going to put  it in my home directory for my own convenience   now that i have the war file in the home  directory i'm going to navigate back to my   terminal window and in my terminal window  i'm already inside of my home directory   so if i do an ls i can see the jenkins.war file is  in direct in the directory and all i need to do to   start jenkins is uh invoke java pass in the  dash jar option and then the name of the   war file so we're going to say jenkins.org  and that will start the jenkins server and   this command is blocking so it's not going  to return back to the command line as long as   jenkins is running and jenkins by default is going  to be running unless we were to specify in the   in the invocation to java pass in parameters  to the program uh jenkins will by default   start running on localhost port  8080 so if we navigate to localhost   port 8080 what we should see is the startup  menu with jenkins and sure enough we see the   first the first startup page for jenkins is to  unlock jenkins so in order to proceed we have to   secure the initial admin password which is  provided to us in the output of the java command   when we start up the jenkins server so you'll  notice here it says please use the following   password to proceed to installation so all i need  to do is just copy that password and this password   is also located inside of this directory so  dot jenkins secrets initial admin password   so i have that copied i'll select continue and then on the next page we can  customize jenkins to some extent   using plugins so as it says here plugins  extend jenkins with additional features to   support different needs and it's best to install  the suggested plugins if we wanted to we could   select a subset of plugins ourselves but in our  case we'll just select the suggested plugins   so jenkins is now installing all of the most  popular plugins that are available with jenkins   and a couple that i'd like to point out here is  the pipeline plugin which we will use in this   tutorial series the git plugin which will  install git on this jenkins server and will   allow jenkins to communicate to source  code management systems that are using git okay so now that the plugins have been  installed we can create our first admin user   and the first admin user that i'm going to  create the username is just going to be moss i'll type in a password here and for   the full name i'll just continue with moss  and then i'll enter an email address here okay and then i'll click save and  continue now if we wanted to change our   our instance configuration and select a  different jenkins url we could do so here   i'm going to leave it as the default localhost  8080 and i'm going to select save and finish   and then i click start using jenkins and we  should be brought to the jenkins dashboard so now we're at the jenkins homepage and i'd  like to get you familiar with the jenkins ui   and introduce you to some key components  of the jenkins interface so to facilitate   automation tasks jenkins has what  are known as jobs or jenkins jobs   and anytime you need to perform a task using  jenkins you need to create a jenkins job   and there are various types of jenkins jobs that  you can create and those jobs are listed under the   new item page here so let's go ahead and navigate  to the new item page so on this page we can create   a new item in jenkins and we can provide a  name for that item we can say test job here   and after we provide a name we can select what  kind of a job or what kind of an item this is   and the first option that we have is a freestyle  project and a freestyle project is really for   simple uh software development workflows it's for  simple projects and code bases where you might   not have a large number of steps to build and test  your software the next item is pipeline jobs and   pipeline jobs are good for more complex software  development workflows workflows that include   build tests and deployment steps but also might  include building multiple code bases rather than   just a single code base or a single repository  and one of the benefits of the pipeline jobs   is that we can store these complex steps as code  and we can write a file called a jenkins file and   that jenkins file will outline all of these steps  that we need to take in our software development   workflow in order to build test and deploy our  software product and later on in this tutorial   series we'll have a dedicated video covering  pipelines and how to create a pipeline job   the next item are multi-configuration projects  and these jobs are meant for pipelines targeting   multiple baselines or multiple platforms  so for instance you might offer a software   product like a mobile app but that mobile app  is offered on not only android but also ios then   you need to uh then you need to have a software  development workflow that includes testing on both   ios platforms and android platforms and this  multi-configuration project would help you do that   the next item in the list technically isn't  a job it's simply a container that stores   other items it could be a nested folder  or it could be a job inside of a folder   it's a way of grouping jobs into a directory-like  structure the next item is the github organization   item and i'm not sure why it shows up twice here  but the github organization item comes from the   github branch source plugin and this plugin  allows jenkins to scan a github user accounts   repositories or a github  repositories in github and for each   repository in the organization or the  user account that contains a jenkins file   jenkins will automatically create a job for that  repository and the plugin will also scan branches   of a github repository and for each branch that  contains a jenkins file it will also create a   dedicated job for that jenkins file and for the  last item here the multi-branch pipeline job   it's somewhat similar to the github  organization because it allows you to define a   code base which might not be located in  github maybe it's located in gitlab or another   source code management system for instance and  when you set the code base that you want this   job to scan it will scan each of the branches in  that code base and for every branch that has a   jenkins file in it it will automatically create  a jenkins job dedicated to that branch so now   that we're somewhat familiar with the types  of jobs that we can create in jenkins let's   navigate back to the dashboard and the next  key component of the jenkins interface that   you should be familiar with is the manage jenkins  page so in the manage jenkins page we can control   just about every aspect and every configuration  there is associated with our jenkins instance   so let's go ahead and take a look at  what's contained inside of this page   so in the first section of the manage jenkins page  we have the system configuration and under system   configuration we can select configure system the  configure system page is the central place where   you can update the configuration of jenkins and  you can also update the configuration of plugins   as well on this page so one configuration  here that is worth noting is the number of   executors and you can see here that the number  of executors is set to two when a jenkins job is   ran we call a running instance of a jenkins job  a build so a jenkins job can produce a build   and in order for a build to execute it needs  an executor an executor is essentially a thread   that can execute the instructions outlined in  a particular build right now i've configured   this jenkins instance with two executors which  means that i can have up to two builds running in   parallel so it could be two builds of the same  job or it could be one build for one job and   another build for a different job but i can have  a maximum of two builds running in parallel so   if i were to increase the number of executors to  you know like 50 then i would have essentially 50   available threads that i could schedule builds on  and i could have up to 50 jobs running in parallel   now do you want to set the number of executives  that high probably probably not unless you have   a large amount of resources when i say resources  i mean cpu and memory and and disk space as well   because as you increase the number of executors  you're also potentially increasing the resource   usage that's happening on your jenkins  instance so you need to be very careful   with this configuration setting and in my  case i'm going to leave it at two another   configuration that's also worth pointing out is  the home directory or the jenkins home directory   which right now is in home moss and then dot the  dot jenkins folder the dot jenkins folder contains   all of the data related to this particular jenkins  instance it contains all the jobs metadata the   the artifacts of particular builds um and  if i were to delete this dot jenkins folder   i'm essentially blasting this jenkins instance  away all of the uh all of the data associated   with this jenkins instance would be gone and if  i were to delete that dot jenkins folder and then   rerun that java command to start jenkins it  would essentially be a fresh uh jenkins instance   and as i scroll down in the configure system  page you'll see other settings like the jenkins   where jenkins is located in the jenkins  url but you'll also see configurations for   particular plugins for instance the github plugin  which allows me to add github servers these might   be the github server out on the public internet  github.com or it could be an on-premise instance   of a github server you know on your inter  internal corporate network so let's navigate   back to the manage jenkins page and then we'll  navigate to the global tool configuration page   and the tool configuration page allows us  to specify installations or versions of   tools that we would need in order to build  and test our code inside of a jenkins job   so for instance we can specify a particular  version of the jdk to be installed and we   can even add multiple versions of the jdk so that  we can compile java code inside of a jenkins job   and we can also specify a particular git  installation so that jenkins can clone down   a code base and build a particular code base from  a source code management system that's using git   uh and there's other tools as well that we can  add installations for like gradle and and maven   so coming back to the manage jenkins page uh under  system configuration we can also manage plug-ins   that are installed in jenkins so from the managed  plug-ins page we can update plugins we can see uh   what plugins are installed on the machine and we  can also search for plugins that are not installed   and there are hundreds of plugins that extend  jenkins functionality that we can install   through this uh through this page so let's  navigate back to the manage jenkins page   and then under the security section i'm  going to select configure global security   and on this page we can specify the method by  which users would authenticate to this jenkins   instance and in our case jenkins is using its own  user database to authenticate users and right now   in this database there are only two users there's  the admin user and then there's one other admin   user uh called moss which i'm currently signed  in as and these two users exist uh inside of this   jenkins owned database now we can utilize other  methods of authentication for instance if this   jenkins instance was inside of a corporate network  that utilized ldap we could also integrate this   jenkins instance into the the ldap service  in our corporate network in addition to   configuring an authentication method we can  also select a uh an authorization strategy   like matrix-based security which  will allow us to define more granular   authorization levels inside of our jenkins  instance now let's navigate back to the manage   jenkins page and under the security section  we also have manage credentials and the manage   credentials page allows us to store credentials  that jenkins might need in order to to communicate   with various services during a particular  jenkins build for instance when a jenkins   build starts it typically has to clone down a  code base from a source code management system   and in order to do so it has to authenticate  with the source code management system and   oftentimes it'll do so using an ssh key pair and  you can store the private key of the ssh key pair   in this manage credentials section of jenkins so  that when the build runs it will reference the   private key stored in in the manage credentials  page uh in order to authenticate with the source   code management system and clone down the  repository to the local jenkins workspace   and from the security section you can also create  delete and modify users that are able to log into   the jenkins instance and from the manage jenkins  page we can also monitor the jenkins instance   uh using the pages listed under status information  we can look at the system log we can also   review the resource utilization  like cpu and memory that the jenkins   server is using on this particular machine and  we can also perform administrative tasks like   uh restarting the jenkins instance or shutting the  jenkins instance down so hopefully you're a little   more comfortable now navigating the jenkins  interface and in the next video we're going   to install and configure the maven plugin so i'll  see you in the next video and thanks for watching hey what's up everybody my name is moss and  welcome back to our tutorial series on jenkins   by now you should have a good understanding of  what jenkins is and you should also be familiar   with some of the key elements of the jenkins  interface in addition you should have a jenkins   instance up and running if so you're ready  for the next topic in this video we're going   to install and configure the maven plugin on our  jenkins server so first what is the maven jenkins   plugin the maven plugin is an open source jenkins  plugin that supports maven 2 and maven 3 projects   and this plugin adds an additional job type to  jenkins called a maven project if you recall from   the previous video i walked through the new item  page which listed all the possible job types you   could create in jenkins and this plugin adds an  additional job type to that list on the new item   page called maven project and this maven project  job type provides an advanced integration between   maven and jenkins but the question is why use  the maven plugin when we could also invoke maven   inside of a freestyle project the maven plugin  greatly reduces the configuration required for   maven projects inside of jenkins and the plugin  enables post-build deployment of binary artifacts   that are produced during a maven build in addition  the plugin provides graphical test reports   directly in the jenkins ui it also supports  incremental builds so as not to waste resources   we can also build modules in parallel and  distributed across executors and jenkins nodes   the plugin also provides an additional  build trigger in the maven project job type   if it's enabled jenkins will parse the pom  files of the current project if any of the   dependencies listed in the pom file are built on  this jenkins instance then jenkins will set up   a build dependency relationship so that any time  the dependency job is ran any downstream jobs that   have that dependency listed in their pom file will  also be triggered to run now that we have a better   understanding of what the maven plugin does let's  go ahead and install it on our jenkins instance   okay so the first thing that i want to  do is navigate to the manage jenkins page   and under system configuration i'm going to  navigate to the global tool configuration page and i'm going to scroll down to maven and  i'm going to select add maven installation   and we'll give it a name we'll call it maven 3   and i have the option to either install this  automatically or i can specify a pre-existing   installation of maven on this machine so in  the maven home field i would just specify the   directory where maven is installed on this machine  i don't have maven installed on this machine so   i'm just going to select install automatically and  the default installer is installing from apache   and then i have the option to select which version  i want to install in this case the latest version   is already selected so i'm going to leave it as is  and since i selected install automatically during   the first build of a maven project on this jenkins  instance jenkins will download and install maven   so the first build of a maven project on this  jenkins instance might take a little bit longer   because it's downloading maven but all following  builds won't take as long because maven is already   installed on this jenkins instance so now that  i've specified install automatically i'm going to   select save and in the manage jenkins page i'm  going to navigate to the manage plugins page and i'll select the available tab and  we're going to search for the maven plugin   okay and it's the first result here called  maven integration i'm going to select that   and then i'm going to select  install without restart   and now that it is downloaded and installed i'm  going to select restart jenkins when installation   is complete and no jobs running and so now  jenkins is restarting and when it reloads   we should be able to see the maven project  job type shown in the new item page okay so i'm going to go ahead and log back in and i'm going to navigate back to the dashboard and from the dashboard i'm  going to select a new item and you'll notice below the freestyle project  we now have a maven project job type in addition   to the other jobs that we can create in this  jenkins instance so i'll go ahead and create   a sample project and we'll just call it test  maven and i'm going to select the maven job   maven project job type and  i'm going to select okay and there's a couple of things i'll point out  in this configuration page the first is under   build triggers so the first option here is the  additional build trigger that i was talking about   in the powerpoint slide so we can add a build  trigger to build this maven project whenever   a snapshot dependency is built and so this is the  dependency that is listed in the pom file for this   particular maven project and the other key element  of this configuration page is under the build   section and here we specify where the root  palm is located and we can also specify   maven goals and options in this field so  for instance i could say clean compile   and test here we can also add pre and post steps  as well and in an upcoming video we'll use an   actual maven project with this jenkins job type  but before we do that we'll want to integrate this   jenkins instance with our github account so that  jenkins can clone down the maven project code base   locally to the jenkins workspace and build  it so what we'll target in the next video   is setting up the integration between this  jenkins instance and our github account we have   the maven plugin installed and ready to go on this  jenkins instance so i'll see you in the next video hey what's up everybody and welcome back to our  tutorial series on jenkins in the last video i   walked you through the installation of the maven  integration jenkins plug-in before we utilize   that plugin we're going to build some additional  automation on our jenkins instance so let's take   a look at the next topic in this video we're going  to integrate our jenkins instance with our github   account you probably remember this diagram from an  earlier video describing the jenkins architecture   what we'll be focusing on in this video is  automating the trigger build and publish results   steps in this sequence diagram after automating  these two steps when a developer pushes a commit   from their local machine to a github repository  or when they make a commit directly in github   what should happen is github should trigger a  jenkins build for that github repository after   the build is triggered jenkins will clone down the  github repository to its local workspace and once   the build finishes jenkins should automatically  publish the results back to the github repository   so what are some of the benefits of integrating  jenkins and github some of these benefits you may   have already guessed but the first benefit is that  it gives jenkins the ability to clone a code base   down to the local workspace so that it can  build and test the code the second benefit   is that github will push new changes to jenkins  and trigger builds rather than having jenkins   consistently pull github repositories for new  changes polling for new changes too frequently can   be quite resource intensive on the github server  and the github api will actually rate limit you if   you pull too frequently the third benefit is that  jenkins will be able to perform github operations   which would include doing things like publishing  the build result to a github pull request   and to individual commits there is one key  prerequisite that i would like to point out   for this integration in order for the integration  to work as described github must be able to send   data to jenkins and since i'm using a github.com  account that means that the jenkins instance   must have a public ip the only exception to this  is if you're working on a jenkins instance that's   inside of your corporate network and you have an  on-premise github instance also inside of your   corporate network now that we've covered those  details let's get started with the integration   okay so the first thing that i want to do  is navigate to my github account which i'm   already in my account and then once you're in your  github account you're going to want to go up to   the the profile drop down and then select  settings and under settings we're going   to navigate to developer settings and we're  going to navigate to personal access tokens   and we're going to generate a new token a  new personal access token okay we'll call it jenkins and after giving it a name we have to  select scopes for this token and what scopes   do is they provide a level of authorization  to uh your to what your github account has so   for instance your github account has access to  all of your private repositories for instance   and you can grant this token access to your  private repositories or you may not want it uh to   have access to your private repositories maybe you  only want it to have access to public repositories   and as we select each one of these scopes we're  adding additional levels of authorization to our   github account for this particular token and  for this integration to work i'm going to be   using a public github repository so the only  scope that i need to select in this case is   the public reposcope which will grant this token  access to public repositories i'm going to leave   all of the other scopes unchecked and then  i'm going to select generate token now on this   next page the token value is presented to us and  you're going to want to copy this value maybe   save it in a in a text file temporarily but not  permanently but at least copy it to your clipboard   because we're going to use it inside of jenkins so  i'm going to select the clipboard here which will   copy the token value and i'm going to navigate to  my jenkins instance which i already have pulled up   and in jenkins i'm going to select manage jenkins   and then under system configuration  we'll go to configure system and then i'm gonna scroll  down to the github section   okay and this github section is for the github  plugin which should have been automatically   installed if you selected install recommended  plugins when uh installing jenkins so the two   plugins that are required for this integration uh  are the github plugin and the git plugin and both   of those are installed as part of the recommended  plugins when you're first installing jenkins and   in the github plugin configuration we can add  github servers that we want to authenticate with   and uh the one that we're going to authenticate  with is already automatically filled out here   so we're going to be authenticating with the  public github api and i'm just going to call it   github and once we specified a name and the  api url we also have to specify credentials to   authenticate with our github account right so we  need to provide credentials to authenticate with   our github account and this is going to be  our personal access token that we just created   so i don't have any credentials created on  this jenkins instance right now so what i'm   going to do is i'm going to select add and i'm  going to select the jenkins credential provider   and on the add credentials page i'm going  to specify the kind of credential that i   want to create which in this case is going to be  secret text okay so the personal access token is   considered a secret text credential okay i'm  going to leave the scope the same and then   for the secret i'm going to paste in the personal  access token value and we'll give it an id we'll   call it github p-a-t i'll leave the description  blank and then i'll select add okay now in the   drop-down we have uh the p-a-t credential that  we can select so i'm going to go ahead and select   that credential now once we've added our  credentials we can test the connection   to the github api so i'm going to  go ahead and click test connection and you can see that it says credentials verified  for user tech with moss and then it specifies the   the rate limit and how many uh api calls i  have left so uh it's set at 5000 and i just   used one to test connection so i have 4999  api calls remaining before i'm rate limited   and the last configuration here is the manage  hooks configuration and when this is checked   jenkins will uh basically automatically add the  jenkins web hook endpoint to the github repository   so that github can then send data back to jenkins  what we're going to do is we're going to manually   add the web hook to our github repository  rather than have jenkins automatically do it   so i'm going to uncheck this and then i'm  going to select save for this configuration   and now that we've added that personal  access token to this jenkins instance   jenkins should be able to communicate to github  and publish build results to a github repository   but one thing to note is that i used my personal  github account credentials uh so that jenkins   could authenticate with github but it is a best  practice to use something like a service account   or a generic account maybe an account that's  called like jenkins-ci or something like that and   then use that account's credentials to facilitate  this particular automation the next step is to   create an ssh key pair and we're going to add the  public uh key to our github profile and then add   the private key to this jenkins instance and that  will allow this jenkins instance to clone down   repositories to the local jenkins workspace so  any github repositories that my github account   has access to this jenkins instance will also  have access to and be able to clone down when   a build starts so i'm going to open up a git  bash session to generate the ssh key pair   and in my get bash window i'm going to  use ssh keygen to generate the key pair   and i'm going to accept all of the  defaults it asks me to overwrite my keeper i'm just going to do  that and so i generated a key pair   and i'm going to cat the public key okay so there's my public key and i'm  going to copy that uh that public key   to my clipboard and then i'm going  to navigate back to my github profile   and in the settings menu of my github profile  i'm going to go down to ssh and gpg keys   and i'm going to select new ssh key  i'm going to paste in my public key   and i'll give it a name i'll just call it jenkins public key okay so that added the uh the public key to my j  uh to my github profile and i'm going to navigate   back to jenkins and i'm going to open up  the get bash window again now i'm going to   cat the private key and i'm going to  copy the contents of the private key okay so this is the private key and i'm going to go ahead  and copy that to my clipboard okay and in jenkins i'm going  to navigate to manage jenkins   and then i'm going to go to the security section  and i'm going to select manage credentials   i'm going to select jenkins i'll select global  and then i'm going to select add credentials okay   and the kind is going to be uh ssh  username with cr uh private key   i'll leave the scope as is and  we'll call it github private key okay i'm going to leave the username blank and   then for the private key i'm  going to enter that directly okay so i'm going to paste the private key in  there i didn't create a passphrase for this   particular key so that's all i need i'm going  to select ok ok so we added the private key to   jenkins so jenkins when a build starts for a  particular github repository it should be able   to clone down the repository using this private  key that we just added so i'm going to go back to   the jenkins dashboard so now that we've added the  ssh private key to this jenkins instance let's go   ahead and create a new jenkins job so i'm going to  go to the new item page and we'll call it test job   and i'm going to select freestyle  project and then i'll select okay all right and in the source code management  section i'm going to select git and then it   prompts me to enter the repository url so this  is going to be the url of the github repository   and i have created a very simple github repository  all it has is a single readme file in it   so i'm going to navigate back to my github profile okay i'm going to select my repositories and then  i'll select this repository which is the one that   i created it's just a very basic repository  with just a readme file uh in the repository   and i'm going to select the uh the code tab here  and it gives me the url so notice how i can select   different urls i can select an https url or an  ssh one make sure that you select the ssh one   because we're using an ssh key pair to clone down  this repository so we need the ssh url and not the   https url so i'm going to select the ssh  url i'm going to copy that to the clipboard   and i'm going to navigate back to jenkins and  i'll paste that in okay so it says fail to   connect right now and it says permission denied  so i haven't specified the credentials that i   want to use in order to clone this repository down  so in the credentials section i'm going to select   the private key that was just created and it  defaults the username so i remember that i didn't   enter a username here it automatically entered a  username for me and so i'm going to select that   and notice that the error disappears after  selecting the credentials which means that it   was able to authenticate uh successfully so it can  clone down this repository the next configuration   is under branches to build i'm going to specify  the main branch since that's the the only branch   i have in my github repository and once i've  specified that i'm going to scroll down to   the build triggers section and what the  github plugin provides is an additional build trigger and it's this one right  here it's github hook trigger for git   sem polling and it says here if jenkins  receives a push github hook from a   github repository that's defined in the git sem  section it will trigger the get sem polling logic   okay basically what this means is that if  github uh receives a push from a developer   so a developer pushes a commit up to a github  repository github will then automatically   trigger this particular jenkins job to  build if this build trigger is selected   so we are going to go ahead and check that  build trigger since we want that behavior   the next configuration that we're going  to set is under post build actions   i'm going to select add post build action  and then set github commit status okay   and what this action will do is when the build  finishes jenkins will communicate the build result   back to the github repository and it will set the  build status on the individual commits that were   built during this particular jenkins build as  well as the pull requests if there is an open   pull request for the commits that were  built in this particular jenkins build   so i'm going to leave these two fields as their  default values and then under uh the what section   i'm going to specify the status result based  on one of default one of the default messages   and statuses so this status result basically  allows me to manually define like what the   status message will say uh based on the build  results so if i select it here result better   than or equal to let's say um you know success i  can provide a custom success message but in for   my case i'm just going to select uh the default  messages in status uh we don't necessarily need   uh custom messages in this particular case  and this back reference will provide a link   directly back to the build inside of github so  that should be all the configurations we need in   this particular job so i'm going to go ahead and  select save so now that we've created the jenkins   job there's one last configuration that we need to  make and it's inside of github repository so i'm   going to navigate back to my github repository and  in the repo i'm going to select the settings page   and under settings i'm going to navigate to  the web hooks section okay and web hooks allow   external services to be notified when certain  events happen in this particular repository so i'm   going to select add webhook and the first field  that is required is the payload url and this is   the url endpoint that github will send data to and  in our case it's going to be our jenkins url with   a special endpoint added onto the base jenkins url  so i'm going to navigate back to jenkins i'm going   to copy the base url which is right here i'm going  to paste that in and then i'm going to add the   the endpoint for github web hooks which  is github hyphen web hook okay and then   forward slash the content type i'm going to  select as json i'll leave the secret blank   and then it allows me to select  which events i would like to   trigger this web hook for and i can use just  the push event i can say for any event send me   everything or i can say let me select individual  events so maybe instead of the push event i want   this web hook triggered anytime a pull request  is opened and this particular selection will   allow me to do that so if i select this then any  time a pull request is open closed reopened etc   an event will be triggered for this web hook and  that data that event will be sent to jenkins and   it will be sent to this particular endpoint  that we provided here okay and i'm going to   leave it as is with just the push event being  sent to the endpoint that's really all we need   right now and i'll leave it as active and  then i'll select add web hook okay and when   you select add web hook it should automatically  test the connection so i'm going to go back into   the web hook and you'll notice that under recent  deliveries we have this little check mark and it   tested the the web hook and that was successful  we got a successful uh 200 response back from   jenkins so we configured this web hook correctly  so now that we've added the hook we should be   able to make commits locally on our machine and  push them to github or make commits directly in   the github interface and both of those actions  pushing commits as well as making commits in the   github interface should trigger our jenkins test  job to run a build for this particular repository   so let's go ahead and try it out i'm going to  navigate to the code tab and i'm going to make a   commit directly in the github interface and i'm  going to just edit the readme file and just say okay and then for the commit message  i'll just say testing jenkins and before i select commit changes i want to  show you that our test job currently doesn't   have any jobs that have ran or any builds that  have previously ran right so after i select   commit changes we should see a build running  in this uh jenkins job okay so i'm gonna select   commit changes and let's take a look back at our  jenkins job and you'll notice that we have a build   currently pending in the jenkins job and it's  already finished so if i select refresh we can   take a look at the jenkins job and uh it success  successfully cloned down the test repo from github   there wasn't really anything in the repo and there  wasn't anything to build any code to build or test   so it didn't really do anything except clone down  the repository and then it performed the post   build action of setting the github commit status  in github so what does that look like in github   let's navigate back to the github repository  and i'm going to select the latest commit here and you'll notice that next to the commit  message here we have this small check mark   that check mark indicates the status of that  commit and it says all checks have passed   and then we also get a back reference   to the jenkins job itself uh so this details um  hyperlink goes back to the jenkins job so let me   click that and you'll notice that you'll see  here that it goes back to to the build in jenkins   and i'll go ahead and navigate back to the github  repository so this works for individual commits   if i were to look at the list of commits  for a particular branch you'll see that   commits that triggered a build if the build was  successful they'll get this small check mark next   to them and it says you know build number one  succeeded in this amount of time and you get a   back reference to the jenkins build itself this  also this status also pops up in pull requests   uh so if you have a pull request for a commit or  multiple commits then jenkins will also post the   status inside of the pull request and let's try  that out real quick i'm going to navigate back to   jenkins and in jenkins i'm going to navigate to  the job and i'm going to make one configuration   change it's going to be to the branch specifier  and i'm just going to leave this blank so   this will build for any branch not just  the main branch if i leave it blank   so i'm going to navigate back to github  and i'm going to go to the code tab   and in the code tab i'm going to create  a new branch we'll call it jenkins   test branch i'll select create branch and i'll  make a small modification to the readme file and i'll commit these changes and as soon as i  commit the changes this should trigger a build   in our jenkins job and it will trigger a build  since i just modified the jenkins job to build any   of the branches in this repository not just the  main branch so i'm going to navigate back to   the code tab and in the code tab i'm going to  select compare and pull request for my branch   i'll leave everything as is and  then i'll select create pull request and you'll notice that on the pull  request page if i scroll down here   we have this status check and it says all  checks have passed one successful check and if i   click show all checks it will show me the  test job name and it specifies that build   2 succeeded in half a second and then it  provides a back reference back reference to   the build itself so if i click details it  takes me directly to build number two okay   so that about wraps it up for this integration  as you can see it's a very useful integration for   optimizing the ci cd process i hope you enjoyed  this video and i'll see you in the next one hey what's up everybody and welcome back to our  tutorial series on jenkins in the last two videos   i showed you how to integrate github with jenkins  and i also walked you through the installation of   the maven integration plug-in which laid the  groundwork for this video so let's take a look   at the next topic in this video we are going  to create a jenkins job for a maven project   i'll quickly recap some of the key  features of the maven project job type   the maven integration plugin provides a new  job type in jenkins called a maven project   and the maven integration plugin significantly  reduces the manual configuration required to   set up a maven project in your jenkins  additionally the plugin provides test   reporting directly in the jenkins interface  and with that let's go ahead and get started okay so the first thing that i wanted to do was  quickly walk through the maven project that we're   going to build in our jenkins server and i have  the maven project pulled up in my github account   it's called sample maven project and it's a very  basic java application and i actually generated   this code base automatically using the maven quick  start archetype and if we take a look at the quick   start archetype it will generate a sample maven  project for us with a very simple structure   and if we take a look at the directory structure  it begins with a top level palm.xml file then we   have the source directory and inside of the source  directory we have a single app class and then we   also have a test directory which contains the  app test class that tests this app class so if   we navigate back to the github repository we can  take a look at the app class and see what it does   so if i navigate to the app class it's very simple  it just prints out hello world to the console   and then if we go back to the test directory  and open up the app test class this is also   very simple it's a junit a single junit test it's  a assert true with an always true condition so it   should always pass so this is the code base  that we want to build in our jenkins instance   so let's navigate to jenkins and we will create a  new maven project so opening up jenkins and then   i'm going to select the new item page and  i'm going to call it test maven project and then i'll select the maven project  job type and i'm going to click ok and the first thing that we want to do  is set up our source code management   and so i'm going to scroll down to the sem  section and then i'm going to select git and   then we need the repository url and remember that  this is going to be the ssh url for the repository   so i'm going to navigate back to the github  repository and i'm going to copy the ssh url and i'll paste that into the uh repository  field okay and i get a permission denied   error which means that i haven't selected  the credentials to use uh to clone down   this repository so i'm going to select my  ssh private key that was previously added   and you can see that the error disappears  after selecting the private key   the ssh private key the next thing that i'm going  to update in this configuration is the branch   specifier i have only have the main branch in this  repository so i'm going to just update that to   main and the next configuration that we're  going to update is the build triggers   and you may remember me discussing this first  build trigger build whenever a snapshot dependency   is built when we were first installing the maven  integration plug-in but essentially whenever   this build trigger is checked any snapshot  dependency that is defined in this project's   pom file is built on this jenkins instance so if  any other dependencies defined in the pom file   are built on this particular jenkins instance then  those builds those dependency builds will trigger   a build for this job that we're defining now  and although there aren't any dependencies being   built on this jenkins instance for this particular  project i am just going to leave it checked since   it's checked by default the next trigger worth  noting is the build after other projects are built   trigger it's somewhat similar to the first build  trigger but when i have this trigger selected   it allows me to specify a list of jobs on this  jenkins instance and when those jobs are built   it will trigger a build for the current job that  we're defining and there is a second condition   that i can specify here to trigger only if the  build in the project specified in this field   are stable or i can trigger it even if the build  is unstable in these projects that i defined here   or i can trigger it even if the builds fail in  the projects that i defined in this field this   trigger might be useful if you have multiple jobs  and each job only performs like a single task   and it's like a pipeline of jobs that you want to  run in succession but in this case we don't really   need that and even if you did have multiple jobs  like that that are kind of like a pipeline of jobs   it would be preferable to just write a pipeline  script and have a single pipeline job to handle   those individual tasks so i'm going to  go ahead and uncheck this build trigger   the next build trigger is build periodically and  this trigger allows you to schedule this job on   a predefined cadence and you can use cron style  syntax in this field to specify how often this job   will build so for instance if i wanted this job  to build every minute i would use this syntax and   you can see here it says do you really mean every  minute uh when you say you know five stars there   uh perhaps you meant uh hourly okay so this would  be the syntax for running this job hourly and i   think this build trigger is particularly useful  when you use it for your release cycle cadence   so for instance you might want to have like a  nightly build of your main branch or you might   have a weekly build of your main branch or you  might have a release at the end of every sprint   so maybe you want to schedule this for every two  weeks or every three weeks and i think that's   where this build trigger is most useful in this  case i'm not going to utilize this trigger so i'm   going to go ahead and uncheck it the next trigger  is the github hook trigger for get sem polling   and this is the trigger that we set up in  the previous video so when this is checked   anytime a developer pushes a commit to github or  makes a commit directly in the github interface   an event is sent to jenkins that will trigger uh  this job for the code base that they they push   that commit to and i am going to utilize it for  this maven project so i'm going to go ahead and   check it and the last build trigger is pull sem  which is kind of the opposite of the github hook   trigger because it allows you to schedule a  predefined cadence for jenkins to pull your   source code management whether it be github or  another source code management system for new   changes i've mentioned this in previous videos but  using this build trigger is kind of a bad practice   you don't necessarily want to pull your scm  system uh if you do you want to do it infrequently   maybe like once a day probably once a day would  be like the most frequent maybe once an hour but   in general polling the sem system is bad because  you're consuming resources on the sem system and   you're also consuming resources on this jenkins  instance to query or make a query to the source   code management system to check if there's any new  changes so it's really a best practice to always   set up if possible the github hook trigger if  you're using github or if you're using another scm   system use some sort of web hook that will push  new changes to jenkins rather than having jenkins   pull the sem system so i'm going to go ahead and  uncheck this pull sem option now remember we do   have to set up the web hook inside of the github  repo if we select this option so after we save   this new maven project we'll go ahead and add  that web hook to our github repository the next   configuration that we want to set up is the build  configuration and under the build section we can   specify where the root palm is located and here we  can leave this value as is since the pom file is   at the directory of our maven project and on the  second field here we can specify goals maven goals   and options so for instance i can specify clean  install in this field and if i were to leave the   field blank it would perform an install by default  it would perform the install life cycle but i   could also specify deploy here if i had a remote  repository that i wanted to send the generated   artifacts to but i'm going to go ahead and leave  this as clean install so that should be the bare   minimum configuration that we need for this  maven project so i'll go ahead and select save and before we build this project for the  first time i want to add the web hook to   our github repository so i'm going to copy  the address here so we'll copy the port and then in the github repository we'll navigate  to settings and in settings we'll select webhooks   add webhook and then the payload  url will be the url of the jenkins   server and the github web hook  endpoint okay and i'll select json and we'll leave the default selection so  this will send the push event to jenkins   and we'll leave it as active i'll select  add webhook okay and if i select the webhook   we can see under recent deliveries  that it was a successful delivery   so we know it's configured correctly and now  let's navigate back to our jenkins instance   now let's build the project for the first  time so i'm going to select build now and let's take a look at build number one so it performed maven clean it compiled and  then it ran the single test case and the apt   uh the app test class and then it performed the  uh install as well so it archived the snapshot   jar in the local repository and what's  neat if we take a look back at the   project page itself after the build  runs we get this test result trend   which shows the duration of the tests on the  y-axis and then the build number on the x-axis   so if i hover over this single data point here  we can see that build number one one test pass   and zero tests skipped and zero tests failed  and additionally if i go into the build itself   we also get kind of a test report inside of  jenkins so if i take a look at the test result   we can drill down into the module the package and  then the test class itself and then look at all of   the tests within a single within a single class  and take a look at whether or not they pass or   failed and if the test failed then we can also  get the stack trace directly in this interface   as well so we could see you know the stack trace  and get a little bit more details on the failure   and from the build page we can also access  build artifacts so as you can see here under   build artifacts i can download the  snapshot jar or the snapshot palm   locally to my machine and this would be especially  useful if you wanted to perform manual testing   after automated testing occurs on the generated  snapshot and just for the sake of it let's update   our test case to fail just so that we can see what  the test report looks like when a test case fails   so i'm going to navigate back to the github  repository and i'll navigate to our test directory and this will also exercise our web hook as well  or it should if our web hook is working correctly   i'll open up the test class   and i'll directly modify it in the github  interface we'll change that condition to false okay and then i will commit that change so let's  take a look back at our maven project in jenkins and let's see if it uh successfully  triggered a build and it took a little   while for the jenkins interface to update  but it did trigger a build and as you can see   for build number two it is unstable the yellow  mark indicates that it's an unstable build   and unstable means that one or more test  cases failed and if we take a look at   the test result trend we can also see that  build number two is represented in the graph   and in build number two we have one failed test  case and i'll quickly open up the build number two   page and if i select the test results you can  see the number of failures is listed up here   that there was one failure and then it lists  the failed tests and if i expand this out i can   expand the stack trace for this failed  test and it's a pretty simple one all   it says is that there is an assertion error  because the condition that we selected was   or that we added to this test case was false  so that's all i have for this video i hope you   enjoyed it and in the next video i'm going  to walk you through the process of setting   up a jenkins controller and jenkins agent  nodes so i will see you in the next video hey what's up everybody and welcome back to our  tutorial series on jenkins in the last video i   showed you how to create a jenkins job for a  simple maven project and our jenkins server   has no problem running this job or really any of  the jobs that we've created in the previous videos   but in a corporate environment jenkins jobs might  consume a significant percentage of resources on   the machine that's hosting the jenkins server  especially when there are many jobs ran in   parallel so let's take a look at our next topic  that will help mitigate this challenge in this   video we are going to configure jenkins controller  and jenkins agent nodes and before i walk you   through the configuration steps i first want to  discuss the problem that we're trying to solve   as software teams and their organizations  grow so does the need to scale jenkins   in addition software teams may need to build and  test software on multiple platforms like windows   mac os and linux and a single jenkins instance  will likely be unable to meet these requirements   in order to meet these requirements we must  scale jenkins up and or out jenkins can support   organizational growth as well as the need  for multi-platform build and testing through   the use of agents jenkins agents are machines  that the jenkins controller can delegate the   task of building projects to in a distributed  manner an agent node might be a windows machine   a linux machine or some other platform that  supports java this allows multi-platform build   and testing using the same jenkins server and it  also allows us to scale our jenkins out as needed   all we need to do is add additional agent nodes  if we take a look at the architecture shown here   the jenkins controller node is the jenkins server  that we've already configured in previous videos   we will be adding a windows agent node in this  video after we've configured our jenkins agents   the jenkins controller will be able to schedule  builds on jenkins agents according to the jenkins   job configuration now that we've covered those  details let's get started so the first step to set   up our jenkins controller in jenkins agent nodes  is to create an ssh key pair i have my terminal up   already i'm using git bash and in git bash  i'm going to use the ssh keygen command to   generate a new key pair and i'll call it azure  jenkins agent and i'll leave the passphrase empty   okay so that generated a new key pair  and i'm going to cap the private key i'm going to copy the contents of the private key and then i'm going to navigate to jenkins and in  the jenkins controller i'm going to navigate to   manage jenkins and then we'll  navigate to manage credentials we'll open up the jenkins credential store  and then global credentials and then i'm   going to select add credentials and then the  kind of credential that we're going to use   is ssh username with private key and we'll give  it an id we'll say it's azure jenkins agent and   the username is going to be azure user and  then the private key will enter directly   and i'll just paste the private key into  that field okay and then i'll select okay the next step is to copy our public key to the  machine that is going to be our jenkins agent   hosted in azure and i already have an ssh session  open with that machine so i'm going to open up   that session and i'm currently on the logged  onto the machine or the vm that will be running   as a jenkins agent and i'm going to modify  the authorized the authorized keys file and add the public key that we just created okay so in the other terminal window  i'm going to cat the public key and i'll copy the public key value and i'll add this public key  to the authorized keys list and then we will save that file so now we should  be able to ssh or this jenkins instance should   be able to ssh to this vm hosted in azure so we  can begin configuring this azure vm as a jenkins   instance inside of our jenkins controller so i'm  going to navigate back to the jenkins dashboard   and from here i'm going to navigate to manage  jenkins and then i'm going to select manage nodes   and clouds so we're going to create one additional  node and i'm going to use this windows pc as the   second node so i'm going to select new  node and we'll call it windows agent   i'll select permanent agent even  though i do have the option to   copy from an existing node and i'll select ok  i'll leave the number of executors at one and   then i'll use the same remote root directory  i'll just call call it jenkins agent data and for labels i'll use the windows label usage i'll leave  usage as default and then the launch method   will use launch agent by connecting it to  the master and one uh reason uh besides just   to demonstrate this launch method uh one other  reason to use this launch method is because   i'm on this windows pc is on my home network  and the azure vm is has a public ip and it's   accessible so i can use the the ssh connect to  via ssh in that case because port 22 is open   and the ip for that azure vm is public but for  this windows pc the ip address is not public and   by using this launch method  i can start the agent program   locally and then specify the ip address of the  controller and establish a connection an outbound   connection to the controller that way since  the controller does have a public ip address   i'll leave the rest of the configurations for this  launch method as default same for the availability   and then i'll select save so now you can see that  the windows agent has been added but there is an   x over the node indicating that jenkins is  not connected with it and if we click on   the agent it says that the jnlp agent  port is disabled and agents cannot connect   this way and it even tells us where we need to  go to update uh the configuration uh so that this   uh so that this works correctly so i'm going  to follow this hyperlink and this takes me to   the global security page of jenkins and if i  scroll down to the agents section i have to enable   a tcp port for inbound agents  and i can select random or fixed   and i'm going to select fixed and i'm going to  enter just you know a high value port number   that i know is not being used  and then i'm going to select save now i'll select manage nodes in clouds  and i'll select the windows agent again   and now you can see that it tells us  at the top we can connect to the agent   using one of these ways and the first we can  run from the aging command line this command   and this is the command that we're going to use  to run the agent program on this windows pc so   i'm going to copy this command to my clipboard and  i'm going to download the agent.jar file locally and now that we have that downloaded  i'm going to open up a git bash session   and i'll navigate to the directory  where the agent.jar file was downloaded and i'm going to paste the command   into the session and we'll run it and  see if if the agent connects successfully okay and it looks like it did connect  it says that it's connected right there so i'll refresh the page and it says that  the agent is connected i'll navigate back   to the dashboard we'll create a new item  and we'll call it test windows agent   i'll select freestyle project and then i'll  select restrict where this project can be ran   and i'll use the windows label this time and in the build section i'm going to add a single   build step execute shell and we'll  say echo hello world to helloworld.txt and we'll select save and then  let's go ahead and build this job okay so i'll navigate into the the job console  output and you can see at the top of the console   output it's building remotely on the windows agent  using the windows label and then it specifies   the workspace that it's using on my local  windows pc and it looks like it was successful   with the echo command as well we can quickly  verify that in git bash supply cd to downloads   and then jenkins agent data i'll cd into the  workspace and then i have the job directory if   i cd into the job directory we can see the hello  world text file did get created if i cat that the hello world is in that text file hey what's up everybody and welcome back to our  tutorial series on jenkins in the last video   i showed you how to create a jenkins job for a  simple maven project and our jenkins server has no   problem running this job or really any of the jobs  that we've created in the previous videos but in a   corporate environment jenkins jobs might consume a  significant percentage of resources on the machine   that's hosting the jenkins server especially  when there are many jobs ran in parallel   so let's take a look at our next topic that will  help mitigate this challenge in this video we are   going to configure jenkins controller and  jenkins agent nodes and before i walk you   through the configuration steps i first want to  discuss the problem that we're trying to solve   as software teams and their organizations  grow so does the need to scale jenkins   in addition software teams may need to build and  test software on multiple platforms like windows   macos and linux and a single jenkins instance  will likely be unable to meet these requirements   in order to meet these requirements we must  scale jenkins up and or out jenkins can support   organizational growth as well as the need  for multi-platform build and testing through   the use of agents jenkins agents are machines  that the jenkins controller can delegate the   task of building projects to in a distributed  manner an agent node might be a windows machine   a linux machine or some other platform that  supports java this allows multi-platform build   and testing using the same jenkins server and it  also allows us to scale our jenkins out as needed   all we need to do is add additional agent nodes  if we take a look at the architecture shown here   the jenkins controller node is the jenkins server  that we've already configured in previous videos   we will be adding a linux agent node in this video  after we've configured our jenkins agents the   jenkins controller will be able to schedule builds  on each of the jenkins agents according to the   jenkins job configuration i'll briefly summarize  the environment that i will be using in this video   the jenkins controller and one of the jenkins  agent nodes are hosted on azure one additional   agent will be ran on my windows pc the jenkins  controller and jenkins agent nodes hosted in azure   both have public ips additionally port 22 is open  for inbound traffic on the azure agent one thing   i'd like to point out is that we won't be using  the azure vm agents or aws ec2 jenkins plugins   these plugins allow you to dynamically provision  agents in the cloud you may be wondering why we   aren't using these plugins and it's not because  they aren't useful these are very useful plugins   but the method that i'm going to show should  be cloud agnostic and can be applied in an   on-premise environment or without a cloud  account you could even set this up to run   entirely on a single machine now that we've  covered those details let's get started so the first step to set up our jenkins  controller in jenkins agent nodes is to create   an ssh key pair i have my terminal up already  i'm using git bash and in git bash i'm going to   use the ssh key gen command to generate a new  key pair and i'll call it uh azure jenkins agent and i'll leave the passphrase empty   okay so that generated a new key pair  and i'm going to cap the private key and i'm going to copy the  contents of the private key and then i'm going to navigate to jenkins and in  the jenkins controller i'm going to navigate to   manage jenkins and then we'll  navigate to manage credentials   we'll open up the jenkins credential store  and then global credentials and then i'm   going to select add credentials and then the  kind of credential that we're going to use   is ssh username with private key and we'll give  it an id we'll say it's azure jenkins agent and   the username is going to be azure user and  then the private key will enter directly   and i'll just paste the private key into  that field okay and then i'll select okay the next step is to copy our public key to the  machine that is going to be our jenkins agent   hosted in azure and i already have an ssh session  open with that machine so i'm going to open up   that session and i'm currently on the uh logged  on to the machine or the vm that will be running   as a jenkins agent and i'm going to uh modify  the authorized the authorized keys file and add the public key that we just created okay so in the other terminal window  i'm going to cat the public key and i'll copy the public key value and i'll add this public key  to the authorized keys list and then we will save that file so now we should  be able to ssh or this jenkins instance should   be able to ssh to this vm hosted in azure so we  can begin configuring this azure vm as a jenkins   instance inside of our jenkins controller so i'm  going to navigate back to the jenkins dashboard   and from here i'm going to  navigate to manage jenkins   and then i'm going to select manage nodes and  clouds and as you can see from this list there's   currently only one node configured for this  jenkins instance and it's the machine that this   jenkins server is currently running on and  we're going to add two additional nodes   the first node is going to be our azure vm and  to add an additional node i'm going to select   the new node button here and i'm going to provide  a name for the node we'll call it azure agent   and then i'm going to select that this is a  permanent agent and this is a permanent agent   as opposed to an ephemeral agent which could  be dynamically provisioned and decommissioned   and after selecting permanent agent i'm going  to click ok and it takes us to a configuration   page where we can specify additional  configurations for this new jenkins agent   and it includes a description as well as the  number of executors on the jake and jenkins agent   so the same as we can control the number of  executors on the jenkins controller we can do   so on the jenkins agent so that it can have  it can perform multiple builds in parallel   i'm going to leave the number of executors  as one in this case the next configuration   is the root directory and this is going to be  the directory that the jenkins agent caches all   of its build data and this can be a pre-existing  directory or directory that's created on demand   and in my case i'm going to create a directory on  demand and i'm going to call it jenkins agent data   and this directory is going to be created under  the home directory of the azure user which is   the user that jenkins is going to be using to  log into the azure vm the next configuration   is very important it's called labels and labels  are a way of categorizing your jenkins agents so   for instance you might have windows nodes and  linux nodes and it might be important to have   certain jobs only run on the windows nodes  and other jobs run only on the linux nodes   so for this agent i'm going to specify a linux  label since the azure vm is running a linux image   the next configuration is the usage configuration  and we can specify whether to use this node as   much as possible or only build jobs with label  expressions matching this node and that simply   means that anytime a job specifies to build  on certain labels for instance the linux label   then uh those builds could be scheduled on this  agent if it's online and we will use the label   expressions but i'm going to leave it as the  first option use this node as much as possible   the next configuration is the launch method  and the launch method is a really important   configuration uh when you're setting up a new  jenkins agent and we have several options for   launching a jenkins agent and the first option is  launching an agent by connecting it to the master   and the master is the older verbiage and  now it's called the controller so this is   launching an agent by connecting it to the  controller the jenkins controller and the second   option is launching an agent via execution of a  command on the controller and then the last one   is launch agents via ssh and for this particular  jenkins agent i'm going to use a third option   launch agents via ssh and for the second  node which is going to be my windows pc   i'm going to use the first option launch agent  by connecting it to the controller when we use   the first option jenkins allows us to download  an agent jar file and that jar file is running   the java application that is the jenkins agent so  we have to download the jar file to the jenkins   agent and then run the java application inside of  that jar file which will establish a connection   with the jenkins controller and with the second  method launch agent via execution of command   on the controller uh this allows us to specify  a launch command in this field and the command   if i expand the uh the help uh text here  the command can be very simple we could   specify ssh and then we could ssh  onto the jenkins agent node and run   the agent jar file directly or we could invoke  a script on the jenkins agent node in the event   that we had we have to set up environment  variables on the agent node in order to   run whatever builds that agent will be running  but in my case i won't be using that launch method   i'm going to use the launch agents via ssh for  this first jenkins node and uh in the host field   i'm going to specify the ip of the jenkins um  the jenkins agent node which is a an azure vm   and i've already copied this vm's ip address so  i'm going to paste it into the host field and   then for the credentials i'm going to select the  private key that we created earlier in the video   for host key verification strategy i'm going  to select non-verifying verification strategy   and then for availability i'm going to leave it as  default keep this agent online as much as possible   we have a couple of other options that we can  use for availability like bring this agent online   according to a schedule or bring this agent online  when in demand and take offline when idle so if i   were to select that i could specify an in-demand  delay and essentially what this means is it's the   number of minutes as it says here in the the help  section it's the number of minutes for which jobs   must have been waiting in the queue before jenkins  will attempt to bring this agent online and then   we can also specify an idle delay and this  is the number of minutes that the agent will   remain idle before jenkins will take it offline  so this availability strategy might be useful   if you're trying to optimize your costs on the  jenkins agent nodes but in my case i'm going to   select the first option keep this agent online as  much as possible i'll leave the node properties   unchecked and then i'll select save and now  you'll notice that the azure agent node has been   added to our nodes list and under the build  executor status we not only see the controller   node and its two executors but we can also see  the azure agent node and its single executor and   if we take a look at the logs on this azure agent  we'll be able to see that the jenkins controller   successfully made an ssh connection to the azure  vm and after it authenticated if we scroll down   a little bit it copied the remoting.jar  and this remoting.jar is essentially the   jenkins agent program that will run on that azure  vm and you'll also notice that it created the   jenkins agent data directory since that didn't  exist on the on the vm and after it copied the   jar file uh it cd into the jenkins  agent data directory and it started the   the agent jar file and in the final  line of the console output here   you can see that it specifies that the agent  successfully connected and is currently online so   let's navigate back to the jenkins dashboard and  i'm going to create a very simple job and we'll   call it test azure agent and this will be  a freestyle project i'm going to select ok and in this job i'm going to select  restrict where this project can be run   and when i select that i get a field label  expression and this allows me to specify a   label that was previously created uh like  a label that was created when we were   configuring the jenkins agent for instance uh  and when i specify a label this job will only   be ran on nodes that match those labels so for  the agent that we just created i created a linux   label so as i start typing linux it autocompletes  it recognizes the label that i previously created   and i'll backspace here and then it says  label linux matches onenote so anytime   this job runs it should run only on our azure  agent node and now that we've specified that   label i'm going to scroll down to the build  section i'm going to select add build step   i'll select execute shell and i'll do something  really simple here i'll just echo hello world into a new text file and then i'll select save so now that we've created this job  let's go ahead and select build now and i'll go ahead and open up the first build  console output and then you can see at the top   of the console output here it specifies that it's  building remotely on azure agent with the linux   label and it's building in the following workspace  and it looks like the echo command did work   uh correctly but let's just verify it uh by  logging into the vm and checking the workspace so   if i take a look at the vm i have the jenkins  agent uh data directory so i'll cd into that and then i'll cd into the workspace and  i can see the job name test azure agent and i see uh helloworld.txt so if i cat  that and it does contain hello world   inside of inside of that file so it looks like  our first jenkins agent is working correctly   so that's pretty much all i had for this video  and in the next video we are going to discuss   jenkins pipelines so i will  see you in the next video hey what's up everybody my name is moss and  welcome back to our tutorial series on jenkins   until now we've only created  freestyle jobs and maven jobs   in our jenkins server but in industry  you'll need more than a single freestyle job   to facilitate your full build and release  process and for these more complex use   cases jenkins allows you to define your  build and release process in groovy code in this video i'm going to introduce you to  jenkins pipelines let's begin with a brief   overview as a team software development  process matures it can become difficult to   model the build and release process using only  freestyle jobs in jenkins in order to mitigate   the complexity of managing multiple freestyle  jobs the jenkins pipeline plugins allow you to   model those complex build and release processes  as groovy code in what's called a jenkins file   so what are some of the benefits of using a  jenkins pipeline if we check in the jenkins   pipeline code contained within the jenkins file  into version control then we immediately get the   benefit of traceability and control over changes  made to the pipeline the other benefit of version   controlling the jenkins file is that the pipeline  exists independent of a jenkins server which   means if for some reason the jenkins server was  accidentally destroyed the pipelines would not be   destroyed since they're managed in a source code  management server the other benefit of jenkins   pipelines is that they are highly extensible  through the use of shared libraries and plugins   in addition pipelines support logical operators  such as conditionals loops and parallel execution   let's also quickly discuss some key pipeline  terminology the first is the concept of a pipeline   a pipeline is a user-defined model of a build  test and release process for a software product   the second term you should be familiar with  is node a jenkins node is a machine that the   jenkins controller can execute pipeline jobs on  the next concept that you should be familiar with   is a stage a jenkins pipeline stage is a logical  grouping of steps that typically align with the   build test and release portions of the software  development life cycle so for instance you might   have a build stage a test stage and a deploy stage  you should also be familiar with the concept of   steps steps are individual tasks completed within  a stage of a pipeline for instance a single step   could invoke a jenkins node's shell and execute  a command like ls or make directory on it   it's also worth mentioning the two types of  jenkins pipelines there are declarative and   scripted pipelines declarative pipelines are a  newer feature of jenkins pipeline and declarative   pipelines have richer syntactical features than  scripted pipelines which makes them easier to read   and write now that we have a general understanding  of what jenkins pipelines are let's get started okay so i'm inside of my jenkins server  and the first thing that i want to do   is install a new plug-in called the  pipeline maven integration plug-in   now to write a jenkins pipeline you don't need  any additional plugins all of the plugins that   you need to write a basic jenkins pipeline are  included when you install jenkins on a machine   but if you have a maven project that you want to  build inside of a pipeline script it's good to   install the pipeline maven integration plugin  because it adds some additional features to   the pipeline and we are going to build a maven  project later on in the video so i'm going to   go ahead and install that plugin and to do so  i need to navigate to the manage jenkins page and from manage jenkins i  want to select manage plugins   and i'm going to select the available tab  and then i'm going to search for pipeline   maven and this pulls up the pipeline maven  integration plugin so i'm going to go ahead   and select that and i'm going to select  download now and install after restart and i selected the restart after it's been  restart automatically after it's been installed so   jenkins is restarting and once it's restarted  we'll be able to utilize that plug-in so now that that plug-in has been installed let's  navigate back to the dashboard and we're going to   create our first pipeline job so to create  a pipeline job i'm going to select new item   and i'm just going to call it test pipeline job and then i'll select the pipeline  item here and i'm going to select ok now from here i'm going to go down to the  pipeline section and we can define a new pipeline   in the pipeline section and we have this drop down  here and the first selection is pipeline script so   we can define a pipeline script directly in this  job definition or we can define a pipeline script   outside of jenkins and version control it in an  scm tool like github we will use this option in   our second example but in the first example i'm  going to build the pipeline script directly in   the job definition now it's possible to write  a pipeline script completely from scratch   but if you're not too familiar with groovy or  pipeline syntax then it can be a little tedious to   write a script by hand and what's convenient  about jenkins is it gives us this tool the   pipeline syntax tool to automatically build um a  pipeline script so i'm going to go ahead and open   up the pipeline syntax tool and this tool allows  us to automatically generate pipeline code and we   have two options we have the snippet generator and  then we have the declarative directive generator   with the snippet generator we can define  individual steps and remember that steps   are individual tasks within a particular stage of  a pipeline and let's take a look at some of the   steps that are available to us so the very first  step that we can do is we can archive artifacts   we can build a job that exists on this jenkins  instance we can also echo we can use the echo   command to print a particular message to the  console and then we can enter the message here   okay and then when we uh enter the message or we  fill out whatever parameters are necessary for the   step that we selected we can select generate  pipeline script and then this automatically   generates the uh the command that we would enter  into the jenkins file so you would use the snippet   generator for automatically creating individual  steps within a pipeline stage but if you want   to define the overall structure and the stages  of the pipeline we would use the declarative   directive generator instead so from here it  gives us a sample directive drop down list   and in this drop down list we can select various  directives and the one that we're going to want   to use is the stages directive so we want  to define multiple stages for our initial   sample pipeline once we've selected the  stages directive i'm going to add a stage and the first stage that uh i'm going to  define is the build stage okay so this stage   is going to be responsible for building  the code base whether it be you know a c   plus code base or a java code base and the  build stage is going to contain steps so   the build stage could contain parallel stages so  we could run other stages in parallel of the build   stage sequential stages or a matrix but in this  case for a simple pipeline we're just going to   have steps included in this stage and then after  specifying that steps are contained within this   stage i can also specify additional options so for  instance i can specify that this stage should run   on a particular subset of agents maybe linux  jenkins agents or windows jenkins agents   and then i can also add when conditions to the  stage i can add post stage or build conditions   i can add environment variables for this  particular stage so there's a lot of options   that i can utilize in a stage but i'm going  to leave it as that and create a new stage   and we'll call this the test stage and this will  also contain steps and then we'll create one   last stage and we're going to call it the deploy  stage okay so the testing stage of course would be   the stage where you would test the  code that was built in the prior stage   and then in the deploy stage ideally if the  test passed this would likely be a conditional   stage if the test passed in the testing stage  then you would deploy your changes to production   or maybe if you had like a qa environment  or some sort of promotion process you would   initiate that promotion process so these three  stages represent the skeleton of a new pipeline   and to generate the uh the code for these stages  i'm going to select generate declarative directive   and you can see here that it uh automatically  generates these stages okay so we have the build   stage the test stage the deploy stage and then  within each stage we can have one or more steps   so what i'm going to do is copy this generated  code into my editor and i'm going to begin a   new jenkins file so i'm going to open up vs code  and before i paste this code in i need to define   a pipeline block so i'm going to say pipeline  and then brackets and directly under the pipeline   block i'm going to specify agent any and what this  um statement means is that this pipeline can run   on any available jenkins agent and  then after the agent any statement   i'm going to paste in the uh stages block as well  okay so now we have our stages defined within the   pipeline block but right now this pipeline doesn't  have any steps defined within the stages so let's   define a step within each of the stages so i'm  going to use the snippet generator in jenkins and let's use an echo statement so i'm going click  echo here and then let's just say hello world generate pipeline script and then let's  copy this into our new pipeline script   right so i'm going to delete this comment here i'll paste in the echo statement  and here we'll say building and then for testing as you can imagine we'll say that we're testing and then same for here we'll  just say that uh we're deploying and then we'll add one more to  indicate that the pipeline is completed   okay so this should be a valid  pipeline so let's go ahead and copy it and navigate back to our  our pipeline job definition   and i'm going to select configure  and go back to the configuration page   and then under the pipeline section  i'm going to paste the pipeline script   into the uh into the field here okay  so let's go ahead and click apply   click save and i'm going to go  ahead and build the pipeline and it looks like the build failed so let's  go ahead and open up the console output   and i can see here that it looks like it  was expecting a closing bracket at line 23   so let's navigate back to the  editor and okay it looks like i   i missed uh the closing bracket for the  pipeline okay so let's move that in one alrighty and then let's go ahead and copy this updated uh script and then we're going to  configure again and then we'll paste it back in   all right let's go ahead and  save and i'm going to build again okay and now the build is passing and in  addition to a passing build you'll notice that   this stage view also uh popped up after the build  completed and the stage view uh basically gives us   an overview of what happened in any particular  pipeline build and it separates it out by stage   so we can see here the status of the build stage  the status of the test stage and the deploy stage   we also can see the average stage stage  times which in this case there's only one run   so it's the same as the actual the  the current execution of of this build   so it didn't take long 391 milliseconds  these are pretty uh quick stages   and for each stage we can see logs contained  within that stage and you can see our echo   statement building in the build stage and same  for the test and deploy stages as well so you   can access the logs uh through this interface but  you can also of course open up the console output   and see roughly the same but maybe a  little bit more verbose so as you can see   writing a basic jenkins pipeline script is pretty  straightforward but of course you want to build   test and release code in a pipeline script and  in this example we didn't have any code that we   built tested or released so let's go ahead and  create a new pipeline and we'll actually build   a maven project in our new pipeline and we'll  also utilize the pipeline maven integration   plugin and before we create the new pipeline job  i want to real quickly show you the um the github   repository that we're going to build and it's  this one the sample maven project and it's a   very uh basic java application that just prints  hello world to the console and it has a single test class using junit and in the  single test within that test class   it just uh runs an assert true on a condition  true so the test should pass and this is the   same project uh that i used in earlier videos  when we were discussing maven jobs in jenkins   so what we're going to do in this repository is  create a new file and it's going to be a jenkins   file that contains our pipeline code and we'll  keep it at the root directory of the project   so i'm going to select add file create new file  and then we're going to call it jenkinsfile   with no extension just plain like that and  then we will define the pipeline block here and agent any statement okay  and then we're going to generate   automatically generate the rest of the  pipeline so let's navigate back to jenkins   and we'll create a new pipeline job so  i'm going to navigate to the dashboard and from here we'll select  new item and we'll call it   test maven pipeline and then i'll  select a pipeline and then select ok and from here we'll scroll down to the pipeline  section and for the pipeline definition instead   of defining the pipeline script directly in this  field we'll pull the pipeline script from scm so   i'm going to select pipeline script from sem  and then for the sem system we'll select git   and then we're going to specify this repository's  url so i'm going to select open a new tab   and i'll copy the ssh url of this repository and we'll paste that in and it gives me  an error code since i haven't selected uh   credentials so i'm gonna go ahead and select  uh an ssh key pair that i previously created   okay so it looks like it uh authenticated uh  correctly and then under branches to build   i only have the main branch so i'm going to update  the branch specifier and then under script path   we have to specify where the jenkins file is  located in the repository since we're keeping it   along with the code base and in our case we  have the jenkins file at the root directory   of the repository so we can leave this as is but  if we had the jenkins file in a subdirectory we   would have to specify that path in this field and  now we're going to define the actual pipeline and   i'm going to use the pipeline syntax tool and i'm  going to select a declarative directive generator   and i'll select stages so i get a stages block and  then i'm going to add a new stage and we'll call   it maven install since within this stage we're  going to invoke maven and we're going to run the   maven install command on our maven project so this  stage will contain steps and this is going to be   the only stage that we really need since maven  install is going to you know complete the full   life cycle so i'm going to select  generate declarative directive   and then we get our stages block i'm going  to go ahead and copy this text i'm going   to navigate back to our jenkins file in  github and we will paste the stages block within and let's navigate back to jenkins and  i'm going to scroll up to the snippet   generator and we'll define a new step and this is  where the pipeline maven plug-in comes in so i can   utilize a step called with maven okay and  then with maven provides a lot of options so   with a specific maven environment i can perform  maven operations so i can specify a particular   installation of maven or of the jdk i can  specify uh maven settings as well so there's   a lot of options that the pipeline maven plugin  provides for this use case i'm going to use   essentially all of the default selections and i'm  going to scroll down to generate pipeline script   and generates a very simple with maven  block and i'll go ahead and copy that and then i'll navigate back to the jenkins file   i'm gonna paste that into uh  the steps section okay and in the with maven block i'm going to  invoke the sh utility and then i'm going to invoke maven so i'll say maven clean install  all right and that should uh run maven   perform a clean and then perform the install  lifecycle and this should be all we need since   the jenkins file is stored with the code base  itself if the jenkins file was stored in a   different uh code base than the maven project then  as an additional step we would also have to invoke   git here to ins uh to clone down the uh the  code base but in our case since the jenkins file   is uh in the same repository as a code base we  can leave it as uh as is and this is going to run   maven clean install on on our java  application so i'm going to go ahead and commit this and let's go ahead and navigate back to  our pipeline and i'm going to select save   and then let's select build now okay it looks like the maven install stage  failed and i get a message here that says   maven not found so i think that the maven  installation might not be selected so i'm going to select configure and let's  open up the pipeline syntax   i'll select with maven again and then with  the maven installation drop down i'm going to   take a look and see okay so i should  have selected maven 3 i have a a maven installation that i can specify so  i think i was supposed to select this uh   so let me generate the pipeline script and  as you can see when with maven is invoked   now uh the maven installation is specified as a  parameter so i'm going to go ahead and copy this   and i'll add it to our jenkins file okay and then let's go ahead and commit those changes alrighty then i'll navigate back to our pipeline and i'm going to select build now okay and it looks like the maven  install is working correctly   okay so uh just to kind of recap what  i did i specified the maven insulation   that the pipeline should use  when we invoked the with maven   code block and if i take a look at manage jenkins  and i look at the global tool configuration   under the maven section in maven installations  i have a maven installation that i install   automatically called maven 3 okay so this is  what was showing up in the drop down menu in   the snippet generator so i'll navigate back to our  pipeline and as you can see when the pipeline runs   successfully the maven installation command  ran and let's take a look at the output here so the full life cycle the full install life  cycle ran so it built the sample java application   it ran the single test and we can see  that the test ran here with zero failures and then we get successful artifact publishing  to the local repository the local repository   in this case is just the jenkins workspace so  if i take a look back at the the job home page   you can see the last successful artifacts are the  snapshot jar and the snapshot palm files that are   contained within side of the workspace of build  2 so these are the build artifacts of build 2.   and from the build page we can also evaluate  the test results of the pipeline in this case   there weren't any failures and there was only  a single test case that was ran and it passed   if i take a look at the home page we  also get the junit the test graph with   the results of each builds test so as you can see  when we use the pipeline maven integration plugin   it makes writing a pipeline script for a maven  project a lot simpler because all we need to   do is specify that with maven block  and then we can pass in parameters   to the environment the maven environment that  is generated by the pipeline maven plugin   okay so we were able to specify an insulation as  a parameter to the with maven block and we could   run other installations of maven as well if our  maven project was dependent on an older version of   older or newer version of maven so that's pretty  much all i had for this video i hope you enjoyed   it and in the next video we're going to continue  our tutorial on jenkins pipelines and if you'd   like to read about jenkins pipelines be sure to  check out lambda tests learning hub for jenkins   the learning hub has a lot of great resources  related to jenkins and real quickly i'll uh   navigate to the learning hub and show you so  it's lambdatest.com forward slash learning hub   forward slash jenkins and uh you have these  chapters which are separate articles on   various topics related to jenkins and if you  were particularly interested in reading about   jenkins pipelines you could check out the  creating your first jenkins pipeline article   but again thanks for watching and  i will see you in the next video hey what's up everybody my name is moss and  welcome back to our tutorial series on jenkins   in the last video i introduce you to jenkins  pipelines which are a key feature of jenkins   and now we're going to take the  knowledge that we've learned over   the previous videos and put  it to use for the next topic in this video i'll show you how to do cross  browser testing with the lambda test jenkins   plugin before we get started i'd like to quickly  go over the prerequisites the first is that you'll   need a lambda test account secondly you'll  need python 3 installed on the jenkins server   and finally you'll also need to install  the python selenium webdriver module i'd also like to give a quick overview  of the lambdatest jenkins plugin   the lambdatest plugin adds additional features  to jenkins jobs such as embedding test results   in the jenkins build you can also configure your  lambda test credentials for your jenkins jobs   and the tunnel also provides automatic  setup of the lambda test tunnel binary file   which is required if you intend  on testing locally hosted sites   now that we have a high level understanding of  the lambda test jenkins plug-in let's get started okay so the first thing that i want to do is walk  you through the code base that we're going to be   testing in jenkins and i'm currently  inside the github repository   and the repo is called sample to do app and  this is basically a very simple web application   and it only has the index.html file here and  the the web application is actually hosted   using a feature of github called github pages  so if you're not familiar with github pages it   basically allows you to use github to host a web  page from a github repository and to navigate to   the the page where this code base is hosted  i can navigate to settings in the repository   and if i scroll down to the pages section and  select pages under the github pages section   you'll see your site is published at this url so  if i go ahead and open up this url i should see   my application and it's the lambda test sample  application and it's just a to-do list with   a few items already filled out but if i wanted to  add additional items to the to-do list i can just   fill it out here and then select add and i can  also mark items as completed using the check boxes   as well now in order to test this web application  i've written a very simple python selenium script   and if we navigate back to the code base we can  take a look at that script it's called test sample   to do app.pi so i'm going to go ahead and open  it up and we'll quickly walk through this code   and the first thing that we do after importing  our modules is define the url for our remote   web driver and we get the url from an environment  variable that's predefined called lthub url which   is short for the lambdatest hub url and we'll get  the value of this url in a little bit and after   we define the url we then define the desired  capabilities using a dictionary and the first   capability dictionary key value pair here  is the build name and we actually retrieve   the build name from an environment variable that  is used by the lambda test plugin or rather it's   defined by the lambda test plugin so this is  the lambda test build name environment variable   and also notice that we set tunnel to be true  because we're using we are going to use the   lambda test tunnel binary file and the lambda  test tunnel opens a connection from my computer to   the lambda test platform and it allows the lambda  test platform to connect to a locally hosted site   on my computer so after we specify the desired  capabilities we then instantiate a remote driver   using the desired capabilities as well as the  url which is the lambda test hub url and after we   instantiate the driver we can then make a request  to localhost and here we're making a request to   localhost on port 8081 which is where we're going  to run our sample to do app when when it's running   inside of a jenkins job after we connect to  localhost the driver finds an element by name   line item 3 and it clicks that line item so  it should select the third item checkbox here   and after it selects that checkbox we  then get a reference to the text box   directly below so we'll get a reference to  this field here and then we send the keys   testing in that field and then we select the  add button so we should get something like   this right and i'll add that to the to-do list and  then we assert that no results found is not in the   page source and directly after that we use a  very useful command here we execute script and   then lambda test status equals passed and this  will actually update the status of the tests   inside of the lambda test dashboard to pass  so this is a very useful uh script here   and then directly after that we uh quit the  driver okay so now that we've walked through the   code base and the test script uh let's navigate to  jenkins and we're going to install the lambda test   jenkins plug-in on our jenkins instance so  i'll open up jenkins here and then from jenkins   i'm going to select manage jenkins and  then i'm going to go to manage plugins i'll select the available tab then i'm  going to select i'll type in lambda test   and the first plugin listed here lambda test  automation is the plugin that we want to install   so i'm going to go ahead and check it and then  select download now and install after restart   okay so it's pending and then i'll  just check the box restart jenkins okay so the plugin should be installed and now  i'm going to add uh my lambda test credentials   to jenkins so i'm going to select manage jenkins  and i'm going to scroll down to manage credentials and i'll select the global credentials  and i'm going to select add credentials and from here the lambda test plugin added  a new kind of credential so if i open this   drop down here you'll see lambda test credentials   and i'll leave the scope as global and then the  username my username is techwithmoss gmail.com   and then the access token i can retrieve from my  lambda test profile so if i navigate to lambda   lambda test and then i can very quickly  uh access the uh the key if i just select   configure tunnel here and then the key is the  the option here the key option right here so   this value is the key that i would need to copy  into jenkins so i'm going to go ahead and do that and i'll go ahead and paste that in and we'll call it lambda test credentials and then if i select verify credentials   it says successful authentication okay so let's  go ahead and select ok and add those credentials   now remember that our test script references an  environment variable called lthub url which is the   lambda test hub url and this environment variable  still needs to be set so let's go ahead and   set that environment variable in jenkins  so if i navigate back to the dashboard   and then manage jenkins and then  scroll down and select configure system   i'm going to scroll down to global properties  and i have environment variables checked and   i'll select add okay and we're going  to call it lt hub url just like it   how it's referenced inside of the python script  and to get the value of the hub url i'm going   to navigate back to lambda test and in my lambda  test account i'm going to select configure tunnel   and down here where it says more details  here this is a link to documentation and   in the uh documentation pages we have  a link to the capability generator   okay if i select the capability generator it takes  me to a tool where i can automatically generate   desired capabilities this also automatically  generates the hub url that's unique   to my account and if i expand username and  access keys you'll notice down here hub   this is my hub url so if i copy that to the  clipboard and then navigate back to jenkins   i'll paste that into the value field and that's  going to be my hub url which my python script   is referencing and now that we've saved  that variable i'm going to select save and we're going to create a new jenkins  job so i'm going to select new item   and i'm going to call it lambda test job   and it will be a freestyle project so i'm going  to select freestyle project and then select ok and the first thing that i want to do is configure  source code management so i'm going to select git   and then i'm going to navigate back to github and  i'm going to retrieve the repo url so remember   that it's ssh so i'm going to copy the ssh url and  then back in jenkins i'm going to paste that url   in and i'm going to select credentials  that i previously configured okay   and the branch specifier just have the main branch  okay and after we've set up source code management   i'm going to scroll down to the build  environment and you'll notice that we   now have a new option here called lambda tests  so i'm going to go ahead and select lambda test   and this section is our interface  with the lambda test jenkins plugin   if i take a look at the selenium test section  i can actually add selenium test capabilities   directly in this interface so if i click add  here i can select an operating system browser a version and the screen resolution  as well and then i can add   as many test capabilities as i need here  the one caveat with using this interface   to define the capabilities is that we  have to reference an environment variable   that is created by the lambda test jenkins plugin  inside of our python script and to find the name   of that environment variable that the plugin  creates i'm going to navigate back to the   capabilities generator and i'm going to navigate  back to uh well actually i'm going to navigate   back to the documentation here and if i scroll  down to the section lambda test integrations   and then integrations with ci cd tools  whoops i'll select the jenkins plugin and in the jenkins plugin page if i scroll down to the environment variable for  lambda test jenkins plug-in section   it gives me a list of environment variables that  the the lambda test plug-in will use one of those   variables is called lt underscore browsers  and this is an array of of combinations of   desired capabilities so you can see here when that  environment variable is printed out to the screen   it prints out the desired capabilities and  these are the capabilities that are defined   within the jenkins job here okay but in  order to use these capabilities we have to   reference uh when we're defining the  capabilities variable in the python script   we would reference this environment variable  now i'm not going to do that here i'm going to   use the desired capabilities as they are  inside of the python script but it's good   to know that if you do want to utilize this  feature in the interface you can utilize   that environment variable in your script to  do so so i'm going to remove this capability   and the next configuration is use local tunnel  and the local tunnel or the lambda tunnel binary   file is a feature that allows you to test your  private server urls or locally hosted web apps   using the lambda test cloud servers and  since our web application will be hosted   locally on this jenkins server when we're  running this job we will need to utilize   the the lambda tunnel so i'm going to go ahead and  check that box and then in the next field we have   the tunnel name which we can leave blank and it  will have a it will be given a default name and   we won't make this a shared tunnel and we're also  not going to use the legacy lambda test tunnel   but under optional tunnel commands we are  going to utilize one additional option   when the plugin starts the lambda test tunnel  and that additional option is dash dash info   api port and then the port that this will  run on so what this option will do is it will   provide an end point the the lambda test tunnel  will publish an endpoint on port 8000 that we can   can connect to and we can perform operations on  the tunnel for instance we can close the tunnel   if we want to or we can get status the current  status of the tunnel as well we can also specify   a download path for the tunnel but i'm going  to leave this blank and then i'm going to check   use workspace path to download the tunnel uh into  okay and then for the lambda test credentials we   already created those credentials so i'm going  to select them here and then for the last step   as a post build action we want the test results  embedded so from the post build actions drop down   you'll notice a lambda test report i'm going to  go ahead and select that and this action will   generate a lambda test report inside of the  jenkins build and it uses the build name from   the environment variable lt build name which  is created by the lambda test plugin so if you   remember in the python script if i navigate back  to the code base and i open up the python script uh where the capabilities are defined in the first  key value pair i set the build key to the value of   the environment variable lt build name okay so it  pulls the value of the lt build name and that's   how we get the test report generated inside of  the jenkins build so i'll navigate back to jenkins   and we're going to add a build step  and we're going to execute shell   and we're going to add a few commands here to  start a simple python server that's basically   serving the current directory the directory of the  workspace then we'll also shut down that server   after invoking the python script and we'll also  shut down the lambda test tunnel using that info   info api port endpoint so the first thing that i'm  going to do is invoke the sleep command and i'm   gonna sleep for five seconds that just gives the  tunnel a little bit of time to make a connection   and then after invoking sleep i'm going to  start the python server so i'm going to invoke   python3 and then the module http.server  and we're going to run it on 8081   since this jenkins server is running on 8080 and  then ampersand so that this is a non-blocking   shell command it'll just execute this process in  the background and then we'll invoke python again   and this time we'll run our python script so i'm  going to say test sample to do app dot pi and   once we've once that script has completed  we'll be able to stop the http server that's   serving this particular workspace and to do  that i'm going to use p kill so uh p kill   and then dash f and uh this is kind of like  a fuzzy match for the name of the process   and we'll put in quotes http server  so that should kill the python server   after killing the server i'll invoke sleep  again and sleep for 10 seconds before   uh shutting down the lambda test tunnel and  to shut down the lambda test tunnel i can   curl and end point the info api port endpoint  with uh the delete method and then localhost 8000 and then the endpoint is api 0.0 and then  stop okay so this endpoint should uh shut down   the lambda test tunnel after the tests have  been completed and that should be all we need   so i'm going to go ahead and save this job and  let's go ahead and build it and see what happens i'm going to open up the console output okay and  we can see that it's sleeping for five seconds   it invoked the server as well as the test script and now it's killed the http  server and sleeping for 10 seconds and then we curled the stop endpoint and you  can see here that status equals success so   it successfully stopped the the tunnel and  if we want to check the output of the tests   we can do so by navigating to the lambda test  dashboard and from the dashboard if i navigate to   the automation section i can see in the very  top of the list here jenkins lambda test job   started a couple minutes ago and if i open that  up i can see that the test pass and i can even   watch the output of the test so if  i open this up and maximize it you   can see the tests run pretty quickly but you  can see what what the test did so right here   it selects the third item and then it adds testing  as an additional item to the to-do list and then   we quit the driver so it was a very quick  test and if i navigate back to jenkins   i'll navigate to the build page you'll notice  at the bottom left section here the lambda test   report if i click on that i get kind of  a high level overview of of the tests   and i can navigate directly to the automation page   lambda test by clicking these links  right so it takes me directly to   the build here as well okay so that's how we  can utilize the lambda test jenkins plug-in   inside of a freestyle jenkins job but we also  want to do this from a jenkins pipeline so the   next thing that we're going to do is create a  jenkins pipeline uh and inside of that pipeline   we're going to invoke the lambda test tunnel so  we can connect to the lambda test platform and   run the tests from a jenkins pipeline so i'm  going to go ahead and navigate to the dashboard   and we're going to select new item  and i'm going to call it lambda test   pipeline and i'll select pipeline  type and i'm going to select ok   and from the configuration page i'm going to  scroll down to the pipeline section and i'm going   to select pipeline script from sem because we're  going to store the jenkins file along with the   code base and then for scm i'm going to select get  and then let's copy the url of the code base again okay i'm going to paste that in   and select my credentials okay and  then the branch is going to be main   and we'll also leave the jenkins file in  the root directory of the repository so   we'll leave the script path as it is now and  we'll go ahead and start building the pipeline   i'm going to use the pipeline syntax  tool like we did in the previous videos okay and i'm going to open up the  declarative directive generator   and we're going to generate some stages okay  so scroll down here to the stages directive   and select that one and then let's go  ahead and add a stage and the first   stage that i'm going to create is going to  be called setup and it will contain steps   and then the second stage is going to be called  the test stage and this will also contain steps   those are going to be the only two stages that  we create and i'll go ahead and select generate   directive and i get the stages generated down  here i'm going to copy them to the clipboard and then i'm going to paste them into vs code   and before we paste the stages i'll want to  define a pipeline block and i'll specify agent any   and then after that we can go ahead and paste  the stages block into uh the pipeline block   and in the setup stage we're going to run several  shell commands to download the lambda test   tunnel binary file and then run the lambda  test tunnel so that we can make a connection   and then execute our our tests so we  need to get the download url of the   lambda test tunnel and i'm going to navigate  back to lambda test the lambda test documentation and from the lambda test documentation i'm  going to scroll down to uh or scroll to the   testing locally hosted pages section  and if i select the first link here   how to connect lambda test tunnel for  linux my jenkins server is running on   linux so i'm going to select this link but of  course if your jenkins server is running on   windows you would want to select  this link and i'm going to open up   this page and if i scroll down i have a  link here to the lambda test tunnel binaries   so i'm going to go ahead and copy that link  address and then i'm going to open vs code back up   and we'll invoke the shell command  and i'm going to use wget to download   the lambda test tunnel uh binary file and then  in the next shell command we're going to download   a utility to unzip the lame to test  tunnel binary file so you only need   to use this shell command on the first run of  the pipeline and then you can remove it for   future runs but i'm going to go ahead and  apt-get install zip unzip and once that is   installed i'm going to uh invoke it to unzip  the uh lambda test tunnel uh zip file okay dot zip and once we unzip the tunnel binary  file we can go ahead and invoke the program so   i'll go ahead and do so and we have to pass in as  parameters the username the lambda test username   and we're going to use environment  variables for the username and access key   just to as a best practice so i'm going to call it  lt underscore username and then dash dash key for   the key parameter and we'll also reference  this as an environment variable so this is   going to be lt underscore access underscore key  and then we'll also give this tunnel a name so   tunnel name is going to be jenkins dash tunnel and  finally we'll also set the parameter info api port   running on port 8000 and one last thing  is we are going to run uh this program   as a in the background so i'm going to add  ampersand and close off that shell command   now these environment variables haven't been  created yet so we will need to create them so   i'm going to do that now and i'll open jenkins  back up and if i navigate to the dashboard and i'll select manage jenkins and then i'll select configure system and if i scroll down to global properties   again we can add additional environment  variables so i'll go ahead and select add here   and the first one will be the lt underscore  username which is tech with moss gmail.com   and then the second environment variable is  going to be the lambda test access key okay   and uh this value we can pull from the lambda  tests test dashboard actually we can pull it   from the um hub url so this is in the hub  url we pass in the username and access key   in the url so i'm going to go ahead and copy  the access key here and i'll paste it in   and then i'm going to go ahead and select save and then let's open vs code back up and  that's pretty much everything that we   need for the setup stage in the test  stage we'll essentially copy the same   steps that we took in the job that we  just created in the in the build section   so i'm going to go ahead and delete  this comment here and i'll invoke i'll invoke the shell command and uh just  like we did in the previous jenkins job   we'll go ahead and sleep for five seconds uh just  to allow the the uh tunnel to make a connection   uh and then immediately after that we'll  invoke python and we'll run the workspace   uh we'll serve the workspace on port 8081  and we'll do so as a background process and after that we'll invoke our our test script  so test sample to do app dot pi and once the   test script completes we'll then kill the  server so i'm going to invoke sh and then p kill http dot server okay once we kill the  server we'll uh add a small sleep statement okay we'll just sleep for 10 seconds and then   we'll curl the end point to stop the lambda  test tunnel okay so delete and then uh localhost or 8000 and then api v 1.0 and stop okay so this  should work or behave essentially the same way   as the jenkins freestyle job that we previous  previously created now what you'll want to do   from here is copy the pipeline script and then  in github create a new jenkins file in the root   directory of the repository now i've already  created a jenkins file so i'll just go ahead and   open it up and i'll select edit and i'll go  ahead and paste the pipeline script into the file okay i'll go ahead and commit those changes   and then let's navigate back to the pipeline  configuration page and i'm going to click save and then let's see if this builds  successfully so i'm going to click   build now and it looks like it failed pretty  quickly so let me go ahead and open up the   console output uh and it looks like i was missing  uh an end quote there so i'm gonna open up github and let's close off that end quote okay and then let's go back to jenkins and we'll  build again and let's see if we have any errors all right so i'm going to go ahead  and open up the console output   and let's take a look at what's going on  so it download downloaded the lambda test   tunnel successfully and we can see that here  and then we also install the zip unzip package   which in my case has already been installed  and then after that package has been installed   we unzip the lamid test tunnel and once  we unzip the tunnel we then invoke it with   the parameters that we had previously that we  set previously in the pipeline and we sleep for   five seconds invoke the python server and then  we invoked the test script but it looks like it looks like it failed here so we should  probably take a look at why it failed so it   says build name should be of type string so  let's take a look at why that didn't accept   our build name so let's take  a look at the test script   in the code base and see if we can  troubleshoot uh what's going on okay and it failed at line 18 but um i think i know  what the issue is here because the build name   in the in the capabilities is still using the  environment variable lt build name which since   we're uh using the pipeline job that environment  variable isn't defined so if i navigate back to   the pipeline script we can actually define that  variable within the pipeline script so if i edit   the pipeline script and then below agent any i'll  invoke the environment block and inside of the   environment block i'll define lt underscore build  underscore name and then we'll say it's equal to   the name of the pipeline job which is lambda test hyphen pipeline okay and then we'll  close off that environment block   and let's see if that fixes the issue okay and i'll navigate back to uh   jenkins and let's uh run the build again  and see if we can get it passing this time okay i'll go ahead and open  it up and let's navigate okay it looks like it's running  the test script successfully okay so the test script uh ran without any errors  and we killed the http server it's sleeping for 10   seconds and then it should close the lambda test  tunnel which it looks like it did successfully   and let's take a look at the test results in the  lambda test dashboard as well so i'll open up   lambda tests and i'm inside of the automation tab  and then you can see at the top of the list here   the lambda test hyphen pipeline which completed  so i'll go ahead and open this up and it says that   the test pass and if i take a look at the video  as well you can see that the tests ran the same as   they did in the freestyle job so those are the  two ways that you can integrate jenkins with the   lambda test platform you can utilize the lambda  test jenkins plug-in within a freestyle jenkins   job but you can also run the lambda test tunnel  within a jenkins pipeline job in the event that   your release process uses a jenkins pipeline i  hope you enjoyed this video and the other videos   in this series and thanks for watching if you'd  like to learn more be sure to follow our blog   at lambdatest.com forward slash blog as well as  our lambda test community at community.lambda you can also earn resume worthy  lambda test selenium certifications   at lambdatest.com forward slash certifications you

Channel: LambdaTest

Views: 6,790

Keywords: Cross Browser Testing, Jenkins, tutorial, testing, what is jenkins, devops, How Jenkins works in real time, jenkins devops, pipeline, full course, integration, selenium, maven project, github, integrate jenkins with github, jenkins tutorial for beginners, What Jenkins is used for, Jenkins Job, Maven Project, windows, jenkins controller, jenkins agent node, basics of jenkins, fundamentals of jenkins, jenkins best practices, jenkins installation, jenkins configuration, azure, automation

Id: nCKxl7Q_20I

Length: 172min 44sec (10364 seconds)

Published: Thu Sep 23 2021