Run Selenium grid tests on cloud without any 3rd party SaaS - In build pipeline

hi guys my name is karthik and i am from azureautomation.com and today we'll be talking about how we can run distributed selenium grid test on cloud without using any third-party paid sas services this is one of the most important and interesting thing that we could able to do within our organization to save a lot of dollars so that they don't really have to hunt around with any other third party sas services and use the existing pipeline that we already have within our infrastructure so let's see how we could able to achieve this in this particular video so usually we start writing our code within our laptop and then we try connecting with our selenium grid which is running either within our machine or within our infrastructure in place something like this so we connect our selenium grid within our client code it can be a java code or c sharp code or whatever it is so we try running our code by connecting the sendium grid and run in different browsers or different nodes something like this while there are some companies don't try running the test just with the local synonym grid they also try hosting the selenium grid on either an aws or azure machine so that they could able to do the distributor test much much easily and some companies even go a bit further to even run the selenium grid directly by using a third-party services like sas or software asset services to perform these operations like cross-browser testing and running the test on different browsers much much easier some of the popular sas services are lambda test and sauce labs that's what many companies use to perform the selenium grid test to run the test in multiple different browsers and multiple different nodes that's how they do it and these tests that you are seeing over here they might not be running in your local machine rather they may be running in a jenkins pipeline or github actions pipeline or azure devops pipeline to perform this operation but now the challenge here is how do we actually run these tests instead of running this in a sas environment but run them on the cloud without having a sas infrastructure in place like lambda test or browser stack how do we actually achieve that well this solution can really really helpful if you already have an existing cloud-based solution in place for example you have a better jenkins node and that has a good computing power or you're already hosting and running your test inside a github and you're using a github action the solution which i'm going to be proposing over here is going to be running the test on the github action pipeline where the github action uses an ubuntu machine to run our code and perform a build that's exactly the same machine that we'll be using to set up this infrastructure that you are seeing over here like serium grid and its various different nodes so let's see how we could able to achieve this in this particular video and in order to start writing the code from the scratch i'm actually gonna use the intellij ide over here and i'm gonna use the selenium project template and this is something which i have already discussed in my earlier video where we discussed about how we can write a framework grade code in under five minutes that's some video which i've already had in my youtube channel you can go and watch that but this particular video is gonna focus on how we could run the distributed test using this same project template so i'm gonna use this project template the senior project template in my intellij ide and i'm gonna create a new project over here i'm going to name this as selenium grid demo or something like that is the name of the project i'm going to use java as the language build system is going to be maven and the test framework is going to be testng and the group name i'm just going to give us com.ea i'm going to use the latest jdk which is 17 and i'm going to be using this execution and reporting i'm going to use selenium allure report selenade or whatever and then i'm going to use the web driver i probably don't need the selling need i'm going to remove that and i'm going to use assert j and then i'm going to hit create so this is something very very interesting in this particular plugin because it is automatically going to generate some of these scaffold code which we can use to demonstrate the distributed test execution on the cloud so you can see that currently we have only one particular test class over here which has got the before method after method and there is a test over here something like that and i need this test to be running in the distributed mode so in order to do that i'm going to create a new base class file and this base class file is going to hold some of the most important chords that you are seeing over here which is going to be these chords so i'm just going to cut the speaker code and then i'm going to paste it over here and you'll understand why i'm actually trying to do that because i wanted to create two different test class files which i can run them in parallel in test ng using the test ngs on threading option so i'm going to even duplicate this test into main page test to something like that so that you will see that we have two different test class files and they are going to be basically extending the base class that i have just created so i'm going to extend the base in this particular class file as well and once i do that i also need to ensure that these members are public so that it can be used within my other test class files over there so you'll see that the main test as well as the main test two are right now all happy so i'm just gonna try removing some of the test from this particular main page test and then i'm also gonna remove some of the test from here so that we will have two different test sets which are gonna be running the test for us so you will see that we have just one test in each class file and they are going to be basically referring the base class file over here and then they are going to be running the test for us so let's try to run these tests and see how they actually work at least one of the tests to see how this actually works and while i run this particular test you will notice that it is going to not open any of the browser the reason being the web driver manager has to be set up so that it can be executed but i'm not going to use the web driver manager or the chrome driver which is being specified in the test on the scaffolder code as you can see over here rather we are going to be using the selenium grid setup to make that thing happen and once again the selenium grid setup that i'm going to be using is going to be using the dynamic selenium grid as mentioned in the documentation over here so you can see that we have the selenium grid option you might be very very familiar about it and in the getting started of the selenium over here you will notice that you can run the selenium grid either by using the standalone mode something like this or you can also run in the distributed mode something like this you can create your own event bus session session cues distributor routers and then you can configure the node on this particular selenium hub to do that or you can also use what is called as the dynamic serial grid and that's exactly what we are going to be using in this particular video as well as in the pipeline so if you just go and search dynamic selenium grid i already spoke about that like a year before like understanding all new dynamics in name grid four and that's exactly what we are going to be using even right now because we could able to run this code as a container run them distributedly and make the ephemeral containers to be executed on demand so you will see that there is something called as docker compose v3 over here so this is the one which we are going to be using this time so i'm just going to copy this whole docker compose file from here and i'm going to our project over here and i'm going to create a new file i'm going to call this as docker compose dot yml or the yaml file i'm gonna paste the copied code and i'm not gonna run in different browsers like the edge or firefox something like that so i'm just gonna get this guys removed from here so i'm just going to run on the chrome browser something like this and the most important thing to note about this particular chrome node is that by default every chrome image which runs as a container will run only one session of the chrome browser but if you want to override the session and then you need to run like multiple sessions of the chrome node you could do that as well and the way you could do it is using this se node override and you can set the maximum session to true something like this and similarly you can set the sc node max session to 5 so that you will notice that it can run until five sessions simultaneously like concurrently so that you can run the test so this is something which i need to set because we could able to run five different threads same time in this particular containers while we spawn it so now that our chrome node is ready and we also have a selenium hub we wanted to run the speaker docker compose and see how it actually works and the way we're going to do is is within this ide over here i have a terminal and if i just do an ls you will notice that i have the docker compose.yaml file that i just created i have already installed my docker within my windows the docker for windows and you will notice that i don't really have any containers and things up and running but once i do the docker compose off up you will notice that it is going to go create the selenium hub and also it's going to start running the containers for me so there is two containers up and running one is the selenium hub and the other one is the selenium grid demo chrome and you can see the whole selenium grid up and running in the localhost colon 444 slash ui and notice that we have five sessions running that's why there is a five over here all right so we have this selenium grid up and running we also have the code like a scaffold code up and running but now we need to change the code a bit to point from instead of chrome driver to the remote web driver that is the only thing that we need to do the change right now so the first thing we're going to do to make this code from chrome drivers like a normal code which is going to just look for the chrome driver to the remote web driver is we're going to add the remote web driver option over here so i'm just going to say driver is equal to new of remote web driver and within this remote web driver we need to pass the new url and this url is going to be the url that we just saw the http colon double slash localhost of colon4444 and then we also need to pass the chrome option and that chrome option is going to be something that we need to create over here so the chrome option is something optional at the moment but i'm just gonna set the chrome options just because there are some cases that while you run the chrome driver in the distributed environment you might get some error due to shm usage and that is something i wanted to set by disabling them over here and this is the chrome option that i wanted to set here within my code and once this is done you will see that there is a screw design over here to have an unhandled exception i'm just going to add and try catch block so that we don't get that exception anymore and let's get rid of the chrome driver from here and also we can set the maximize in this place and on the timeout you see that there is a crossover here so we could able to just remove this n second to actually a duration seconds to 30. that's the maximum timeout that we can set within our chrome driver to be completely loaded so that's it this is the chord change that we are going to be doing so this way the code is going to basically run within our remote web driver instead of the local machine and you will notice that once i start executing the code this time the code is going to basically work so i'm going to go to any one of the tests the main page test this one and if i try running the test this time and if i go to our sessions over here you will notice that we have one session over here on the queue which means one of the test is currently trying to run you see that there is one session and it is running for us we can also sometimes see the session by the video but the test has already been completed so you can't really see that that's okay our tests are currently running but now i want to run these both tests together as a distributor test so that we can run them in parallel so how do we do that and the way we are going to do that is by creating another file i'm going to call this as testng.xml file and that is the one thing which is going to help us to run the tests in parallel so i'm going to say testng.xml file and this testing dot xml file is going to be an xml file something like this and i'm going to set the utf 8 over here and i'm not going to be keep writing the whole xml file over here rather i'm just going to copy paste the code that i have already written and the only thing that you need to notice here is that this particular suite is going to run in parallel for all the tests and each test is going to hold the classes as you can see over here the main page test and the main page test 2. so if i run this particular testing.xml file from the ide something like this you will notice that this test is going to run two tests together and if i go to the browser this time you will see that there are two sessions and two tests are running same time in parallel so this is the way that we could able to do it and you see that the two test has got passed that's it we have already achieved the parallel execution of the test within our code the next operation that we need to do is to also ensure that this particular code is running from the command line and the good news is that because we already have a scaffold.code generated by the seleniums plugin you'll notice that it also have the maven surefire plugin automatically configured which is very very good we could able to run the whole code with the maven surefire plugin itself but we also need to ensure that in the maven surefire plugin we point the testng.xml file so that we could able to execute that and the way we're going to do that is by using the sweet xml files tag something like this and then we're going to say sweet xml file and the path of the file and the path of the file is going to be src slash test slash java slash testng.xml file that we just created so this way the code is gonna execute even from the command line so let's see how we could able to execute from the command line so that we are gonna use the exact same command within our pipeline to execute it so i'm going to go to the terminal over here and let's create another terminal something like this because we don't want the grid to be down and over here i'm going to say mvn clean off test so this way you will notice that it is going to perform a clean before the test is going to start and you see that the currently the test is up and running and if i go to the grid and if i see the sessions you will notice that there are two sessions up and running which means the test is currently running as well which is great so our command line code is also working fine the last operation that we're going to do is to push this code into remote cloud so that we could able to execute the code and the cloud that we're talking about over here is gonna be the github action so i'm gonna be using my own personal github execute automation account to run my test that you are seeing over here and i'm going to use the github action to perform those operation for me even in the github's free account you get 2000 minutes of free execution time which means you could just use it without any problem at the same time if you're going to be using the github for organizations you should have a paid account already so you don't have to worry about that if you already have it so i'm just going to assume all these things in place right now and i'm going to create a new repository and i'm going to call this as selenium grid execution and i'm going to choose the git ignore by selecting the language as java and i'm going to create the repository over here so once we have this particular wrapper we can then start pushing our code into this particular repository and we can see how we could able to achieve everything over there but in order to do that i'm actually going to use the time machine to push all my code and once the code is pushed into the speaker wrapper i'll be back and now if you come back after my time freeze you will see that i have uploaded all the code like an initial commit in the github repo over here so we have all the code right now and we also have our repository over there so it is pretty good to go and now the final operation that we need to do is to run this in the github action pipeline using the dockercompose.yaml file but in order to achieve that you need to have some information on the github actions which i'm not going to be talking about in this particular video because we already have a video series in youtube just talking about the github actions you can go over to exit automation channel the github actions playlist and then you can learn it from there but for now i'm gonna go to the github actions over here and you will see that we have an option called as java with maven github action so i'm going to use this particular option over here and then i'm going to try running it so you notice that the steps are automatically generated for us to perform the operation but you can see that it is saying the maven hyphen b package file as palmetto xml file instead of maven b of the package something like that we can use the maven clean test command that we just executed but you'll also need to run the docker compose file that we spawned in our local machine to spin up the selenium grid setup so how do we actually do that because that is the most important thing that we need to achieve and the way we can do that is using a run command within the github actions over here so once we have the maven execution completed like the restoration is completed we are then going to say i need to perform a start selenium docker grid setup something like that and i'm also going to be let's give a space there and i'm also going to be using a run command to perform a docker compose run so the way i'm going to do it is using the docker compose of hyphen ef and then i'm going to give my docker compose file name which is going to be the docker compose.yml or the yaml and i'm going to say up hyphen d in a detached mode and i think we don't require the build but i'm just going to keep the build over here that's it that's going to be spinning up the docker for us over here and there's going to be a bit of problem if we don't do an additional step there but we'll talk about that later on but i'm going to do a commit the new file and once i do that you'll notice that within the github actions over here we will have a action running the create maven.yaml file over here so this is the first place where you will notice that our code are going to be start executed on the cloud which is going to be on the github actions cloud and you will notice that if you go to the build it is going to set up a job where it is going to download all the code for us in the ubuntu operating system of the github repo and you'll also notice that now it is starting to run the selenium docker grid setup so you will see if i just make this font a bit bigger you will see that it is trying to pull the chrome node for us and it is also pulling it from the selenium node chrome there and once it is done it is also going to pull the selenium hub for us and then it is going to run things for us you will notice that right now in the maven job it is trying to run the one that we just did the mvn clean test within our local machine that's exactly what it is doing in the remote machine as well this time and now it should start running the test by pointing our selenium grid setup but while i did that you'll notice that the execution has got failed there so let's see what is the failure so if i just come back all the way down ah you see that currently we have a problem over here it says the invalid target release of 14. i guess the reason why this is happening is because we are using java 17 rather the java 11 within our code execution so if we just go and view the workflow file once again you see that we have set up a java 11 there so i think we need to use the java 17 so i'm probably going to set up the java 17 i'm not sure if java 17 is there in the github actions but i'm just gonna use this java 17 or you can also search java 17 github actions something like this and you can see that within the distribution of the timorin there is the java 17 version as well so we do have the java 17 which means it is quite right so i'm just going to use the java 17 and now i'm gonna do the commit changes so this way it is going to do the build for us in the java 17 instead of java 11. so if i go to the github action once again and let's go to the build and fast forward we directly see what's going to really happen in the maven and you can see that the execution has been completed for us over here so let's go to the build with maven step of our github actions and let's go back all the way down because the log is too huge and it is showing a lot of detail and we also need to see that whether our test execution has been done or not so you will notice that our test execution has been completed so it has really connected with the selenium hub you can see that there was two session being created and there was two run happened and the execution got completed so that's it this is the way that we could able to run our distributed test on the cloud in the github actions over there and we could able to complete our test execution without any problem without using any third-party sas services but there are some cases which is going to be like an edge case where you might have a situation where the synonym grid would not have been ready and the test is fast enough to start using the selenium grid you might encounter that particular problem in order to resolve the problem you need to run an sh file to wait for the selenium grid to be up and running in fact that particular solution is already available within the docker selenium hub page itself so if you just scroll all the way down over there wait for the grid using the bash script this one so if you just copy this particular bash script within your code and then run this particular shell script as one of the command within the step within the github actions you could able to achieve that as well but that is not something very hard for you to do you can just copy that paste it into a repo and call it in a bit of actions but you already got the idea right now we could able to run the distributor test directly in the github actions without any problem and the build is also successfully completed so every time while there is going to be any check-in going to happen within our repo over here you'll also see the github action is going to run and it is going to run the test as a disabled operation but you could do even more operations within this particular github actions and run this in docker container like generating the videos creating the artifacts attaching the reports and also make sure that your code always runs without any problem the github using the docker containers like dynamic synonym grid you could do that and all these details are available within my course understanding docker and using it for selenium automation there are a lot of enrollments already happening a lot of rating there you could able to enroll this particular course and it will help you understand how you could do and create your own ephemeral containers and run it on the same network and also use the github actions like the basic subject of actions to perform the same kind of operation that we just spoke but this is it this is how you could able to achieve that in the cloud without using any sas platform to run the selenium grid test in a distributed manner thank you

Channel: Execute Automation

Views: 7,284

Keywords: Selenium, java, intelliJ, Ide, 5 mins, Framework, page object model, pom, testing, ExecuteAutomation, video series, under 5 mins, getting started, introduction, github actions, pipeline, distributed test, selenium grid, grid, docker container, docker compose

Id: n0ipIP_gNHM

Length: 24min 37sec (1477 seconds)

Published: Mon Aug 01 2022