Pyautogui basics tutorial part 1: how to automate mouse operation using coordinates

hello by the programmers so this is the part one of our automation of mouse operation using pi auto gui video series actually this is a two part video series in this first part we'll see that how we can automate the mouse operations that are click drag point and all the mouse operations that you can normally perform by your hand we'll try to automate that and it will be done in two ways in the first part or in this video we will see that how we can automate the mouse operations using the coordinates using the x and y or the height and width coordinate of your screen and in the second video we will see that how we can automate the mouse operations by providing the image okay so let's start off with this part so these are the commands that we're gonna explore actually one more slide is there okay so these are the commands that we're gonna explore i'm gonna show you that how they are applied practically on our uh mouse so uh let's start off so let me first show you that how you can install pi auto gui so what you'll do is uh you'll open the command prompt and simply write pip install uh pi auto gui now if you are able to hear the click sound of my keyboard this is because i bought a new keyboard which is a semi-mechanical keyboard if you want the link of that keyboard the link will be provided below and full exposure it is an affiliate link okay so let's install our pi or to gi package so just write pip install pi out of ui and click enter and this package will be installed for you for me this is already installed so it is saying requirement already satisfied so let's close this and let's open a python file and let's write a command so first of all we're gonna import pi auto gui uh it should be pi auto gui and then let's start off with the first command so if you want to control the mouse operations by the coordinates then you should know that what is the size of your screen now the laptops or the desktops come in various screen sizes so this command can be helpful to uh let you know that what is your screen size now this is not the most important command but yes you can say that it is pretty important okay so there are some spaces so let me remove them so what we'll do is let's print it so we'll print this size okay let's run the command okay so pi auto gui the spelling is wrong the spelling in my slide was wrong this is why it's showing an error okay so this is the width one nine two zero is the width and the height is one zero eight zero this is the width of the screen of the laptop that i'm using okay so now let's move on to the next command and not youtube okay so let's move on to the next command which is to get the position so let's say that you have placed your mouse pointer at a specific location so this command will help you to get that location okay so let me copy this and open and what i'll do is i'll replace this with the command and and then replace all these spaces okay uh so let's run this and here you can see that this uh at that moment my mouse was pointed over my mouse was present over here now let me move my my mouse over here and then uh run the program again so here you can see that the position are changed let me uh place this at the very starting of the screen and i hope i will get zero zero as uh as the output yes zero one which was pretty close so yeah this is working fine so you can use this command to get the position of your mouse pointer now you must be wondering that these are simple commands i mean these things are not that amazing that you are showing to us so yeah i know these are very simple thing getting the screen size getting the position but if you are working on an automation project project let's say that you are working on a facebook auto like a project or let's say instagram auto like a project or anything like that in which you want to automate the mouse operation or even a gui automation project then you can use these use these commands and then just automate your uh project it will be a fun little project so let's move on to the next command and things will now spice up a little bit so uh this will show you the uh position of your mouse pointer live it means that let me open the command prompt so that we can get the full uh flavor of this uh of this command so uh first of all we'll import pi auto gui okay and now let's provide a command so what this will do is it will display the live positions so let me run this uh okay so hi auto it should be pi auto gui okay so here you can see that this is showing the live x and y coordinates and even the rgb rgb is the red green blue which is the color of the position at which my mouse pointer is here you can see that 12 12 is the shade of this black and if we move to white here you can see it is 255 255 255 okay so this is not jet black this is something 12 12 12 but this is jet white or we can say that it is pure white so you can see that this is for red this is not exactly that this is why it is not 255 and others are non-zero okay so you got the idea right so now let's move on to the next command so this can be very helpful very helpful to get the live positions of your cursor or your mouse okay so ctl plus c to exit it's written here okay so now let's move on to the next command so let's say that you want to drag your mouse pointer irrespective of you of the current position let's say that you want to drag the mouse pointer or you want to move the mouse mouse pointer at a specific location so you can use this command and what you can do is i'll write this in the program only so uh yeah so let's let's do this so let me write this here and let's say that i want to move this now this uh now this command accepts three parameters first is the x coordinate second is the y coordinate and third is the z coordinate so x coordinate let's say 0 and y coordinate let's say 0 so we want to move to the very top of the left hand position which is the top of the screen and let's say it will take time of five seconds so that it's visible to us so this third parameter is the time taken by prior to gy to move your mouse cursor to a specific location okay so let's run this program let's save this first and now let me run this and here you can see that i am not touching the mouse at all it's moving by its own and here you can see the program is executed now okay so now let's move on to the next command which is how we can click triple click and double click so this command will help you to click which is a very basic mouse operation so let me copy this and paste it here i am commenting this out so that you can see all the commands so uh now click accepts three parameters i guess three parameters it accepts no actually it can accept five parameters uh the the first one is the x coordinate second one is the y coordinate third one is the click how many times you want to click if you select this triple and double click you don't need to specify this uh parameter then the fourth parameter is interval how much time will the click take and the last parameter is the button right button or the left button so let's do this so first coordinate is y let's say that we want to move to zero okay zero zero and the third parameter was uh click so let's say that we want to click uh three time okay so fourth parameter is interval so let's say that we want to take time of four second okay and the last parameter is button so let's say that we want this to be a right button make sure that you're writing this inside a string and it should be capital uh small sorry so let's save this and run our program uh okay so we got an error which was pretty much expected when i'm writing a code uh okay so click okay so it should be clicks i guess okay windows key and again i'm getting an error okay so once again this there have to be a smelling mistake okay so we haven't written x is equal to and y is equal to uh because in this in this click command we have all these five independent parameters like you don't need to write all these parameter all the time you can select that you need only the buttons command you just want to click the right button you you don't need the interval or you don't need the x and y coordinate it will click at the present position so you can use this five command independently you don't need to write all the commands so this is why we are specifying that this is x and this is y and this is why we are not we were not specifying x and y here because these three these two actually were uh pretty much required so let's save this let's run okay so once again okay so yeah i have to stop the video because the error that we were getting before this fail safe error was happening because i was moving the mouse mouse cursor or the mouse pointer to the very top and this is the condition specified or this is the uh parameter defined by pyoto gui which says that if we want to cancel the automation let's say that you are performing an operation and you want to abort that operation or you want to exit that automation what you can do is move to the very top or the zero zero uh coordinate and that program will be stopped this is called fail safe condition in piotr gy and this can be disabled also like let me show you it is working for uh if we specify this okay so uh there is an error now because because because because because let's uh let's change these values let's change these values to x and y okay so here you can see that our program is working because we have changed the uh coordinates of our program to uh 110 okay so now let's make this 0 0 and see that what error we are getting so let's save this and let's run our program so here we are getting the fail safe error which means that uh the prior to gui fail-safe condition has been triggered and it has stopped the program so these kind of uh this security measure has been provided by prio to gy so in case your program is not working as you want it to be you can stop the program or stop the automation in between okay so and if you want to disable that uh fail safe security condition then what you can simply do is i have written that in my ppt i was saving this for the last but the condition came here so uh what you can do is you can simply write fail safe is equal to false okay so prior to gi dot fail safe is equal to false so what will happen is even if we move to the very top our program won't be terminated here you can see that it is clicking right button three times in the interval of four seconds okay after four seconds you will see that program is finished okay so the program is finished okay so this is how the click command of pyoto gi works these are the five parameters and we have also seen how fail safe works so let's move on to the next command which is which is uh drag so let's say that you want to drag your mouse pointer to a specific location you are selecting a specific lines or you simply want to drag it for any purpose so python gui has a command for that also so uh let me show you that how we can do this so uh it accepts three parameters just like the move to first is the x coordinate we don't need to write that it is x or y so let's say that we want to move to 100 uh 100 and 100 and the duration that this uh automation will take is let's say four second okay so let's save this and let's turn our program and here you can see from the position at which the cursor was present it is moving to hundred hundred in four seconds and i was not touching the mouse at all it was all the automation task okay so this was how the drag two command works so let's move on to the next command which is drag relative and move relative so here uh in our previous commands we were specifying that move to this coordinate move to a specific coordinate now what we can do is we can provide a relative position let's say that i am present at this position where my mouse pointer is and i want to move relative 200 coordinates or 100 units so over here so i don't need this coordinate or the position or the x and y coordinate of this position i just need to tell okay move 100 units of x and 100 unit of y from the respective position or the current position okay so let me show you the example so let's say uh we will use the drag one so let me copy this and we will comment this out okay okay so it should be c l plus v okay so it accepts all the parameters that we have seen before the x coordinate let's say 100 the y coordinate let's say 100 and the duration uh which is let's say four seconds okay so i guess the duration spending is right okay so let's save this and let's run our program and here you can see that our mouse pointer is moving very slowly to 100 so let let me change the values so that we can actually see the movement so okay and there should be a comma okay so let's see let me save and run our program and here you can see that this is moving and as we can see that the lines are selected we can see that it is dragging also okay so this is how you can use this drag deletive uh piotr gui command so and the last command of this priority gui using the coordinate uh video is the scroll command now as you know that your mouse pointer the physical mouse pointer has three buttons the right one the left one and the middle scroll button if you're using a laptop you will not have that scroll button obviously but if you are using a mouse then you will be having three buttons the left the right and the scroll button so we have seen the automation of the left and right buttons but we haven't seen anything for our scroll button so this is the command that will be used to uh do the scrolling task okay so let me paste this here okay so this will have three parameters first is the clicks the number of units that you want to scroll this uh mouse up to so let's say five okay so let's say five now actually it's unit so let's say 500 so it is visible to us it's not five scrolls it 500 units okay so we need to specify big amount let's say that we want to uh position our mouse pointer at 100 and actually we don't know we do not need these values let's see with the uh units only so let me open a web page so that we can see this and by the way if you want to explore the pi outage ui mouse operations you can go to this uh official page of piotr gui this is the documentation of piotr gui you can just explore this package over here and okay so we'll try this command on this page only so let me minimize this so that we have we are at this uh page so let me run the command okay so we are having an error we are having an error so okay let me see what that error is uh no attribute as scroll okay so there is definitely a spelling mistake over here uh yeah this should be small it should be small uh yeah okay so it is working let me save this and let me move to uh the web page so that we can see okay okay so let's move up and let's see that how this thing works let me run the program okay so we were not able to see anything okay so uh so let's do this once again let me minimize this actually this is working we can see that the program was finished but we were not able to see the output because we are not present at a very scrollable uh page or something let's uh let's make this file scrollable okay so we are here and let me save this and let me execute the command okay so we can see that we were scrolling okay so this is what this command can be used this command can be used to scroll the uh page let's say that you're working on a instagram or a facebook automation project and you want to like the pics so you can use this scroll automation command and then scroll down the page you can like all the images i guess i have used this command on my twitter auto liker project yeah yeah i guess i have used so yeah i'll provide the link below you can watch that project also and you can see that how i've used this prior to gi package to actually create an automation project i have used selenium and pio2gi for that project okay so yep this is it this was a short video in which we have explored the uh automation of mouse operations using piotr j this was the part first in which we have seen that how we can use the coordinates for this task in the next video we will see that how we can use the images and then just say to prior to gui okay so just look at that image and drag that or click that or move that whatever it is just do it with that image okay so yep meet you in my next lecture bye

Channel: I know python

Views: 7,122

Rating: 4.9756098 out of 5

Keywords: python automation, python, python tutorial, pyautogui tutorial, python automation tutorial, python automation testing, python programming, python basics

Id: ThrOhLJv8lw

Length: 21min 40sec (1300 seconds)

Published: Sat Oct 31 2020