Automate Your Bot Script Creation: PyAutoGUI Video Game Bot Tutorial 3/4

let's supercharge our bot I'm gonna teach you how to record our in-game actions and run them again later I'm Ben and welcome back to learn code by gaming in the previous video we learned the very basics of Pi auto GUI and what our first spotting script if you haven't seen that video there's a link in the description to the full series and phase 2 here I'm gonna show you how we can automate the creation of your time about actions so we can start to build a bot that's capable of many different activities and it won't take us an eternity to do it to do this we're gonna install another Python module high input I tested a couple different methods for capturing our input and I find I input to be the easiest PI input can also be used to generate mouse and keyboard inputs just like IATA GUI but we won't be using it for that here we're just going to use it to capture input events and we'll use PI auto GUI to play them back lighting and with that let's hop back over to the code okay so at this point in the tutorial you should find yourself dr. earth station and what we're going to do today is we're going to automate the process of selling our goods here at our station buying the next set of trading products and then we're going to fly that back to low key station just like we did before but instead of manually typing out all the commands for that we're going to create a script that will record our inputs in the game and then we'll make another script that will simply play back those inputs and so this is going to greatly speed up our ability to create these scripts so we can eventually create a whole set of actions that our character come before him so your character should begin here dr. earth station and if you look around you'll notice that we don't have a NPC here that we can easily talk to you like we did back at loci station but we still need a consistent starting location and the best spot I found was here in the main lobby you turned right by this Xavier Domingo guy and kind of wet yourself behind him against the wall and then just like before chat with the character so that we get our character oriented correctly and then this will be our new starting location here at her station all right let's get started writing the code let's begin by just creating a new folder for this code recording but we'll call it I don't want that to be inside the timing bat I want that to be outside there we go and the first script we need to write will be a descriptive that records our actions in the game so I'm going to call this recorder and then using our main script from before I'm just going to steal this syntax for setting up our main function alright and then the first thing we need is we need to install PI input so that's just going to be another simple pip install it's py and beauty so no I they're on input and it looks like it installed successfully yeah I see PI input there so now in our script we can let's do a from PI input import I think it's a mouse and keyboard now what we want to do is we want to set up just a basic listener to figure out you know if we can detect when the keyboard is receiving inputs so I'll make a run listeners function here to contain the setup for listening to all these inputs what we want to do is we want to collect the keyboard inputs until the script is stopped till release let's say and then use the link to the documentation of how to do this so let's go check that out okay so I'm in the PI input documentation monitoring the keyboard and we're going to grab that section of code so this will set up a simple keyboard listener and the parameters in here it expects two functions a function on what we should do when a key is pressed and another function on what we want to do when a key is released let's go ahead and write those functions on press release and then let's go back to that documentation just to get these simple on press and on release function definitions so the main part here is when we press a key we're just going to print out what key was pressed if it was a letter from the alphabet it's gonna go through this first try statements alphanumeric key so we can grab that character of it but if the special key we're just gonna print the key itself and then when it's released we'll also print out the fact that it was released and then also in the on release it's going to detect if the key that was pressed was escaped that'll stop the listener which will go ahead and stop this infinite loop that's listening for inputs so let's go ahead and run this and see if it's working okay so I had a little issue with my indentation there initially make sure you get the indentation right anyway statement let's try it again okay you should see it down here in the terminal just running and when we press the key on the keyboard it should tell us what was key was pressed and then when it was released so if I like hold II what it'll do triggers lots of presses and just released once and then of course we're gonna hit escape to I stop our program okay I'm also going to format these for my strings the motor that I prefer and the next thing we need to do is need to capture the mouse input somehow so to do this I'm going to collect mouse input events will need to manually start this listener and then when we receive our escaped statement through the keyboard will need to manually stop it just make a mouse listener equals no stop listener then click is the name of the parameter here will create an on click function handler for it and then to start it here's just Mouse loose node R dot start and then we also want to wait for that start process to finish before we bind the keyboard wait for the listener to become ready okay so I think do a great job the first time explaining how the keyboard and mouse listeners work this is probably the most complicated part of the code so let me try to explain it again so in our run listeners where we have with keyboard listener as listener this is essentially initializing this keyboard listener and it's starting it and then we going to join it to the main thread so listener is a subclass of thread and thread is from the bass Python threading library and what joined us is it joins it to the main thread so the main thread is going to wait for this this listener thread to finish before it continues so threading is usually used to run multiple in parallel but in our case we just want to wait for our collection of the keyboard inputs to finish before we continue on at the main thread so that's what this code is doing now to add our mouse listener you might be tempted to you know look at the documentation here we could also add this with Mouse listen air blow it but the problem with this code is it's a it's not gonna run initialize this listener and then join it to the main thread until we release the main thread from my keyboard listener so it would just capture our keyboard inputs first and then it would listen to our mouse inputs and that's not what we want we also don't care for the movement of the mouse or the scrolling of the mouse we just want the on click is really the only thing we're interested in so instead what we have to do is we have to start the Mouse listener manually and that example is also from the documentation that says non-blocking fashion we can create the mouse listener like this and then we can manually start it now we can manually wait for it to become ready and what this will do is it'll start our mouse listener in a separate thread from our keyboard listener I'll be running at the same time and then when our keyboard listener stops we'll need to manually stop our mouse listener as well so that's what we're doing in our on release when we detects with the Escape key has been pressed we're gonna go ahead and stop our mouse listener before we stop our keyboard listener and if you want a better understanding of how threads work might be interested in this documentation page for threading the threading library in Python and you can see this start that we're using we're also using this joint function and you can be more about them there okay and the next thing we need to do is create our on click handler on click let's look at the documentation for this again let's go to handling the mouse monitoring the mouse let's grab their odd click okay so you can see they're giving us the XY coordinates they're also giving us what what button was clicked and this pressed and this pressed is a true/false boolean flag and when pressed is true that's the click down event and when it's false that's the released click event so we're actually only going to want to listen to the release event so if not pressed let's go ahead and print this and if we don't want to stop the listener in that case because we're gonna use the Escape key still to stop it so let's remove that part of the code and then we don't need to check if it's released or not because we know it's always going to be released all right and so the last thing we need to do is when they click the escape key we want to stop the mouse listener as well as the keyboard listener so we want to stop Mouse listener this will be stopped keyboard listener so the mouse listener dot leave its listener nope just mouse listener that stop will stop it and then this will still work to return false to stop the keyboard listener we could also raise an exception though and that's just a little more clear to me what exactly is going on so that'll be keyboard dot listener dot stop exception but of course Mouse listener isn't defined here and so if we want to get the mouse listener out of the run list News function and be able to use it on release we need to make it a global variable so let's just go ahead and do that I'll keep it simple start with Mouse listen there is none make a note to declare Mouse listener globally so that keyboard on release can stop it and then when we go ahead and wherever we use that mouse listener we need to make sure we use the global keyword so that we know we're referencing the global mouse listen they're variable now let's test this and see if we're capturing the mouse inputs as well as the keyboard imports so a still works and then clicking is working to cool I went ahead and updated our on click print statement just a little bit so we can see what button we're clicking on and then we're on the screen it's being clicked ok so now we're able to detect whenever a keyboard key is pressed or a mouse button is clicked the next thing we need to do is figure out what time those inputs are cared at so start with importing time that was from time import time and then we're going to be referencing the start time of our script all over the place let's just make that another global variable start time declare our start time globally so that callback functions can reference it okay so first we need to set our start time and let's start it after our lish listeners have been initiated so down in one listeners I guess before we start the thread let's set the start time and then on our input listeners instead of displaying the full timestamp let's display it relative to when the script got started so why don't we make a another function for this put it here under main go call it elapsed and that'll just give us the current time - the start time so global start time return the current time - the start time so now in our event handlers we can figure out exactly how long after the start of our script that input occurred let's go ahead and put that in the print statements okay so now when we run it again it should output not only what buttons we clicked but when we click them one thing to keep in mind is when you're testing your recorder whatever you have selected when you type in it's going to receive that input so be careful if you have like your code selected and you push some letters are your codes gonna get those letters too so be careful there not to mess up your code doing that and another thing that might happen is if you have your terminal selected when you're making your test inputs with the keyboard your terminal all is actually going to collect those inputs but they might not actually appear down here in your terminal until you click another letter and when you do you'll see something strange like this and when this happens if you try to run your terminal again you could see this air you know strange characters command not found and that just means that your terminal window capture those inputs and so you should be careful to you know create a new terminal window every time you run your program or after you've captured some inputs and ended it go ahead and just hit like backspace few times to clear that out before you try running your recorder again all right now we're getting I clicked there five seconds after the service script press H that was about nine seconds after the start of our script and so on perfect so now I think would be a good time to go back and fix that issue where if you hold down a key it records multiple pressed events so let's see that again real quick yeah so remember if you press like hold down e you get lots of rest events in just a single released event so to correct for that let's go ahead and make one more global variable will keep the track of the unreleased keys just in our list and so whenever we press down on a key we'll add it to this list and then once we release it will pop it off the list so down in on press let's make a note we only want to record the first key press event until that key has been released so if that key is in our unreleased keys list again that's global then we're just going to return gonna exit this function else let's go ahead and add that I'm really ski using the append method to the list and then of course in on release we need to remove those keys from the list once they're released mark key and it's no longer pressed global unreleased keys let's do a try-catch here all these keys that remove the key if we get a value err then we'll go ahead and print out an error statement all right let's give it a try see if that worked so I'm holding down D and perfect we have the pressed about four and a half seconds in and then if you lease at almost six seconds that's what we want okay so we're almost done writing our recording script the next thing we want to do is remember when each one of these events occur and we're going to save that to JSON and then just output that to a file so let's create a simple list for storing all input events and then let's write a function for recording those events record event so we're gonna give it an event type you know it's a keydown keyup or click we're also gonna record the event time for four key down we want to record what key was pressed and then for the clicks we want to record what button was clicked so let's just call that button in general for both of those and then four key clicks we're also going to have an XY coordinate as we'll call that POS for position but of course keyboard imports won't have positions let's default that to none for them so we're going to add this to the input events list so need to once again tell the compiler that that's a global variable and put events that end to add this new event and formatting let's make this a dictionary so time will be the event time type with the event type the button the button and the position to be the position and then when we record one of these events let's go ahead and also print out what happened and then after we add the print statement here we'll remove it from our event handlers so here's the print statements that came up with so if the event type is a click then we're going to go ahead and print out everything including the position but if it's a keyboard input we don't have a position so it's a little simpler of a print statement now we can go down to our on press function and instead of printing out what happened we'd go ahead and record the event so let's a record event for the event type so this is on press I could just put like key down here so let's go ahead and create look it I mean um for this call it rent type key down the event time we'll just use the elapsed time to get that and then the key character get rid of our print statement of course if it's not an alphanumeric key we don't have that key character so we'll keep this exception statement instead of passing over the character it will just pass over the key and then let's go ahead and set up this enum now I'm just gonna make a class for it hook you down will just be the string key down do a key up I'm gonna click looks good to me so on release it's going to have very similar code to unpress of course the event type is on key up instead of key down and then on clicked record event this event type will be click still want the elapsed time I will just give the whole button and then for the position get it the XY go ahead and delete that print statement and then let's go ahead and run this now make sure we're still getting all of our credit statements that we expect okay I've got the keys down key up and the clicks perfect so the next thing we want to do is when I convert that to JSON so we'll hit boy JSON and then inside of our main let's first print out how long our total script ran for recording duration give it the elapsed time at this point and this will be in seconds and I'll put all of the input events that we that we recorded let's put them in JSON format so JSON dumps that's going to take our list of input events confer to JSON a JSON string and so that JSON string is what we're going to print out let's run that okay and we get this error type in error type of button is not JSON serializable so sometimes we're passing in a button here then about an object that comes from hi input and not a actual string so we could just convert that button to a string that should solve that problem click a few things all right here we go so we recorded for almost three seconds and then here is the output of everything that we recorded so you can see we got that click we got that pressing a pressing E and then the Escape key so the final final thing we need to do with our recorder is output our JSON to a file that way we can read that file later when we want to run back our actions so let's let's define a file named output file name call action is test one for now and of course you could use like a command light input to get this file name I'm just going to hard code it in for now that seems like a simple way to do it and then we're going to create this file inside of main let's say after everything's done running so first thing we need is a path to the file that we want to output let's get the directory first and then we can add the file name to it you know a step path doctor name of the current file I'm going to create a folder inside of this recording bot to hold all these output files we also need to import OS and then the full file path will be where's path join let's join the current directory with the folder that we want to make let's call it recordings and then finally let's put the name of our file smell the output file name and then we want to add JSON an extension to the end of it it's just a string format to do that it looks alright before I've again let's go ahead and create that Recordings directory and now we can open the file and write to it so in Python they would like you to use this with the syntax so with open your open that file path to open it for writing now we can use JSON dumps to write write to that file to take the input events and we'll put it to that out file we can also format this a little nicer which I think we should do because once we record these files you might want to actually go in and edit them some times just to make adjustments so we can do in debt equals four and instead of outputting the JSON all and single line to this file it'll now do a carriage return at the end of each line and do indentation for us it'll make it look a little nicer so finally let's run that and see if we get a file I put our file is not defined I think that should be lowercase huh let's try it again Tom's only takes one positional argument so it must be the output file okay so I looked it up and when you're using an output file you don't use s SS for string it's just JSON dump and then you can put it on to a file let's try one more time okay no errors this time let's check our folder and we do have a JSON file in there and it looks like it has on button clicks keyboard inputs and all the times that they occurred at so that's perfect this is all we need now that we have these inputs recorded we can then run these later okay so for the playback let's go ahead and make a new script for that call it playback that py and then I'm gonna go to our old time bot I'm gonna use this as the starting point I'm just going to copy it all paste it in get to lead all these action functions we're not gonna need to report the mouse position delete those calls we won't need any delay between the commands because we're just going to do it exactly how we recorded it station will keep we'll keep the countdown timer and you know I don't think we'll even need hold key because we're just going to fire the peyote GUI key down and the key up as we're doing our playback ok and for playback the first thing we're going to need to do is open our file and read it so we're going to need to import OS to find the file and then import JSON to parse it let's create a new function for this call it play actions and I think I'll pass it in the name of the file that we want to run so this is actions test a1 dot JSON then let's go ahead and write that function what did I call it play actions I gave it the file name so let's go ahead and open the file just like we did in our recording script so I'll go and steal that code because we're still gonna want the same directory we still want instead of this output filename we actually just want to use the file name were given and then we're not opening for reading or writing anymore we're opening for readings as our and we know this is going to be a JSON file and we don't want to dump it if you really want to load it into a variable so we'll just call that available data json dot load for loading a file and then let's just print out that data for now and see if we are opening it correctly so I'll run that we've got our countdown timer so it's going to wait 10 seconds and then there we go it looks like it hoping that file just fine and it I'll put everything for us so inside our play action so we don't want to just open the file and print it out we want to actually do all the actions that are in the file so add some comments here it's parse the JSON first then what we want to do is we want to loop over each action so that would be for action in data but thinking ahead here a little bit I'm actually gonna I want to get to actions at a time the current action and the next one so that I can determine how long I should sleep in between each scene so let's go ahead and do an enumeration so let's get the index as well as the action and enumerate on that data so what are we looping over the actions I guess the first thing we want to do is look for that escape input cuz that means we want to exit so let's look for escape and put exit so if the action button equal to kita escape then we just want to break out of the loop but if it's then an escape key we just want to be perform that action so let's perform two action and how we do that it's going to be different depending on what type of action it was so if the action type let's check the type let's see if they're going to be the key down or key up or click so check for a key down that'll be one thing if the type is KeyUp I'll do something different with my or degree or if it's like click so to recreate a key down action with peyote GUI that'll be just like we did before peyote GUI has a key down function and we just wanted to know what key to press down and we have that in the debugging data same thing for the key up rtqe key up give it the action in the button and then for the click pilot GUI wants I want the X the Y and the iteration so the X we can get from action position and let's say to item topple there so the XS in the zero position and the Y will be in the one position one index of that Tuffle and then for a duration we can just set it to something pretty quick quarter second again sounds fine now if we were to run this code right now it would simply zip through each one of our inputs and that pause at all in between which is definitely not what we want to do what we want to do is we want to execute these actions at the same time that they were initially performed relative to when we started recording our script so we need to sleep for some amount between each one of these inputs and we can get that by looking at the next input that's coming up and seeing how long of time there is in between the current action and the next one so this part we're going to be sleeping and sleep until next action should occur first we need the next action and that's just going to be the data the current index would be the current action but we want the one after that but of course if this is the last action then this index plus ones going to throw an air so let's just put a try except around that should be an index error right if that was the last action then we don't need to sleep at all so we can just once again break out of this loop and so that now that we have the next action we can calculate how long we should sleep for it's called that elapsed time that's just going to be the next action time minus the current action time right because if we clicked a at five seconds but we release a at 10 seconds then we want to wait for five seconds so that would be ten seconds here on the next action five seconds here on the current action 10 minus 5 is 5 and that should be the correct amount of time to wait so we'll just sleep for that elapsed time and just in case there's a chance that our recording will record these actions out of order I'm so I just want to check make sure that elapsed time is greater than 0 if it's less than 0 it's going to throw an exception let's just check it just in case so if elapsed time is give it any equal to zero just go ahead and sleep else let's print out an air I changed my mind let's actually raise an exception and we'll say unexpected action ordering ok and that should be all it takes to play back our actions but of course we're gonna want to be careful about this let's not just play back the actions that we just randomly made earlier so let's go back to the game I'm still here yeah I'm still here still in a starting location let's record some actions in the game so I'll go back to my quarter run it click on the game window click inside the game I just look around turn left turn right then we can escape in the game that'll end our recording just a cancel on the dialog so we have a new recording and then let's go and run it so it's gonna give us our 10-second countdown but I'm not gonna focus on the game because I believe the first thing I did was click on that game when don't let's see if it clicks on it to get focus on it yeah it does alright and there my character is turning left and turning right just playing back my actions that I took earlier so that's perfect another thing to note here is we do have access to this JSON file so that initial couple of clicks makes me nervous which you probably should you can go in here and just easily remove these delete those commands you can even if you want to go down and delete the escape commands that way it'll just go through each one of these actions and when it's done this key up on D it'll just stop stop doing actions and end of script so let's see if that still works too so let's play it it's giving me that 10 seconds this time I do definitely want to select the window we'll see if I care good turns left and right there it goes perfect so I think now even though we saw it behave incorrectly when it played back the actions let's go ahead and add some print statements just to make sure that it's absolutely doing what we expect so when we perform the actions you let's also print out how long we're sleeping for so we obviously need to do that before we actually do to sleep sleeping for elapsed time and let's just run that same recording again see if we get those print statements working okay and there we go in the key down and a that we waited 1.3 seconds before pulled up on a then in between a and D we just had three tenths of a second wait time and then once we did a key down in D second and a half and then the key up once we did that we were done that's exactly what we wanted to see now the next thing we need to consider is the special keys keys like shift or caps lock these aren't going to be exactly the same between peyote GUI and PI input so when we call say keydown and keyup and we give it that if it look like key down escape priority GUI it's not going to know what that is so let's go ahead and write a function to do that conversion for us so what we want this to look like is we want to get a key is equal to convert key I'm going to give it the action and then from this pi input action we want to return a PI otic GUI value so once we have that key can do a key down on that key we also want to report that same key do the same thing for a key up and we shouldn't need to do anything for clique because here we're just using the X&Y position and that should convert finals or just into juice and then to create I prefer key function it's going to be given the button from PI input we really need to look at the documentation for both PI input and peyote GUI and compare them and make any translations that we need to so they work together so here are the the links to those parts of the documentation let's go take a look so I can show you exactly what I'm talking about so here in high input these are all the different keys that it expects for its button button values and for a pie out of GUI down at the bottom here here's the values that it expects when you give it to AI keydown and keyup so we just need to do a simple conversion here so let's make a clean key and in most cases all we need to do is we move the the key that command or something or key dot F 1 and just make it into F 1 and that will line up just fine so I'll line up with everyone over here so something that looks like key that f 9 should return f 9 but if the button is just W that should return as W those will be strings.already so what you just take that button string replace any time we see this key dot I just deleted it so that only work in most cases if we just return that clean key and then there's just a few instances where we do need to change the complete name of the key so here I've gone through a documentation I found all the instances where whatever PI input is using to refer to a certain key is different than what a PI OTO GUI uses and so on the left here is the PI input value like caps underscore lock and the PI auto GUI value for that is just caps lock no underscore so now that we have a map there I can just check that map so if I think key not the the key portion of this string has been removed that is in our map called that PI input special case map then we want to return the special case map value for that cleaned key but if it's not in this map then which is going to return whatever was created by deleting the key dot portion and that should work just fine and this is important for our bot because we are going to use the shift key when we're buying and selling our goods and so we need to do that conversion correctly so we're just going to run it quickly make sure everything still looks good my recording of course doesn't have any shift inputs on it it's almost like just checking to see if I introduced any syntax errors or something like that here now it looks like that finished fine so I think our recorder and our playback scripts are in a pretty good spot right now let's try recording a large longer action in the game and see how well it does so look at the recorders script ready let's just go up here to the platform so if we can go to the center of the room here and record that all right so I'm going to run the recorder I'm just going to run forward into the room turn around in a circle go this way pretty good of course escape to end my recording there I'm gonna edit these actions quite a bit I think get rid of those escapes get rid of these last few button clicks I don't want any of those get rid of those initial button clicks all right let's go back to where we started here and then run our playback okay not too bad let's do a recording I guess from our starting spot so we're in this little corner here all ready to go we're not recording I guess I'll back up first I turn to the middle of the room go there TURN RIGHT yeah I think that's a good initial recording okay let's play it back and see if I get similar results oh of course I need to get to my starting position well I'm not gonna do that in time I'll use my handy failsafe there upper left-hand corner stop my scripts all right back to the same location let's run it okay that was pretty good but you may have noticed that it seems like each one of our actions lasted a little bit longer than what we actually recorded it wasn't a perfect match so I think this is happening because if you look at our code not only well let's look at our initial code before go to hold hold key let's take a look at hold key so before we were simply pressing down on the key sleeping for a certain number seconds and then releasing up on it but our playback function is a little more involved so after we open the file we start running through everything there's a few checks we have to do before the keydown keyup we're also doing some calculations to get to get the elapsed time and all that takes just a little bit of time and I think that's adding just a little bit of extra extra milliseconds to each one of our actions so what we could do is we could record how much time is happening in between each running each action let's try that out actually and start and then the it's going to perform the action it's going to do some calculations action and time let's print out that difference execution time I'll be the action end time - the action start time let's get an idea of how long it's taking to process each one of these input events so if we turn to starting location there it is let's round the playbacks group our time is not defined sometime with imports leave and Lamport time we'll get back to the code where we're printing that out and we'll try it again let's look at our results and we're seeing pretty consistently here the execution time is almost a full tenth of a second which is definitely enough to throw off our script especially when we're turning left and right you know it's pretty pretty finicky so why don't we go ahead and subtract that execution time from the amount of time that we're sleeping so let me be organized discord here a little bit let's first check if the elapsed time it's less than 0 and if it is then it will raise this unexpected ordering err otherwise let's adjust the elapsed time elapsed time we subtract the processing time which is the action endtime which we don't have is that here so we can just call time here my name is section start time and then we'll end up sleeping for that amount of time but of course if we have a really quick like keydown keyup this could potentially be negative again let's say F the elapsed time it's less than zero go ahead and make it Cl let's put our print statement back there so we can see how long gonna be sleeping for before you between each action but I'm curious to see if this uh runs a little more true okay I think that was a little more accurate to what I initially did so I think that's pretty good improvement there so now our recorder in our playback scripts are in a pretty good place I think the next step is just to record each whole section of actions and then string them together and play them back and see if we can create this whole bot here so when I create a bunch of different actions for that so let's go go to trader I guess this is an earth station when I have lots of these earth station go to trader let's sleep in between each set of actions say two seconds and then once we're at the trader we want to do the trading - trading and then we want to go back to ship and the last thing we want to do is fly back to low key station so I'll be the final step in our recording and when we don't get on our ship before we go to low key station remember we have that loading screen I think we gave ourselves 10 seconds last time for that loading screen so let's do that again and we don't need to sleep at the end here because we'll just be done so now I need to go create these four recordings so I'll copy that first step and put it in the upward file name of my recorder then in the game remember to go to my starting location this guy over here okay and then we'll make our first recording so when you're making these recordings you still kind of act like you did before don't make too many minor adjustments you're more likely to be off with your bot so it's better to try to make as few long movements as possible and you want to try to get into the middle of everything as best as you can oh man I got stuck in the door I guess that's all right should I go that way aim for that same spots before and we'll stop our first recording here I think I need to turn left a little bit to get that guy on my screen all right so we should have any new recording we this go to trader 1 then I go ahead and remove those initial click events the escape event and then let's play it back to see if it worked go ahead and comment out these other actions that we haven't recorded yet of course I need to go back to my initial starting location let's see how we did you alright that was perfect now one of the good things about doing these recordings I haven't get go to a JSON file so if I want to make any minor adjustments I can go ahead and do that it's like I paused for quite a while after I made that at that last step when I made that last turn towards the trader so I can go ahead and just adjust the time here so my last forward step was done at 34 seconds in let's make this a turn it happened at 35 seconds what you need to keep the down of the up relative to each other of course this is an easy calculation as they pull 39 so I can just change the ball to 35 and that should be no problem all right so our first recording is done now we can go and record the second step this is where we're going to do the trading so I'm changing the output file name here on the recorder again you could create a command-line argument to get the output file name if that's your preference so I will start recording and then in the game I want to talk to the solicitor guy I'll be careful to try to aim for the middle of these buttons and then I'm gonna hold shift and click shift and click for all of these and I'm making sure to pause in between each one because it does take us a quarter second to move the mouse in between each step here and I'm clicking on these empty slots even though we have nothing in them because eventually we will have full trading cargo and then when I take this step to move the inventory to the right I don't want to click and drag because I recording doesn't have the ability to record drags so I'm gonna have to just click on the arrow one two three times now go ahead and sell these other items a little tedious this first time but once we record it we'll be able to play back these actions as much as we want and we'll have a fully functional bot let's get this last row perfect just closed click done and that is the end of that section so now we should have our I do Trading JSON complete I'll just open this up and remove the first couple clicks as usual this gets a little tricky which ones where let you to make clicks and which ones work I think it's these first two I'll go ahead and cut them so I could paste them again later if I was wrong but I'll remove the escape from the end yeah as usual I guess since I'm doing this all the time I could just not record these escapes as well that would be a little smarter and then we're now ready to play it back let's go back to our starting position you kind of really messed up that one I think maybe it's because there's so many players in this station right now that gonna affect the load timing a little bit I thought look better to get quite get the trait that time you I so I made a new script door I should say I've made a new recording for doing the training part part of our script so now I'm running it and let's see if I does any better you okay I made it through at that time so the third section of our recording is going to be going back to the ship so once again I'll just update the recorder to use that file name I'll not go ahead and start recording back in the game I know I want to turn this way center of the room turn to the exit walk through there all right don't left you can go out the exit turn towards the ship thanks the Starbase and that's perfect I'm just going to edit that trip back to the ship this once again get rid of those clicks once again get rid of the escape I'm pretty confident that was a good recording now to test it I could go back through the whole process again if you're having difficulty on this particular section you might just want to start from the trading position run this through a few times yeah I'm pretty confident that was good so I'm just going to test it right back from our initial starting location you this section here selling might be a good candidate for adjusting those times a little bit to go through a little faster and as you saw I'm only selling four items they're just trying to get that working first but the final recording of this I want to sell all 28 items in those slots remember to click the arrows and not try to drag that par oh it looks like he's getting back to the ship all right though it looks pretty dead center perfect all right last step it's just to fly up here from her station back to local station and we can record right from this starter location copy over that new file name let's record it so that's one click two clicks I want to delete and I'm going to open the navigation menu click on hi earth as centers I can get it click see to do the work make sure to give it an extra second here just in case the work does take a little longer sometimes but next I'm going to click on entering the gate alright still recording give it a second to get through the gate and then I'll open the navigation map click on lucky station see again to warp and all I need to do is dock back lucky station and you can escape your recording there I'm just editing this JSON one more time to remove those first two clicks remove the escapes in the game I am at Loki station and that should be it in my recorder I'm going to go in and set this back to my actions test a one just I don't accidentally record over what I just created but everything looks good I'm gonna fly back to earth station manually and then try to run this whole script see if it goes all the way through okay I'm back at station I'm going to get into our starting position and before I run this script for the last time just some housekeeping items you'll notice in the recorder when we are recording the events for the button clicks we are saving the type which is just a click but we're also saving the button and that'll be like left or middle or right mouse button and in our playbacks we're just doing a basic click in peyote we know if I click or middle click supported but if you're playing a game if your body a game that needs that I'll leave you to implement that part of the playback also if you're developing this and your script crashes at any time you might end up with some zombie Python processes so you'll want to do control-alt-delete go to your task manager and delete those let me sure what that looks like I'll just run the recorder here if I do a ctrl Alt Delete in my task manager if you scroll down you should see something says Python oh yeah now I've got a bunch of them here so we're going to go in and all these tasks these are zombie processes that are just sitting in the background you may see your system get a little sluggish with these things hanging about so you'll definitely want to go and close them down all right perfect and then if you've missed anything in the record or the playback I've updated the github repository so you can find the code there just case you missed anything and the link to the github repo is in the description of the video so let's play this back one more time see if our entire script works you all right so that worked pretty flawlessly so now you can see it'd be pretty easy to record actions for all kinds of stations you could set up all kinds of trade routes relatively quickly just by playing through them once or twice yourself making a few adjustments to those JSON files to get them just right yeah this will greatly speed up your ability to create new BOTS alright good job keeping up with me for this part of a tutorial to get more practice I encourage you to use our new recording technique to recreate the first part of our trading route after landing at loki station we turn to our starting position and then have your bot go to the trader trade the set of alpha rads by the new manufacturing robots head back to your ship and fly back to earth station at that point we'll almost have a fully automated trading loop but of course we still have this huge handicap of needing to manually walk to each of our starting locations at each station up to this point our bot has been completely blind ignoring all feedback from the game also our bot is very fragile it breaks whenever we don't begin at the perfect starting location or something simple like a doorway takes longer to open than usual in the next video we'll attempt to solve this I'll show you how to use Pi Auto gooeys simple image recognition features to automatically orient our bot when we land at a station I'll also show you how we can stop our bot when it gets off track so you'll definitely want to join me in the next video where we'll complete our beginner trading bot you

Channel: Learn Code By Gaming

Views: 11,834

Keywords: pynput, pynput tutorial, pyautogui, video game bot tutorial, pyautogui bot, pyautogui game bot, automating the boring stuff with python, recording bot, gaming bot code, earth and beyond game, python, mmo bot, simple game bots, basic botting, beginner video game bot, video game bot python, video game automation, how to write a bot, learncodebygaming, learn code by gaming, python video game bot, vscode bot, learn python by playing games, pynput listener, gui automation

Id: ixwVSsKydIY

Length: 75min 40sec (4540 seconds)

Published: Fri Nov 22 2019