AI Aimbot | YOLOv5 Tutorial | Tech Breakdown # 2

our ai is now ready to take on anyone crazy enough to challenge hope you're ready because on this episode of tech breakdown we're creating an ai aimbot [Music] before we get started today we're going to need to know a few things i'll leave timestamps on screen as well as in the description in case you want to get straight to setup although foundations is highly recommended to ensure the project can be understood at a fundamental level firstly what type of ai exactly are we dealing with this is an extremely important concept because many people seem to think ai means full-fledged terminator t800s and that is not the case ai is a broad term with many layers and subdivisions the scope of which is beyond this video but here is a general overview in ai there are three basic processes first is a learning process based on some given data your algorithm will produce an estimate about a pattern in the data in other words it will predict what you are trying to show it second is an error function this is used to evaluate how accurate the ai's predictions are lastly there is model optimization this compares multiple trial runs to see if our model is getting more or less accurate then using that data will adjust its weights or prediction parameters to try and guess more accurately this can get extremely complex but as you can see from a basic standpoint machine learning takes input and given a function tries to predict output as we get to the innermost layer deep learning the model takes input passes through multiple different neural layers then predicts an output today we are going to be using deep learning in an ai subfield called computer vision this field is dedicated to helping computers make sense of images and understand them the way humans do the algorithm we are going to be using is called you only look once more popularly known as yolo developed by joseph redmond yolo works by applying a single neural network to the full image dividing the image into regions and then predicting probabilities for each region this can get extremely complex so for more research i'll leave a link to joseph redmond's website as well as his youtube channel which is filled with in-depth computer vision lectures it's worth noting although joseph redmond was the creator of yolo he abandoned working on the project after concerns over military applications and abuse potentials the last version he was part of was yolo v3 this however didn't stop progress and in april of 2020 yolo v4 came out lastly only about a month later ultralitics released yolo v5 which despite its massive popularity to this day does not have a formal research paper creating controversy on if it really deserves to be called yolo v5 with that being said 5 is what we will be using today it has great performance speed and is extremely friendly to beginners as well so let's get into it [Music] for setup the first thing we need to discuss is dependencies yolo v5 has a few dependencies two of the more important ones to remember are opencv and pytorch opencv as defined by themselves is a library of programming functions mainly aimed at real-time computer vision if you are in this field for any amount of time the name will come up and it's worth researching further secondly is pi torch an open source machine learning framework if you're interested in ai pytorch is a framework worth learning about it is used by industry leaders such as open ai comma ai tesla and more lastly when training neural networks it's important to know that it takes a massive amount of processing in order to train and teach our computer anything by default this can be very time consuming in order to increase efficiency and speed we can leverage our gpu to speed up the process significantly in order to do this with yolo however we will need an nvidia gpu i realize that most people will not have this so the following method i'm showing will work for everyone then at the end i will show how to install what is needed for those with nvidia gpus so to get started we navigate to the yolo v5 github repo you can search for this or find the link in the description as well as all the following commands if you have git installed we can clone this repo by scrolling down and copying these commands if you don't have git installed however you can simply click the green code link in the top right and download the repo it is the exact same thing [Music] after that is done we open up a terminal or command prompt and change directories into wherever we downloaded yolo to do this we type cd in the path to your yolo v5 directory next to install the requirements we type pip install requirements.txt this will install all the requirements in text file ultralitics has left as you can see torch and opencv are two of the names listed next is scuti a program i recently wrote to help automate the capturing and sorting of images for our ai model data gathering this is optional but will help ease the pain and speed up the process of data collection for all computer vision based projects note that i plan on continuing to build and upgrade this as time goes on so if you have any ideas or suggestions on additional functionality that would help your daily data collection feel free to let me know and i will work on implementing it to install we navigate to the github repo and similar to yolo we can either click the green download link or git clone the repo again all commands will be left in the description below once cloned we cd into wherever this was downloaded and type pip install dash r requirements.txt this will ensure all dependencies are installed [Music] to run this we type python gui dot pi if everything was installed correctly a user interface will pop up lastly we need to clone label img created by zoodalin this will allow us to create labels for our images in the training process there are different commands depending on what operating system you are using i will show on screen but know they can be found on zoodalin's github which i will leave a link for in the description as you can see for windows we need python pi qt and lxml if you installed scudy you will have pi qt and all we have to do is install lxml to do this we go to our terminal and type pip install lxml once that is done we get cloned the repo by typing git clone in the link to the repo next we will need to cd into label img lastly since we are using the newest version of pi qt we will copy the following code and paste into our terminal to run this program we type python label img dot pi if everything is successful a user interface will pop up with that being said if you don't have an nvidia gpu or don't want to enable gpu acceleration that's it for setup and you can skip to the runtime segment to set up gpu acceleration we need two more dependencies nvidia developer toolkit and anaconda the nvidia developer toolkit will enable us to activate our gpu's cuda cores which in a nutshell is essentially a processing core while a cpu may have six or eight cores a gpu may have hundreds of cuda cores so select your operating system and be sure to install the newest version because version 10 of nvidia developer toolkit isn't compatible for gpu acceleration with yolo next we need to install anaconda this is a distribution of python that helps with package management once this is installed we navigate to pytorch's website and copy the command to install pytorch for anaconda once that is done we go to our anaconda terminal and paste this command [Music] note that in order to use the anaconda distribution of yolo we will need to pip install the requirements.txt file in our anaconda terminal once that's all finished to verify that everything was set up successfully we go to our code editor i'm using vs code make sure to select conda then type torch dot cuda dot is available and if everything worked correctly an output of true will be given now that everything is installed let's get right into it i'll leave template code on screen this can also be found on yolo v5 github but it is fairly straightforward we import torch create a variable with our model add image pass or video path to a variable then tell the model to analyze those images with that being said all we need to do now is add results.show to print out the image with the model's predictions as you can see for a few lines of code this is not only extremely accurate but also easy to use now if we type results.print our program will give us back what it believes it found in the image while this is cool this is not enough data to track in real time this leads us to our next line of code results.xyxy this will return a tensor with the following information the x-min the leftmost point of detection the y min the lowest point of detection the x max the rightmost point of detection the y max the highest point of detection the confidence that the model has with its prediction and a corresponding class number for different prediction objects [Music] this is more than enough info to add some basic formulas to and create a click point for our bot with that said let's load up a while loop and see if this model will work out of the box in a new file i wrote a script that will capture the screen location of where our game will be if you installed scudy during setup all of these imports will be available otherwise you will need to pip install mss as you can see the code is pretty similar to before we create a model we use mss and define where we want our screen to be captured then in a while loop we create a variable and grab our screen image then we pass that image to the model as before we then use opencv to display those results it's worth noting in order to have those results displayed properly we need to use numpy's squeeze function now let's see how this looks [Music] as you can see for a base model out of the box this provides pretty impressive results recognizing zombies as humans it might seem intuitive to just create an aimbot that targets humans this however will not work as you see we are getting tons of unnecessary info as well as the player's hands getting registered as human this will break a bot program by constantly aiming at itself this is not yolo's fault however yolo was trained on the coco dataset which has lots of generic categories zombies unfortunately didn't make that list so to get a list of objects we desire we will need to train our own neural network note if you're wondering why this video is so slow this is due to how our program is set up everything being inside one loop is causing us to get a delayed result averaging only about 6 frames per second [Music] believe it or not training a neural network is extremely simple with yolo v5 the only caveat is it requires a sizeable amount of data if you're wondering an exact number the answer is usually more an effective amount of images for a neural network is usually in the tens to hundreds of thousands but understandably that is unreasonable for a single user to gather so just get as many pics as you can with a target goal of at least a thousand this is where study comes in handy whether you want to have images auto captured every few seconds want to designate a keyboard key to take a screenshot whether you want to take a video and convert it into numerous images it is all easily done with scudy i don't want to self promote too much but this tool personally has saved me hours of time and i hope it helps someone out there with that being said let's gather some data [Music] [Music] so after collecting a decent amount of images i have distilled my categories to four first basic zombies second crawler zombies third demon dogs fourth broken barriers we will now take this data label it and be able to start training to label our data we cd into our label img folder and type python label img.pi this will bring up the label img program now that that's open we have three things we need to do first open the directory where your images are second choose a save directory i recommend using the same folder that your images are in this will make things easier later on third make sure to change the label format to yolo that is absolutely critical once that is done we are good to get started to label an object and press w on our keyboard this will prompt a two-dimensional axis to be centered on our mouse simply click and hold around the object to be labeled and then when done release the mouse the box will pop up prompting you for the label name there will be auto-generated ones but what we are interested in is our own so simply type the name of your object and hit enter or click ok once done labeling all objects you want to train for inside a photo either click the save button on the left or press ctrl and s to save to move to the next photo click the next image or hit d on the keyboard note this is an extremely tedious task so knowing the key commands is important for saving time as you go through the images add labels as needed to different objects and don't be afraid to add labels inside of labels once we're done with that if we view the classes.txt inside our image directory we get a list of all label classes created we could have deleted these auto-generated classes before we started but that's okay because we did not label anything with these there will be no return results during training now we must create a yaml file in case you don't know yaml is a data serialization language if we go to ultralitics github we can find a template yaml file taking this basic model we apply the following changes first we add the base folder to where our images and labels are next we identify the subfolder where our images are third we add the subfolder for validation data this is important because validation data is what helps us know if our model is truly getting more accurate or if there is bias in itself to create validation data we simply take a portion of our images in their corresponding labels and put them inside a separate subfolder if you're wondering how much a good validation set is again the answer is usually more but because we are working with a small data set about 10 percent will work lastly looking at the classes text file we copy the items and paste them in our names list each class separated with a comma also be sure to add a number of classes or elements in that list where it says nc now that all of that is finished we're ready to start training when training a neural network there are many different factors that can drastically change the outcome of our ai model this is beyond the scope of this video but i highly recommend looking into this by using a site like kaggle it is free and has great hands-on computer vision and deep learning lessons i'll leave a link in the description to train our neural net we can copy the code on ultralitics github and simply change the coco128.yaml to the yaml file we've just created a brief breakdown of this code img is the resolution we will be training at batch is the amount of data we show to the neural network at a time larger batches means our ai gets to see and make predictions over a wider data set epochs is the amount of time our ai sees all of the data data is what our model will be making inferences on and weights are the weights we start off with i am choosing yolo v5 small because it is extremely fast try playing around with these settings and you will see how different parameters result in different performances with that said i will enter this code and let the training begin while this is training we can go into our yolo v5 directory click on runs and select train here we can find all kinds of data my favorite is the ability to see inferences for each batch during training once training is done a weights folder will appear inside will be two files waits for our best batch and waits for our last batch now if we copy this path we can reload our loop from earlier and see our newly created ai as you can see we've not only got our custom labels being shown but we've eliminated the extra noise at the same time for a data set of just a few hundred images in a small training run this is pretty impressive with that said we move on to our last segment aimbot [Music] the first step in any aiming process is identifying what we want to aim at if you recall from earlier when we created a dot yaml file we have a specific class for every item we labeled to figure out what the class number for our desired object is we simply count from the top of our class.txt file down to the bottom and subtract by one we are subtracting by one because in python list elements start at index 0. therefore we can correlate zombies with the index of 15 barriers with the index of 16 demon dogs with the index of 17 and crawlers with the index of 18. now that we know this let's write some code to start you may or may not have pi auto gui installed so if you get a module not found error simply type pip install pi auto gui next we will start by using the same code we've been using throughout the tutorial but this time we will call results.xyxy.2 list in doing this we convert our tensor object into a list object that is simple to write parameters for now if we run this whenever a detection is made results will be given in a list format just like the results above now what we need to do is write a method to get the center point of our detections and return the coordinates which will be used as our click points note that the following code we're about to write is an extremely fundamental way of writing a bot script and is only scratching the surface of our newfound knowledge and capabilities because this video is getting long i want to make sure everyone can write working code to run in games but know that i'm creating a full length body tutorial which will go in depth and allow for optimization as well as modularization so to get started inside our while loop we write an if statement and say if there are any detections proceed we then can optionally assign a parameter to specify a confidence threshold and say if the confidence of the detection is above a certain percent continue this will help deter false positives next we define what we want to look out for remember those class numbers this is where they come in handy if you're wondering why there are two parentheses next to our rl variable this is because our list is two dimensional so we must specify the row and column next we create two variables for our x max and y max values to get the width and height of our detection we simply subtract the max's from the mint for both our x and y values now here's where things get tricky different games have different mechanics to prevent cheaters so as you can see in our expose and y post variable we are multiplying by 0.37 and 0.3 this is to account for the difference in what coordinates should be versus what they are you may have to play around with this setting to find what works for you all of this being said the hardest part of this code was to figure out to move back the same amount we move forward after clicking on our target without this single line of code our aimbot will spiral out of control and will leave you on an all-night vendor wondering where you went wrong in life but don't worry i did that for you so we can move on this will provide us with a working aimbot that will automatically find targets and click on them two final things to note are one our debug video will be choppy but this will not affect our actual aimbot like i said earlier i will be coming out with a full length botting tutorial to apply specific parameters in-game logic as well as increase our fps by a magnitude of several factors lastly while this method works on many games lots of newer games have advanced anti-cheat and regular mouse libraries will not work don't worry though i am also working on a video that will show us how to bypass this at a hardware level so for all that more if you enjoyed this video make sure to like subscribe and let me know what i can do to make these tutorials better for you this is trevor satori signing off

Channel: Satori Digital

Views: 66,322

Keywords: AI Aimbot, Deep learning, Machine learning, Yolov5 aimbot, object detection, object detection python, object detection pytorch, yolo object detection, yolov5, yolov5 custom object detection, yolov5 explained, yolov5 object detection, yolov5 train custom data, yolov5 tutorial

Id: ilsn-TvryyA

Length: 20min 39sec (1239 seconds)

Published: Mon Apr 11 2022