A.I Learns to Play TOWER DEFENSE

this is Pogo Pogo is an artificial intelligence agent and he has a problem everybody wants to touch his meat and as many of you might know there's only one real way to protect your tiny meat from predators fortified Archer Towers Pogo will have to learn all by himself what is the best strategy of placing Towers in such a way that his treasures remain intact here is how it all works the map is divided in a 20x 20 grid with one spawner and one mid card the spawner will well it will spawn waves of enemies which will try to find the shortest path towards the mid cart in order to stop them Pogo will have to build towers on the empty tiles which will cost him 200 coins or upgrade the existing Towers to a higher level which slightly cheaper 150 coins killing a monster will give Pogo some coins depending on the level of that enemy however since the monsters just want meat and Pogo is the one actually killing them it back the question who's the real monster the longer it goes the more difficult the waves will get so Pogo will have to find the perfect balance between positioning amount and the level of his Towers a level one tower has a range of two tiles and will require two shots to kill an enemy shooting one projectile every 2 seconds it can be improved to a tier two Tower which has twice the damage twice the range and twice the attack speed of a regular Tower the last wave will be a boss which has way more health than the other monsters but is also much slower so Pogo will really have to focus on having a solid plan in case you're new on this channel you might be wondering how exactly will Pogo learn all of that and the answer is simple reinforcement learning in simple words Pogo will get rewarded for correct actions such as building a tower while being punished for the wrong actions like losing his meat or simply for existing yeah that last one as cruel as it might sound actually serves a real purpose beyond having Pogo feel miserable for our entertainment punishing the little dude for every second of his existence will motivate Pogo to complete the task as quick as possible resulting in better strategies if you'd like to find out more about reinforcement learning artificial intelligence and computer science now you finally got a chance to do that thanks to the sponsor of this video brilliant brilliant is where you learn while actually doing the thing with thousand of interactive courses in math data analysis program and AI each lesson is crafted by an award-winning team of teachers and researchers and is filled with Hands-On problem solving for sharpening your critical thinking skills brilliant short and concise lessons allow you to gather bits of knowledge whenever you have some free time which will later Stack Up in huge results get hands on with real language models as you explore how they build vocabulary understand the importance of training data by comparing different models learning how to tune a language model to generate different kinds of outputs whether it's poetry or a cover letter go to brilliant.org apps or click the link in the description and enjoy 30 days for free as well as a 20% discount on their annual subscription thanks to brilant for being a long-term sponsor of this Channel and now let's start with the training at the beginning I had everything spawned randomly on the map the enemy Castle the meat cart and even Pogo would spawn on different cell every time I thought that would help Pogo learn better the concept of the simulation rather than remembering a best set of actions for that specific map however it didn't seem to really work like at all the only thing Pogo Learned was to spam Towers at his spawn positions in hope that the enemies will pass by I assume it has to do with POG randomly getting spawned right next to the enemy's path so I tried fixing by having predefined spawn positions now our AI will have to move closer to the enemy path in order for the towers to reach them so hope hopefully the results will be better initially Pogo kept on doing his Pogo things spamming towers and doing other dumb things soon however his curiosity won and he actually started moving closer to the enemy line but not Towers were being built all he did was run in circles like a broken Roomba that's when I started thinking that maybe the spawning wasn't the problem after all perhaps Pogo just had a bad idea of the entire environment I know it might sound counterintuitive but in order for PoGo to flourish in this simulation he needs to have a quite clear picture of what the hell is going on currently he was able to see thanks to a set of rays that he would shoot all around himself when another such a ray would collide with a tower or an enemy Pogo would receive the necessary information about that object such as the type and the distance towards that object well this is usually enough for my other simulations in this example pogu seems to require more data about the environment so I did just that I made it so Pogo has exact information about every single tile and now he knows whether the tile is occupied by a structures and knows exactly what the structure is he knows what tile he is currently on Etc spoiler alert that didn't work either I even tried doing what I do best in these situations and they added more punishment for Pogo but that didn't work either at this point I've been struggling with this project for about 2 weeks and I was already getting quite desperate it was similar to when you open the fridge for the 10th time in the same day hoping that something new magically appeared just to be disappointed by the same results over and over again I even tried using a different machine learning algorithm and after endless hours of tinkering around I finally got it working and they might be using this solution quite often in future projects as well now pogo doesn't need any raise or environment info because he gets all of it using a grid sensor long story short Pogo can now get all the necessary information in a 20x20 grid around himself so basically a mix of previously attempted Solutions anyways that seemed to be a good idea so I started training again he started by spamming Towers once again placing them in between the meat card and the enemy spawner in what proved to be quite a good formation this time however it seems Pogo actually managed to get better at it cleaning the waves without hesitation but let's keep in mind he was playing at reduced difficulty he had 4,000 coins which is basically infinite ammo and he was unable to upgrade Towers just yet also there was no Boss Wave for now once it was clear the grid sensor is actually working I returned the default difficulty and that's when I noticed another small issue apparently it's impossible to win the game in the current state LOL yep you would basically run out of money no matter how you build and that won't be enough to complete the harder waves I quickly fixed the spawn frequencies and gold drop to make it feasible and started my last training by the way I know the towers are being placed a little of greed and they have no clue why that happens in editor mode everything looks great but when I click play they move by a bit misaligning themselves hope that doesn't trigger your OCD but if it does make sure to smash that like And subscribe button to let me know about it the training started with Pogo recklessly spamming Towers whenever he had enough money the greedy bastard doesn't even realize most of of them can't even reach the enemies well seems like it didn't take him too long to realize it makes more sense to build the towers closer to the enemy path I mean he's still confused as don't get me wrong but at least this time either due to huge brains or sheer luck I wonder which it is LOL Pogo built Four Towers in what looks like one gigantic building pretty cool and what is even cooler this allowed Pogo to secure his first win and that's only on round three three sure enough Pogo failed to repeat the success placing Towers all over the place in the next round he did have several interesting attempts like this one on round 12 where Pogo succeeded at redirecting the enemies a little bit winning himself some time but he still failed to secure the round on round 13 he built something that resembles Customs or a gate of sorts it proved to be quite effective against the level one and level two enemies however the gates failed to stop the boss from passing right through them and getting a taste of pogo's meat another interesting strategy can be observed on round 21 Pogo after placing two small Towers in the middle which will win him some time and money approaches the spawn point and starts building right next to it I wasn't sure what's the benefit in that but then I remembered the existence punishment this way the enemies are eliminated faster thus less existence punishment see I told you it serves a good purpose the boss almost gets away but it is stopped by the small Towers Pogo placed at the very beginning nice okay so you get the gist of it but before we get to the final results I want to quickly mention that this video idea was suggested by Seldon on my Discord Channel where we had the video idea contest congratulations to Seldon and thanks to everyone who participated and shared their cool ideas anyways I've kept training Pogo for about a day and here is the final result and to make it even more interesting I multiplied all enemy Health by two including the boss but also doubled the gold drop from them Pogo starts safe by placing a tower in the middle and immediately upgrading it to tier two which should be enough for the first wave notice how now it requires two hits from a level two Tower to kill a level one enemy Pogo places a second tower right next to the first one and attempts to upgrade it despite not having enough money which I expect shall change quite soon with enough money his whs play upgrades the second tower and after getting some more he builds a third Tower and once again upgrades it in preparation for wave number three now we have level two enemies which require a total of six hits from a tier 2 Tower and 12 hits from a tier one Tower so the fact that Pogo upgraded all of his Towers now proves to be quite useful not a single enemy has passed yet and the money problem seems to be Vanishing away with each enemy dropping 100 coins after earning about 1,000 coins Pogo finally starts building more Towers which shall prove crucial at the Boss Wave this wave takes quite some time so I've sped up the video we can see Pogo using one of his old technique of building a square of towers but this time it is improved this time Pogo won't limit himself with a square and is building an entire fre wall forcing the enemies to walk around it taking massive damage in the process very nice here comes the boss it gets close to the towers and oh no it changes directions instead of going down as intended by Pogo the boss has decided to go from the top and oh never mind the boss is dead LOL that's it for this video but make sure to check out this other video of mine where Pogo has to learn how to properly handle big balls in my favorite childhood flesh game Bubble Trouble bye

Channel: ZuzeloApps

Views: 26,646

Keywords: a.i., a.i. learns, a.i. learns to, ai, ai lea, ai learning, ai learns, ai learns to, artificial intelligence, deep learning, deep learning ai, deep learning tutorial, machine learning, ml agents unity, neural network, neural network explained, neural network projects, reinforcement learning, unity, unity ml agents, deep reinforcement learning, tower defense, tower defense simulator, ai tower, ai defense, ai tower defense

Id: Gxovw3EE7zU

Length: 11min 32sec (692 seconds)

Published: Fri May 31 2024