Neural Network Learns to Play Snake
Video Statistics and Information
No captions available for this video.
Channel: Greer Viau
Views: 3,048,276
Rating: undefined out of 5
Keywords: java, snake, java snake, snake ai, ai, processing, genetic algorithm, genetic, algorithm, machine learning, machine, learning, deep learning, deep, neural net, neural, net, train, training, programming, visual, video game, game, video, coding, computer, science, computer science, feed forward, vision, computer vision, neural network, network, networks, neural networks, neural nets
Id: zIkBYwdkuTk
Channel Id: undefined
Length: 7min 13sec (433 seconds)
Published: Sun Nov 04 2018
Please note that this website is currently a work in progress! Lots of interesting data and statistics to come.
Neat project with a nice visualization. Good stuff!
It's not surprising the snake can't improve significantly from generation 30. The input features + model you've chosen don't have enough information to determine whether a path will be closed by a given action so the snake will happily spiral in on itself. You'll probably see a smarter snake if you give it a view of the entire board or use a stateful architecture.
As for future projects, it would be fun to see multiple snakes in the same arena.
How did you know to use 2 x 18 hidden neurons? Would it look much different if its more/less ?
If you're just going for high score (ignoring time), one optimal algorithm can be stated very simply:
>Start:
>Start in the upper right corner.
>Trace the wall.
>Two spaces before entering the upper form a switchback across every interior square not in the rightmost two columns.
>Use the penultimate rightmost column to return to the upper right corner, goto Start.
The reason this algorithm is optimal is because it *plans* for the base case where there's only one open square on the board (note that it also completely ignores the location of the food).
It seems to me that when you train a NN, you reward it for seeking food while avoiding harm. It *could* accidentally arrive at something like the aforementioned algorithm and do quite well as a result, but it seems to me more probable that it is going to evolve a *food seeking* function, thereby *eliminating* the possibility of arriving at answer that aligns with optimal planning for when tail management is the main object of the game.
You might run your simulation forever, and never throw your "genome" out completely such that you'd have a chance at accidentally discovering the "optimal" algorithm.
(Is there a name for this in AI? I haven't taken an AI course in 10 years. I think evolution skeptics call it irreducible complexity?)
Is there any framework or shared code that people can plug in their own algorithms to try other approaches to this problem?
How did you feed the GA into the NN? What information did the genome encode? Activation probabilities? How did you perform crossover? Also, your video is showing a single individual per generation; Iβm assuming this was the fittest individual. How big was your population?
What advantages does it have (in this case) over simple back-propagation?
From the video it seems obvious that the NN fails to grasp the rules of Snake: it almost exclusively performs right turns, and it only eats food coming from the top. Both of these are very typical cases of overfitting, which surprises me: Iβd have thought that GA would be more robust against such overfitting. Maybe the mutation probability was chosen much too low? (Or, conversely, much too high so that all but one individual are pathological cases, and the single fit individual per generation barely evolves at all; Iβm purely guessing here).
Have you considered using Q-learning over the entire game state instead?
I'm really new to programming, I just want to ask what is Genetic Algorithm and How is it used?
Like Derek Zoolander, it can only turn right... Except for that time it turned left.
Very neat! I was into Machine Learning like 2.5 years ago in my job - at that moment I was very excited about it. But day-to-day work disenchanted me for a long time. It is great to see people doing fun things with the machine learning