Arduino Neural Network Robot - Complete How-To!

hey what's up everyone so today's video is actually a video series that I did for make media a few years back and it's on building an Arduino based neural network robot and I'm really proud of the series and now that maker media it no longer exists then I figured I would compile all three versions put them into one video and post them to my own channel so I really think this is a great video series and I think you'll enjoy it so if you haven't seen it before check it out let's go so this week's gonna be a little bit different than the videos on my channel I'm gonna be doing a detailed process on designing your own Arduino base robot but not just any robot this robots gonna run on a neural network it's gonna be pretty cool I'm gonna go through the design steps of prototyping making the circuit board and the schematics ordering it and then assembling it and then finally we'll program it you'll see all the steps and hopefully you'll be able to make your own at the end so with that said let's gather up some components and let's get started [Music] okay so these are the basic components we need to get the robot kind of tested beforehand before we start making any schematics you want to make sure that the pin outs are gonna work you know even though you can read the documentation you just want to make sure that everything's gonna work before you start making the circuit board so you've got an R do we know this is actually a Sam D 21 Sparkfun board this is the microcontroller that I like to use these are some photo resistors for detecting the light two motors will probably only have to test one this is a motor driver it's the drv8825 I believe this little board from Pulu of course the battery wheels and I may put an OLED screen on it so it will display some information when the robot is doing things so I'm gonna connect all this stuff to a breadboard and then we can kind of test it out before you know we make any schematics and when we're making the schematics we can refer to this and we can also change some pins if it makes it easier to connect them you'll see later on when we're working on it [Music] okay now that I have all the jumpers connected to the Arduino and to the components I just need to write a quick Arduino program and test out everything and make sure that it all works so writing programs like this can be really helpful for later on you can define all your pin outs and you can test them out and you can save all those for when you're designing the program later on you can just copy and paste them over it's especially helpful if you're working with motors because sometimes microcontrollers especially using like RT we know you could have pins that behave weirdly in certain situations like when you plug it in and you don't want the motors to spin when you first plug it in you only want them to react when the program tells them to so this is a time when you can test all that so I wrote a simple program that just tests all the photocells and the motor driver and then it also displays it to the OLED display so each one of the photocells outputs how much light they're seeing on the OLED screen and then the motor is controlled by this one here and it will speed up and slow down depending on how much light we have so now I can see that everything works I can go on and I can transfer this physical schematic into an actual one in kicad and I'll show you how to do that so the nice thing about using you know that module like the motor driver from Palolo and the sparklin board is that they have their schematics available that means you can integrate them into your own even though it flows is just a PDF and Sparkfun is in Eagle not kicad you can still use them to reference and you can use them to create your own schematics you don't have to go digging through the documentation for say that motor driver just to see how it needs to be set up because you can kind of copy what others have done already and you know that it works and you've also tested it that's the point of these boards and Arduinos and is that you can test your designs before you make your own custom circuit board so now that the schematics are done and we're about to begin designing the actual PCB and the traces and doing all that stuff I like to design the board in some CAD software first and lay out some of the things like the motors where the screens gonna be where the buttons are gonna be and the actual outline of the circuit board the reason for this is it makes it a lot easier to import the board outline into kicad if you have a DXF file of it already and a lot of CAD software most CAD software allows you to export the excess now when you first open up kicad to start the PCB design process I import the DXF file and then from there I then import all the components which when you first open it's gonna be a big mess of wire so the first thing I like to do is separate the components into the groups of things they're connected to so the capacitors that are connected to the microcontroller or the group of things that is involved with charging the battery or the photo resistors with their resistor I like to separate them all so that they're isolated that you can kind of create smaller tight packages of components before you actually place them on the board you'll probably have to move them around but this is the way I do it just to start off once I have pretty much all the components on the board then I will start to route the components that are the most sensitive and require the direct line and of communication so say the OLED display you know the clock line or something like that you don't want interference on those because it could mess with it it's really unlikely and you could probably just wrote them however you want but that's just the way I do it I just gives myself a starting point so and generally the last thing I do is the power lines if you take your time work away at it and slowly you get everything on will be good to set all the traces and then we'll be able to send this board design off to some PCB manufacturing place and we'll go from there [Music] alright so the design is done that's step one to this whole thing so I'm gonna send this off to a PCB manufacturer and we should get it back and maybe a week or so we'll continue on so one of the things I like to do after I finish it is loaded up which kicad has a mode called 3d viewer which lets you view the PCB in 3d and this lets me know if some components are gonna be weird to install or things are gonna bump into each other just gives you another viewpoint of what the board could look like when it's finished it's really helpful so with that that's part one down we'll get these boards we'll get the components put it all together and we'll have our assembled robot next video so stay tuned subscribe check out my channel you maybe you'll see some things you like there and yeah everyone be good and have a good night [Music] hey everyone my name is Sean Hodgins and welcome back to the multi-part series on building your own arduino based neural network robot so last week we explored early development stages so we prototyped some components using an Arduino some motors made sure everything worked then wrote a program to make sure that everything worked together we also created a schematic and a custom circuit board and sent that off to be made well they're here now and we've got all the parts to assemble it so I'm gonna go through the steps of putting the surface-mount components on this the through-hole components finally putting a bootloader on it and putting some firmware on it and testing it out so let's get to it [Music] so I mentioned that I already have the components for the board ordered you're more than likely going to order the components from digi-key which will have everything that you need so with in kicad there are a number of exporters custom ones that will extract the parts and put them into a CSV file if this is your first project I suggest putting each part manually just from the bomb in DG key so going through and searching for each part say you search for the microcontroller the Sam D 21 when you add the part to your cart and you add you want to put in the customer reference the name that you used on the circuit board so in this case it's u2 and it will say this in the kicad bomb and when this component arrives from digi-key it's gonna say u 2 here so it'll make everything a lot easier when you're assembling the board a lot of people forget to do this the first time around and it will cause a lot of headache so the next thing that I like to do as you saw me doing in the intro is separate the components from the discrete components to more complex the ICS and the microcontroller LEDs anything in these silver anti-static bags I will separate them out and then also the surface mount components will separate out because what you're gonna be doing is you're going to be installing all the resistors and the capacitors first while the surface mount ones the second set of surface mount things will be the ICS like the microcontroller and the motor driver and then once you reflow these if you're doing it in an oven afterwards you can put all the through-hole components on and this is just the way I do it and it makes sense to have the capacitors on the board before just to eliminate any possible chances of ESD but it's never personally happened to me I've never fried or do we know with ESD so I don't know I don't even sure it's possible again this is where you have a few options the solder paste so for most cases I like to order a stencil for the board because they're actually really cheap there's a lot of stencil services that you can get them done but if I'm only doing a couple of boards in this case one or two then I'm just gonna do it by hand by using a very fine tip on the solder syringe what I'm gonna do is on each pad where there's a surface mount component I'm going to apply a little bit of solder and this is going to take a lot of trial and error you're gonna end up putting too much solder almost all the time especially when you have really small pins you'll find that less is often more but again trial and error and you're gonna see how to fix it afterwards because you're definitely gonna have bridges no matter what if you don't use a stencil even when you use a stencil you often have a couple so trial and error let's get going so I've got all of the solder paste on the board now for the components that I'm going to be doing this will be a good example of when you have too much solder paste especially notice on the microcontroller so we're gonna have to clean that up for sure after with the soldering iron but generally have to anyway so that's another technique you're gonna have to learn and with the really small pads on the motor driver I actually used a little piece of solid core wire and just took some solder paste on the tip of it and spread it out you notice you don't need to go on each individual pins you can go along them because when the solder paste heats up it actually spreads out anyways it spreads out on its own so you can go between them as long as there's solder paste touching the pin and the solder paste touching the pad you should be good the next thing you're really gonna want to have is a really good pair of tweezers oh these will make your life so much better when dealing with really small resistors and components because they'll let you grab onto them so much easier [Music] oK we've got all the components on so we're ready to reflow it it's really simple just pick it up carefully kind of near the front so this is honestly the cheapest toaster oven I could find it was like twenty dollars you don't need anything fancy though it is nice to have one it has a fan in it this one doesn't anyways close it up it's set to as hot as it can go and we'll just turn it on and watch it just wait for it to reflow you visually watch it reflow and then I like to leave it in there for like another five or ten seconds after everything is flowed also I like to have a little fan going for when I open it and also something that you can put the hot surface on normally I put it on like a heat sink or something and some neon those pliers once I'm sure everything is solid and there it is some components have obviously shifted around because we've used too much solder paste which is normally the case it's rare that you get a perfect board though it does happen so we're gonna use the soldering iron to fix all these the idea here is you're going to take your flux and wherever you need to move a component you're gonna apply some flux to that solder pad so if you've got a resistor that's slightly off and it's only on one of the pads you don't need to fix the ones that are like crooked but they're still connected but if it's only connected to one pad then you're gonna apply the solder paste to the side that is connected and then you'll heat that side while grabbing on to whatever you're moving this case a resistor it's gonna come free and then you can move it over so that it's touching both pads like so and now it's only connected on the one side you have a little bit more flux on the other side and just touch it like that and it's connected so that one is moved and it's fixed now since we use a lot of solder paste on the board there's a lot of bridge pins on the microcontroller the way to fix it is to take a dry soldering iron which means there's no solder on the end apply a bit of flux to where the bridges and then it will naturally wick the extra solder up and leave just two pins with a little bit of solder remaining but now they're separated it works really well and it's really simple once you understand that the soldering iron wants to wick solder away from the pins so every time it gets to full you have to then clean it off again the brass sponge or else it won't be able to pull any more solder away so that's the easiest way to get the pins from stopping bridge together now that we've fixed all the surface mount components we can move on and install all the through-hole components [Music] so now that all the components are soldered on actually realize that the mounts that hold these are fuse holders but they weren't ordered for some reason so that happens so I'm going to make some ones that you can 3d print and those files will be available in the github as well so I'm going to 3d print the motor mounts look at those mounted on we get the bootloader put on and we're rocking [Music] okay we just have one last step to do before we can load some firmware on it and that's to load a bootloader on the chip so what the bootloader allows you to do is program it using the arduino ide it enables the USB so that you can plug it into the computer and use the Sam D driver to be detected as a comport so since we use the Sparkfun Sam D board to test it we might as well use that bootloader to put it on so what I'm using to program for the first time it's called an AB male ice but you can also use and already we know itself to do it so just plug this into USB and then we connect this cable and also the device does not power the chip itself you have to give it a power supply so we'll attach the battery green light means we're good so now we're going to open up Adam else to do I'm using ML Studio seven it's free to download of course and we're gonna go tools device programming and C we're using the add Mel ice so you hit apply which enables that programmer and read to make sure you can see now you don't really know if this is okay but if anything shows up it's probably fine actually go to memories and you have to locate the spark fund bootloader so what you're gonna do is percent data which brings you here but you actually want to go one back go to local or do we know and packages Sparkfun hardware sam d whatever version is there boot loaders 0 and here it is sparkling sam d unhex so that is our bootloader and you find that simple as hitting program and if everything works you get that and that is it it's now program so we're gonna connect to the USB we're gonna write some firmware we're gonna test out the robot okay so everything is assembled we got the battery hooked up at the OLED screen on there that are connected and we're gonna run a test program that we made originally so all we have to do is hit upload on this hopefully the motors do something and then we're good programming alright that's what it's supposed to do so our programs that's all we need and that is it for this segment which is assembling the neural network robot so stay tuned for part three where we'll be discussing neural networks we'll be talking about running neural networks on Arduinos it's totally possible we've done it before we'll be installing a neural network on this robot so it can navigate using one and the whole project will be open source so anyone can download it and make their own circuit board and do the whole thing and assemble it that is the whole point so be sure to subscribe hit that like button and also check out my channel where I do a whole bunch of projects that are like this as well anyways everyone you know the deal be good and have a good night [Music] hey everyone my name is Sean Hodgins and welcome back to this multi-part series on building your own arduino based neural network robot in the first part we took off the shelf components and built a custom prototype and then we designed a custom circuit board in the second part we took that custom circuit board added surface-mount components through-hole components a bootloader and then some firmware just to test it out and now finally we'll be installing the neural network firmware and talking about it and making sure you understand how it all works let's get to it [Music] so before we develop the neural network program I've put together a really simple navigation program using the light sensors but no neural network just the light sensors controlling the motor and I will break this program down so you can understand what's happening in each section he uses the UHG to library for the screen that's what that's for this include make logo is the very beginning make logo screen that's all that does and this enables the ohlet's well that stuff is just for the display then you've got photo sensors and then the motor outputs and the buttons and then here this is defining the analog inputs for those photo sensors and enabling the buttons here more OLED definitions and then this is how the first logo is set up you can look into that library it's pretty simple once you understand it that's just the setup and then you can see here in the loop it's just depending on what menu case is running then it will run that so simple light avoid that's the program that will make the robot navigate avoiding light hopefully in this simple light avoid routine it reads from the four sensors then it Maps them from you get a value from 0 to 1024 depending on how much light but then it switches them from 400 because I find they still get light to 0 to 48 for the for the top sensor in the bottom sensor and then from 0 to 64 for the left sensor in the right sensor it takes those values and it feeds them into something called draw falter so it's drawing the ball direction on the OLED screen so what this does is it takes those 4 inputs and then it defines them in x and y-coordinates based on how much more light each sensor has so if the left one has more light than the right one then the ball slightly goes to the left and if the top one has more light than the bottom one then the ball is going to move to the bottom one it's kind of like a joystick on the screen and that's what's controlling the robot so once you have those x and y's those are actually sent to the motor control well this is to draw that ball on the screen and this is the motor control that will drive the two motors so if we see in this motor control subroutine i've defined the motors that 50 is not moving zero is completely backwards as fast as it can and 100 is completely forwards as fast as i can so what it does is it takes or takes the y offset and sets that as how much power each motor has and then it takes the x offset so the x coordinate the left and right sensors and then offsets that power from one motor to the other if it's fully to the back sensor then it they would both drive straight at full power but if they're slightly more to the right then it would make it move to the left so it takes that offset sends it in and then finally motor a and B this is how you can control them so you have a motor routine a where it takes the speed between zero and a hundred fifty being not moving zero being backwards 100 being forwards and that's how it defines it here this is how it controls the motor driver so it changes the the phase of the input and then gives it a PWM input depending on which direction it chooses there's a hundred ways you could do this probably a lot of ways you can do it better than I've done it this is what works so it just keeps doing that over and over and over again so let's see how that works so you can see I've made a little menu here but there's only one program on it you can switch between them using the left button or the other to just a blank anyways if you go to the simple program hit the right button to run it then now we have a ball in the middle now the motors aren't connected at the moment there they've only got power when you have this switch that moves over that puts power to the system from the battery and they're powered through the battery as you can see it's pretty dead even right now because they all have the same amount of light it might be a little bit to the left here because there's a little a shot on this one if I put a shadow over this then the ball moves up to the top if I put the shadow over the back and the ball moves back and if you can imagine that being like a joystick then you could see that it's gonna make the robot move in those directions so let's test it out so now the motors are connected to the power source we'll run the program so as you can see it's trying to stay in the shadow but one thing we could implement is have it drive forward when it's fully in light let's add that right now okay so what I just added is if the sum of all of the sensors is greater than a value which would seem like they're all being lit where it's not in the shadow then it's told to just drive forward slowly until that's not the case anymore so now we can see it's happy when it's in a shadow not happy when it's not in the shadow and it looks for the shadow as well so as you can see that program works really well and it's pretty simple doesn't take that many lines of code to get the robot behaving the way you want it to but it's not as cool as a neural network and what's cool about a neural network is we can defined certain extremes and then it can work out the points in the middle on its own so let's develop the neural network program now so for the neural network we're actually going to be adapting code that is found on the website robotics at Hobby scene comm they have an Arduino artificial neural network a really great example code that is actually designed designed to take a 7-segment input and translate it into only four bits so you're taking seven bits and converting them down to four it's a really cool interesting program but what we're going to be doing is we're going to take this set of code that they have because they've developed all the math behind it and then we're going to modify it so that it can run on the sensor inputs and then output to the motors so we're gonna be defining we're gonna be having this run continuously after it's trained to control the robot it's pretty cool so again with this program written and everything modified let's go through it and I'll explain everything again and we'll just make sure you're familiar with everything that's happening within the program we've gone over some of this stuff with the inputs they're all the same now we have some variables that are for the neural network we're gonna define the extremes of the program so the lights the extremes of the inputs are going to be defined between 0 & 1 & 1 being the most possible light on that sensor and 0 being the least possible light on that sensor so here are the inputs so here we have a matrix of 4 inputs by 16 so each one of these is the sensor input and we're defining these now as the maximums so as you can see I've labeled them here on the right so with this one 0 1 1 0 that means there's fully light on one on the left and right but no light on the top and the bottom sensors and we go through every possible scenario that we can think of that will allow it to properly learn how to behave because we're gonna correlate these with motor functions and you don't actually need this many you can experiment with removing some and then seeing how the robot behaves and how the behavior adapts to not having as many inputs this is what we're using we've got 16 of the inputs and these are the correlating outputs now as I mentioned before we gave the motors a value between 0 and 100 so now we've mapped that down to between 0 and 1 and these are the outputs to the motor so when the left light is on and the right light that's full light then the motors are going to move we're gonna give the motors a value of 75 to drive forward but not superfast and as you can see each one of these scenarios is kind of broken down if there's light on the left then have slightly more power on the left motor and the right motor stationary and then if we have light on the top left in the right then we want the robot to drive backwards so right now all we're doing is defining the extreme cases because we have all these extreme cases defined there's going to be situations where they're not all fully light and not fully light in fact it's almost never gonna happen but what the neural networks going to do is going to define an equation that determines what the output of the motor should be and fill in all the gaps between all these extremes and that's what's interesting about a neural network is that it can learn and adapt different scenarios just by training at these specific extremes up here we have some variables that I should explain pattern count that's just the number of patterns possible for the training input nodes we know we have 4 sensors we have 4 input nodes hidden nodes it's kind of up in the air what you should do but I always just do input nodes plus 1 and that seems to work for now and you can always play around with these and see how it affects how long it rains anything like that because the training can take a while output nodes we know we have two motors so we have two outputs learning rate that's the amount it's allowed to modify the weights between the nodes in the neural network so the nodes are the inputs the hidden layer and the outputs each one of the neurons is consult a node and what you're doing is modifying the weights between them that are trying to get the output so what's happening when you're training is you're taking you're taking the predefined outputs and the predefined inputs as I showed you and developing basically a very complex equation to make those two equal on inside of the equation so it's going to go through all the training inputs and the outputs and make sure that they all work and they're very close to what they need to be they're not going to be exactly Oh point seven five for the outputs if we tell it that but it's going to try to get as close as possible so it's gonna modify its going to start with a random set of weights between each connection and then it's going to modify these weights every to every iteration of the neural network so the learning rate is how much it's allowed to change those weights and if it's too high you will constantly have an increasing error or never find the proper equation and if it's too low it could take forever and then if it takes forever it'll give up as well and then finally success is the mean square error which is the basically it's trying to see how many how close it is to the inputs equaling the outputs and that's kind of how you define when a neural network is trained so once the success gets below the mean square error of this 0.0009 then we consider a trained and it should be able to operate properly but you could increase this and see if it works and you can decrease it to see if it works even better it's not necessarily the perfect number here but you need to play around with it and see what happens so this is all part of the program that we took from hobby zine website so they explained it very well there but basically it's a program that is checking this is the training subroutine trained neural network this is basically their code that trains the neural network what it does is it checks each time the air and then it modifies the weight slightly and it sees which direction it should increase or decrease the weights between each node so all these are our iterations for each node between the hidden layer and the output layer and it does this iteratively and that's what takes so much time and that's why parallel operations on neural networks work really well that's why GPUs are used for neural networks because it can do a lot of small calculations at the same time so our we know has to do each one of these on its own in each routine which takes a while and you can see it takes about three to five minutes so it's checking each time and this report every thousand actually changed it to report every 100 and what this does is it outputs to the graph every hundred iterations of no changes to see what the error is and once the error gets below that threshold that we just just redefine the 0.0009 then it stops training it's done and all the weights have been saved in the neural network so now that it's been trained we can select the dried neural network function subroutine what this is gonna do is make sure that it has been trained if not it won't let you run it and now instead of the training set that we use it's going to use our light sensors as inputs so again we read the light sensors we this time we map the light sensors to between zero and a hundred because it's an integer and then we convert it into a float between zero and one and those are now our neural network inputs so we've got this routine input to output and it takes the four inputs that we just measured physically measured from the robot these four inputs are entered in uses those inputs with the neural network and the weights to determine the outputs and that is the output that you get from the neural network and the outputs on the motor control we end up with the outputs with two outputs from the neural network that gives us a value between 0 and 100 and then all we do is send those the same way we did before to the motors and then because of that the motors will be driven and it will console we refer back to the neural network and will constantly give you outputs so let's train this neural network and then we'll run it and see how differently it behaves than the other way we've got two more items on the menu so we've got simple which is the one we ran before which is still in this program we've got trained neural network and run neural network before we can run it we have to train it so when we do this you'll see this will light up and the graph and the graph will give you every time a hundred iterations goes through the graph is going to display the mean square error and it's going to make sort of a logarithmic curve as it gets closer to the value that we need okay so now that it's trained let's make the room really dark and we can see if we can chase it around with a flashlight and see how it behaves so that'll be cool so all of these motor decisions are taking place based on the training data so it works yeah it works pretty well it's trying to avoid the light and find places of shadows now if we're to train it again it may behave differently because each time you train it the equation that map's the inputs to the outputs slightly different because the beginning is random so that's what's really cool about neural networks now you can also experiment with like I said removing some of the training inputs have larger gaps between the extremes so that the behavior could be more erratic or it doesn't actually want to avoid shadows they could just drive around in circles you know anything could happen because it'll be less defined about the different amounts of light it could see so why don't we try that again with the simple program and we can see how that behaves on the shadow you can see that the behavior is similar to the last neural network training now as you can see a neural network is not exactly necessary in this specific type of robot we were able to design a simple program that almost behaved the same way but the interesting thing is to see the different behaviors that happen each time you train it and also that you're not specifically giving it every scenario but that is defining the scenarios between the extremes within the neural network and I think that's really cool and you can adapt this all different types of applications and it's just a great tool to experiment with neural networks and see how they work and you realize that they're not that complicated they're basically an equation that is defining one set of parameters to equal another set of parameters that's what a neural network does so if you enjoyed this project and you want to build one yourself head over to the github there you'll find all the files needed to recreate the entire thing it's fully open source if you like this video series hit that like button and if you have an idea for something you want to see done in the future leave a comment below because I might be able to make it happen if you want to check out my channel I've got all types of projects that are fully open source similar to this they're pretty cool you might enjoy it anyways everyone you know the deal be good and have a good day [Music] we lost a wheel you

Channel: SeanHodgins

Views: 151,331

Keywords: SeanHodgins, sean, hodgins, seanhodgins.com, timelapse, diy, make, build, electronics, arduino, raspberrypi, 3dprinting, idlehandsdev, neural network, neural link, robotics, ann, arduino neural network, robot, programming, artificial intelligence, ai, machine learning, how to, how-to

Id: wtNaPLmpy1I

Length: 43min 35sec (2615 seconds)

Published: Sun Jul 28 2019

Reddit Comments

[u/SoberGameAddict]

What does he use the NN for? I didn't want to watch the whole thing to find out.

[u/[deleted]]

Edge computing is here lol

[u/PM_ME_YOUR_HOG_PLZ]

That was great, thank you! You explained everything so well. I learned so much, keep em coming!

[u/Maminizer]

I got this video on top of my youtube home page and I foumd myself watchimg 30 min even I don't have any experience or knowledge with embeded systems...

[u/LangFree]

Just got around to looking at the Youtube. It is really amazing.

[u/emekaboris]

Awesome, thanks for sharing