Building a Touchscreen Smart Robot Remote

a few weeks ago i built this dedicated workstation out of my old pc and a 3d printed and aluminium case this is an all-in-one transportable computer which basically is running ubuntu 1804 with ros on it the robot operating system that makes my life a lot easier than running on a vm or windows 10 which is what i was doing before in this video we're going to do the same thing but with my everything remote as i named it and this remote control i've had kicking around since open dog version one i use it to control all my robot projects it's just an arduino mega and an nrf24l01 chip that talks directly to the robot it's got two three axis joysticks and a bunch of toggle switches and an lcd that doesn't display any useful information so when i'm driving a ros robot around manually doing mapping perhaps i have to use a keyboard to steer at the moment using the keyboard teleop program now roz is a network operating system so what we really need to do to make a remote to drive the robot instead is put a single board computer in now be really good to have a touch screen as well so we can assign as many buttons and things we want on an interface although it's still good to have some physical switches so let's have a look at the parts we're going to get printing and see what's [Music] inside [Music] [Music] [Music] most of the parts are printed for this project but before we have a closer look at those thanks to 3d fuel for the filament for this project don't forget to check out my channel for more 3d printing projects and check out 3dfuel.com so we've got this box with a big hole in and a lid on the bottom that screws on and we've got these lids as well for these little boxes on the side it's a bit like a nintendo switch other than these are going to be fixed on permanently obviously i've got two three axis joysticks here we've got these buttons which are bit like the menu buttons and the select button on my existing remote and this side where i'd normally have the motor enable i've just got a big emergency stop switch i mentioned putting a touch screen in so we can configure different interfaces so this is the raspberry pi official 7-inch touchscreen and on the back i've already mounted a raspberry pi 4 now that's the monitor cable that goes off to the display so you don't use the hdmi sockets up and these are the cables which are i squared c that goes to the gpio pin and that's for the touchscreen i've bolted these sections together and i left bolt holes to just put a nut and bolt in and obviously we've got to do some wiring for the joysticks and buttons eventually and my screen and raspberry pi should neatly drop in like that and i've left a hole here so we can get to the ports of the raspberry pi if we need to now obviously the monitor is connected anyway with that cable so we don't really need to plug anything apart from power in although we could get to those extra hdmis if we need to the only things we can't get to are usb which are on the back here but i've got a plan for that which we'll look at later there are several approaches we could take to getting the data in from the joysticks and the buttons and sending those as messages over the network to other rolls nodes now roz itself is basically network enabled so we have a ros core that runs somewhere it could all run on the same machine but in my case it's probably going to run on this workstation and roscor is a bit like a directory where nodes can register so they can find each other they connect with topics and send messages over those topics so we could go and get the analog joysticks and the buttons into the raspberry pi now the raspberry pi doesn't have any analog ins so we'd need some sort of extra interface that turns that into serial perhaps we could build something with an arduino or find an i squared c chip with lots of analog ins or something like that then we can write a script on the raspberry pi which will go and take those and turn them into ros messages in our case it will be a twist message sent over the command velocity topic for the sticks and something else for the buttons what i'm actually going to do though is use the arduino ros library the ros serial library and i use that in my really useful robot to connect the arduino that deals with the motor control so we run up a ros node on our single ball computer in this case a raspberry pi we can connect with the usb serial to an arduino and then we can basically have that arduino publish the messages so that will read the analog ins that would actually broadcast the message over serial to our single ball computer in this case a raspberry pi and that would relay the message on to wherever it needs to go or at least publish that topic that things can subscribe to now the raspberry pi itself has got this touch screen so that's gonna have other interfaces so that can still publish its own messages and it could also subscribe to the messages from the joysticks if we actually wanted to display that data on the screen or display the button state for instance or anything like that and we can also get data back to display on here from the robots as long as that robot publishes its message over a ros topic we can subscribe to on the remote i fill in my touch screen in with these plates either side that screw onto the box and they screw onto the mounting points on the touchscreen so what we've got attached to this now is a teensy 3.2 this is an arduino compatible microcontroller with lots of i o that we can use to read all of the buttons and those analog joysticks on the back of my proto board is a piece of header strip and that allows the board to neatly mount onto the gpio header of the raspberry pi now it's not actually connected to that i'm not going to use those connections it's just to mount it and that's because it's connected with usb so that makes a usb serial port and we're going to run the ros serial arduino library and the code on here and then connect over raw serial to the royal serial node running on the raspberry pi and that means we can publish those messages straight off the arduino over serial and then use the raw serial node on the pi to publish those over the network right we're all wired in say that we've got the analogues from those sticks wired into the six analog ins on the team c and we've got the four digitals going to my four digital switches and that includes that emergency stop switch i've just written some arduino code to read those analog joysticks and the digital switches and for now that teams he's plugged in with usb to my computer and we can look at the serial terminal on the arduino ide so we're not using the raspberry pi yet now these joysticks are pretty cheap they cost about 10 pounds and they've got kind of this spot in the middle where the spring doesn't center the joystick properly so we get kind of this place where the value can just vary as you wave the thing around so as well as having read in the analog ins and all the digitals i've also made this little function that takes out any values that are in between 50 and minus 50 replaces them with zero and then takes off 50 from the resulting value so that we still start at zero but we just get rid of that dead spot so i'm just reading those out and putting those out to the serial terminal so now we can see that we don't have a nasty kind of wobbly value in the middle there but if i turn it it still starts at zero and all of those values we can see there for the analog and digital and the other ones are for the digital switches which are just four digital switches so that's all very well and good but what we need to do now is link it to the raspberry pi that's in here and use the ros arduino serial library so that we can publish that data on the ros serial node running on the pi over the network and control ros robots so i've written some code which i'm not going to go into in too much detail although it is open source i'm going to be putting all this code on github as well as the cad for this if you want to build one and that code basically does just that it scales the values down because we don't want to be driving at 500 meters a second generally one meter a second will do so we need to float from zero to one meter of seconds and probably the same for your as well for rotating the robot again we need sort of one or two radians a second and that's about it really when a robot is doing mapping we don't want to be driving really fast one meter a second is probably too fast in fact that's why it's so good to have those analog sticks so it takes those values and it publishes a command velocity topic with a twist message now i'm going to publish the arduino library as well that i've hacked which works with all team seas and i had to hack that for my really useful robot and i've also increased the buffer size so it can in fact send that message over the rolls serial to the roll serial node running on the pie we probably are going to need keyboard access and also a mouse we've got a track pad on this one so i've plugged my wireless keyboard dongle in that can just stay inside and then we don't need to worry about the usb ports i've launched the ros serial node on the raspberry pi here so that's publishing my command velocity topic and i've got the rods core running on my rods workstation we built a few weeks ago so we should see now that if we move these sticks around and doing a rostopic echo for command velocity we should be to see those values changing for both the translation and the rotation axis we should probably try driving a ros enabled robot with this now but before we have a go at that it's time for a quick ad from the video sponsor and that is arduino who are providing the building blocks for innovators to power connected experiences for the future with the opla iot kit and i've got one just here the arduino opela iot kit allows anyone to build internet of things devices and manage and control them from multiple interfaces such as the opela hardware device itself custom web dashboards or the app which is available for ios and android this functionality has been made possible by combining a wide variety of electronic boards easy to use tools a collaborative community and practical project examples to suit all levels the opera iot kit allows you to add connectivity to devices around the home or office and build your own iot enabled projects in the box is the opela iot device itself and piggybacked on the back of that is the provided mkr wifi 1010 board the opla device fits into this plastic housing that's been provided and there's three screws in a bag to screw it in and that means its display will shine out through the front there's also a back here to wall mounted which clips on so you can make an iot device and put it anywhere in your home it comes complete with a set of eight internet of things self-assembly projects ready to show you how to turn everyday appliances into smart appliances you can start with no soldering so it's really good for beginners but the arduino opera iot kit is also a really good resource for more experienced users too the kit is compatible with amazon alexa so you can expand your existing home automation and it comes with a 12 month subscription to the arduino create mkr plan with premium access to the arduino iot cloud you can get it now for 99 euros and i'll put that link in the description to this video right now let's check this remote out and get a ros robot out so let's drive the robot with this obviously we've got that your control which is on my left hand stick when i rotate it which is the same as robot dogs and then the other stick is for driving so that is going to be so much easier for doing mapping at least than driving with keys which tends to sort of turn it on and off you don't get any analog control really with the default keyboard teleop program this is really smooth and i can drive much slower and rotate as i go or i can punch it and uh shoot off to where i want you want to be quite careful when you're doing mapping of course because it needs to look around with a laser scanner and build a map of the entire area but this is worlds away from driving with keyboard teleops so that's extremely good bit of a lump in the carpet there and there's no filtering on this really at the moment it's just driving the wheels the speeds i say so if i stop dead then it still stops there just have to be a bit careful we could put some filtering on but for now i'm pretty happy with that i think that's going to be pretty good it's going to make my ros life a lot easier at the moment roscor's running on this workstation but it doesn't have to of course it could run anywhere it could run on the remote or it could run on the robot for that matter so that means we could just drive this around and not have the workstation as long as we've got wi-fi access points somewhere but that could be a sort of 3g router in my pocket or it could be actually an ad hoc network running between the two but we do need these other functions implemented the switches and hopefully something on the touch screen so we can control other features of not only that robot but robot dogs and all sorts of other robots that i might want what i've done with the buttons of which there are four is actually used one topic with one piece of data in and that's an integer and basically i've assigned each button a value that adds onto that integer which are basically one two four and eight so you can see on the remote workstation now if we look at that topic called buttons and you can actually see that published on the raspberry pi screen as a 16-bit integer if we press those buttons we should see that that data goes up and finally eight and then we can obviously strip those bits out on any of the robots we want to control to find out which button it is that's been pressed we could use the one two three and four but obviously one two four and eight are individual bits in that variable so we could just strip the bits out and then we'd have the data straight away typically i use those buttons on the legacy remote to drive a menu up and down through the options and then press select to select the option and that puts the robot dog into different modes like walking mode or kinematic demo mode or initializing the motors and all the setup i've also got my motor enable and disable switch on the other side which i've replaced with that big red button on the new remote so that's all very well and good for robots that just need a simple menu with several options but actually in the future i want to use this remote for any robot so it could need all sorts of settings and all sorts of other functionality and that's why the touchscreen is so useful because we can draw any type of display that we want and make embedded menus and settings and all sorts of graphical options and even get data back like a webcam view and anything else we want to display on there with a rolls message so for now i don't have much to control but i've written a proof of concept in python that uses the ros pi library and i'm also using pygame which is actually for writing games but it's really good for doing simple graphics drawing rectangles on the screen and putting text on the screen and i'm using that to make the touchscreen work so basically it works just like a mouse and when you click in fact you don't get a mouse button down action anymore in play game but play game supports touch screen so this finger up and finger down for touch screens so i've written a very simple script that displays a few buttons one makes it quit and the other two send some strings which are rolls messages i'm running that touchscreen script on my remote and i'm running the roscore again on my workstation so we can see those messages over the network so if we do a ros topic list we should be to see that in fact as well as the standard ones that come up when rollscore launches we can see one called touchscreen and if we go and look at that topic we should see that if i click on hello it should echo hello as a message on that touchscreen topic and the same with goodbye so that's just a really simple proof of concept at the moment that notices where i'm touching on the screen and we can define any coordinates so we could have a big graphical picture and we could click on different pieces of the robot for instance to activate different functions they don't need to be square rectangles but that's a proof of concept for now and that seems to be working pretty well and of course we can subscribe to rolls topics and display those on the screen in our gui as well so we could have the robot publishing its battery voltage as a rolls message and then subscribe to that topic and display that on the screen and typically with the menu options the remote acts like a dumb terminal so as we scroll up and down with these physical buttons or we use the touch screen to set menu options those menu options will be kept by the robot and it would then publish back the menu state we can display on the screen that means if the remote gets rebooted when we reconnect the robot still kept its mode and that information is displayed straight back on the screen so there's still quite a lot more building out to do as i develop the robot so don't forget to subscribe to check out the really useful robot on the robot dogs and how i integrate rods and how i integrate this remote and build that gui out the finishing touches on this are that it's completely wireless now so i've put a 20 amp hour usb c power delivery spec usb pack on its back and that powers the raspberry pi and it should do for many many hours i've also put this stand on so that it can sit up and i can use it like a terminal if i need to type on the keyboard but of course we can ssh into it as well so that'll make things much easier if we have to do anything too complicated that's the end of this video this project is entirely open source i'll be publishing the cad for this and also the code so far that proof of concept code and the code that operates all the buttons and publishes all the roles topics all of that will be published on github my really useful robot and open dog version 2 are also open source and as those projects proceed you'll see this getting developed as well as the robots and all that stuff will be published so if you'd like to support me through patreon or youtube channel membership those links are in the description below alright that's all for now [Music] so [Music] you

Channel: James Bruton

Views: 54,278

Keywords: robotics, robot remote, building a ROS remote, robot operating system, how to use ROS, ROS robotics tutorial, communicate between Arduino and Raspberry Pi, Raspberry Pu project, Python code on Raspberry Pi, Raspberry Pi touchscreen, smart remote, controlling Arduino robot, controlling Raspberry Pi robot, building a robot, useful robot, make a home robot

Id: ATQblGOjMWQ

Length: 17min 53sec (1073 seconds)

Published: Mon Dec 07 2020