ESP32 Web Server - Async Micropython Tutorial

hi everyone last time we made this awesome rgb lamp over here and we were able to control it over web apple using micropython but today i want to build upon that and go one step further be able to control this from your phone by running an asynchronous web server on the device itself now for those of you don't know this is running in esp32 with the latest micropython build which is 1.18 at this point and what we're going to do is install the micro dot python library which is very similar to flask if you use flask in python and so if you're familiar with that it makes it really easy to quickly add a web server and play around with things so that you can control your lights or your project using your phone whether it be android or ios instead of using commands from your web repo terminal on your laptop so let's get right into it i want you to go to github.com and microdot by miguel greenberg it's a fantastic library and the library is just two source files so just go into source files you have micro dot dot pi and micro dot async io dot pi and we're gonna need both of them because we're gonna make an asynchronous web server so that it's compatible with all our crazy light patterns so you want to download those and put them in your project folder so here i have the micro dot async io.i and micro.pi now this is just in your project folder we need to actually send to the microcontroller so if you see my last video there is instructions on how to set up web repo and all that so if you are confused about how to do that please check out my last video the link is here um in the meantime i'm just going to connect to it so that's the ip address of my particular esp32 and i'm going to click connect and i'm going to type in my password and web repo is connected so right now i want to send over those two files that's what you want to do as well you want to send your library files over to your project so here micro dot async io dot pi it's uh 12 kilobytes i'm going to click center device okay it's sent that's awesome and we're going to send over micro.pi 32 kilobytes cool send it all right it's sent so now those are on the device so we can use them uh let's get into our code so we're going to open up our thony editor and this is the code i basically have from last time we're going to expand upon it today so we can add our web server in what we need to do first is of course import the libraries before we do that i just want to quickly say this code is just on my computer right now and every time we make changes we want to try things out since i don't have my project connected via usb to my laptop it's actually connected over the wi-fi i'm going to be sending my new file over websockets or web repo every time and we test out the code live by restarting the microcontroller so let's get back to importing the library so we're going to import uasync i o and that's not the microdot library that's the async io library that's built into micropython and we actually do need that if you want to play and interact our led patterns with the web server so that they don't clash and collide with each other so i'll show you how to use that soon and of course you want to get the micro dot library and specifically the async i o compatible version of it and we're going to import this micro dot object and that object is basically the web server it's a class that helps you use the web server let me increase the font size for you there fantastic okay so this is our neopixel setup from last time i'm not going to change anything there and these are the functions we have so we have the ability to set an rgb color we have the ability to convert an hsv or hue saturation value style color to rgb and we have this pattern that i called niagara and it allows you to do these crazy rainbow fun animations so i'm super excited to bring a web server to this what we need to do is before we try to set any colors or try to do any magic on the over the web let's just do the very first thing that any program should do which is hello world right let's do our hello world so let's make our web server basically respond to us with hello world when we go to the main page so i'm going to call this section here set up web server and what we're going to do so we're going to create an instance of the micro dot application and we're just gonna create a route so this is you create routes in micro dot using python decorators it's exactly the same as flask if you use flask before so this is going to decorate this function i'm going to call that hello request so it takes a request object and we're not going to look at the request object we're just going to return hello world boom that's it so if you go to the ip address of this microcontroller it should return you a webpage that just has the words hello world on it and we're going to try that out and so before we do that of course we need to actually start the server so let me define a function that lets you start the service we're going to call it start server surprise surprise and can i just say print starting micro dot app just so we can track that from the web repo console and usually the code is just this so app.run and you supply a port number uh the reason i'm supplying a port number is because the default port number is 5000 but i don't want to type in 5000 every time if you use port 80 that's the default http port so you don't even need to put a port number in your browser it'll just work right but but think about this what if i start the server how would i stop it to do further development how would i pause it that's an important question right so let's have the ability to pause it by basically allowing us to press control c so if you press ctrl c it kills the server you can develop some more code start the server again right so we're going to say and this is just a little hacky python trick but basically this accept clause here is catching any kind of exception which includes keyboard interrupt which is ctrl c so when you press ctrl c it gets an exception it shuts down the app and then you can play around with it and start again so this will help us debug instead of having to constantly unplug and replug the device we can just press ctrl c and gain control of it again so okay this is it this is everything i hope it works let's send it over to the device so i'm going to save that and we're going to send it over just the same way as we've sent everything over so we're going to choose file choose main.pi and we're going to send to device okay sent main.pi 2068 bytes fantastic right now it is running a server so i just hit ctrl c there and usually i would in previous videos do something like this to restart the machine and i realize that is really tedious right like i pretty sure you don't enjoy writing all that every time you just want to restart it and guess what luckily there's a very simple way to do it and i just learned this you just press ctrl d that's it ctrl d which is the end of file which for some reason means reboot the microcontroller and that's awesome so i'm going to go ahead hit ctrl d there and i got disconnected of course i did it's switched off okay so it comes back on online pretty quickly i'm gonna hit connect hit my password right there bam okay we're in this thing has stopped changing colors it is it is stuck of course it's stuck because the server is dead right so let's import main and we're gonna do main dot start ah it has auto complete fantastic main dot start server so let's hit that starting micro dot app okay let's start it now we want that hello world uh you don't actually have to use a browser for this you can use curl but you can use the browser so you know what let's let's use the browser let's see what happens there so let's go 192 168. i believe this is my server right bam hello world oh my god okay it worked and uh the esp32 can now talk to the world amazing you can also do this using curl if you prefer command line because you're a very linux person or something so you can go http um you can just curl that and you get hello world perfect so there's two ways to test it and of course you can do this from your phone as well but i'll show you that at the end when we have all our patterns coded in it's gonna be super awesome all right so next up we are going to now expose one of the functions right so hello world is not very interesting let's expose set rgb so that we can control the color of rgb just by sending an http request so i'm going to create a new route called rgb surprise surprise and we're going to say rgb request and what we're going to do here we are going to set rgb of course there that we just need to call that right so we need to take the parameters from the request and set them there and these are going to be like get parameters so just to kind of give an idea what i'm talking about i'm going to put in a comment here like someone's going to call this so ip address slash rgb and you might have seen this if you use the web before if you're used to apis you just do like a question mark to show that these are your query parameters so r equals some number and g equals some other number and b equals yet another number so this is what you're going to have to type into your browser or if you're using your apps it'll be a bit easier than this but this is a classic http get request and here we can actually use that request object so this request object is going to contain a list of arguments and those arguments are going to be this r g and b value so let's do that i'm going to leave that in there all right so we want to go set rgb call this function up there basically and do you have to provided a tuple or a tuple or a tuple of rgb values so it's gonna look like that and uh the thing about http requests you have to realize is the values actually come in as strings so we need to convert them to integers first right so we're going to say int request dot args r r and so on and so forth so i'm just going to copy that g and b and let me just make sure our brackets are good perfect cool we're gonna do that and with every request whenever you use a web server you should usually return something return something so that the client that made the request knows that hey everything was cool the server didn't freak out so i'm just going to return something simple just return ok ok cool you have said rgb right so let's try that out so what we're going to do is we're going to send one of these from curl and we should in theory it should set the rgb to those three colors and then should return an ok which we should see in our window i'm going to save that now that our file is saved we need to send it over webrepo so we're going to click choose file and that's our new version of main.pi make sure to click choose file again even though it already says main.hi there from last time i'm pretty sure that's the old main.pi so do it again send to device sent main.pi fantastic okay let's restart this thing so we hit ctrl d okay so it's actually running the server right now so hit ctrl c first to kill the server and oh disconnected what's up and hit ctrl d i see it comes back up pretty quickly i like that all right and so we're gonna start the server this time we're going to test out if our rgb works cool so let's hop into the terminal again and use curl so this time we're going to say rgb r equals let's just try setting it to red so 255 and g equals zero and equals zero cool hit that oh okay yeah i saw that out of the corner of my eye fantastic it works oh cool so that's taken care of so we have a working server it serves rgb requests and it gives you a little hello world let's take it up a notch let's do hue saturation value so what we're going to do there is basically the same so i'm just going to quickly bang out the logic for that um so we want a new route route route called hsp make sure to put these leading slash there and i'm going to call it hsve and honestly let's just let's just copy that code because it's almost the same with the slight difference of course we're going to call a different function so the example there i really want to be able to say hsv equals h saturation and value h actually can go all the way up to 65 535 right so let's just put in a bigger number here just to make it very clear uh what's going on so instead of setting rgb directly we're actually going to convert the input parameters into hsv first using this function up here it takes a hue saturation value converts it to rgb and then spits it out and actually i think that's it that's the entire function right there um yeah let's try that uh send it over choose file i mean not by let me connect there i'm not sure why it disconnected okay let's send to device okay and hit ctrl d over here to restart that device connect again all right run the server at this point i'm pretty confident in the server but i'm just i still do like the ability to run it from here but eventually when we're done we're just going to start the server automatically on boot up so that we don't have to keep hitting main.start server so let's do that starting macro.app awesome let's curl it with a new request so instead of rgb we're going to do h equals let's do my favorite color which is a very nice minty color and saturation equals full and brightness just so that i don't blind you i'm just gonna turn it down to maybe around 100. go internal server error aha i forgot to change this to hsv internal server error again okay what's up it doesn't like it why is that hsv did i do something wrong here i've got root hsb hsb aha the code is wrong who would have thought copy pasting things results in probably 90 of errors doesn't it all right let's try that again so i'm going to kill the server ctrl c send over the new file so it's actually awesome like i'm sending over new code like over there update and not having to like flash it and compile it and all that like i would have to do an arduino so honestly this is really fast okay cool import main start server let's go let's try our previous minty color again yes ah fresh minty fresh okay great so that's all the basics so you can already use this web server you can change colors you can do all this just from simple http like api commands which is great now i want to bring on something much more complicated which is that niagara falls kind of animation and all the parameters it has so that animation the difference between that one and this one for example with this one we just send one value to the new pixels and then that's it it just the code stops interacting with the new pixels whereas for the niagara falls one there's every few milliseconds it needs to send a new pattern a new pattern a new pattern right and what's going to happen is if you try to do this the normal way so by normal i mean synchronous so you send a request to the web server hey start the niagara falls animation and if i just ran the niagara falls function it would block the whole microcontroller so the microcontroller's entire scheduler like time and resources would just be spent on sending new patterns new color patterns and the web server would not be able to take any more requests from me anymore right what we want to be able to do is we send a pattern and then the web server is still running and they're coexisting side by side and that's where the asynchronous comes in so if you've worked with javascript before you might already know how to think like that but in python it's not very intuitive necessarily so i'm going to explain how asynchronous works by running this web server so what we need to do is first of all you notice that the function set niagara is a normal synchronous function when you use the library async io you actually get new keywords async and await that allow you to tell the interpreter that this function is asynchronous right and it's very simple asynchronous function niagara and why do we need to do that basically we need to do that so that this time dot sleep down here this is the part where we can get asynchronous this is the part where we can say instead of doing time.sleep and blocking the microcontroller what we're going to do is in this point we allow other stuff to run so we're going to allow the web server to run right here so while this function sleeps other functions can do stuff and the way to do that is actually very simple you go uasync io does sleep milliseconds and you need the await keyword right there what does this mean this means wait till we sleep for these many milliseconds you know and by weight i mean go do something else say go do whatever you want run a web server or run some other function and then once this few milliseconds have passed come back here and then run this loop again that's what we're doing so we're really just saying like forget about me forget about me for like 50 milliseconds don't worry about it you know that's what we're saying um in order to use that down here we also need an asynchronous call to it so i'm going to make a new route i'm going to call it niagara and we are going to take in our request and do something similar as before so i'm going to be much more careful copy pasting this time what we want is ip address and then over here it will say niagara and these parameters are all totally different right so a couple of things here we don't actually want the user to have to set all the parameters every time because we have defaults right we have a default saturation default value default of everything actually let me go change that to something faster and smoother 10 milliseconds let's say you know default hue get happen hue cycle speed so really if if i want to just say http ipad niagara i could do that and how how are we going to do that so basically we need to treat these optional parameters separately so let's try to start by grabbing those so here just as an example i can set the saturation but i'm just going to put in there all prams are optional okay so how do we do that we're going to say we're going to create a dictionary of arguments args dictionary equals the reason i'm doing this is because so the request.args is technically a dictionary but all its values are strings not not numbers right and actually it's not even really a python dictionary it's like a separate class called multidict so i just want to avoid all that and just bring it into like a normal python dictionary so we're going to go for key in request.args.keys so for every key such as sat val hue delay cycle whatever all those keys we're going to say we can now just put it in our dictionary so artistic key equals request and with the only difference being of course that we need to convert that to an integer so we're going to say integer of that so for example you get saturation equals 230 in quotes like a string this will just put saturation equals 230 into this dictionary and why am i putting in a dictionary though right that doesn't this doesn't this function doesn't take a dictionary uh so this is where some really cool python syntax comes in it's similar to the spread syntax in javascript where basically you can take a dictionary and spread it as the function parameters using asterisk asterisk and i'll show you that it's pretty simple so we're going to say this is where it gets very asynchronous so use it create task so when you have an asynchronous function basically something like that you don't call it directly you don't just say set niagara you want it to be called asynchronously right so you're creating a task so that it gets called by the scheduler and the task is set niagara and this is where we're going to put in our rx dictionary but again like i said it doesn't take a dictionary so you need to use asterisk asterisk syntax to convert that dictionary into a list of keyword arguments so it'll look like this and that's all there is to it it'll work and we of course want to return okay okay right uh lastly don't forget to define this as an asynchronous callback so yeah as soon as the user goes there it's gonna get this request we're gonna convert all the parameters in the request to integers and then we're going to call the set niagara function using uasync io.createtask so that is called asynchronously and return okay this is almost good there's one problem with this but i'd rather show you the problem than try to do this so i wonder if you can guess i wonder if you can guess what the issue is all right let's try it out let's go to our web repo i'm gonna ctrl c to stop the server there ha sent to device okay it's sent so i'm gonna control d to restart it all right import main main start start server let's go okay moment of truth so we're gonna this time curly niagara request and let's just do the default niagara quest like we don't have to worry about the parameters too much for now or actually i'm gonna set the brightness loader so i don't blind the screen the camera okay that's fantastic look at that okay it works right it works right right okay let's try what if i wanted to change the brightness seizure alert okay better stop gonna stop the server ctrl c ctrl c okay i've lost it okay i'm gonna go unplug it okay it's off cool what happened there basically what happened is is this i'll show you in the code this line right here uasync io.createtask we're creating a task to set the niagara led pattern right which is great and then it goes off and asynchronously does its thing and like changes the colors now when i go hit this root again so if i want to say change the brightness or change some parameter it creates another task it creates another task so it's actually running two set niagara's in parallel and this is kind of where like you need to be careful with concurrent programming like this right what we need to be able to do is every time our new request comes in we want to delete the previous task right we don't want to set niagara's running at the same time and so the way to do that is fairly straightforward we need to just track the current task so i'm going to create a variable called current task and that is for now none and the micro dot library is actually pretty cool it allows us to apply a certain function to every request that comes in and basically what we want is we want a function that takes every request and kills all current tasks or the kills the one current task before servicing that request and for that we have a decorator called app.before request which is very well named in my opinion and this is going to be an async function and i'm just going to call it pre-request handler even though we're never going to call that function ourselves so yeah whenever we get an http request if there is a current task then we're going to current task dot cancel and cancel is again something from the async io library so just how we have a create task we also have a cancel task and this is how you do that so that's that's that so hopefully this time when we run niagara we run it again a second time it cancels the first one but just make sure that we store this so we need to do global current task and the reason we need to do global is because we're assigning to a global variable so you need to do that in python otherwise you'll think you're creating a new local variable called currenttask so we're taking our current task which is the same as the one up here that's why i have to put the global keyword and then we're going to assign it this new task that's created so the way it's going to happen is http request is going to come in it's going to go here it's going to cancel the previously running task if there is one and then it's going to hand it off to here if it's in niagara for example and we're gonna just do our regular thing where we create a task all nighter should work should work let's try it all right let's connect hello i have missed you sorry about what i did to you last time okay sending the new file and import main i mean let's start server let's go one last time all right so let's set it pretty low brightness okay very good very good now let's set it higher brightness or actually let me leave the brightness but change something that you may be actually able to see much easier which is let's go let's change the hue gap let's change u-gap to like very small gap maybe 50. no more crazy flashing so it cancels the previous task and started a new one and now we have a totally new pattern and so this is great you can actually play around with this a lot you can change hue cycle speed make it negative which is really interesting so you can make it the pattern go backwards let's try that so yeah not i mean i don't know if you were tracking but now it's going upwards instead of downwards so anyway that's great so that's actually working lastly what we want to do is make sure that we start the server on startup so that i don't have to go into webreport every time that i have to unplug this for example so that's pretty straightforward just right at the bottom there before anything happens start server on boot up this is all it does just starts it that's it we're going to go here oh and kill the code ctrl c send our final and latest main.pi and ctrl d and this time i don't even need to control connect to web apple again that server has started so we can try it let's set it let's change the hue gap to something much bigger and that's fantastic it works awesome so that's everything from the computer side now i did promise you that i'll show you how to control all this from ios and android so i'm going to go ahead and do that and do some screen recordings for you to check out okay so on ios you have this app called api tester and i chose this one because it's free and it doesn't have ads or any payments or anything so you click on open that and what you can do is you get to make like a list of requests that you would like so you don't have to keep typing in again and again the entire thing so for example let's go into this hsv one and what i've done is i put my ip address over here and then i've put my query parameters over here hsv and it automatically populates those fields with that so for example let's change that to 80 the brightness and let's reduce the saturation down to maybe 230 and i'm gonna go to my favorite minty hue right and just hit play in the top right corner there aha done and you get your okay response over there just like you did on the computer so that's great and you can do the same with niagara and you can actually store all the five parameters here which is a lot more convenient than having to type it in every time similarly on android you have this app called http http http er and again it stores your the ip address and the endpoint that you're hitting also you can have your parameters down here so i'm going to go ahead and yeah i can just send it as is just so we can see this animation so you hit set request at the bottom and there you go you get an ok response and the leds are working so there you have it you learned how to use microdot with asynchronous programming to make complex led patterns callable from your phone or from your laptop and you could use this kind of thing to build apps to build web apps to control your lights really any way that you want the http protocol allows you to send in your parameters as you wish and also if you want to build upon this even more you can make the code more secure like it could actually check the values of the parameters make sure you're not sending numbers that are out of range to protect you from crashing the the code right so there's a lot of lots of ways to build upon this it's really awesome i think like having the esp32 gives so many possibilities of connecting your project to the web and connecting it to tweets to weather apis to whatever you want you can you can control it exactly how you wish i hope you enjoyed this tutorial and make use of it to make awesome projects and if you have any questions please let me know down below if you'd like to see more videos like this please subscribe either here or on my website where i explain in text all the details of all these projects and i'm really happy to have taught this today thank you so much till next time bye

Channel: Bhavesh Kakwani

Views: 10,875

Keywords:

Id: pOB3mH8G2q4

Length: 34min 6sec (2046 seconds)

Published: Sat Mar 19 2022