MicroPython - MQTT tutorial on ESP32

hello everyone and welcome to my channel where i teach micropython tips tricks and secrets today i want to announce something very special i just hit 10 million subscribers in binary so 128 and actually we've been growing a lot since and we've been growing so much that i got myself a white board and we're going to be looking at that today for today's topic and what is today's topic you may ask well today's topic is a fancy protocol called mqtt and what mqtt allows us to do is to create an internet of things and internet of devices that are all communicating with each other and these devices can even be tiny microcontrollers because the protocol requires so little data overhead that we can use it on the tiniest devices so let's all head on over to the whiteboard where i can explain the protocol explain what we're going to try to do with it and get right into the code so how does mqtt work well the trick is with mqtt you don't directly connect the microcontroller to say your phone or your laptop you need a broker right here what the broker's job is to do is to capture all messages that come out of any device and send them onto the device that wants them now how does a broker know that how does it know which device wants a certain message let's say we have the blue esp32 over here wants to send a message to the green esp32 to switch on its led well the way to do that is the green esp32 will have a topic what's called the topic in mqtt and that's just a string like a name and that topic could be let's say slash set my led the blue esp32 publishes on that topic and the green esp32 subscribes and the meaning of these words publish and subscriber are actually sent to the broker so the broker knows that the blue esp32 wanted to publish on that topic and the green esp32 want to subscribe and what it does is it handles the trading of those messages so it takes that message from the blue one and sends it to the green one but also notice that you don't only work with two devices in mptt you can actually connect as many devices as you want so for example if you want to connect a laptop and control everything from your laptop in your home for example you could totally do that by also connecting it to the broker but what is this broker like where does it run where does it live and the answer is the broker can live on any computer it can run on your laptop you can run it on a raspberry pi you can even run it on a server on the internet the choice is really yours based on your needs uh for my example since i'm just going to be teaching you i'll be running it on my laptop but if i were to do a home iot setup where i wanted to connect every sensor and every light in my home all to each other i would probably run it on a small raspberry pi and tuck it in the corner and the broker would be passing the messages around one last thing i do want to mention the medium of communication between all of these is just good old wi-fi so this is talking to a broker over wi-fi as is this as is everyone else um the only difference here is that they're using the mqtt protocol to talk to each other rather than http as we are used to from previous tutorials okay so let's get started right away for our broker we're gonna use this awesome software called mosquito with two t's in it so make sure you get the two t's there mosquito.org and we're gonna go click on download if you're on windows get one of these if your mac or linux like i am we're going to use the command line to install it so i'm just going to go ahead and close this for now and just going to type in brew install mosquito since i'm on a mac remember the two t's never forget the two t's all right so for me it was already installed so not very much going on there before we can run it actually let me run it just so you can see what kind of output you get so mosquito mosquito right there version 2.014 starting using default config starting in local only mode this is actually the most important thing here so connections will only be possible from clients running on this machine and that is not okay because we actually want to connect our two esp32s here to our laptop and to be able to talk to each other and they're not local to the machine so we're gonna have to dig into the config file and just change two settings very simple and i'm gonna show you how to do that and then we can get back to coding in micropython and connecting microcontrollers to each other so first off we're gonna kill that we're going to open in a text editor the mosquito configuration file on mac os it's in opt homebrew opt homebrew et cetera mosquito mosquito.conf it might be different for you based on your system so just google around i'm sure it'll be pretty easy to find and once we get in there we see lots and lots of things going on here i have no idea that's fine it doesn't matter go to the very bottom of the file so let me just show you the line numbers here go to the very bottom of the file and just add these two lines so listener 1883 which it just means that we're telling the mosquito broker to listen on port number 1883 which is the default port number for mqtt and allow anonymous true allow underscore anonymous true that's actually super important because that is what allows us to connect any other device on the network to this broker and the broker will allow that connection it's kind of like a cutting down the firewall in a way what's important though here i must say is we're just doing this because we're developing really in a production mqtt system you wouldn't want to allow anonymous connections and you are actually able to set user names and passwords and connect from micropython with that username and password but just because we're developing for now we're just going to set these two settings make our lives a bit easier so go ahead and do that and save the file and we should be good to go now so now when we run mosquito we're going to purposely pass in so dash c means pass in a config file and we're going to pass that config file in and great so as you can see it says configuration file loaded from homebrew etc mosquito et cetera et cetera and it's opening a listener socket on 1883 and it's not showing any message like it did before about not allowing connections that are not on the local machine so we are good here and what we're going to do is open up another terminal instance down here so let's let that mosquito broker just kind of run in the background and let's just give it a shot let's see if this even works so we're gonna try it out before we get into micropython by just setting a simple subscriber and a publisher from the command line and seeing if they can talk to each other note that this would actually work regardless of whether we allowed local connections or not because it's all on my laptop right now so luckily for us we don't need to write any python script yet the mosquito program actually comes with two command line utilities to do a publish and subscribe and we're just going to use those so let's just set it publisher mosquito underscore so yeah subscriber publisher subscribe and you have to give it a topic so v v means just verbose so we get to see all the messages that it logs t means topic and the topic in this case let's say we get a topic of hello simple topic name and you see up here it says new client connected from blah blah blah blah as blah blah blah blah very long address names so this is just referring to the fact that we got this subscriber over here so as you can see the broker is kind of aware of everything like it's aware of every new device that shows up on the network that connects to it it will log a message for that so let's get back down here so we have a subscriber it's not really saying anything we need to open yet another terminal and in this one we're going to create our publisher so we're going to go mosquito underscore pub and same as before we're going to give it a topic so we want to talk on the topic hello so that the subscriber can listen to it and we want to send a message dash m uh my name is blah boom there you go hello that's the topic name and my name is blah and we can actually send a different message on the topic hello so we can just say how are ya and again on topic hello we got the message how are you so that's great our broker is working our subscribers are working our publishers are working so now we can actually now that we're sure of that we can go into the microcontroller realm and try out things over there so as you can see down here i've got two esp32s connected over usb to my laptop just for power though they're not communicating over usb and uh these esp32s are actually just the most generic cheapo esp32s you can get you don't need to buy anything special you don't need to go to the fanciest websites these are probably about 10 canadian dollars each so even less than that in american dollars and they work perfectly fine with micropython and you can just install it using the regular instructions for installing micropython so what we're going to do is we're going to have them communicate with each other and the way i'd like to do it is such that if you press a button on one of them it lights a blue led on the other and vice versa so let's open up our thony editor and let's create a new file and one cool thing about sony actually is you can connect two esp32s to it and just choose between one or the other so you don't they're not really connected at the same time just go over to settings to make sure you know which one you have connected so if you go to tools options and if you go to here interpreter you see my interpreter says micropython esp32 which is great and then over here it gives you a choice of devices to connect to down here on your port or web repo so what you can see here for me is i have actually two potential devices oops two potential devices to connect to and they're just on different usb serial ports so let's just choose one of them for now and stick with that we'll see down here in our shell we can do our regular python coding which is great okay so we're working let's write our main file so i'm not starting from scratch just to let you know if you've seen my previous tutorials i do teach you how to connect these devices to wi-fi first so before you write any main.pi code you want to first connect it to wi-fi using a boot.pi and a secret stop pi boot.i looks something like this where you import your wifi name and password from secrets and then you do a connection using the boilerplate code if you don't do this if you don't know this or if you've never done this before i will put a link down below to my previous tutorial which shows you how to do this so don't worry about it too much for now you can just visit that if you've never done it before for now i'm going to jump right into my main and i'm going to pull in the mqtt library that is used in micropython and it's actually called u mqtt where u is it stands for micro like the greek letter but we don't actually want the entire library we only want something specific which is so we're going to go umqtt.simple import mqtt client we just need the client so the the client is what connects to the broker it's a client broker kind of architecture in addition since we want to work with buttons and leds we're going to bring in the machine library and specifically the pin and also as always we always need our time dot sleep so we're going to do it from time import sleep and what are we going to do first so the very first thing is we want to test if we're even able to connect to our broker right so let's set up the broker connection so to connect to our broker we use the command mqtdclient.connect and it takes three parameters so one is the client name you the this machine can give itself a name and since on the whiteboard earlier i was describing it as blue and green we're just going to call one microcontroller blue and one of them green right so this is the first parameter next is the broker address so this is the ip address of the broker and one good thing about mqtt is that this is the only ip address you need to know in the entire protocol the microcontrollers don't need to know each other's ip address at all they just need to know the broker's app address so in that case this is my laptop's ip address and that is this on my local network if you want to find out yours at least on a mac you just do oops ip ip why keep typing ipa there we go just just hit ip uh it seems like i've been drinking too much beer but here you go that's my ip address and that's what you're going to put in over here right and lastly you need what is called a keep alive time so the way the mqtt works is it has this concept of keep alive in which the broker basically is watching all the devices that are connected on the network and trying to determine whether they are currently connected or disconnected right so if you example unplug your microcontroller it would be good if within a certain amount of time your broker knew that it's disconnected so it's not going to expect messages from it anymore and it can like delete that connection handle and the way we do that is with the keep alive timer so basically we say to the broker hey my keeper lifetime is like 60 seconds if i don't message you in about that much time it means i probably disconnected and let's just say we set it to 60 seconds for the purpose of this tutorial but you can tweak that number to any number of seconds that you like so i'm going to create a client object i'm going to call it mqttc which means mqtt client and it's going to be of type mqtt client which we imported earlier and we're going to give it the name which is blue we're going to give it the broker address and we're going to say keep live equals 60 seconds don't omit the keep alive because it's actually required as a security measure and you'll find that it doesn't really work without it okay next up um there's a lot of logic we want to do with buttons and leds and all that but before we even get to any of that let's just try something a bit simpler so let's try just publishing something in a while loop right let's not even subscribe to any messages so let's just say while true um we're to take our client we're going to publish a message on a topic so that would be let's say oh this has to be a bytes object by the way so we're going to send it as bytes object and we're going to say blue slash button slash 0 and we're going to send it the value of the button so if you remember from our previous gpio tutorial button is just a gpio and actually we should first set that up so we're going to say the button is on pin number zero in my case for this particular device that i bought it might be different for you but i have an i o zero button right on gpio pin zero and uh in fact let's just encapsulate this in in a in a variable name so i'm going to cut that out i'm going to call that button topic and notice that since this blue is kind of like repeated from before i'm just gonna capture that so we're gonna say client name dot encode and then concatenate with that with button slash zero just to say it's button on gpio zero uh the encode function by the way if you don't if you haven't worked with bytes and strings before it just takes a string and encodes it into bytes using utf-8 encoding and if you don't know all that that's totally fine just think of it as turning strings into bytes so we're going to publish on button topic and we are going to send the value of the button so how do we do the value of a button or a gpio in micropython very simple we just go button.value right but again like i said this needs to be a bytes value and well button or value will give you an integer right it will give you a one if the button is high or zero of the button is low but if you want to see it on our laptop if you want to see it be printed out to the mqtt subscriber terminal we actually want to send it as a string so that we can see whether it's a one or zero and so what i'm going to do is actually convert that to a string for that purpose and don't forget you got to encode it so encode so that turns it into bytes and we can that's it we've published a message and we can let's say sleep for half a second okay so let's just do a quick recap what we've done is we've created our mqtt client and we've given the keeper lifetime and connected it to the broker we are creating a button gpio and a topic to send messages about that button on and then we are publishing it so that the broker can see it on this topic and we're sending it the value of one or zero based on whether the button is high or low and we just do that every half a second sweet let us save that as main.pi onto our device i'm going to call that main.pi this file is already open in thani if you want to save it this name close the existing editor first what is going on okay sorry about that i seem to have run into a strange glitch with tony so i just restarted it and we seem to be good now so okay i've saved main.pi let us restart the microcontroller by hitting control d down here all right an error attribute error non-type object has no attribute right and this is line 134 great what's happened ah yes i've forgotten to connect so let's go mqtt client.connect that's all we had to do okay save that control d to restart seems a bit happier and look at that we get a new client connected to our broker as the name blue so that clearly works let's see if it's actually publishing its button value and if we can read that value so i'm going to do a mosquito underscore subscribe on the topic instead of hello it's going to be blue slash button 0 and here we go okay cool cool we're getting on topic blue button zero again one one one one one and that's because this is an active low button so in this case if i press the button i don't know which one it is there now we're getting zeros i let it go it's ones press it zeros fantastic okay that's really awesome one interesting thing we can notice is if i try to kill this code so hit ctrl c to kill it we actually getting nothing down here which makes sense if i actually open a separate terminal and i try to publish on the same topic so i say blue button zero and i publish the message hola will actually get that message so this is the trick with mqtt you have to be very careful about how you architect your system because anyone can publish on any topic there's really no control over it and so it is up to us as the developers to make sure we don't have multiple devices publishing on the exact same topic and confusing us about what's going on so just something to keep in mind but anyway we have our system working and it's really great and it's all working over the wi-fi in my home which is awesome now let's get to the next step let's add a subscriber in the micropython code so that when it receives its system microcontroller's button value it switches on or off the led based on that right so i'm going to go back into thoni and i'm going to now create a subscription topic and how do we do that first let's set up the led so let me add some comments around here so this is the mqtt client setup just so we don't lose it this is the button setup and let's do an led setup the led on my microcontroller happens to be on pin number two so i'm going to say led pin 2 and this is an output pin so we need to explicitly say that so it's a type of pin.out and in addition we are going to create a topic for it so in this case the topic is we're actually just going to listen to the other microcontrollers button so we're going to subscribe to the other microcontroller's button topic let's do that i'm going to call the other microcontroller green so the top is going to be green button zero so whenever on the green button zero we get a one which means no one's pressing the button led is off when you get a zero someone's pressing the button led on and remember we're getting these as bytes which we need to decode into strings so we're going to do that soon next thing we need to do is to create a subscription but when you create a subscription we need to actually set a callback function that gets triggered when a message appears in that subscription so we need to do both of those so let's start by creating the callback function and the callback function is we're going to call it blink led and it's going to take a topic and a message and all your mqtt callback functions need to be of this format it takes a topic and a message two parameters um in this case since we only have one topic i'm not going to check the topic but usually you would you would do something like if topic equals equals something then do this otherwise if it's a different topic do that for now i'm just going to say if message equals one and actually remember again message is actually a bytes object so we want to decode it first into string message message.decode is a one that means no one's pressing the button so we're gonna take our led and just set it to zero otherwise we're gonna take our led and set it to a one so set it switch it on basically that's the callback function here we can set up our subscription such that whenever we receive a message on the green button topic we will blink that will blink or not blink the led so we're going to take our client and we're going to say set callback to the callback we just wrote which is called blink led and we're going to subscribe oops subscribe to the topic to this topic here green button zero that's that's almost all we need to do we have one more step just because we are subscribing and setting a callback doesn't mean all of this really happens asynchronously so that's one of the things about the default mqtt library in micropython it doesn't automatically [Music] run the callback function when a message is received we actually need to manually check for that and the method for checking and running the callback function interestingly enough is called mqtt.check message right so mqtce client dot check let's go message that's it that's everything that is needed to do what we want to do which is listen to the other microcontrollers button and switch on the led now let's just do a quick recap we have just created an led on pin number two which is the pin on my microcontroller where the led is located and set it as an output pin we also have created a topic on which to listen which is the other microcontrollers button since we are microcontroller blue the green one will decide when to switch on the led or not we created a callback function that when it receives a one it switches off the led and when it receives a zero it switches on the led and we've set these up using our mqtt client methods and finally this is very very important you need to do a check message in your main loop notice that in this case a check message will happen every half second which is totally fine and so what we're really doing is every half second we're publishing the status of blue microcontrollers button and checking the status of green microcontrollers button and so we can tell it to sit on or off its light and we also switch on off on or off our light based on its button status obviously for half second so that's that let's try to save it and run it see if it explodes okay so to run it we're going to go to the bottom down here and send a control d seems fine to me we're getting ones if i press the button we start getting zeros and so on very good now of course we don't really get to see this magic actually we do we could pretend to be the green microcontroller just for now and uh published from our from my laptop a message on the green button topic just to test it out so it's actually one of the nice things about the flexibility of mqtt like anybody can publish on that topic so when you're doing testing you don't actually need all your hardware you can just simulate it from your own computer so we're going to do a mosquito pub and instead of blue we're going to do on green green button 0 and message sender one and that shouldn't really do anything because one means no one's pressing the button let's send to zero aha awesome i hope you can see that in the camera yeah so the blue led is on so perfect this the subscriber works we can turn it off and we can turn it on great we are so close we are on the final leg of this we just want to basically copy all this code change all the blues to greens and greens to blues and put them on put them on the other microcontroller and then we'll be able to kind of remote control each other so i'm super excited let's just go for it i'm gonna go in here grab everything copy it i'm going to go into options and change over which macro controller we're working with to the other one using the thawney interpreter options so we're going to go down here and choose the other one hit ok and just close this file just to be on the safe side new file paste everything in change blue to green so that we publish on green now it's the other microcontroller and we listen to blue and everything else is exactly the same so very simple to make changes i'm going to click save and it is going to save to micropython device and again you notice i've already set up boot.buy and secret supply here so that microcontroller is also already connected to wi-fi i'm going to save it as main.pi and yeah that's it now let's restart it okay and it seems to be running we also see a new client connected from this ip address as green so the broker sees it finally let me create a new terminal and subscribe so we can watch kind of what's happening oops mosquito tt sub cool let's subscribe to green button awesome we're getting messages from green button and if i were to switch it off we actually see both we're getting zeros in the terminal and the light comes on the blue light comes on its system microcontroller if you go the other way around if i press the button here the blue light comes on on its friend over there and we also see zeros in the terminal for blue button and finally just to have fun let's just do both of them and check it out they both work so it's a pretty awesome very robust protocol very low overhead works between microcontrollers and any other device my suggestion is if you're ever making a big system where you can have this master node like a laptop or raspberry pi or some kind of central controller that is able to talk to all of your microcontrollers such as a home internet of things system or robotic system or anything like that highly suggest using mqtt because it can get like a lot of fast messages a lot of flexibilities with this concept of topics that uh microcontrollers subscribe to and publish to and uh yeah you can actually build out very complex systems just using this very simple powerful concept of brokers and publishers and subscribers so i really hope that you enjoyed this tutorial i really hope it was helpful to you it was very exciting for me to get these super cheap awesome microcontrollers and do something so advanced with them if you do have more questions or if you would like to learn more please let me know in the comments below or on twitter or on my website where i actually post all of my tutorials not only as videos but also as written articles if you want to reference them later finally i want to say that if you enjoyed this please please subscribe it really helps me because i get more understanding of what people's needs are and what projects they'd like to make so i can make the best projects and also helps you because every time you join and you give your feedback i make projects that will help you accomplish your goals with micropython thank you so so much for watching until next time take care see you bye

Channel: Bhavesh Kakwani

Views: 7,925

Keywords:

Id: BkXWInr-KWM

Length: 31min 43sec (1903 seconds)

Published: Fri Jun 03 2022