Mosquitto MQTT Broker - Explanation and Setup

hey everybody and welcome back to Jim's Garage in this video the second in the home automation series we'll be setting up an mqt broker with mosquito now what the hell does that mean well mqtt is an open standard protocol specifically designed for iot internet of thing devices it's really lightweight and it acts Upon A publish And subscribe model so you will have some basic sensors that use mqtt because it's extrem extremely lightweight and then you'll have a broker that collects that data and then you'll have clients that subscribe to that data hopefully that's pretty straightforward now in this video I'm going to give you a little bit of a background and some diagrams about how this works and then we're going to be spinning up mosquito in Docker and I'll also share the kubernetes Manifest file for that as well but I'm not actually going to show you how to use it in depth we'll do that later once we set up home assistant and then we start to connect devices to mqtt and then subscribe with home assistant hopefully that makes sense if it doesn't think of it like this you push a button on a switch it sends an mqtt message to the docker container that's waiting and listening it picks up that message home assistant is also connected to the broker so the broker sits in the middle when it sees in the logs a requests come in from a switch to do something you then program within home assistant what that click does so switch one registers a single click you've configured it so that that switch with that ID when it does a single click it turns on a light it turns on multiple lights it turns on your heating whatever you want it to do and we'll come on to all of that later in the series so some of the cool things about mqtt is the fact that it's so lightweight and efficient so using it on iot devices makes a lot of sense where there's often very limited processing power on these microcontrollers and they're often battery operated so it kind of Mak sense you don't want these batteries to run out frequently you'll have to change them bidirectional communication is also enabled which is great for sending and receiving messages across the network it can scale massively to millions of things reliability of messages is also embedded into the protocol it's why it has three defined quality levels zero at most one at least and two exactly once it has support for unreliable networks for persistent sessions and this reduces the time to reconnect the client with the broker and for anyone who's accidentally turned off things like their smart lights or smart device or the batteries run out that's really important because as soon as you put things back in replace batteries Etc you want things to be up and running swiftly another massive benefit is TLS security which is embedded into mqtt so that signals can be sent securely and there's no risk of them being intercepted across your network again very important for things like iot where we don't necessarily always trust the manufacturer so as I mentioned we're going to be deploying an mqtt broker courtesy of mosquito what's mosquito well is probably the most popular open- Source broker for mqtt now as I've said before this isn't necessarily the only product that does this this is what I've been using for years and it's worked really well for me so let's hop over now to VSS code and let's have a review of the docker compos file and I'll discuss some of the options that you can tailor and tweak further if you wanted to now thankfully the docker compost file for this deployment is pretty simple but the configuration file is a little bit more complicated so whilst I'll give you a working config file for this video do go and read up on it and I'll flash that up on screen now so that you can see it but basically there are a whole host of options and I can't guess exactly what your setup needs to be like if you're learning this like I assume you are let's keep it really simple and we can add complexity as in when we need it some of the more complex items are things like authentication TLS certificates having this different listener groups different user groups Etc so that you can start to break up your iot devices probably not required for a home lab setup but if you're doing this in Enterprise or industry or something like that um you probably want to have those feature sets for security and other conveniences so going through the docker compos file pretty straightforward I've called it mosquito I'm using the latest version I want this to always restart because we want to be able to control things even if I server needs to reboot I've put some constraints in there for my deployment so I've set it so that it can only use 256 Megs of ram I think that's fine for a lightweight home lab setup remember as I said at the beginning of this one of the key advantages of mosquito is that it is lightweight but if you're wanting to scale this up or maybe you've got hundreds thousands of devices I don't know what you're running do bear in mind you can increase that or you could remove it if you just want to allow it to dynamically expand mosquito only uses two ports for the deployment that's 1883 and 901 and then we get on to the volumes so the mosquito. comp is the complex one which we'll have a little look at in a moment but the other two are basically where it's going to store its data and where it's going to keep its logs so map those folders to exactly where you need them and I've also set the security option of no new privileges to true you could obviously add things like a Mac VLAN here if you wanted to put it onto its own dedicated subnet I'm not going to do that in this video but I provided examples of that in previous videos so now we've gone through the compos file let's have a look at the config file now the configuration file looks pretty straightforward but there's a whole host more that I'm not even touching in this setup just to make it less intimidating the first thing you'll notice is the allow Anonymous false now what does that do well it prevents anonymous users connecting to your broker you might wish to set this to True when you're just starting up that will remove any form of authentication to your broker so any device will be able to connect and post things to it obviously from a security standpoint that could be a problem very likely for a home setup especially if you're living somewhere detached whatever away from everybody else probably not an issue but we're going to cover the security aspect through using authentication anyway just to make sure the next is to set up two listeners and if you've been following any of my videos you know what they are these are ports that the broker will be listening to now in the docker compose we Expos the two ports the 1883 and 901 So within the application configuration we set up mosquito to use those ports to listen for things coming in those will be connections coming in connections going out Etc next we're going to set it up to use a protocol of websockets which should be straightforwards we're going to say that we want persistence to be true then we're going to set some volumes to be mapped so the idea is that it's going to write this stuff to disk so that it maintains persistence during a shutdown so you don't need to have to change this and reconfigure it every single time like I said this is a very minimalist setup and there are a ton more things under the hood one of the key things here you'll notice is that password file and once we spun up this container we'll go and create that so that we've got a username and a password for connecting devices to our Network and it is recommended that you use TLS certificates for this I'm not going to go down that route for this video that's because often a lot of devices certainly in the home lab space they don't offer you that privilege of uploading certificates to the device and also it might be a case that some devices don't actually allow you to use a username and password as well so you may end up having to just do Anonymous equals true the good news is in a sort of heterogenous setup you could have some devices on one listener that are all authenticated and you could have others on another that aren't authenticated so it can get around that problem anyway let's now deploy this container and we'll create the authentication mechanism okay so I've remoted into my Docker host now I've copied over the docker compost file and I've also copied the MOs mosquito. comp file I've put that into the location in the mosquito folder under config and then the mosquito. file so going back to the dock compos file it's this volume here that's where I've copied the config file because when we run this with the pseudo Docker compos up- D command it's going to expect to find that configuration file but one thing and I almost forgot we need to create the password file so here you'll see that PW file which is expected to be in the config file config folder we need to create that so let's go over to the config let's right click I'm going to create a new file and I'm going to call this one PW file so I'm going to save that it's going to be blank we're actually going to EXA into the Container once it's spun up and then we can create this file I.E we can populate it with prompts from the container itself so we don't need to manually edit anything in here now that we've created that file we actually need to map it so back to our compos file you can see I've added this line here so I've basically mapped that password file from config PW file which is up here I've mapped that to inside the container so that's there and now everything should match up SL mosquito config PW file that's what's in here and that's what's mounted from here which is the folder I just showed on the left so now hopefully when we run the sudo Doos of- D that's going to go away and create and hopefully we'll see some successful log output let's go and check so now you can see that mosquito version 2.0.8 is started we've got a little error message which is just about the permissions of the PW file that should have been set to 700 so do go and fix that but you can see that it's listening on ipv 4 and 6 to 188 3 and the web socket is running on 901 so that's pretty good we should now be able to execute into the container and populate that password file so you can do that using the docker EXA command interactive mode with mosquito running a shell and that will take you in or if you wanted to and keep it simple like I've done previously things like Porta for example you can actually just go in through the web browser which is pretty easy to do all you need to do is click on the console button choose the sh and then click connect and it's exactly the same thing so just choose what you want to do I'm going to do it in vs code because I've had a few people saying why do you use portainer so we're going to do it that way this time but the commands are going to be exactly the same so just choose whatever you like so now we need to create our first user and when we do that it's going to prompt us for a password so I've done mosquito password that's the command it's looking for the config file file so that's the password file the one we specified and I've just called this user one so when I hit return it's now asking for a password I'll do user one just to keep it simple reconfirm and again it's complaining about it being readable so I'm going to have to go and fix that for later videos but that should now be populated and with any look if we look at the PW file you'll see here yeah we've got user one and that's a hashed password so that's great if we want to delete a user we can use a similar command for that we just do the mosquito password command again but with A- D instead of A- C sorry I think I said that was config before it's actually create so we're going to do that and that should have deleted and now if we reload this page we can see that it's now empty so I'm going to recreate the original user just to make life easier for this and now we have a user that we can use for our mosquito broker now as I said at the beginning of this video we're not actually going to be using this just yet we've again built another building block that's going to get us to where we need to be in the end so in the first video I showed you how to connect up the hardware and a little bit about what zigby is in this video we've kind of put the middleware in replace so that that Hardware can send signals and something is listening in the next video we're going to spin up home assistant and I'll give you an overview of what home assistant is and then in the next video after that we'll then start to connect things like home assistant back to mqtt so that when an object is clicked say a switch we can listen for that event and then we can make home assistant do something within our lab by sending a command back so a switch sends an event to the mqt broker home assistant set up as a client it's got some logic built in it will then say home assistant send back to the broker and it will send a signal then back to the device so you can see that whole Loop and we can do whatever we want within that space which is why home automation is so powerful anyway we're going to get on to that soon thanks very much for watching this video please let me know if you've got some other products that you use and how you use them in the comments below but if you've liked this video give it a thumb Thumbs Up Hit That subscribe and I'll see you in the next one take care everybody [Music]

Channel: Jim's Garage

Views: 10,038

Keywords: home automation, home automation system, smart home automation, home automation project, alexa home automation, home automation ideas, home assistant automation, iot based home automation, home automation using esp32, home assistant, home assistant dashboard, philips hue, sonoff, automation, docker, deconz, zigbee, phoscon, mqtt, z2mqtt, alexa, ring, cctv, frigate, camera, homelab, linux, mosquitto, mqtt broker, what is mqtt, mqtt protocol in iot, mqtt tutorial, mqtt protocol

Id: 2S_kZo_ElxY

Length: 14min 38sec (878 seconds)

Published: Sun Apr 21 2024