Run Your own LOCAL MQTT broker to control your IoT devices in Home Assistant

today we're talking mqtt [Music] so recently i had a request to make a video about cloud-based mqtt services and i tried i tried to use those services there were a couple of caveats to that though which drove me to do my own local mqtt server the first is you need to make a tls connection from your devices over to the mqtt server that presents a problem because some of my devices will not do that the second is cost some of these services are free however some of them require you to pay for them for the paid for services if you're capable of doing this yourself and you don't have a use case for having a cloud-based service then you don't need to pay for it because you can do it for free now i realize that the services are in business to make money that's fine i will support them when i need to but for what i'm doing i don't need to pay for a service like that and third it's cloud-based which means that if you lose internet connection you lose access to your mqtt broker and then all your stuff potentially will stop working keeping it local is the way to go so why do we want to do anything with mqtt well mqtt is the backbone of many iot devices and how they communicate to their controlling hubs or whatever else in our case home assistant so you need an mqtt broker to handle that communication between the devices and whatever is controlling them as an example let me show you a little bit of what i'm running with mqtt most of these items here these topics are coming from mqtt based devices so we have a lot of taz moto devices here that i run for sensors and and controlling different things i have my nest devices running over mqtt all of my z-wave devices talk over mqtt and that you can see is a huge portion of my network and there's some other stuff here that we also use such as frigate for uh my cameras and notifications of of things that go on with the cameras mqtt plays a big part in my iot and home automation setup so it's important to have a good mqtt broker now you can run mqtt directly in home assistant there is an add-on for that and in fact that's the way i'm running it but i do want to show you today an alternative for running an mqtt standalone server on another piece of hardware within your local network so we're going to make a couple of assumptions here the first assumption is that you have what's called portainer installed and i'm running pertainer on a vm so if you look right here this is my pertainer instance and i'm running a number of containers here what i'm going to take you through today is i'm going to take you through actually building out the mqtt container and how to configure it so that you can talk to it from your iot devices so let's get started into that and i'll go through a step-by-step process for doing that because this is a docker container that we're talking about the first thing we want to do is create some storage so that if you remove and recreate this container for some reason that all of that data is persistent on your uh your machine or whatever you're running the docker container on otherwise when you destroy a container you will lose all of the settings and all the retained values that are in that container if you don't have some sort of persistent storage so to do that we're going to come over here and we're going to create some volumes so let's go ahead and build out some volumes and you want to build out some specific volumes because you're going to want to have data you're going to want to have configuration and you're you're going to want to have logging so we're going to add three different volumes the first one we're going to call mosquito or mqtt for short config and all these options will remain the same there's nothing to change here we'll create another one called mqtt data and we'll be using these later on that's why we're doing them now and then you want to do a final one mqtt log and we're going to create those volumes let me make this a little bit smaller all right so now we have three different mqtt volumes that's where our data is going to go our configuration is going to be read from etc this is all very simple if you're using pertainer because you can just follow through the ui settings or follow through using the ui you can also build this directly on the command line if you want to ssh into your vm or whatever you're running this on and build it that way but i'm going to show you the ui version since a lot of us are used to using the ui and home assistant and other things this makes it a lot simpler all right so we're going to click on containers over here and we're going to add a container and this is where it gets a little specific on some of these options so just pay attention here i'm going to call this one mqtt and the registry is docker hub and this is where it's going to pull the container from or the docker image it needs to have an image in order for it to know how to build it and the the uh the docker hub has tons and tons of different images basically a library of getting all of your different stuff and we can search for let's just say mqtt for example and this is the one we're going to use but there's dozens and dozens of different types of mqtt related containers so we're going to actually use the eclipse mosquito it's a dash this is the container that it's going to go get from docker hub and put onto our local vm we always want to pull the image and of course it says here if you're using an anonymous account you will be limited to 100 pulls every six hours we're doing this once so you shouldn't have to worry about that unless you've been doing other stuff i'll leave this all the same uh we're going to come down here and do a couple things number one i want to have an interactive console so i'm just going to select that at the beginning volumes is important this is where we're going to we're going to map those volumes that we created to locations within the docker container itself so that this docker container knows where to get or store the data so we're gonna map three different volumes so i'll click it three times and the first one is going to be slash mosquito slash config and we're going to select a volume from those that we have already created that's why it's important to create the volumes first this will be config the next one of course will be a mosquito and mosquito is spelled funny so make sure you spell it correctly it's two t's is that how you spell it in real life well anyway two t's for mosquito on this anyway data will be this container and we are going to choose that volume and then finally mosquito with two t's log and we're going to select the log container so those are the three volume mappings that we're going to create for this the network we want to change one setting here and that is to make this the host the environment we're going to leave alone labels alone restart policy we want to restart this anytime the system restarts unless we have explicitly stopped that mqtt container you can do never always on failure or unless stop if you wanted to always restart no matter what let's say you stop it and then you reboot something uh in your system here and you want it to automatically restart you want to do always but i'm just going to set for unless stopped all right runtime resources and everything else will stay the same then you just click on this deploy container button you should have a nice pretty green uh successfully created box up here and we see that it is running so we have now created volumes we've installed the container and we've started up and running in specific mode what i want to show you first actually is i want to show you the console access here i want to show you what it's doing or we're going to open a console with the bin shell environment and here it is right here let me shrink it a little bit and all i want to show you now is i'm using a command called netstat and it's going to tell me what ports are currently being opened on this docker container for things to listen to and i want to look specifically for port 1883 because that's the mqtt port and you can see here that this port is currently open as default settings are set up in the the broker or the mqtt container these this only listening on the local host that means if i try to connect to this with any application or any uh device from outside of the the local container here the local host it's not going to do it it won't reach it because it's not listening to anything but the local loopback interface so what we need to do now since we're currently running default settings we need to go into our configuration and we need to set up the settings we want to run with this mqtt instance or this broker's instance so we're already on the the system in a shell for and we're in the docker container environment for the mqtt broker so i'm going to first see where i'm at i'm in the root directory i want to change to the mosquito directory and i want to change further into the mosquito config directory and now if i do an ls which is a directory listing i can see that i have the mosquito.conf file that is created by default from the container when you download it from the docker hub it's going to be full of all kinds of stuff and we can look at that real quick let me just do less on this one you can just see there's tons and tons and tons of settings in here that you can set up with within this mosquito broker configuration we don't need to do all that stuff right now we're going to make it simple i'm going to show you configuration that i use that will allow you to connect to this mosquito broker anonymously from anywhere that it can be reached in this network so the first thing to do is i'm going to copy that mosquito dot com file to a backup so that if i want to look at the settings in there i still have that as a reference so that's now copied over i'm going to edit the original configuration file and i'm using vi and there's a shortcut in vi to delete delete all of the or delete lots of lines at once i'm going to type 1000 so 1 0 0 0 dd i don't know how many lines in there well there's 901 lines so i just deleted 901 lines from the file i mean it's a shortcut i use all the time so you if you have your favorite way of editing things or using nano or something else that's fine so now i'm going to hit i for insert and i'm going to copy the configuration that i'm using and i'm going to place it right in here and then we'll talk about it briefly so i've got that copied and i'm going to paste it and this is my current configuration persistence is true persistence location is mosquito slash data that's why it's important to map those volumes because that's where it's going to store that data if you remove this container and you map the and you create it again and you map all of the volumes the same all of this data will still be in that same place as long as you have this persistence location set we're going to run as user mosquito we're going to listen on port 1883. this one here allow anonymous true that lets connections come from anywhere on any interface if you set it this way also it doesn't require any kind of credentials that connect to it then our destination for logs is here so i'm going to hit escape i'm going to hit colon wq and i'm going to save it now we have saved that but it's not using it yet so what we need to do next is we need to i'm going to go ahead and control d and disconnect out of here go back to my mqtt container up here and i'm going to restart it so click on restart and i always like to look at the logs to see what's going on and it'll tell me whether it's starting or if it has any issues if you mess up that config file it will show here that it didn't start correctly you can either go log into the terminal of the host that's running this docker container or you can just start over you can delete the container and start again which might be your best way to deal with an issue all right so everything's up and running so let me go back into the console again and i want to show you that i have the ability to listen to it or connect to it from everywhere so now if i do that same netstat command we're going to grab 1883. now instead of local loopback interface it's now listening on all ip addresses and you can actually control this with the listen command if you only want it listening on a single interface you can control it with the listener command you would put the port and then you would put the ip address after that of what you would allow it to connect from or where connections could come in from okay so now let's take a quick check here this is my production but let me disconnect from this one uh there's no username or password here this is the host name or the ip address and the port of my new mqtt instance or broker that i just set up i can click on connect and now i'm connected to it and this is without any kind of authentication it just lets me connect to it because that's the way i have it set up so now you have a successfully deployed mqtt broker running on a docker container in your virtual machine or wherever you want to run it all right so let's just say that we don't want to allow access from anywhere without some sort of credentials so what i'm going to show you now is how to update your configuration to provide credentials so that you can restrict access from people connecting to this and it's a simple command we're going to use a command and we're already here on the console so we'll just stay on the console this is a command called mosquito password pass sswd and we're going to create a new credentials file so dash c and we're going to put it in our configuration directory and we're going to call it the file credentials and you need to remember the file name because you're going to use this in a minute in the configuration and then i'll take a space and i'll give it a username so we'll just use the username mqtt for now hit enter it's going to ask me to create a password and verify it i have now created a credentials file so we can look at that in the config directory actually so you can see here that i have a credentials file and this credentials file contains a hash password and a username so that you can set it up to only allow connections uh with that password and our username and password and while we're in this configuration directory we're going to make a couple of changes to our configuration file so let's go back into vi uh of or let's vi them configuration file and we're going to first change this allow anonymous we're going to change this to false so i will just make this false it will now no longer allow anonymous connections it requires the password and the password file we're going to define that and we're going to give it the path name relative to the docker container so mosquito config credentials and we're going to write those changes and we're going to get back out of here again and we're going to restart the container so let's restart it there we go we're restarting the container and now if we look at the log let's make sure we didn't mess anything up all right so it's up and running everything's good to go and what we want to do now is i'm going to pull up the mqtt explorer again i'm going to try to connect there's no username and no password here so let me try to connect and now you get a connection refuse not authorized so now we put in the username and the password that i generated with that generation command and that's saved in the credentials file we're going to connect to it and there we go now we're able to restrict access to that mqtt broker uh without a username if it doesn't have a username or password now there's further options you can set up if you want to do certificates tls encryption or anything else that will prevent snooping on there this is running on my local network so there's a little modicum of security there because it's not exposed to the outside world but you can set up encryption between the mqtt broker and any devices now you have to have your devices have to be able to do uh tls connections or whatnot so how do we know it's working well first of all we can see that it's working here but now what i want to do is i want to show you one of my devices that i use i've got a little um temperature sensor that's got tasmota on it tasmota is a big user of mqtt for all of its communication so i'm going to configure that to show you how it does talk to my new mqtt broker so let's go into that device so we'll open up a new tab here and we'll go into the device itself so this is a one of my plugs my s31 plugs and i'm going to just change the configuration and point the mqtt uh host to a different device and i'll change the username and password as well so this is 172. and 161.1 this is the host of my vm that's running mqtt and because i'm running in host mode on the container it's going to go directly into the support 1883 which is what we have set up and i'm going to change this to mqtt for the user and then the password that i set up we'll save it and let's jump over to mqtt explorer and after that reboots and does an update we will see right here that that device is now talking to my new mqtt broker and we can get you know various stats and things that are coming across on here from that device and you can see what it's sending over to it right now so it's a very quick demonstration of it actually taking in data from a device and putting it into the mqtt broker well let's say we want to use this mqtt broker with home assistant well in many cases once you have this on the network if it's on the same network segment as your home assistant instance home assistant's actually going to discover it it may take a restart of home assistant but after it restarts you should be able to see it and one thing you can do to tell is see this notifications down here and you'll see up here that you have new devices discovered check it out and here it is mqtt so you can configure this directly from the uh the interface you don't have to go into any ammo to do this so we're going to put in our new broker address which is the 153. port 1883 is where we are uh or we're at and then mqtt is the user and then the password that i set up for it and we'll submit it and it's finished now and now we have an mqtt broker and what we can do with this is we can go into configure and we can just kind of listen to everything that's going on within the mqtt broker so if you have a good connection once you do a listen to uh basically everything you'll see that you have stuff coming in from the broker and this confirms that you're now connected to the broker now the one thing you'll have to do with home assistant if you're using this method is in most cases the actual mqtt entries are not built for you so you're going to have to do a little yaml work to actually build the mqtt entries that's out of scope for this video something we can do in a different video if you would like just let me know down in the comments so now you have a working mqtt broker that you installed yourself on a virtual machine that you can put any of your mqtt related traffic into and then also take home assistant connect to that and be able to pull in information from a home assistant and also send out commands from home assistant to the mqtt devices through that broker so thanks for watching let me know if you have any questions in the comments down below i really appreciate my channel members for joining the channel as well you'll see them uh over here over here right here going up past my head and then if you're not a channel member i really would appreciate it if you would join the channel it really helps support what i do here and you can also talk to me on discord if you would like to i answer questions there as much as i can as well so have a great rest of your day or evening or whatever and we will see on the next one [Music] you

Channel: mostlychris

Views: 50,087

Keywords: Smart Home, Home Assistant, mqtt, eclipse, mosquitto, eclipse mosquitto, smart home, IoT, mqtt broker, broker, portainer, docker, install docker, install portainer, eclipse-mosquitto docker, eclips mosquitto docker, smart home automation, mqtt tutorial, smart home technology, home automation, mqtt tutorial youtube, mqtt explorer tutorial, mqtt explorer

Id: juSoczXtlxA

Length: 19min 57sec (1197 seconds)

Published: Sun Apr 17 2022