Raspberry Pi IoT Server Tutorial: InfluxDB, MQTT, Grafana, Node-RED & Docker

in this video we're going to create a Raspberry Pi iot or Internet of Things server which will allow you to collect any kind of sensor data you like send it over your network to this Raspberry Pi server and visualize it on a nice looking grafana dashboard in our example we're going to use an Arduino Uno paired with a BME 680 sensor to collect the temperature pressure humidity and air quality data every two or so seconds this data is wrapped into a Json message and is sent over our Network to the server that we're going to create in this video and that's done via the mqtt message protocol in this video we're going to focus on the server aspects of this application if you want more information about the sensor node let us know in the comments if you want a follow-up video specializing in that we do have the source code Linked In the description as well if you want to have a poke around all in all this project should take you roughly half an hour to an hour to implement from a top level view this system could consist of a series of sensor nodes that send data in a Json message over your network to this Raspberry Pi via the mqtt network protocol and the mqtt protocol is a is basically where a client publishes data to a broker or a server such as mosquito this is the application we're going to be using in this video and that broker pushes the data that has been sent to it to any client that has subscribed to that topic a topic is essentially a way to categorize the data that has been sent for example I have this sensor in my office so I could use a topic such as home office and any client that is subscribed to home slash office will be forwarded the mqt T message that is sent to the broker in our implementation an application called node-red will subscribe to this topic interpret the Json message data and then send the data to a database to be stored and eventually used in this case we're going to use influxdb to store the sensor data and then we're going to use a grafana front end which is going to query the database in order to display the data we want to see and in a nice format so on our Raspberry Pi server we're going to need four main applications this is mosquito the mqtt broker node-red influxdb and grafana we are going to run these applications within containers using docker running an application inside a container means it's really quick to set up and deploy either to deploy again on the same device on different devices and to remove or manage certain applications containers are not quite virtual machines as virtual machines emulate a machine all the way down to the system Hardware whereas a container runs above the operating system it's not as confusing as it sounds if you haven't used Docker before so don't worry as we're going to use a program called iot stack to get the containers ready for us to use and that makes the process super simple before we get into the tutorial covering how to implement this please do consider subscribing if this video is going to help you out we have a number of interesting iot videos coming up such as a review of this drop-in dip replacement for the Arduino Uno which adds Wi-Fi functionality straight onto the board so stay tuned for that okay so we are starting with a fresh install of headless raspbian OS so that's the 64-bit light version and we've enabled SSH we have a video Linked In the cards above if you need to find out how to set that up I should note that you will need a static IP address for your Raspberry Pi and you can do this by adjusting the DHCP configuration as mentioned in that video the first thing I'm going to do is update everything with sudo apt updates and sudo apt upgrade then we're going to download iot stack using the this curl command this will also install Docker Docker compose and some other necessary packages that you might need so if you get a prompt telling you to install something just say yes to those this might take a little time to download and install but once it's finished it will prompt you to restart and you can do this via with sudo shutdown minus r now it's important that you do restart so that all the environment variables are set correctly you'll get a few errors if you try and run this without restarting to note there is a written guide to this video on our website where we list all these commands you will need to get started so by all means head to there and copy and paste those commands once the pie is restarted log back in and enter the iot stack folder open the menu with DOT slash menu dot sh and use enter to select build stack and this is where we can select the packages that we want to use and we can navigate using the up and down arrow keys and spacebar to select just to refresh your memory we're going to be using grafana influxdb mosquito node red and portena portena is something we haven't mentioned yet and that's going to give us a graphical interface to manage these Docker containers when you select node-red you might see a warning press the right arrow key to the enter the options list and the defaults are fine press enter to build these lists and then navigate to return to the menu and finish your choices this isn't quite displaying properly on my screen that's because I've zoomed in on the terminal to make it easier for you guys to see but it should be fine on yours and once you've selected all of your components so all the five you can press enter again and that's going to build a file which essentially contains all the different components we've set and how to configure them if you navigate to the docker commands section select start stack and this will start all of these containers this will take some time as all the images are downloaded with all these downloaded all your applications should now be running and you can test this with Docker Dash compose PS and that should show you your five containers another way to look at this is to open portana by opening a web browser navigating to the IP address of your server and use the port 9000. if you click on the option to see the local Docker containers you can then click the containers and see a list of them this is gives you a way sort of a graphical interface of how to interact with these containers so you can open things like a console in them or you can turn them off if you want to have a play around with this interface so let's start configuring our applications first things first we're going to create an influx DB database in order to store all of our sensor data let's go back to our terminal and use the command Docker exec Dash it influxdb influx and what this is going to do is enter the command prompt that we're using into the D into the docker container so the commands that we run are executed inside that container we use the command create database and then the name of the database you want to make I'm going to call this sensor data and then hit enter and it'll create the database and we can then exit using quit next we're going to configure node red navigate to your Pi via a web browser by entering the IP address and the port 1880 and this is essentially a visual sort of programming thing where you can build a workflow by dragging and dropping boxes from the left hand panel and linking them up we're going to need three units mqtt input change node and influxdb output to configure the inputs we double click on the mqtt input button and we need to add a server by clicking on the pencil we can name the server and type the IP address of the pi into the server address box leave the port as the default 1883. localhost won't quite work how you expect in this situation so it's easier just to type out the IP address of your server press add server and then select the server from the drop down list we can then type the topic of the data that we want to subscribe to in my case I'm going to use slash home sensors um and we want a past Json object from this block you can also name this block it's helpful if you have multiple of these set up and so I simply use the topic I'm subscribing to in the change block we want to change the transmitted message into a Json expression and we select the Json expression from the list and press the button with the three dots and in this expression we're going to change the keys in the original message to something a little bit more descriptive you can obviously adapt this to the specific data you are transmitting yeah you can save and close this window last but not least we have to configure the output node open the console press the pencil next to the server and add the IP address of the pi and use the port 8086 if it's not there as default then we use the database name that we previously created press add and then we need to name the measurement we're writing the data to you can put what you like on here it's essentially a way of naming the table that everything's stored in I just use sensor data again you can close this panel and hit deploy to save and implement this configuration I've made sure that my sensor node is transmitting data and we can test if this node red configuration has worked by checking that there are some entries in the table if we head back to the terminal and enter the influx DB container again we can enter the commands use sensor data then show measurements and then select asterisks from sensor data and you should see now some data being populated and this means that we're ready to set up our graphical interface if we exit out the docker container and open up grafana the grafana dashboard which is the Pi's IP with the port 3000 it might prompt you for some credentials in this case just use admin and admin and you'll be asked to change your password the first thing we need to do is add a data source if you don't see this on the front page you can hover over the Cog in the bottom left select influx DB and then enter the IP address of the pi and the port 1886 you'll need to put HTTP at the beginning scroll down and enter the database name and then the gets HTTP method press save and test and you should see a nice green bar saying that the data source is configured and confirmed next we can create a dashboard which is where we will see all the information we have recorded select create new dashboard and then add a panel in the measurements box in the from parts of the data source you can select sensor data and then under the field option you can pick the variable that you want to plot and as you do this you should see some data being populated in the graph you can name the panel on the left hand side and click apply repeat this process for as many times as you have data to plot in my case that's four graphs you can adjust plot types and various settings in the right hand side panel so I would recommend having a play around with that and once you're happy with your dashboard do make sure to save it because the number of times I've set something up and then exited away from you know close my browser window and it's not saved this panel is annoying so do make sure you save it so basically that's it you've now created a nice iot server which will take sensor data sent from a node using the mqtt protocol and it'll be plotted in a nice graphical dashboard as a reminder all these commands are written out on our website linked below if this video has helped you please consider leaving a like and subscribing and do let us know how you intend to use this server down in the comments below thank you very much for watching and as always have a nice day

Channel: Learn Embedded Systems

Views: 124,872

Keywords:

Id: _DO2wHI6JWQ

Length: 12min 31sec (751 seconds)

Published: Sun Nov 06 2022