#352 Raspberry Pi4 Home Automation Server (incl. Docker, OpenHAB, HASSIO, NextCloud)

the raspberry pi is grown up now to be a perfect home automation server today i show you why and how to set it up conveniently in the end you should be able to make an educated decision for your own project crazy youtubers here is the guy with the swiss accent with a new episode and fresh ideas around sensors and microcontrollers remember if you subscribe you will always sit in the first row the raspberry pi started as a simple learning device in version 1 and came a long way to its newest iteration the pi 4. the two things never changed are the form factor and the price all other parameters followed moore's law which predicts a doubling of capabilities every 18 months for silicon chips today i argue that the pi 4 reached the level of an ideal general purpose home automation server as usual you can comment your opinion but please only if you saw all my arguments in today's video we will focus on a home automation server built with the raspberry pi 4. go through some of the available software choices for the pi for that purpose all used software is free of charge discuss ways to simplify installation and update show that the separation of programs and data has lots of advantages and how it is done look at which hardware has to be used and how to get it running here it will become apparent why the pi 4 is the way to go we will walk through a setup of such a server beginning with a homemade sensor to a neat display of the measured data but first what are our requirements what do we need i propose the following ingredients which of course you will discuss in the comments or on the discord channel the first and most important interfaces to various sensors and actuators homemade and commercial ones without that home automation is not possible we do not want to have a proprietary cloud solution as proposed by many manufacturers a dashboard to see the status and from there to branch into various departments to see the details a logic engine to connect sensors and actions transparently and flexibly primarily we need a programmable logic to get our many jobs done a graphical interface to show sensor data our server should be reachable from our smartphone alarming functions for two cases either something happened or one sensor does no more work both need our attention maybe a secure remote access and maybe also a private cloud with shared data like photos calendars etc let's start with my first argument for a raspberry based server the availability of software on raspberry os you have the widest choice of open source and free software for home automation and with this fact comes the dilemma you have to decide about what to use we basically can go three different ways focus on a single software package like home assistant openhab or fhem they include interfaces to many sensors as well as dashboards smartphone connections etc etc usually they are not very good at visualizing data or sharing data in a cloud but for core home automation they are an excellent choice maybe you share your experience with one or the other assemble your own su of software by installing one after the other on raspberry os this is of course the most flexible way but you need a lot of linux know-how and keeping it up to date can also become a nightmare use a pre-built raspberry image as provided by me in the past this is the most comfortable way because you just burn an image on your sd card and everything works this can be a good starting point for beginners and also a fast start for your own sue unfortunately such images are obsolete after a few months if not maintained by the creator which is the case for the images on this channel the last way combines the good things mentioned above and avoids most of the bad ones and if it is done right it solves the backup problem elegantly separating program and data enables to backup only the valuable data and if we work with docker containers arranged by knowledgeable people we can rebuild the programs quickly and fast no backup is needed for this part i will use the last method which was introduced in video number 295 and is called iot stack this project started by graham garner was further developed let's check it out but before we have to have a look at the hardware i strongly suggest using a pi 4 for our home automation server why because it is fast enough and even in the smallest 2 gigabyte version it has enough memory for the job and most important it boots from ssd yes i know the pi 3 also boots from ssd but it only has usb 2 which is way too slow for this application ssds are particularly important for home automation because first they are much faster than sd cards and second we collect lots of small data chunks over time if the pi frequency writes data on an sd card it will die after a certain time and you will lose everything in addition sd cards slow down your pi a lot who wants that so an ssd is the right way to go i use these small and cheap 128 gigabyte ssds they are fast and have enough space for the next 10 years of my sensor data if they are reliable you ask i don't know this is why iot stack has a smart built-in backup concept two things are essential to know in respect to the pi 4. number one it needs cooling i use passive cooling which is perfect for a home automation server because i do not like nagging fans who die after a short time in video number 288 you get the comparison between different cooling solutions if you are interested you still cannot just connect an ssd instead of an sd card to your actual pi 4 and hope it will boot it will not because the current pies have old firmware in the eeprom this firmware has to be replaced this process became much more comfortable you find the needed information in video number 341 please use the newest write-up if you want to save time i assume that newer pies will be shipped with the appropriate firmware till then you have to do this step before you can boot from ssd for my production server i use a four gigabyte version in a flurk case which is connected by cable to the internet this combination runs now for month without any problems now we can go on with the software as a prerequisite we have to burn a 32-bit raspbian os on our ssd and boot without an sd card next comes the part which makes the difference the iot stack project if you do not know it i suggest watching video number 295 it uses docker which is a brilliant concept for our purpose we get ready-made containers with our favorite software installed and pre-configured that alone is cool but we can do more if you clone the project to your pi using this command and type this command you get a menu that leads you through the process first we have to install docker next we go to build stack here we find all the goodies a lot of choices like in a high class restaurant just without prices and calories how cool is that you select whatever you need and later you can rerun the menu and add extra stuff if you want we find things like portainer to manage all containers or we find mosquito node-red influx db and grafana this is the stack i currently use and which we later will use for our example we find databases interfaces to sensors like zigbee to mqtt tasmota admin espurino hub and blink server and you find openhab if you decide to go this route home assistant in the form of hass.io will be covered later and maybe somebody will also add fhem in the future but you also find cool things like next cloud to build our private cloud server or motion i to attach your ip cameras a vibrating community on discord created this list you find a link in the description if you want to join it we select all our desires and hit ok if you selected node-red you can select a ton of nodes that will be automatically installed maybe you are asked if you want to override the service directories usually the right answer here is do not overwrite in the end this menu creates a docker compose file if you start docker in the iot stack directory it pulls the newest versions of all selected containers from the repository and installs them on your pie if you want to install home assistance you have to go to the menu again and install it in just a few minutes you have your home automation server configured and running the proof is here if you connect to port 9000 of your server you get to the portainer interface here you see all your containers in green and running also home assistance which runs on docker 2. if you have to stop or reboot one of your containers you just do it here much easier than rebooting your raspberry after changes if you click here you see the log of the respective software and if you click here you get a prompt inside the container very handy but how do you use the different containers most containers can be used from a browser by adding a particular port number to the address if you for example add 1880 to the address you get to node rate of course it is empty because we do not have flows defined yet the same applies if you connect to grafana by using port 3000 no data sources nor diagrams are defined next cloud is on port 9321 it also has to be configured to your requirements it is recommended to use mariadb as a database this is why we also have a container with this database very handy i will not go into details of nexcloud but here you see the possibilities of this private cloud of course you need much more storage if you want to use your pie to store music photos or videos i ordered one of those for a test if it works my pie can store terabytes of data maybe i will once make a video about nexcloud what do you think as we see home assistant on port 8123 also works currently it runs outside of iot stack openhab on port 8080 runs fine too of course you have to configure those systems a lot if you want to use them you should find tutorial videos for all these platforms on other channels so iot stack is a good solution if you need more than only home assistant or openhab if you want to concentrate on one of those platforms only it is probably better to use their native install as promised we will go now one step further we will configure a simple scenario it starts with an esp32 light sensor measuring several properties of light every few minutes it transmits the results via mqtt to mosquito the mqtt broker node-red is subscribed to its channel and gets all its messages via a flow it transfers the data to influxdb where the data is stored grafana uses influxdb as a data source and displays the values in a dashboard let's start with the sensor it measures light intensity rgb values ir and uv and packs everything in a json string this string is sent every few minutes via the pubsub library to mosquito which listens on port 1883 pay attention to include user and password in your message the standard is admin admin otherwise mosquito will not react the mqtt topic used by the light sensor is light sensor values if the mqtt broker got a message it transfers it to all subscribed partners one of those partners in our case is noderate it is connected to the mqtt broker as you can see here an essential fact inside docker is that you do not use ip addresses for communications just names of the containers and the name of the mqtt broker container happens to be mosquito also here username and password are important of course you can change the password if you desire this note now gets all messages on topic light sensor values for debugging we connect it to a debug node now we see the messages in the debug window next we have to pack the values into the right form for influx db this is done in this function node called light node red transfers data in messages between nodes so we have to create a message which is compatible with the next node which stores the values into influx db all programming inside node-red by the way is done in javascript here we store the looks value into a global variable this variable can be used by other flows for example to extend the awning when the sun is strong enough and this command shows one value below the node quite handy for debugging too the last node in this chain stores the data into the influx db again its address is just influx db and the database name is light not very creative influxdb also needs a username and password it is node-red node rate as you see in the description of iot stack now the data is stored in influxdb i added two flows to show you the possibility of node rate one is a flow that checks battery voltage it uses a switch node which depending on a value transfers the message to one of many outputs this node only has one output which gets all messages with a battery voltage below 3.2 volts all other input messages are dropped the next node creates a readable message and the last one transfers it to the messenger service telegram which tells me on my iphone when i have to replace the batteries of the sensor the third flow monitors all messages coming from the light sensor if no message arrives for 3000 seconds something went wrong and a warning message is created and transferred via telegram to my mobile very good i assume you are not keen to type all this information by hand this is why you find a link in the description to a json file with all the information you just have to import it maybe node-red complains that it did not find all nodes in your container then you have to add them using the palette function finally you deploy the flow to activate it now we have all our data in influx db you might think but unfortunately this is not true we get error messages in node-red because influx db does not accept new data without the database as we said the database in question is called light but how can we connect to influx db inside a container to create the database potty only connects us to raspberry os fortunately we have partainer if we click on this icon we get a prompt inside the container and if we type influx we are in like flynn we see that there is only the internal database available right now so we have to create one we can do it here or also in a simpler way we download influx db studio start it and add our influx db container as a connection here we need the ip address of our raspberry because we are outside docker and we need username password as usual now we see the same internal database as before and can create our desired lite database and really we see it also here in pertainer now we do no market error messages in node-red very good we are proceeding fast and nearly reached the end after a while we even see some data in the database the last part is the visualization of the data i use grafana for that purpose which listens on port 3000. after login with admin admin we have to do two jobs connect grafana to our light database in influx db and create a dashboard to display the values the first one is easy we select influx db as a source fill out the needed fields and press save and test if all is good we get the green confirmation otherwise you made a mistake somewhere for the second one you are lucky again i also prepared a json file that you can import if your database already contains some values you should see them now nice the sun is shining i was not aware of this fact because i'm sitting in the basement you see a nerd's way of dealing with the real world at least i do not get a sunburn this way the last important part of a home automation server is the backup we do not want to lose all our work and data just because of a technical defect or even worse because of a human error this is why iot stack stores all relevant data in one place on the disk if you backup this data to a different machine or the cloud you should be safe two possibilities exist either save it automatically to dropbox or to google drive now we are done with the best home server on the raspberry pi ever why do i think so the pi 4 is fast enough for such a server already the 2 gigabyte version has ample memory for this purpose usb 3 is fast enough for home automation and also can be used to share data in your family it is for sure not the fastest nas on the market but with nexcloud it has a lot of excellent features it is small and does not consume a lot of energy all software is free of charge and most of it is open source we can build our server by selecting the needed software components iot stack creates our server in minutes because of docker we can separate programs from data this enables a simple data backup as well as a simple update for our programs because we do not need to backup the programs the backup file is tiny we can easily add other programs if we need the price of a raspberry pi with a small ssd and a simple case is in the 80 range one last thing if you want to add an uninterruptible power supply you can join the super power project of the last video where we build such a thing as always you find the relevant links in the description i hope this video was useful or at least interesting for you if true please consider supporting the channel to secure its future existence thank you bye

Channel: Andreas Spiess

Views: 432,820

Rating: 4.953352 out of 5

Keywords: arduino, arduino project, beginners, diy, do-it-yourself, eevblog, esp32, esp32 datasheet, esp32 project, esp32 tutorial, esp32 weather station, esp8266, esp8266 datasheet, esp8266 project, greatscott, guide, hack, hobby, how to, iot, lorawan, nodemcu, smart home, ttgo, wemos, wifi, tutorial, grafana, node-red, influx, influxdb, mosquitto, docker, container, vpn, pivpn, dropbox, Pi dropbox, raspberry dropbox, IOTstack, hassio, nextcloud, openhab, fhem, home assistant, raspberry pi, raspberry pi 4, pi4, Pi 4

Id: KJRMjUzlHI8

Length: 22min 16sec (1336 seconds)

Published: Sun Sep 13 2020