Node Red MQTT on the Raspberry Pi

you may have pressed the node-red icon on your ass replied desktop and being a little disappointed to have seen nothing but a console screen that almost appears to freeze but do not be deterred you are only a few clicks away from what I believe is one of the most exciting environments you could come across today this is your axis for a simple but powerful interface that provides you with the tools to monitor and control the physical world around us it's a convenient gateway to a massive new technology that is now becoming cheaper and more readily accessible I am of course referring to the Internet of Things or IOT as is known and node-red is software that's rapidly developing into a popular platform for device data and information manipulation this video will introduce the node-red editor and how it operates on and controls the pie it will show you how quickly links can be made to and from external feeds and how third-party software can augment the basic node-red installation it will also show you how node-red allows you to rapidly develop graphical apps that will allow you to control your pie from a mobile we will then explain how the pie fits into your iot environment at home and the basics of other key supporting technologies such as no js' Jason JavaScript and MQTT one word of warning this video was filmed at the end of 2016 but as with all such videos it's likely to be out of date even before it's uploaded we've tried to include all of the current techniques tips and tricks that should help you to get a working platform as quickly as possible but do be aware of how rapidly technology is changing this video only uses open source software that can be downloaded onto the PI many of the services can be replaced or augmented by cloud-based technologies these may be mentioned but our aim here is to demonstrate how everything can run from a PI you're learning is based on your individual pragmatic control of every feature ah we introduced the internet-of-things in a previous video our connected world and how this term refers to a future transform by devices that monitor and collect data from almost anything anywhere the data is used for direct control but could also be amalgamated and processed to produce further information we mentioned basic examples such as climate control and security systems but then discuss the more profound implications of massive data processing and information management what is called Big Data these activities are promoted by rapidly developing technology combined with dramatically falling prices the PI can perform all of the functions required by IOT it has inputs that may be connected to reach sensors it has outputs used to control devices it has communications and finally can be programmed to emulate intelligence the control of all of these aspects is when low dread comes in no dread is a little like scratch on steroids in that is a powerful extensible graphical programming environment it was developed by IBM in her sleeve Hampshire and only released in late 2013 about three years ago it's home page load read dot org already has over a thousand contributions for download and there are a rapidly growing number of sites providing service help and support if you put off by terms such as extensible environment and interface then please do stick with it as all should become much clearer in the next few minutes this may be your opportunity to conquer the world without having to become involved in coding at all no dread will let you paddle in the coding shallows or if you wish dive right in there is no need to understand every stage of this demonstration at the moment the idea is to describe the desktop how it operates and demonstrate how easy it is to rapidly design complex systems in a simple graphical interface until now red arrives a good deal of what is shown here would have taken some time and some serious coding to achieve this is the basic default mode red design page it is in fact a web page running on your PI and will be familiar in style if you have already used scratch the palette is the panel on the left it is a list of colored boxes called nodes sorted under headings such as input output functions social on the right is another panel with info and debug tags in the center is the main design area and the basic idea is that nodes are clicked and dragged into this area as many as you want and all are connected by clicking on these small grey circular points and pulling out these orange elasticated wires outputs only connect to inputs but you can collect any number of wires to any point into collecting nodes like this produces what is known as flows clicking on a node and its high lit in an orange glow wires dutifully follow the nodes as they're repositioned and here is a lovely feature nodes with both inputs and outputs may be simply inserted into a connection like this delightful there is a temptation to try and right-click on a node or wire to delete it but this just invokes the response from the browser to delete a node or wire highlight it and press the Delete key on the keyboard clicking and dragging around a group of nodes selects them to be repositioned or again by pressing on the Delete key removes them this happens immediately there is no are you sure confirmation mousing over any node in the palette produces a quick description and clicking a node in the flow displays a fuller description under the info tag here on the right holding down the control key and tapping the plus and minus keys zooms in and out in the usual browser Manor and these sliders around the side of each pane operates in expected way that completes flow connection properties now to achieve something it's traditional to do a hallow world demo and who are we to fly in the face of tradition it will reveal a little more about these nodes and what they do drag out an inject node as a debug node and connect them clicking once on the inject node displays this information under the tab on the right-hand pane and double-clicking it provokes a flyout that provides a menu of options and setup information for the inject node the default payload is a timestamp so let's just leave it at that at present and select cancel double clicking on the debug mode produces a similar flyout with the only option here being output message payload or output message complete message object once again we will leave the simple default in place and just note that term payload returning to the flow editor we see that both nodes have small blue circles on them these indicate that they have been modified in some manner to save and activate the changes we need to click on the red deploy button and this green drop down appears to confirm that all this well to see any activity click on the debug tab which is clear it's caused any activity to begin we need to initiate a flow and here this is achieved by clicking on this side extension of the inject mode this causes a timestamp to be sent from the gray inject node to the olive-colored debug mode this is confirmed by the message appearing on the right under the debug tag and the green confirmation of the action dropping down from the top of the screen in fact we can repeat this process as often as we like this confirms that our desktop design editor is functioning correctly which is satisfying we can mute this debug by clicking on its extension button that retracts it to show it's disabled no debug message appears and enabling it again as a side point the timestamp appears in the debug area this is what's known as the UNIX epoch number of seconds since the first of January 1970 notice that even just the slight clicking-and-dragging of a node causes the blue spot to reappear this can be cleared by clicking on the deploy button there going double clicking on the inject button again and using the pulldown to select string from the payload option and inserting the classic words hello world and pressing done changes the inject the blue circles remind us that change has taken place suppress deploy then click being checked the extension to reveal hello world in the debug job done we've seen how these two nodes operate how to enter the setup information and clear the blue dot created by these changes by pressing the deploy button no changes are recorded until the deploy is pressed all of the debug details are revealed in the tab and these can be deleted by clicking on this clear log dustbin button as another example we could double click on the inject node again and this time select interval set it for say three seconds tick the injector at once start box and name this as periodic complete this by pressing done for completeness drag out the comment box title it nodes and enter a helpful command for future reference click done then click deploy to save the changes and start the flow and now because of the settings in the inject node every three seconds hello world appears without our intervention we have seen how nodes are connected into flows we have seen how loads in a flow are configured and experience the operating style of node read using the deploy button to save and initiate the flow node read as provided with the PI has some specially customized nodes situated at the base of the palette let's drag these in and see what they can achieve for this demonstration I've added a push button to physical pin 13 of the GPIO port on the PI and an LED to physical port 12 I say physical pin as there are rather confusingly three differing numbering schemes for these pins starting with the led drag out the PI GPIO now notice note has an orange triangular warning on it this indicates that it's missing vital setup information double-click on the node and we can see that it needs to know which pin it is talking to the pull down allows pin 12 also known as GPIO 8 so select that I'm not going to set an initial value but let's call it LED and complete by pressing done the orange triangle is now cleared to keep things simple let's have to inject modes 1 to provide a 1 in a string and the other to provide a 0 1 to turn the LED on and the other to turn it off deploy and press the extensions on the inject nodes 0 1 this may not seem to achieve much but there's a warm bloom of success the first time you set it up and get it working notice also this helpful little indication just below the LED node this is really quite a delightful package dragging the button again the orange triangle is visible warning us that it's not set up and so follow the now-familiar routine double click and slip in 13 GPIO 27 call it anything you wish but select the pull-up from the resistor option this ties the input definitely too high or 1 when the button is not pressed and stops it flapping around connect the button for the led deploy the blue circles disappear and forget mine I'll old fingers around this little header I can show you the LED going off as I press the button and the little indicators also showing up beneath the notes on the screen just for good measure we can add a debug mode deploy and Shazam although this is trivial it fulfills the first part of our IOT requirements input output to and from the device obviously we're just using a push-button an LED here but we could be using anything a motor a light sensor GPS device there I say kettle or a toaster quite rightly there's been a good deal in the meter over the last few weeks about IOT and security a lot of scare stories about refrigerators taking over the world but given some of the other options for to us recently and our ability to choose I bet my Dyson any day having mentioned security what if this button were connected to some security device here can I email or tweet it status under social dragging the Twitter node to remove the orange triangle double click and click on the small pen button a further panel appears click to authenticate with Twitter press the button wait for the approval entering your details as required that browser tab can now be closed once confirmed at the option deploy now a tweet is sent when the button is pressed it's that simple it's easy to send an email to in the same manner drag out the email node complete the registration deploy and email I've never seen such a simple yet powerful interface and as they say there's more dragging the Twitter node I'll name it Twitter for incoming bbc1 press done I'll drag down a debug and connect it press deploy to be flooded by incoming tweets go and grab a trigger node this produces a digital pulse select the period I'll choose 2 seconds and now complete press done to confirm grab the trigger over pull down the led and reconnect it to the output of the trigger now when we press deploy the Twitter is shown in the debug or select any subject to search for so the LED pulses for 2 seconds every time a tweet is received extensible NOPA means that this interface can be expanded to include third-party applications here is one that's not in the mainstream node-red development but runs closely in parallel with it and it's a breeze to import an employee click on these bars here to reveal the pull down menu select manage palette to cause the palette to reveal two tabs node and install clicking on the node tab reveals the list of nodes already installed clicking on install reveals the wonderful world of extensibility and all of the efforts others have put into developing the community investigate this later into dashboard into the search bar and click on the install button in the node-red dashboard option when it appears the package will be downloaded and installed press done once this is completed on some occasions I've had to restart load read and restart my browser for these changes to be fully reflected on the desktop but hey-ho what changes there are scroll down to the new dashboard section in the palette to see the new set of nodes that control the UI or the user interface here is a complete set of tools that allows you to set up a web page that includes sliders buttons graphs gauges and forms and map them back to your PI or any other item on the web by dragging these into the flow and configuring them you now have the same control over your PI as we had earlier from the desktop with the in check mode but this is now all available from a web interface when that ball was imported you may have noticed the third tab appearing on the left hand pane click on the dashboard and this button it opens a new window with your button in it click the button if you have a mobile device go to that and into the same address into your browser replacing the 127.0.0.1 with the IP address of your PI the same button appears you now have remote access of the LED on your PI from your mobile explore the dashboard to discover the full range of facilities if a page becomes too crowded you can start another by pressing this plus sign in the corner it opens a new tab and these can be selected across the top of the page pages can be renamed by selecting the page and then the menu button and configure flows add rename and delete only options here but beware delete it's instant there is no are you sure protection you can also share your designs clicking on the menu icon the game reveals the pulldown option from which you can select export and clipboard the complex flow in the graphical design has been reduced and saved as a text file you can see a small glimpse of this in the compact and formatted modes this format is called Jason which is the data former closely linked to the JavaScript language that node-red uses we can Jason and this formatting later but for now you are able to export your design to the clipboard and paste it in the same way as you would handle an image or an mp3 audio file exporting and sharing is therefore easy and encouraged we're going to be doing a lot of this in coming videos importing is just as easy take as an example of design directly lifted from the node-red org website copy it into the clipboard and paste it into node red menu import clipboard you're offered the opportunity to import into the current flow or a new flow either way the imported code is converted into a graphical flow when this code is run it interrogate and presents data in the debug tag taken from this web page that is continually updated with live data about electricity flow in the UK I claim to start this video that you can avoid code entirely and that's fine but for those who want to dabble here is a good entry point it's called Jason JavaScript object notation the complexity of the flow was described in what appeared to be this block of code it's not initially easy to read particularly where they can all be seen on one line in what is called a minified format the minified format can be copied into other web pages that just reformat it still may not mean much even in this more readable format but it's the key component for passing data it is a lightweight human readable format and know that Jason is the notation format for data javascript is the language used in load read and for those who do want to jump straight in you can enter JavaScript directly into nodes like the function load to do this you should cooperate with the whole basis of node read which is that it operates by sequentially passing a mess between nodes in the flow the message carries the payload and the nodes can be used to examine the payload they also with these functions and realize that this is a large message switching center this node changes the content of the message and this one decides which fruit to follow by switching based on the contents of the message these notes split join and then convert the range of messages to summarize this section we've been introduced to node rate we've seen how the graphical design connects the nodes into flows we've seen how the requirements of the nodes are satisfied and how these are deployed into action using deploy we've seen a large range of capabilities such as direct control of the PI using its inputs and outputs node read sequentially passes a message with a payload between nodes nodes can examine manage and direct payloads messages can be read from websites or used to generate websites messages can easily be sent to messaging systems like email and Twitter we've seen access to a fast and growing library of other nodes and flows that are easy to access and integrate and how designs can be imported saved and exported in Jason investigate the wider pallet there are some phenomenally frightening capabilities emerging and if I were to be a more wiki person like in the previous sections we described a typical connection configuration and demonstrate the power of node read causing an LED to flash on the PI each time a term was mentioned on Twitter it wasn't just the fact that this links hyperspace directly to some hardware here on the desk but the ease with which it was achieved I'm not now going to confess that it was all achieved using smoke and mirrors and to prove it you could skip to the next section entirely and carry on using node read as you currently find it on your PI several changes have had to be made to this video even during this production to keep up with some of the more recent changes after all node red has only been available for three years we will press ahead undeterred but suggest that you keep a close eye on all the version numbers updates and upgrades this procedure documents the best attempt to provide an installation to work with or at least provide a solid path to follow to update any installation we start with the most up-to-date installation based on the most recent raspbian image issued by a Raspberry Pi dork on the 23rd of September 2016 insert your newly flashed my Christie Carlin power up the system is shocked into its initial boot and will take a little time on this first occasion but will finally arrive on this impressive new desktop the first step as with any new card is to personalize and secure the installation you have the options in the PI graphical interface of clicking on Mouse preferences raspberry pi configuration or clicking on this terminal icon I'm getting too old for more changes so I am going to use the latter option over the window and enter sudo raspy - config to reveal the trusty old setup screen override these options to suit your local requirements but a standard install would include the following I think option 1 expand file system has already been carried out by the system when it first boots but that's selected to be sure option to change user password is a must select this and enter and reenter your new password option 3 boot option I prefer to set this to be one for a standard text console login option 5 provides the internationalisation option and 9 Advanced Options leads to another array of choices a to set the hostname change this to suit your local requirements I'm going to use no dread in this example by default this name appears as part of the prompt and provides useful confirmation when logged in to several machines simultaneously selection 9 again a for SSH is enabled by default and allows you to use a terminal emulator to log into the PI from another machine this is highly convenient although it's enabled by default enable a5 the VNC remote desktop viewer a6 the spi support and a7 the I squared C support tab down to finish reboot to incorporate these changes log back after the reboot the new hostname displayed in the prompt should confirm this change at least has been accepted as an initial check the version of nodejs is installed is confirmed by entering node - v is interesting that the default version ship with a 23rd of September 2016 distribution is a 1029 an NPN is not installed at all will come back to this shortly okay first we need to do a firmware upgrade so enter sudo RPI - update my PI appears to be a Cray here by the magic of video I'll cheat by speeding this up it takes three of your earth minutes in real time after which every boat is acquired oops I'm sorry I'm not totally sure what happened there back from the reboot do an update so sudo apt - get - why update the - why flag enters default yes - any question presented by the update and saves you having to infer many change individually this takes a minute or two to do unlike the next stage sudo apt - get - why upgrade which can take over 30 minutes depending on your link speed there are normally no problems here apart from this challenge where light diem Kampf is the graphics greeter so accept the default once completed reboot and then get down to some real changes this completes the process of completely updating any standard edition it takes time but at least you know you're up to date with the basic packages the whole process could have been speeded up by removing the unwanted packages before the update but pay ho anyway let's progress our own particular requirements of the node-red install load read uses nodejs and lo jess requires updates and we need a convenient install process we need the node package manager NPM so sudo apt - get installed NPM I should have included - why flag this took two and a half minutes now we use the newly installed NPM to grab an install version to globally so NPM i- g npm at 2 dot x you then add sudo npm install - g npm node - g IP to keep all the versions aligned and up-to-date this takes about three minutes note - fee shows a dramatic leap to version seven and the packet manager - version 3.1 0.9 great we are now ready to go as a side note you can make an image of the SD card in its present state and use this as the basic platform for future installations it will save the hour taken so far so that's it your PI is updating upgraded to within an inch of its life and should now be ready to run load read before we do let's add one further concept that is key to the part of IOT it's another open source development from IBM and it's called MQTT Message Queuing telemetry transport it may not be immediately obvious as to why MQTT is needed as it just sits in the path of sensor data and is another element that has to be learnt understood and manipulate it if we have a PI that's used to read say break temperatures on cars each car has a PI then connected back to a main pie that amalgamates the massive data from all cars as part of a product development program establishing a link between the pies is initially quite simple a few connections is easy to cope with but becomes more and more arduous as the number of sensors increases the task grows and eventually becomes unmanageable it's said not to scale and with IOT involving millions of devices scaling is essential with n qtt all of the sensors are connected to a data broker they become clients of the broker and publish their data to the broker and label it details as topics interested parties also called clients then subscribe to topics that they are interested in in this way the direct link is uncoupled and the flexibility of having many subscribers to many published sources is established the broker acts as a clearing house collecting published topics and distributing them to interested subscribers it's dynamic links can be removed and introduced new topics added and old ones removed it's very flexible a task of the broker is to prioritize data and ensure a quality of services sustained some important later referred to as class two must be guaranteed to get through only once as a single copy but the lower class zero is volatile it does not matter and may be lost in transit only to be updated with the next arrival if MQTT is so important and in the general spirit of these videos of let's learn by doing it ourselves we are going to install and test our own copy of mosquito the chosen open source mqtt server of the time of production the normal installation route for the PI using apt-get is a little behind time so the quickest and most reliable route is to add another more up-to-date repository and download from there the process is first to grab the key of another repository so sudo W get add the key that's been received sudo apt - key add change directory and get to the list CD / etc' / AP t / sources.list d / grab the list sudo w get do the update sudo apt get update you and install the three packages mosquito Oski so clients and the Python link to mosquito and install all pretty painless so far to test the newly installed server started if it hasn't already started it usually does sudo /e g c / int ID develop mosquito start the simple way to test the server is to subscribe as a client and in another screen publish as a client we can do this from the command lines like this so open two sessions the first is to subscribe so sudo mosquito subscribe this is a subscription to anything referenced under the topic hello world the command is acknowledged in the second screen let's send as a client a message with the topic hello world thus pseudo mosquito client topic hello world hello from terminal window to an hour 29 bytes of data has been transmitted and received that ends is slight diversion into MQTT fellow this server is now running in the background means that future testing is simpler though reboot is necessary although Chrome is now the basic browser shipped in a standard additional frost bian and the PI is a powerful device it is probably better if you use it in what is known as the headless configuration this means that the PI concentrates on the task it performs efficiently and offloads all client functions to another machine it means using the PI as a standalone device and controlling it remotely it's a common configuration and we will show you how to set it up here in a PI only configuration we could log into the text-only mode and type statics to start the graphical user interface we need a graphical user interfaces that's where the power of no dread exists to start no red simply click menu and then the node red icon this started this video to see the node red desktop described in the earlier section in this video simply follow the instructions in the console window that seemed to stop open a browser and enter either HTTP colon slash slash localhost colon 1880 or HTTP colon slash slash 127.0.0.1 colon 1880 once assembled 0.021 is the special me address this is the direct link into the node-red desktop although this works is possibly not the best and most nimble solution open the browse on another machine and pointed to the address presented in the console on the PI this will typically be something like HTTP colon slash slash 192.168.0.1 1880 the pi3 has potentially two useful addresses that can be seen by mousing over the interface icon at the top of the desktop if you have local Wi-Fi access you can click on the Wi-Fi link and associate it with your home router this will provide a second IP address for your PI now you can point your browser at either of these IP addresses but ensure that the colon 1880's added after the address as this is a special node red port not the standard port eight you have four HTTP or 443 for secure HTTPS once you've established this link you can control node red now just consider the connection options that already exist if your PI in the remote machine are connected by Ethernet you're both connected via cable if either the PI or your remote browser are connected by Wi-Fi then a wireless connection is also used somewhere if you want to control the PI from a console screen via a Windows machine then it's safe to search Google for putty and install a copy from any of the top three references putty is a secure encrypted terminal connection that uses port 22 when it is opened you can login to your PI from a terminal on a Windows desktop Avel devices have suitable terminal emulators already installed if you're a Linux user you don't need telling you're obviously wise enough to make the link yourself at some point you're going to be irresistibly drawn to control your kettle from the other side of the world and the good news is that this is easily possible with the technology that exists today care has to be taken though in order for this situation not to end up in the bad news column all of the graphics so far have concentrated on the typical setting at home on the land the local area network home Rooter is the central component in connecting the land to the rest of the world the one the wide area network the route has a number of functions one is the share the single real-world IP address that's been assigned by your ISP your internet service provider it does this by generating a different range of private IP addresses and allocating them to the devices in your home by Wi-Fi or cable connection another function is to manage the flow of data across the land WAM interface outgoing access to the web is generally open it may be restricted by parental guards but it's generally open inbound date from the Wan poses a major threat which is why software alarmingly called a firewall is programmed to provide protection a description of the firewall is beyond the scope of this video as is any hope of describing how it's changed there's just too many differing models of Reuters to describe but they are generally accessed by pointing a browser at the default gateway address it's always interesting to search through the options and read through the log files that explain what the Rueter is protecting you from if you don't consider this too daunting you can continue by finding the section on port forwarding port forwarding allows you to map the private internal IP address of your PI to the public web address greatly set up your friends or enemies compat into your PI using your public IP address from anywhere in the world you could also try running a web server by exposing port 80 but do take care you could then expose the IP address of your kettle to the outside world along with any other smart TV baby alarm security system and refrigerator you may own but don't here is a prime example of the use of the mqtt server rather than peppering the file with multiple settings just add one the mqtt service operates on port 1883 and if you follow the standard settings only this message service is connected through the firewall all of devices in your home could then be programmed to listen to this one service to ensure that you add authorization to the MQTT server described earlier in this video before you use it in anger those alarm bells are the execute for this video we're node-red on the PI was introduced I hope that it's been helpful and encouraged you to look more closely at node read the repeated warnings about version numbers should be observed as inexplicable failures can be demoralizing our extended installation description should help you circumvent these playing with nodes and what they can achieve is fascinating importing further libraries of nodes and flows extends the basic capability a huge range of really impressive interfaces can be designed using imported systems but at some point the inquisitive will discover Jason data Jason data structures and object-oriented programming in JavaScript these are the subjects covered in the next video you

Channel: Richard Wenner

Views: 407,534

Keywords: Node Red, MQTT, Raspberry Pi

Id: WxUTYzxIDns

Length: 38min 19sec (2299 seconds)

Published: Sat Nov 19 2016