How to use the Context storage in Node-Red

welcome to a new episode in my another tutorial video series and today I want to talk about how you can use the node red context I think this is an important topic especially for those who come to node red who don't have previous programming experience and they say the node-red interface where you know you have boxes and then you connect them with virus and then you just assume that you can do a lot of things without knowing too much about programming and have any programming knowledge and I think this is certainly true but then you are also missing some of the concepts that are you know required in order to do certain applications and for those I think especially in the noted environment when you deal with messages then you have to have some sort of way to store the information between messages if you want to remember the value of a previous message or a different message that I want to use when you are processing a completely different message and this is what the context is used for so what we are going to learn in this video is you are going to learn how you can store data between messages so how you can store you know part of the data or your entire message and this is what you you use the context for you understand the differences between different contexts and also this is also called the scope of the context we are also going to talk about various storage options um so how you can make your contacts persistent so for example store information so it doesn't get lost when you restart your noted instance and we are I'm going to show you quite a few different examples so some of them are going to be very very basic examples or for example how you can calculate the minimum and the maximum of the values that are coming through in messages or count the number of messages or how you can Implement some timer-based motion detector or a simple thermostat so some of these are going to be like basic examples that you might be able to do using a node and some of these are going to be like more the you know the home automation type of scenarios also I'm going to later on go into some examples how you use the context in the functional so when you are writing your own JavaScript code and I also have a couple of examples for you when you should not use the context or the the type of scenarios where the context is not a good place to store the information there's going to be a lot of content in this video so if you want to do it in pieces I have broken up the video into various sections so you can see the chapters in the progress bar and also you can find links in the video description so as I mentioned in the beginning of the video node red is all about processing messages so everything that we do in node-red is an event based and the event is whenever a message comes from let's say a sensor or some external device or a web request and when these messages are coming through we are processing those single messages and we don't necessarily know about any of the other messages or the previous message that are coming through in the same flow as well and this is why we need the context because we need to store this information so for example I need to store the last data because for example I need to tell whether the the data that I have received is either bigger than or you know or smaller than the previous one so maybe if I need to decide something based on the value whether it's trending up or trending down or I just want to calculate the minimum or the maximum or the count the number of messages that I received let's see in the last hour and this is what the context is for so first of all what I'm going to show you before that let me just move my face out of the way so in the top in the left navigation or in the left bar we usually use either the debug or the status information but if you come here there is one which is called the contacts data and this is where we can view the various contexts and we have three different contacts there is one which is called the global there is a flow and a node so first of all I'm just going um and in this example I'm going to use the flow mainly and the difference between the flow and the global is that whatever you store in a flow variable can only be viewed used or is only visible for nodes in the same flow so I'm going to use everything in the flow because this example is all related to the context and I don't feel like some of the other nodes need to have this information but if you have a global information for example um let's say you have a light sensor and then based on that light sensor you decide whether it's you know daytime or night time and you want this information to be used in multiple flows then you put it into the global area so it will be visible for all your you know flows but we are going to see these examples a little bit later and the node is again is sometimes special it's only used by the function notes but I'm going to talk about that later let's take an example maybe it's going to be easier to understand this so here I have a couple of inject nodes where I can send in various numbers 10 20 25 30. let's assume that these are temperatures that are coming from a temperature sensor and uh so I have the information coming through here in the flow and I want to save the value because I just want to have the last value always visible in the system or always stored in the system so I can do this very simply using the change node and I can say like I want to set and you can see here that usually we have either the flow or what we usually use the message because if you want if you want to use the message topic or a message payload but you also have the option to select flow or Global so again as I mentioned this if you select flow it's going to store the value in the flow so it's visible in this flow only if I put it into Global it's visible everywhere and I have to specify the name of the value and let's say this is the the last value and so I want to store the message payload in this last value variable or context variable so I deploy my flow and nothing is happening and if I inject so I send the data in and it is getting stored in the flow but yeah I don't really see that right now but if I come here and I refresh then I can see oh let me just delete some of my earlier ones so they don't disturb us for the for the time being and with this I also show you how you can delete the flow variables sorry the context variables so if I send number 10 again and now if we refresh I can see that the last value is 10 and if I send the data 20 now it is getting overwritten by 20. so again you have the option to store this value and it is available in your flow so now the message has been processed but this value is still available in the flow so what can we do with it for example we can have an inject node and then we can get this value so using simply a change node again I can say that now I want to put my flow variable which is called last value and I want to put it into the message.payload so if I execute this and then I ascended result in a debug so if I execute this now you can see that I managed to retrieve my last value and it's here so if I send in 30 and I get the last value again it shows 30. so again you have a value you uh you you know you process the incoming let's say temperatures but you have a separate process let's say every hour or every five minutes you want to do something based on the last temperature so you put it into a flow variable and then you retrieve it when you process it using an inject node so that's the simplest example but we can do something with this last value as well so if I connected this to this change no sorry this switch node then I can determine the the change of the value so now I can compare the message.payload into the flow last value variable and whether it's bigger or smaller I can change the value saying that the value is lower or the value is higher so and then I send the result into a debugger so into debug as well so the last value that I sent in was 25 so if I say 20 then well nothing happens and actually I made a mistake here because um you see we are checking what was the last value but then we are setting the last value at the same time so now probably what happens is this gets executed first and then because these are always the same nothing really happens because we have a condition for less than or greater than so what I can do is I can add a small delay node here let's say I set it to 100 milliseconds and that's just going to be enough so this gets executed first before the the value gets stored I'm you know I could have done it in a different way it's not that elegant but I think it's going to work so now let's check the last value so I go back to the context it is 20. so if I go to the debug and I say 25 that's higher if I say 10 it's lower so now this is works we are able to compare the context value with the current value so you can do something with that in the next example I wanted to calculate the minimum and the maximum value so again let's assume that these are temperatures but you are interested in what is the maximum and the minimum value in a particular day then you can again just do a simple switch to say that if the payload is less than the flow.min so minimum then I want to store the floor I want to cut put the current value into the minimum flow variable um so if I do like I don't know 10 then we can again check the flow variable it shows that the last value was 10 it doesn't show the mean yet and this is because here we are comparing the current value with the flow variable but the flow variable doesn't exist so it cannot compare something to something well a value to something which doesn't exist but we can have with that because we can initialize the minimum value so here I put the minimum value to 1000 so now you can see that it is 1000 so as soon as I receive the data now it is getting reset to 10 and if I send 30 then nothing happens it's still a trend because sorry still 20 because that's the lower value lowest value but if I say send 10 then now the minimum which changes to 10 and of course you can change it to you can create another one so if the current value is bigger than the flow Max bam we are going to store the max value so um we put the current payload to Max and also we need to initialize the max value so let's say that the flow.max and let's put it to minus 1000 so it would definitely get over within whenever we receive the the first set of data so if I do set and we have the max anime sorry I should have done this to a number minus 1000 again just to be sure so we are comparing numbers and numbers so okay so this is good now minus one thousand and one thousand so if I say 25 that's the new medium and the maximum and if I send 20 then the minimum is 20 the maximum is still 25 so let's do the extremes 10 and 30. so now we have the minimums at 10 and the maximum is 30. and of course if I send 25 then nothing happens because our calculation works so this would now calculate the minimum and the maximum temperature is of the day and of course let's say you can set this inject node to execute every midnight and then it's just going to reset the values every midnight of course then you need to wait for the First Data to reset the Thousand to the minus thousand but uh maybe you can find another way to reset the uh the first value to the last reading instead of 1000 and minus one thousand in another example um I showed you how you can for example create a counter because I want to show you a new thing here so again everything that I send in here these readings go to this change node and here I set the flow value counter and here instead of setting it to a specific value I specified an expression so this is an expression type and whenever you are editing the Expressions you have two functions that I that you can use it in order to set or get or read or write flow variables so there is a function which is called so everything starts with the dollar sign so there is one which is called Global context and the other one is a flow contacts again it's either the flow or the global variable space so you can do dollar sign flow context and in Brackets and in double quotes counter so this is the name of the variable and you can just say plus one so again this doesn't work because the value itself doesn't exist but I can do this with this reset which just sets the flow counter to zero so now we can see that we have a zero and if I start sending in numbers or data then this gets incremented every time it receive a new data so this is just a simple counter again if you are want to calculate your leverage you need to calculate the number of records that you have received and also the sum of the records so you can just calculate let's say a simple average the next example is more like a real home automation example let's say you want to do a timer based motion detection so you have a motion detector which is going to send you to information whether there is motion or no motion and for the sake of Simplicity I said that if you have motion it's going to send a value of 1 and if you don't have a motion it's going to send you a value of 0. and you want to process this data depending on a counter let's say you only want to do something with this motion detection if it's happening between I don't know 10 o'clock and midnight so for this we can use a timer so this is not part of the standard set of nodes it's called the big timer so you can just install it from the palette and I specified here that this timer starts at eight o'clock and then it completes at 11 45 and when it starts it sends the value of 1 and then when it finishes it sends the value of zero and I have saved this value in this timer function so you can see that I put the the current payload into the timer and actually we can see that the timer is one now because we are between 10 and sorry 8 o'clock and 11 45 so this is one which means that the motion detection is now active so let's go back to the debug messages and if I send in a motion now we can see that the value of one is going through and when we say 0 then it's that that information is also going through so we have created a simple switch node here which set which checks the value of the timer and if it's one then it redirects the value to the first port and if it's zero so the timer is inactive then it just doesn't send the data through so that's how it works so now we can see that the timer is uh the value is one so the the data is going through and let's say if I change this one to um so let's say only nine o'clock which is already in the past so we might have to wait a little bit because I think this only triggers oh no it says it's only off so now you can see that the timer is zero so the timer is deactivated and now if we go back to the debug messages so your motion sensor sends you events that motion is detected but it's not going past this switch node so your motion detection motion detector is inactive and the last of the example could be like a very very simple thermostat so what we need for the thermostat is we need a way to set the required temperature or the set temperature and we also have an input where we are receiving the temperature data from a temperature sensor so this is my set temperature so let's say I can set the temperature between 20 and 20 20 and 25 degrees but of course you can replace that with let's say a UI element where you can type the number or a slider or something like that but we are going to store this information in a flow variable which is called set temp so we put the message of payload into a set temp so I can set it let's say to 20 and then we can say we can see in the context that we have a set temp of 20. and now let me go back to the debug because these are the various temperatures we have so we have the temperatures coming in from the temperature sensor and then we have the big switch which is that's basically the logic of the thermostat so we check the payload and if this payload is less than the set temperature sorry if it's bigger than the set temperature well then we have well it's warmer in the room that we need so we are going to turn off the let's say the radiator and if the if the temperature is less than the set temperature minus one I added this minus one so we have a little bit of hysteresis so it doesn't switch on and off constantly around the room temperature but you can set it to minus 0.5 or something then we need to turn on the heating so let's say that turning off the heating is we are just going to send zero let's say we send it to a relay so that's going to turn off the heater and when we want to turn on the um when we need to turn on the heating then we just send the one so that's like turn something on and I've also added this filter because now what is going to happen that every time when we receive a temperature reading then the system is going to send zeros or ones depending on whether it needs to turn on so let's say if the heating is or the temperature is low it's going to send like every time we receive the temperature reading it's going to send like one one one one and I just said that okay you know block all the all the messages that are the same unless they change so we don't send so many messages to our relay so the set temperature is now I think it was 20 degrees so if it's 18 degrees then we need to turn on the heating so again if I set just keep sending 18 we don't get any more once because of this filter but then if but the temperature goes to 22 degrees now it turns off so if I set the set temperature to 25 now and if it's still 22 degrees at home now we can see that the we need to turn the heating on so it goes to on but then if let's say the temperature goes to 27 which is definitely above the set temperature then it turns off so that's all you need to program a simple thermostat using these two pieces of information in node red well we haven't really finished with all the examples because I want to talk about a few other things which could be important so first of all is um let's say you want to delete the context value so you have a couple of options I mean of course you can use a change value and then you can just set the flow variable to let's say either 0 or like an empty text it really depends on your scenario but what you can also do is you can use this one delete and then last value so you put the name of your flow variable here so that is going to delete it it's going to act same as the delete button again we have seen previously that sometimes it's better well there could be scenarios where it's better that you don't have a flow variable or maybe it's even better that you have the flow variable but it's set to a certain default value so you can do this you can you're going to change it to the default value or you can use this delete function as well it's really up to the next one I want to talk about is contact storage and that's an important thing because uh what I haven't really mentioned up until now is that anything you can store in the context is is stored in the runtime which means that if you restart your node red or let's say there is a power outage so your server goes down and it comes back again these values will be lost but there is a way to go around this because within node red there is a concept of context stores so by default everything so this context store defines how this context information is stored and the default is that everything is stored in the memory but you can configure node-red so that you have a memory storage and you also have a file storage so the file storage means that every time every time you change your contacts it's actually written into the file so when node red starts for the next time it's going to read the information from the file so it is going to be persistent it stays with you when you restart node-red in order to configure this you have to find your settings.js file which is usually in your node-red installation folder so you have to edit that file and you have to find the section where it says you know this context storage probably it is in your settings dot js5 but it's commented out so you need to uncomment it and you have to change it like this so you say that I Define the context storage and I have two options the memory only and the file and the default is going to be memory only and of course don't forget that if this is in the middle of your contact storage so there is some more stuff under it then actually you need a comma here because otherwise it's going to be an invalid Javascript file and your node is not going to start so the comma would be here under the closing curly brackets and once you change that obviously you need to restart your node red but once you've restarted then you have some options so when I store my or when I do anything with the context either it's a flow or Global because I have these two different context options or the contact storage is configured now I have this button here so here I can specify that I want something to be stored in memory only or I want to store it in a file so you can see that it changes from memory only to file and also here in this sidebar it also shows for every single one where they are stored so if I refresh my Global you can see I have you know tons of stuff here then most of it is memory only but I think I'm pretty sure yeah for example these schedules these are stored in file so you can use them both at the same time you just have to make sure that you know whenever you want to use a file storage you have to specify it here because but based on the example I've given you the memory only is the default contact storage and of course since we have seen these examples now up until now I show you how you can store simple numbers and text in the in the storage but you can order sorry in the context variable but you can store anything it can be arrays and objects and or basically you know any data type yeah you can see this one is for example an object apparently now I only showed you how you use the context in simple notes for example in the change node in the switch node but now we are going to see that how you use it in the function nodes as well and I'm not going to write you know complex or actual examples here but I'm just going to show you the various options that you can do and well there is only just a few functions that you would need to use so what do you what do you do if you want to read the context and then just put it into a variable that you can do some you know manipulation with it within the uh your function node so you just Define your variable let's say last value and then equals and you do if you want to read it you basically say flop on get and then the name of the variable so that's going to read the value of the flow variable last value and then put it into the last value it's a local variable and then let's say if I want to do something with one of my Global variable that's also easy because let me just put pick something which is a little bit easier as I said I have tons BT and State so let y value of course it doesn't have to be the same and if you want to read Global then it's just global.get and then vtn underscore state as simple as that and then from now on these you know last value and my value are going to have the value of these flow and the global variables because I talked about storage options let's see how you how this looks like when you are you want to read this not from the memory but for example for the file so let me just take another example again so my Access Control Global variable is stored in the file storage so let sorry my value 2 Global get Access Control and then you can specify the the name of the storage option and then the storage option is stored as a string so my file storage is called fire so it's double quotes file and again if you are just you want a memory only because this is a default you don't have to mention anything so that's fine also it is a good idea on draw control okay also this is uh it's also a good idea to define a default value sometimes it's needed so let's say I if I want this last value but um [Music] if it doesn't exist I want to set it to zero so I can do that so let last value equals flow flow dot get last value and then and zero so we I can do this with diff with some basic types so if you want to default it to a number you can do that but the other thing you can do it and this is what I do especially if it's uh something is is an object and I want to um you know set a I don't know default object that I sorry I want to put it into this section actually let me just change these so you don't get so that's going to be last value 2 and that's going to be my value free so we don't get errors because we use the same variable so um if let's say I want to read this Access Control uh variable from the global and also from the file but let's say this is an object so I can't just use this end to set a default value but what I usually do is I check so if my my value is undefined that means that so this will execute this statement will execute if the Glo uh the the context variable doesn't exist because then the global and the flow returns undefined so let's say that in that case I want to initialize the sorry not the access control the my value my value of three so let's say that in that case I want to initialize the my value free with an empty object so you just check whether it's undefined updating the context is it's very easy so let me just I just copied this example above so this is how we read the value and if I want to update then I simply say that and instead of flow.get I say slope on set and I specify the name of the flow variable and then the value so this is how I write the last value back to the flow and I can do the same with global.sat I think it was my value and then that's it and if you want this and if you want to write this value to a specific storage for example now on I want to store this BTN state in the file context I can also specify the name of the context so the name of the context variable my variable and then the contact storage and I realize that I should have put this example here so that's how it looks like properly and since we are talking about setting the flow values let me just mention something that I myself only learned recently because the way I did my programs is that let's say I have a function node where I need to use the flow variable and then I would read it like this and then I change the last value and then I would always set it at the end when I before I finish but what I learned is that whenever you are reading or getting either the floor or Global you are not getting a copy of the context but you get the context itself so when you make changes to it it immediately uh impacts the context but um so you might say that I don't even need to use these set functions which is technically correct unless you are using the file storage because only when you call the set functions then it instructs node-red to actually write the changes into the file so I think uh well I think and also it was advice to me that um to you know get into a habit of using the set functions because if you I mean if you omit the set functions and you're using memory only then you know no harm is done but let's say you revisit your code and then you want to change it to file storage then if you miss the set functions in the beginning you probably won't remember the points where you want to save the information so I guess it's better just to use the set functions because then you can just easily come back here you know find all your Global or flow set functions and you can just add the context storage if you need to change that at a later point and then finally my last example if you want to delete your context from a function node then well actually the example is here is let's say flow dot set so we are still using the set function and let's say last value and then the last is undefined so that's going to be the same as you would press the delete button here and then finally the last thing that I refer to write in the beginning of this section is what is this node context and the node context is something that you use in a function node so in the function node you can use flow dot set or global.sat and you can obviously use this in change in the change node as well but in the function node you also have context dot get and you also have uh and this should go into variable my value 5 and you also have context dot set last value and let's say my value five so this this will create a context variable which is only visible for this function node and those would be visible up here in this area and um this is great because uh um I mean I use this a lot because if I have a function node which processes messages and again it needs to store some information for something you know for for a message that comes in later then um or if all the processing is done within this function or there is no point putting into the flow and making it visible for everything yes I think it's just much better to keep the scope as small as possible so if I want to store a data which is only going to be used here I would use the context so it only visible by the function node and then you know processing data so for example if I'm creating a function or which calculates the minimum and the maximum and the average I would store all the information in the node because uh you know those intermediate data would not need to be visible for other nodes so I just put everything into the context if I have anything that I need to put into the floor Global because they are R that's data should be visible or useful for other you know parts of the flow as well I can you know store some of them in the context and I put I can put something else in the flow order Global as well the you know it's really up to you but that's the difference between the context values for the function node and the flows and the globals which could be used by anything else and finally I do have some reminder here which says what are the don'ts so then you should not be using the context um you should not be using the context if you want to store something on that message that you want to use later on so something which is specific to that message that you want to use later on and that's actually related to the to the second comment here because when you are using the context so you put something in the context and if an let's say in a change in order in a function or then if another message comes through then obviously that is going to override the changes so maybe you have a scenario where you you know you have the message comes in you put the the value of that message into the flow variable in a change node and you do something else and then later on the flow you are reading that but you cannot be certain that by the time the processing comes to your second let's say change in order to function node another message doesn't come in which would override the original message so this is why I said if you if you want to um just use the context as a temporary storage for message specific information you shouldn't really be using that and of course if you want to do something like that because let's say there is something that you want to store in the beginning that you want to use it later on and that's specific to each of the messages you should be putting it under the message as well so obviously not in the payload because the payload is getting overwritten by every single node but you can put it into at say message dot save so another object within the message object not the payload itself because by default the you know each of the nodes that the message goes through it should only you know modify the payload or some other specific attributes for example the file node is going to use the message dot file name if you want to specify the file name externally but if you put anything else it would just ignore it and then you pass it through the node so it would change the message it will use the sorry it would use the payload in order to do the processing and put the results usually in the payload as well but any other part of the message object is going to get unchanged so you can use that to store message instead of using the context mostly because you can't really be sure that if you have a lot of messages coming through the you know the subsequent message is not going to override the data by the time you want to process it later on in the flow I know this was a little bit complicated especially if you are a beginner and you just want to understand the basics but I wanted to include all of this in a single video but that should be the end of it so let's recap what we have learned today and let's recap what we learned in this video we definitely learned what is the context in node red and how you can use it we still we learned how you can pass information into the store and how you can you know store information in between messages and then we learned the various options that we can use in contact storage well to the main options that are probably going to be the most frequently used and hopefully we understood this whole concept using a couple of examples and these examples uh were on standard nodes so I like using the switch and the change node but we also learned how we use the context in the function notes as well so that will be all for today thanks for watching and hopefully see you next video

Channel: Csongor Varga

Views: 3,026

Keywords: tutorial, how-to, node-red, iot, node-red tutorial, best practice, node red, nodered, using context in node-red, store data in node-red, persistent storage, best example for context storage, flow context, global context, context storage, how to store data in node-red, how to get data in node-red, writing code in function node, file storage, store data permanently in node-red

Id: vW3IZnml4y4

Length: 40min 19sec (2419 seconds)

Published: Fri Mar 24 2023