001- Using Python to read Serial data from Arduino / STM32 using threading and curses libraries

welcome back to this new we episode for the python tutorial actually in the previous episode for the stm32f1 we have been reading the data from the adc and send it to the computer and so far we have been using for the communication with the computer the good very good edge term but one of the things that really not easy to manage it's not really data visualization friendly so even like a minimum it's just a scrolling thing to a certain extent it's quite nice but if you want to use more data and to have something very interesting to read and to use it carefully we need to to go further and to make our own program using python so let me close this one disconnect it and open here the program that we are going to to use it to create within this video so this is our program and as you can see here i be changing and moving the potentiometer the the the value is changing here without really impacting or scrolling so i know exactly what is my value and just to quit i have to tap the key q2 to leave this program so and also we are going to use some threading and some other cool stuff to make this one happen so if you'd like to learn more just stay tuned before jumping to the video let me take a moment and thank everyone for their support i will keep doing my best to provide this quality content so as usual to support the channel you can give us a thumbs up make a comment or subscribe and stay tuned this is the wheel and you are going to learn how to read data from the microcontroller using python so as usual the code is available on github the language we are going to use is only python and the code editor will be vs code finally i think even if you have a beginner level for python that would be very fine but if you are intermediate it would be very easy for you to get it for the hardware that we are going to use so this is as a follow-up for the stm32f1 tutorial series that we do have on the wheel but if you do have even an arduino that will be totally fine and we are going to use here the stm32f1 or aka bluepill and also an ftdi to allow the uart communication between the microcontroller and the computer finally you will need whatever the microcontroller you do have a variable resistance or a potentiometer to make the readings and to play with the readings to see how dynamically your readings are appears appearing in your into your computer so for the tutorial first of all we are going to have a quick circuit and program review then we are going to get the serial data the basic stuff after that we are going to build the cursors yes not the cursors that you know in english as i was surprised but this is a like a base called library in python after that we are going to use the exciting stuff the threadings and to have the readings um and display finally making everything beautiful and make a right sampling and have a beautiful view okay so the circuit that we are going to use first of all we will need a microcontroller we have to connect it to the potentiometer and you have to be sure that you have selected the right pin that has the an adc on it after that depends on the microcontroller as we are going to use the stm32f1 we will need to have ftdi to establish the serial communication with the pc however if you are using um the arduino uno or even mega you will not need this ftdi and you go you can go forward without it the message that we are going to use to send you through the uart will be the voltage measure that we are going to get through the adc plus a new line just to to make this simple for the um python to read it for the program a quick review so this is the program within our pc there will be two stuff the main thread which will manage the user something very easy and simple just quitting the program they will be like a message if you'd like to quit you just push the key q and the secondary thread which will be getting the data and displaying it and why we are doing like this this is because we need the main thread just to handle the user when in the background in a different thread a program that is going to run everything so there is no any interruption or any problem happening so let's see now together and let's jump to the code so this is the first part where we start reading the data the serial data from whatever the microcontroller not even the stm32f1 but also arduino or whatever the peak so what we have to do first of all this is to install the pi serial library so it's quite simple and easy let's just copy this one and pip install buy serial so i just watched it here already install it so no need to stop there too much and after that you just have to import serial and that's all so the magic is quite easy to make but i will call it uh like com3 will be called port do not use magic number and then let's have ser or the serial connection that we are going to use it will be serial dot serial and we are going to put so the port i'm using is the com number three so that's why i'm using that port and the connection at the end of my microcontroller is the 115 hundred and i will use a time out of equal zero okay the amount equal zero after that what we need to do is quite very simple i will put a while true yeah and then we are going to read the data so data will be equal sir dot read line and this is what this one this command will read the whole message until it goes to the end of the line this is how it work it make it very easy and just to let's get the call it ascend data sensor i i know it's a weird name but anyway and it will be equal so what we need to have is data the data what we read and we have to use a decode this one is to make it as normal format for the python so we are getting from something sent in um in binary or something based on char numbers and we would like to have it in a normal string and this is what we are going to use utf-8 okay and after that we print our data sensor okay so let's save and run this code on the um just a moment python here and yeah i'm running this by serial threading and by putting enter i should see some data so now i'll be playing with the potentiometer you'll see the number so we are we're reading the data but the display is quite really not good and this is not helpful at all to to read the data and to have a comprehensive visual or friendly usual um job but at least we established the connection so let me make a control c and stop it interrupted it's really ugly and we are going slowly step by step correct this one okay so this is the first part now we can jump to the next one so in this part we are going to start improving the display and we are going to use this library the windows courses uh if you are using a linux or mac or whatever else then windows so this is a library that you can use within inside python but if you use windows you have to install it again so you have to make a pip install windows cursor so if i put in i already installed it so no need to to wait for that okay so what we are going to add now is import and cursors okay and after setting up the connection let's first initialize the cursor the screen that we are going to use so generally speaking this library is something that you would like to use if you would like to control the terminal and to have more graphical display within inside your terminal to have more control on it so what i have to do first of all is to create a screen here and i will add cursors and init screen this is what will start my terminal screen and after that i will have to refresh i usually do it so for this one refresh okay so and to display you have there is a lot of function but there is one that i usually use because it's simple and quite um intuitive so let me create row equal here zero so you can easily understand the meaning and column equal to zero so we are going to add stuff in the screen here and we are going to put add string not n string but string here and this is the row and this is the column and let's put here value or data okay so in total i'm adding one two three four five six so seven okay so when i'm starting using the uh cursors function i can't use any more this print so i will going going to to comment it first and we are going to read the data and we are going to start displaying it to display it then what we can use is go in here and instead of data we are going to put data sensor okay but for the column as we would like to have display in front of column so we are going to add here eight so from the column eight position the 8th position we start displaying the data and we have always not to forget to add this refresh actually this refresh can be here that would be better and then let's copy again and we do have our refresh so that's should be fine but what i would like to add is before having this let me copy this one and having a small make like this so it will erase the previous data just in case just in case the numbers the total number of digits change okay so these are the cursors it's just controlling um the cursor's position and also having you can add some coloring to this one okay so let's save and go to the terminal and run okay that's not the right place so cd this one and python okay so you see here the number is changing a little bit but that's not as we would like another problem coming from the cursor's library and you maybe notice it if you are following the code and trying to execute at the same time is when you put a control c and you would like to leave this function you are not able to do it because the cursors disable this kind of function that can interrupt the code so one other thing here is to just shut down the the terminal and we i have to run again so i got to open integrated terminal again so the reading here was not really successful so what i can do is just putting like this for the moment and i show you why i added this one so if i put python here and put the code okay there's something getting wrong um python is not recognized here of course because it's not yeah this is not a python okay patch it goes now okay i didn't save that's my bad let's go back again yeah this is one of the problem with the curses library if not used in the right way and we are going to do it in the next part of the code so as you can see i have something crushing it's quite consuming in time and sometimes frustrating so let's just wait a little bit and vs code is not really um so responding with this term at this terminal thing so let's just wait a second okay reopen perfect so i will save this one so we're removing this one and we should let's open this one in terminal and putting python and the code yeah a little bit of improvement but you will see so it goes to almost the one thousand the maximum value but when we go try to go to smaller values something like zero or smaller there is some residual data that stays there let me go slowly and still here it's not really very beautiful yeah it works quite well yeah yeah you have an image but this is really um still fuzzy and we cannot cancel what's going on so in the next part we are going to improve this one so we we have been able to draw some data on the screen but still something not really helpful to to get out of the screen and still not good not good enough so what we are going to do now is to have a kind of main thread that will control like what the user would like to use and another thread that we are going to create where which will be in charge of reading the data and display it in the screen so the user can really like all the program can focus on something and the other thread will be doing something else so first of all we need to create a function that will manage all this reading the data and displaying so what we are going to do is just come in here after this one and we put def here my and we call it my serial for example and yeah and this is what we can do so we just put this while here and also what we can do is to add even this one here so the whole function will manage the serial stuff okay so why this went like this okay so we do have the whole serial managed now but we need to have also another control on this one so we do have while true let's create create a variable like a global variable here somewhere or call it global and let's call it global is correct here so let's call it serial now let's call it uart just to avoid any misunderstanding and we add another one the other side so for um here and we call it global so global and we call it uart and here we say ult is equal to true okay and globally ought and while you are we keep running and doing this so this is the first part so after that we can start creating the mainstream the one that would be interact directly with the user and this one is a while true and we have to read something as an input so to read the input using the cursors library it's a screen and we are going to use getkey so to get directly the key okay not k let's put it equal to key better and if key is equal let's say if the user will type let's say the letter q then what will happen you ought will be equal to false and we are going to break and exit the program okay so that's how the code will be working however in this situation so even if i run my serial here one problem will happen so the first one if i run my serial in this way it will be stuck here without going to this place and the other problem then if we put my serial somewhere here the program will be waiting each time for this screen get key so what we need to do is to work on a threading create this threading that will manage this one and to do that so we have to import threading import threading and we start working and create a threading so let's work on that so first of all we have to create let's call it t1 and equal threading dot thread and we are putting the target equal my serial here where is it yeah and it should be after the main program okay and we are going to add something else this t1 dot daemon and this is if there is an interrupt or something happening equal true this will help to manage a lot of things because when you have a threading running and you interrupt like you interrupt your program this trading will help you to survive and to kill also the other threat else you'll have some troubles and this threat will not stop until you crash your your your terminal and finally we are going to run and start our threading okay t1 dot start okay so these three function together these three functions together will manage a parallel job doing the um reading the data and displaying it where here we do have um some kind of a daemon that will help to interrupt if there is an issue um let's add some indication here so let's add something do this one saying grow plus three and column zero uh insert q to quit for example insert q to quit the program okay that should be fine we can now start and run it actually this is not the one that they want to go for this and here there is we put python and we can have the program okay so you can see now so i'll be moving a little bit the sensor so we are reading there's no issue at all and if i put quit it close the problem without any issue so this is this is really wonderful because i have two thread working separately and doing their job so if i show you again so you see here the number displaying it's not really perfect but we do have some something that we can read and we can work on and in the next step we are improving the display but the program now where we would like to go is working quite better so before to to finish that so let's close this one and manage better the threat so the way that i have been managing here this thread is a little bit um not the ugly way but this is something that we are sure to make but even with this one i would like to add a few stuff so to to really close this thread once for all and to do so i'm importing the signal library port signal you you can skip this one but this is will make your program really clean and also import system library okay so what we are going to do on this one is to create a signal so signal dot signal here again and we are going if there is any interrupt of whatever so just in case there is an interrupt happening so saying and so this is for the if there is any interrupt in this one let's have a signal hendra just like this one and this is a function that will work when we will need to when we will have this interrupt okay and for this signal handle so signal handler we are going to put this inputs signal and the frame so what we are going to input on this one is quite to have a system exit and this is where really clean the whole thing if there's any problem anything additional this is really will create will clean everything and will not leave your system messy free to do not add in it but quite advise you to to use this one so let's run again to see if there's no issue at all yeah so i can change the value i hope you are able to see it's really small apologies but this is the terminal and you can see there the value here okay so this is a step where we worked more on the system structure and now we are going to improve the display so now that we have a better visualization and reading let's improve a little bit the accuracy and um how this program can work and the added value of having other data coming to python so first of all let's go inside the myserial here and our three variables so the first one will be called the average and equal to zero sampling which is the number of sample that we would like to have let's put 200 for start and see how things goes and sample will be equal to zero okay so now each time that we can read the data so we don't do it like exactly so let's remove this one and let's uh work on this so first of all when we read the data let's check if the lan of the data that i'm reading let's put here is greater than zero so it's not greater than zero we don't need to read this one okay and to proceed with the rest means there's nothing in inside the system inside sorry the buffer of the uart communication and so we can start there and after that so we read the data here and as we are getting just the data or the information what we can do is instead of having the decode here what we can do we added a try here and we move this one because we are going to make things a little bit more uh correct and robust so except pass so we can put another thing but here instead of putting the decode only we can put it as int so we are reading exactly the integer from that and after that what we are going to do including average is equal or just be like pythonist plus equal sensor so means the data sensor okay so it means i'm increasing the average each time and sample will be equal sample plus one so plus equal one okay so and then what we have to do we say if sample is equal to sampling okay so if we do have this equal to each other we can start displaying the data so let's move this one like this and what we can do at that time is to say so if we have the sample equal to sampling we put so first of all the let's create a variable we call it sensor it will be equal the integer of average divided by sampling okay so actually it should be average let's put sampling minus one because we aren't okay and after that so by doing this one so screen equals this one actually not sorry we don't need to to put the minus one here as we put the plus so int of this one so after that we we had our sensor so we need to add now so average is equal to zero again sorry started sample to reinitialized equal to zero and from that we can add some stuff here so what we can do so let's go here and put a screen add how to put it now i think we can we can add it here so screen and we can correct it later on uh add sdr not n str and we put as usual so this is row and call plus it's let's say yeah like let's say fifth i'm not sure exactly but um let's put like this eight and then what we can do is to put some spaces here and add millivolts okay and in this one in the data sensor here we are not going to use data sensor anymore but what we are going to use we are going to use seth's f sentence which will be the int let's put this one so it will be the end of the sensor multiplied by 3.3 volt and that's that should be fine so what we are going to do here is to have 200 reading and this will be much faster because we are just going to read read read we are not going to go inside this one so because the display itself takes more time for the micro microprocessor of the computer so we're really reading more and doing a lot of internal job that will not take that much time and not make the buffer quite overflowing and then when we have a sample we are going to display it okay that should be fine so we can save here and quit go back again and see so we do have this one looks i'm moving now the potentiometer and you can see that the value is changing quite well and quite stable i don't have any variability the only thing that's still a little bit um not really beautiful so what we can do we can add here and add screen adds add string here each time and we put instead of row zero we were putting row one we put nothing just an empty stuff just to to make it clean so let's try it so i'm pushing q to quit entering again yeah now it makes more sense all like what i can do is just a zero here i think that that will make much less trouble and you can see now so you see it's a very beautiful very clear uh way to display um in the terminal it's not really very beautiful but it's very easy to read data and you can see it's a millivolt so you can see how many millivolt are applied between at the um peripheral for the um adc that we do have at the stm32f1 okay that that was the last part so we do have a reading from the terminal in a very friendly way um i hope you enjoyed this uh we python episode uh see you for the next one

Channel: WeeW - Stack

Views: 3,077

Keywords: python, threading, thread, Curses, curses library python, arduino, arduino serial read, Python Pyserial, Python arduino, Read data from arduino using serial, Using serial data and python and thread, reading analog data suing Python

Id: z96YH3xGOYA

Length: 32min 12sec (1932 seconds)

Published: Sun Dec 13 2020