Automated People flow tracking | with Computer Vision and AI

security measures please present your passports with your boarding passes hi welcome to this new video I'm Sergio I'm a competition consultant developer and course instructor and I build competition solutions to help companies improve their process efficiency reliability and scalability today we're going to see a people flow tracker I built this solution like a prototype to track the flow of people on Big areas that could this could be airports it could be shopping centers or like big spaces also outside in the city and so on we will see in this specific video if you're a developer how to build such solution from scratch so what are the steps to build such Solutions and also what are the critical points when we build uh tracking software in not control light environment like for example this one that you saw at the beginning and also if you have a company you will understand when it's good to implement such Solutions when what are the limitations also like from the camera that you already have if there are certain limitations when integrating object tracking and so when it's good to invest and when it's not possible to do this we'll see everything right now let's go let's start talking about the software side like what are the course of such program I've identify like three cores to build this specific program the first one is object detection the second one is object tracking and the third one is object Counting of course it's more complex than this because after the counting we can have the trajectory we need to understand the flow and all of these must be packed into a software that can that can be used on different cameras uh different uh different computers different Hardware like there is some complexity behind that but the code is auto detection tracking and Counting first one let's see this object detection object action means associating a bounding box per each object detected I'm talking about object because object detection is in general we can identify count track anything but today we are talking uh specific about people so in this case we'll be put a bounding box identifying each person we have object detection this is pretty simple you might be already familiar with this because we see a lot of this stuff on the internet where there are videos and there are like new algorithm coming every month for object detection and this is pretty common knowledge at least now in 2023. what gets more challenging is the second step and it's object tracking nowadays still this is a challenge because detecting a person and tracking the same person is quite a different task and we will understand now why after object detection let's move what is object tracking object tracking is the part of the software that will allow us to track people or to say it in simpler words associating a universal ID for each single object let's take for example uh this ID right here you see ID 18 so this is a bounding box for a person with the univocal ID Associated which is 18. let's say for example this one somewhere in the middle which has ID 3. when we use object detection on a video or in real time a video is nothing more than a frame after frame so when you see also like for the cameras it's written also FPS which is frame per second where like standard we have 30 frames per second faster camera 60 frames per second and more and so on so it's frame after frame like for the computer it's just picture after picture and there is no connection like no tracking history of what was uh that person like that object before that's why with object tracking we need to follow like the object during time so from the previous frame to the next frame until the object exists in that specific video and so when we associate the universal ID to the object and you see in this example now I'm taking like the three as an example when you go we go to the next frames and that person is moving to have a good tracking we need to still follow that person with the same ID if a new idea is associated uh it gets lost lost it means that the tracking is not working well and so in this case now I'm of course this is a video sample that I'm using for this prototype I'm freezing now frame by frame so that you can follow along and we can check that now that ID3 is always edit three even after like we move on this one uh is still always id8 this is ID uh 11 and so on uh there are different algorithms for object tracking and this is getting like of course this is improving over time like each month each few months you get a newer better angle algorithms but the concept usually of these algorithms is to uh to associate an ID to the object let's now talk again of this ID3 uh the algorithm will uh follow this object for a few frames so it will just predict where this object is going to the next position it will also understand how fast this object is going so if someone let's say uh if the person 3 is here let's say that the person two it's not the case but let's say that the person 2 is running very fast after a few frames the algorithm can predict what will be the next position and where we are going to find that specific objects this is one of the things that this this sort algorithm for object tracking does because it makes an estimation of the next position we have deep sort which is slightly more advanced with OC sort with byte truck we have strong sword we have so many algorithms if you check them and each of them has pros and cons will not get into this specific things in general when we have a big space and a good camera positioning even a basic algorithm can do some good tracking and very high reliable prediction of that so object detection first object tracking is the Second Step what is after object tracking we have object flow or object counting let's see that as you see in this image I have some re region defined you can see like red lines and this is what I use for object counting first we Define a region of interest and it's a specific area where when people are crossing so in this area we put a line where people are crossing that specific line then we add the count uh let's say an example so this is the area that I have we will only detect people that are inside this area why do we need a region of Interest that's very simple because where we take a very big region and we have like such huge area there are Parts where we of course cannot do the tracking so all the parts that it's outside of this region very far uh there might be there are hundreds of people which are so small that we can't uh that we can't identify so the object detection algorithm uh the object tracking algorithm will just give us uh not good not good information because it gives we get a bounding box and also the dids will be pretty much random of course they will give you an idea of how many people are there but they will not be reliable so unless you have a camera placed on top and you have a very small region which is rare the case from CCTV camera it's always necessary to select a region of interest for the area that where we want to do the tracking so what I will do now we have a defined region of interest is in the red I will now I'm showing you the bounding box for everything in the screen what is out of this region will not be detected so I'm going to deactivate from the code all the detection of everything that it's outside of the this region so this one must be removed from the detection I will do that and we will focus more specifically now on the counting right now as you can see from this footage we have just one person inside uh the region of Interest so one person inside this the fine area right here we are only detecting and tracking that specific person to this specific person the ID 2055 was associated and of course if the tracking is working well this ID should stay with the person when how long should this stay with the person as long as the person is inside the region of Interest and what we will do you can see that I have a full region but I have also another like line where the person is crossing like this line is just a center of a region where I'm I'm counting like the specific person so once the person enters inside this we can say I did 2055 is inside so we're going to save like we can save this either on an array if you're working with this and you have time on our database we can say ID person 2055 cross the line and we have like counting one id255 then we have 302 302 didn't cross yet so we have 302 53 so what is now the tracking we added three ID because we check on every frame is ID 302 inside the region of Interest like the specific counting area if yes add these two people that cross the line so we will say uh 2055 53 22 they cross the line how many people uh cross the line today in this specific area three people because we have like Daddy for each of them so once we get the tracking like uh correct tracking then of course like the counting is a very simple programming operation you just need to add the ID to to all the ideas that cross the specific area how about the flow because this is not only counting but it's it's also about flow what we are doing to track the flow it's still a simple operation because let's take into account the ID 302 or maybe let's focus on some other ID so I will find some other that is crossing the area so we will focus step by step and try to understand what is the flow of that so the person is just crossing the area what can we do we can store like and track these over time so in the moment when we see this and ID so we have object detection of the bounding box the ID is associated we save this ID and also we save the position we can say like the bounding box we can save the center point this is the first time we see the 843 let's say that we save the center point this idea is of course Crossing frame after frame let's say a few frames so each frame we have like some new point like this so let's go this way so all the points will be like this of course these are over time so this is the first point and this is the last point so one two three four five six seven eight so this is the eight points okay this doesn't look like eight but it's an eight this is the eighth point uh that we have uh for this how can we understand the flow of course this is basic geometry now because if the first point is right here and the last is right here we know that that direction is that one so very simply the y-axis in this case if the last y-axis of the idea 133 is bigger than the first one then the flow is up if it's the opposite then it's down very very simple we have like the movement of the person over time and with that we can understand it's going up the person is going down it's going left right it's going like Northwest so it's like different like if you have a lot of directions we can also calculate that and we can also estimate if the person is going always straight we can also estimate like where the person is going uh based on this like if we want to make a prediction so there are many things that we can do when we check the flow with this information that I'm showing you right now so storing this over time we can also show like an arrow of where the person is going so let me now activate this function from the script and let's see what information we can display once we have the history and trajectory of the person as you can see right here we can display an error so they are overlapping now with the number but you could see there is a blue error so blue arrow it means up and also change the color to make it more understandable when it's green it's down and I'm saving and displaying few points so you see the line that I'm tracking and drawing behind the object is like the the flow the trajectory of where the person was plus the arrow with the basic geometric concept that I explain going further with this footage we will see now ID 334 we have arrow down and we have like the last few points of where this person was going so you see all the trajectory 334 it's going down and so we have this down arrow down and this app Arrow up and we can easily understand uh their trajectory this of course can be structured in a way that you can save this on a database you can save like uh the moment so you can say like also a timestamp on the database of each specific ID you can also have more big elaboration of of the trajectory if there is some improvement to make for example on on the flow right there and so on so there is a short line but this uh can be longer after web search information we can save them on a database and we can elaborate the information and display that on the screen let me show you a very basic graphic that we did for this uh to show like this demo on the screen and we will talk about that this is a basic graphic that we built with also like basic informations like uh total people of the day today so we have like the people going up this value going down this value then we have a graph this graph is also generated by this software which uh can show like in this case like current month or it can show like they of course this very cost custom graph that depends on the need can be done and of course this is maybe it's not even like the best graphing also distribution to put such information but this was just a demo to display what we can show uh we search software and then of course we have like the average people like per hour from Midnight to 8 in the morning from 8 to 12 from 12 to midnight again and this information of course will be updated in real time when the software is running real time I'm running this on a video footage but of course the same software like this instead of If instead of running this on a video I will run this from a camera it will just work in real time without any problem let's quickly see how this performs it's very simple we have a person which is going up this person right here when this person crosses this line we shall see this value 1412 to increase to 1413. if that doesn't happen it means the software is not working so the person is crossing we see the area so the line when it's counting lights up and we have 1413 and of course it increases also this one and over time you will see also this happening in the graph uh let's see some other person so now it should go to 1414 it does this increases also so a higher number and so on let's now discuss briefly about the hardware that you need for uh such to run such software nowadays with the Improvement of the algorithms also not only the hardware is improving but the algorithm are improving so we need even less powerful Hardware this means that with a laptop we can easily run this in real time uh with also with smaller devices like the Nvidia Jetson nanoc severe we can run this of course for such smaller devices uh they have like limitations so it's also like the they need to be somehow improved like to get a good performance but with lapper laptops or like uh computer with Nvidia RTX graphic cards we can run more than one camera per single computer and with the Improvement of this algorithm we will be able to run multiple multiple cameras with just a laptop and another important factor that I mentioned before but uh I I can't stress this enough is like the importance of the positioning of the camera and the tracking because when the camera is not properly positioned and takes uh people from not a good angle so the more is on the top the easier is to track your people because the more it's close to the ground uh the the angle makes people mix together and it's very hard to uh to track them so that's uh very important and also when of course uh it happens that another ways you can choose where to place the camera because the cameras are way already there so it's good to make a study of the specific specific scenario and choose like what region of interest to focus on so I hope that this information was clear from the point of view of like development if you want to build such solution I have a few other videos on YouTube where I explain like the difference between object detection and tracking where we got also more in depth about uh the code where other videos where I show object detection tracking different scenarios for manufacturing companies to identify defects on an up on objects to uh get the speed of the vehicles and so on so I have many other examples that you can watch on YouTube on the channel if you are interested to watch them if you want to learn how to build easily and quickly object counting and tracking software and you want custom codes uh ready to run for this I have a course called audio detection with opencv and deep learning you will find the link below where there is a structure easy structure to follow to build such Solutions and the co and there are codes with projects provided like to build similar solutions to the to this to detect objects to track objects and to count objects let's talk about the point of view of a company when is it good to integrate such solution and when it's not too good to integrate such accounting and tracking solution with AI we're in a phase where still there is a high cost to develop such software because there are custom software they need to be there there needs to be developers who made such Solutions so there is a high cost to build such solution the advantage is that such solution is scalable so once you build a solution to track from CCTV cameras for your specific needs and if there are there is a lot of flow and traffic that can be easily scalable so to put this in perspective if you have as much shop or if you have like a small activity and now the counting is done manually uh probably it's not worth the investment at the moment to implement such solution if you have a big flow and and like thousands and thousands of hours of video to process like each month then such solution will have a huge return of investment uh over time hope this video was useful uh I will be back with other similar videos regarding object detection tracking counting four different um for different fields like manufacturing like factories and so on this is all for this video see you in the next one

Channel: Pysource

Views: 11,166

Keywords:

Id: COYUiWxthMc

Length: 23min 15sec (1395 seconds)

Published: Tue Mar 07 2023