Webcam Video Augmentation with an Image/Video using ArUco Markers | OpenCV

hello today i will show you how to use the rocco markers to augment webcam media with input images or video using python with opencv let's start with a short plan of what we will do first we will prepare the video setup for our camera next we will create and detect the roka marker after that we will augment the roku marker with an image and we will finish the video augmentation for this project we will need an opencv contract python library the difference between this and opencv python library is that this one has their rooka model also we will need numpy we will start by preparing the video setup first we need to import opencv library import c2 next we need to initiate the video capture most of the time your camera will be auto detected using only 0 and that should be enough next knowing that the video is just a bunch of images of frames we need the while to go through them we'll write while true and unit underscore comma frame equal to cub.3 to read all our frames i'm also adding one additional length rotate a video by 180 degrees use this line if your camera is rotated 180 degrees like mine to show the frame we'll write c2 dot i'm sure and in brackets we add window name input and source of what we want to display frame what's left is to add a couple of lines which will help us close the window and stop the code we add if condition to close the window when the escape button is pressed if the condition is true it will break the loop and we finish by destroying all windows let's run recorded tested we can see the input window and that the outcome is working let's move to the next part in this part we'll create the roku marker okay so the roku marker looks like this rocco markers usually are used for camera pose estimation but we can be used for any tracking we want rocco markers are used because the detection is robust fast and simple the author of this page is ola kolacho check the video description for a link to this page here you can simply create the roku marker you just need to select the dictionary the bigger the size the more unique ids can be generated in that dictionary will generate the smallest 4x4 next we need to choose the id let it be 7 and the size 100 millimeters at the bottom you can save it as svg or pdf let's move to the code ok first what we want to do is to import aruka import c2 dot aruka as zeruko next let's define the marker id we have ad7 after that we can define the local dictionary rocco don't get predefined dictionary in brackets you write roku.dict besides we choose the same we created 4x4 and you can choose how many unique values you want let's choose the lowest 50. what else we need is to define where parameters aroka detector parameters create now let's move to the main while loop here we will detect our aruca marker to detect a marker first we need to convert the frame colors him to grayscale and after that we can detect the marker recall dot detect markers in brackets we define the image dictionary and parameters as you can see we will get three outputs corners its corner coordinates of our marker ids and rejected markers we will use the first two corners and ladies okay so before drawing the detected marker we need to check if we detected the right one for that we will create the if statement we will check if we detected anything and if we detected something we will check if it is the id we are looking for if so we draw a detected marker in brackets we define the image we want to draw on it will be the frame and what we want to draw corners and show stays the same because we are drawn on the same frame let's run the code okay now there is nothing let's bring a marker okay it is detected and you can see that the top left corner of the marker is marked with a red square we can change the position of a marker however we want it is detected very well let's move to the next part in this part we'll use a rocco marker to augment the marker with an image okay so basically what we want to do is to change our marker to the chosen image will red detected the marker in its corners what we need to do is to warp our image to the marker image position let's start by importing the numpy library as np next let's define the image we want to warp on the marker okay now let's move to the main while loop here we already detected the markers corner coordinates let's print them to see how they look before running the code let's change the weight key from 1 to 0 it will stop from looping let's run ok you can see that it isn't clean list workload we will need to clean this list before starting you can see that coordinates into nested lists and coordinate pairs are in separate lists okay let's redo the code as it was and command to draw detected markers line we will create a new function called augmentation the inputs will be our markers corners with np array we will read off a text in a list but we will create an additional list to restore the number of lists we will select the first element we will list in nixon next the input will be the frame and the image we want to augment on your marker okay now let's move up and create a new function the same inputs as bounding box for marker corners our frame will be image and image augment first we need to unpack the bounding box we will start the top left corners x coordinate you will get it with list indexing the first zero index will get us in remaining list the second zero index will get us in the first coordinate pair and the third zero index will get the x coordinate value the white coordinate will get the one next we get top right bottom right and bottom left corner pairs next we need to know the dimensions of the image we will use document for that we use the shape method and unpacking okay now we need to create new numpy arrays with coordinate pairs for our bounding box and the image we will use to augment next we need to transform the perspective from the imagery augment in the actual bounding box for that we use the final homography method which will calculate the perspective transformation matrix knowing it we can warp our image with var perspective method we just define the image we use to augment the perspective transformation matrix and the output dimensions we still aren't finished we get only the image on marker position we also want to see the camera view outside the marker for that we will fill the marker position in the camera view with black color and after that we can stick these two images together by adding one what's left is to return my image out let's run the code the marker is replaced with the image we have and we can move the marker however we want the marker stays detected very well let's move to the last part in this part we'll augment the marker in the video first let's define our video i will use give format because it runs smoother than a video file but you can use a video if you want next we need some new variables the first one will be detection and the default value will be false the second variable frame count is to count video frames and loaf a video the starting value will be zero also we need marker size variables we can choose any values you want because we need this value just to initiate a process the size will be changed to necessary in the augmentation function after that you can read the video augment to get the one frame and resize it to the marker size okay that's it for the start next we need to move to the main level here we first create an if statement to hold the video if the marker is not detected if detection is false the video shouldn't start so the frame is set to zero and the frame count too else we get the image video but first we need to have one more if statement to loop the video we check if the number of frames we counted is equal to the frames the video has if it is we reset the video in frame column if not we get the image video and resize it to the marker size and what's left is to write detection is true when you find the right marker change image augment to image video increase the frame count by 1 at the end of the while loop okay we have a marker augmented in the video let's turn your code looks good that's how you can use aruka markers to ignite webcam video with input images of video hope you liked this video if you have any questions leave them in the comment section you will find my code in the video description if you like this video please leave a like and hit the subscribe button for more python videos

Channel: Giovanni Code

Views: 1,988

Keywords: python, pygame, code

Id: D1biUn9X7H8

Length: 10min 10sec (610 seconds)

Published: Tue Jul 12 2022