Python OpenCV 3D Scanner

hi my name is ulysses ordaz and this is my project presentation for advanced embedded systems of renter 2021. uh my objective was to create a 3d scanner using opencv now the reason i picked this project is because sometime last year i picked up 3d printing as a hobby and i chose this project so i could produce work similar to what you see on the picture on the screen so there are many methods of generating 3d images so the challenge i chose was to to generate these 3d images using opencv but what exactly is opencv opencv or open source computer vision library is a library of programming functions mainly aimed at real-time computer vision now before starting the project i had no experience with opencv or python which is the language i chose to put this project together i did know a little bit about 3d printing rendering objects and solid works and i have a good c and c plus plus background i didn't know where to start but after a brief consultation with brother grimmett i found that i had to find a way to generate a point cloud i had a few different tools at my disposal but i ultimately opted to use an intel realsense d415 depth camera to help me with this objective so with this in mind i was able to put together two programs they use a combination of opencv and an open source library available on the intel website that allows you to access the depth camera's functions and i'd just like to give you a live demo to show you how my project works so i'll just give you a brief rundown of the project uh first we set up all the hardware functions of our depth camera so we enable the camera we enable the depth sensors then we have this while loop that consistently feeds the camera's output into an open cv generated window and then we have commands we can give to our program once we run it so we have q to quit e to generate a point cloud and v to view set point cloud now these are functions that are a part of the intel realsense camera so we just access those using python and i just like to show you the output of the camera to the right we have the camera's depth map perception rendered in blue and to the left we have a real time color rendering so if i press e it freezes up for a little bit but after the ply file is generated it'll start moving again we press v we can see the ply and again this part uses opencv to be able to view this ply file so you can see i'll take another picture from the front press v to view it and there you go the quality is pretty good for this version of the program it's the difficulty is that it is generated as a depth map so it's kind of hard to tell what is what if you're working with multiple files then we press q to quit now the second version of the program is a little bit more complicated because it uses a special algorithm that i'll go through later but as soon as before we we go we activate the depth the camera's depth sensor activate the camera we have these functions to project our our point of view at an angle just so we can tell better tell uh what we're taking a picture of and then we generated point cloud and in color so this specific algorithm i got it from the from the intel realsense software development kit and it just places that color in in a uv light rendering onto the ply file you generate and this program does generate ply files with a with a higher resolution so same as before we have our our old commands p to pause e to export or create that pi file v to view the ply file so i'll just show you let's take a shot let's press e and this one doesn't freeze up and as you can see the quality on these shots is much higher and you can very well tell that's me let me take another shot at an angle you can see me in greater detail and this version if you want you press p to pause like i mentioned and then it freezes up your point of view you can press e to create the ply press v and it takes that freeze image and generates a ply file for that as well i'm going to press q to quit we successfully exited now one thing that this project falls short on is generating a render of whatever object you're taking pictures of real time but for that we use this project called mesh lab so with mesh lab you take the ply files and you manually glue them together so here i have the scan of my head i took a few pictures of my head and then put them together using this program then we can export that into a stl file generator like cura and we can generate those files for 3d printing later on so i'll just show you how the scan how the 3d printer will slowly print my my head in this case this has an application for many things we could print out cards you can print out figures you can print out models of of anything you would like to scan as long as it's uh somewhere from a small to a medium size object i'd say somewhere up to car is the safest we can go with with scanning and there you have it now uh a few of the challenges i did run into was understanding the software development kit from intel but after getting the hang of it and how i could generate point clouds using the intel realsense camera it really took a lot of layers of complexity away from a project uh i did have trouble of figuring out what to do with the pi files but meshlab took care of that issue as well and it was very convenient that those ply files could be uploaded into cura to later generate stl files that can be 3d printed some possible improvements would be to increase the size of the objects in the ply file if you noticed um i had to zoom in a couple of times when i would take those pictures and while uploading those those 3d images in cura i get the error message that the images have to be magnified greatly so that's a potential fix for the future another improvement i could add to it would be to generate a 3d render of the object in real time so that way i could um just avoid using other software to process my files aside from aside from cura to generate the stl file and another thing i tried to implement it but didn't work was to create a simple for loop just to avoid overwriting the output ply files but it seems that this action isn't supported by the intel software development kit so i just went down with my project now from this project i was able to achieve my my objective of generating these 3d 3d images i learned how to how point clouds are generated i learned how to use opencv i learned how to use a intel realsense d415 i increased my knowledge of python programming and i feel the project was relevant to my career because i'm interested in going into test engineering and working with embedded systems and with canvas with cars so i feel that learning about opencv would help me in this with self-driving cars and just computer vision used in vehicles and other app applications and with this video submission i will include a link to the to the intel library that i use to support myself and a github with my projects but i thank you for your time and thank you for for attention during this presentation and i hope you enjoyed it

Channel: Ulyses Ordaz

Views: 28,770

Keywords:

Id: WZCeX1GbPl0

Length: 10min 12sec (612 seconds)

Published: Thu Apr 01 2021