3D modelling with the DJI Mini 3 pro (using photogrammetry)

foreign [Music] thank you [Music] hello everyone today new video a little bit different than what I'm usually doing today I'm going to talk about photogrammetry and more specifically how to do photogrammetry with the mavic Mini 3 Pro it actually applies to any kind of drone or almost even every type of camera I'm going to talk to you in this video about how you can do this effectively with such a drone so uh let's go foreign [Music] so first of all we need to Define what is photogrammetry maybe you have heard of it but you probably don't know what it is so let me just clue you in quickly photogrammetry is the process of taking pictures with either a drone or any kind of camera your phone also works and to create a 3D model with a texture of an object or a place a person whatever subject you want to shoot just solely from photos so you could potentially use a lidar to do this more precisely but they're very very costly they use a lot of power and it's generally speaking not something that the everyday guy is going to have access to foreign [Music] so there are many two uses of photogrammetry the first one would be more scientific when you want to capture all the 3D aspects of a place like in archeology you have a tomb that you have just a couple days to study then it's a good idea to just come and take a lot of photos and it allows you to have like a digital twin of a place that you want to study another thing that you could use it for is inspection of I don't know a telephone pole or whatever something higher up something hard to reach you could go there take a bunch of photos recreate the 3D aspect of it and see if things moved or if everything looks right and it gives you more than just photos because it has the 3D aspect of it which is very important now the second use of photogrammetry is more for an artistic purpose so it allows you to capture real life objects like vegetation or rocks or things that are typically hard to model and to create a 3D model that can be used either in video games or in a 3D render blender or whatever short movie whatever it is that you want to do [Music] so when you're doing photogrammetry there are a couple things that you need to take care about and you need to keep in mind when you're taking the photos it's typically different at least from a drone perspective it's different than taking a video so that it looks good in photogrammetry typically you're going to want to take a lot of photos of the subject from all the angles all the angles that are possible to take you should take that and something very important is that you need to have an overlap between successive photos so it's typically recommended to use 70 overlap meaning that you're going to see 70 of the previous image in the new image you're just going to have a small band that is new most of it is the same that's because in order for photogrammetry to work it needs to recognize the patterns in both photos and match them so you know this rock is that Rock This Cloud is this Cloud whatever it is that you see it needs to be able to match the two to try to solve what the camera angle was at the time of the shoot now you could from a drone capture a video and use the frames the steel frames extract that from the video and use this as a basis for photography and that works but you have to be careful about the motion blur that you may have because you're just moving quite fast and it's also better to take pictures with the Drone because the geo location is actually encoded in the file so you have the GPS coordinates in the file and that really helps the photogrammetry software to know what the relative orientation of all the shots are it uses that as an input and it converges much faster and it gives overall better results so you should prefer taking photos rather than taking a video and then extracting the frames if you can another thing that you need to be careful about is the shutter speed now typically when you shoot video you want a shutter speed that is about double the frame rate so if you're shooting at 25 FPS you should have a shutter speed at around 1 50th of a second that gives natural motion blur and looks more cinematic and and it usually requires you to use an ND filter which is a neutral density filter it's like sunglasses for your camera so that you can lower the amount of light that comes into the sensor otherwise it's just completely blown and Overexposed and unusable now for photogrammetry you want to do the opposite you actually want the photos to be as sharp as possible so you're going to want the shutter speed to be as fast as possible while still having a good exposed photo so typically the way that you're going to want to tune the camera is going to be to have the lowest ISO possible and then a good exposure that you either can let the automatic mode do it or you could just adjust the exposure it's better if all the photos are evenly exposed that really helps the photogrammetry software to work well and then there is one last thing that you need to consider is the white balance I highly recommend that you just set the white balance manually so that it doesn't change and shift if you go into the Shadows or if you go into the sunlight for example [Music] foreign [Music] okay and one last thing when it comes to capturing photos for photogrammetry you should definitely go out in a day like this where it's overcast why is it good that it's overcast it's good because it's evenly lit you know when you have like a bright sun and then a blue sky it looks beautiful in video and it looks nice in photos but for photogrammetry what you want is an evenly lit subject so that you're not going to have some light baked in the texture and it's usually better and then you can re-light it in post so to recap use ISO 100 prefer High shutter speeds do not use any ND filters prefer an overcast weather if you can choose the day of shooting slower means sharper images prefer using a manual white balance so that you can have consistent colors between all the shots and finally prefer using photos to a video that it would split foreign [Music] [Music] so the first thing to do is to take a look around and make sure that you're not going to have any obstacles that you could run into because of course Safety First you don't want to run into objects with your drone and lose it you also have to make sure that you don't have too many people around try to avoid flying over people directly as per the rules so here a little bit of settings you can use autofocus but I highly recommend that you're going to use a fixed focus and set it to Infinity because here we're far enough from the the subject so that it's just going to work well and for the rest of the settings you're going to want to put jpeg only you don't actually need the raw files for this set the white balance accordingly you could try to have it do automatically and then maintain it static so here we're going to use 5100ks I think it's about right maybe it's a bit too cold for my taste and I like the 550. it's okay you can see this guy up there it's mostly neutral at least on my screen so okay we're going to keep it like this make sure that you have enough space on the storage device and use four thirds because that just uses the entire sensor instead of just focusing on a 16x9 aspect ratio so you want as much detail as you can like I said before ISO 100 which is the minimum and here I'm going to leave the shutter speed in Auto and it should have a consistent lighting at least for the subject it doesn't matter if the sky is blown up because I don't care about capturing the sky right now okay so basically what you're going to want to do is an orbit around the main subject and ideally you're going to want to do the Norbit multiple times at different heights so the closer you are the better resolution you're going to get but be mindful of obstacles so one thing that you can do with the Mini 3 Pro is to select a Zone and then kind of Wiggle around because right now it's just detecting that zone and now it has converted this to a GPS point and now I'm just going to be rotating around that point and no matter where I go with the Drone it's going to keep pointing at the right spot which is what we want here so now we're just going to start taking pictures okay so one thing you could do either you're gonna take a picture then you move a little bit take another picture move a little bit so on so forth or you have this new functionality here that is called interval shooting but you can't use it in Focus track so I'm going to keep taking photos manually but if you do not really care about using Focus track to use the spotlight mode which I'm using right now that's the little green uh marker at the at the middle you would be able to use the interval the interval capture so that you can just have like a smooth motion and it's just going to capture every maybe two seconds or whatever you set it to so I'm just going to do the captation manually so it does take a while make sure that you have a full battery before you start doing this because it is it is going to take a lot of battery so you can either try to stop every time so that you have a stable picture here I have one four hundredths of a second so I could actually probably maintain my speed like this two meters per second seems all right you get to capture photos what you just want to avoid is having motion blur on the pictures now the more photos you have the more time it's going to take to process but the better quality you're gonna get and if you have too many photos you could always skip some of them in post now note that right now I'm just aiming at this Castle ruin but I'm also actually capturing part of the village in the background and this will show up in the final render once again I want to bring your attention to the fact that you do not have side obstacle avoidance sensors on the main E3 Pro only forward backwards and bottom so be very careful where you're flying make sure that it is free of any obstacles that you could run into and if you're not sure just stop the process turn the camera around make sure that you have enough clearance okay so we are arriving at almost one full Revolution which is good what you might want to do then is to get a bit closer keep capturing photos and you might have noticed that right now I've only captured the main part of the castle so I'm gonna undo this and I'm gonna start taking more photos but right now it's going to be top down so I'm just gonna go straight up right there and I'm just going to start capturing photos from above and that's going to allow me to have an idea of what's inside the towers as you can see here since there's no roof anymore we can see inside and it's a good thing to have an idea of what's inside and it is going to show up in the final result so do take some time to do that it's going to drastically improve the quality of what you produce I'm going to show you how you can do this with interval photos so you just select the relevant option then you're going to want like a relatively fast come on okay so now as you can see every two seconds it takes a photo which is much less annoying than doing it manually now like I said try to capture as much photo as you can and it's better when you follow a continuous pattern so either a grease pattern or an orbit like I did earlier so for example here I'm having a 45 degree angle more or less so it allows me to capture some detail that I would not be able to see directly vertically the castle walls of course if you really have the time I would encourage you to go lower down and capture more detail into the wall structure something that I did not do needs to go directly above but on the right side now remember that what matters for photogrammetry is The Parallax that means there needs to be motion between two shots for it to make any sense if you just rotate the camera without actually moving the Drone it won't actually add any detail it's just going to be wasted basically so I did one like this I could also do one the other way so note that here I'm closer so it still can be useful to take these photos although I'm going much faster now I'm angling the camera 45 degrees so that I can capture some detail on the walls all right stop the automatic capture and now we can just uh probably go back still have 46 percent of batteries so I could just keep going again the more stuff you capture okay let me just do a pass like this lower down to the ground they might follow canopy and trees okay like this and slowly travel capture the detail on the wall like I said if you're not sure just take a look turn around okay it's okay no trees are in the way also if the sound is annoying you can actually disable it I just like to have it to know when it's taking a picture even though there's a countdown in the middle of the screen it can be useful to have this information okay now I'm done and I can just go back to where I come from let's go and that's how you capture something for photogrammetry [Music] [Laughter] [Music] so here we are in front of the computer and I'm now going to show you the workflow using 3D apps Dev here but the process is usually very similar no matter what software you're using so the first step is to create a new project and import the photos for this example I'm only using a portion of the photos that I took I took a total of 287 photos but I hear I'm using a subset of 50 photos so it's going to go faster just for the purpose of the video and at the end I'm going to show you what the final 3D model looks like with all the quality that it can have so one important note is that when you're taking individual photos it usually recognizes what camera was used and the focal length and that helps the software converge faster so that's also why you want to use photos instead of frames extracted from a video but this is something that this software allows you to do or you could just use an editing software video editing software to extract the frames manually now these settings have to be adjusted depending on what scene you're using and what scene you're trying to model usually because this is a drone we can check this sequential box it usually is going to go faster because it knows that the photos have been taken in a sequence instead of randomly trying combinations of photos another important point about using individual photos with the DJI drone is that the geolocation is embedded in the file so here you could see that every photo it hasn't done any alignment yet but it knows that the photos have been taken at these places because it's embedded in the file and that is going to help the software converge now like I said this is just a subset I had like multiple orbits and I had many many more photos here in the middle but for the purpose of the demonstration it's going to go much much faster okay and here we are it has successfully matched all the photos which is very nice sometimes some of them will not be aligned and it's not going to be taken into account in this case you can try to rematch some of the photos with specific ones that you know are closer but it doesn't always work so here we have our sparse Point Cloud as the name suggests it doesn't have that many points especially because I didn't use all the photos but it gives you it may not really show in the video with the YouTube compression but if I de-zoom uh you could see that the shape of the castle is mostly there which is a good sign of course some parts for example in this region as you can see these are the photos these are the camera positions in blue so if you double click on them it shows you the initial picture and you can see that here this whole wall has a huge hole in it and this is because there are no cameras here I do have the photos I just didn't include them in this data set so this part is going to be pretty bad the rest seems to be okay we see that we don't have any resolution here either because it was just the top down photos but the photos that we took later on with a 45 degree angle are not included but this is going to be good enough for now so the next step is going to be to generate the dense Point cloud this basically uses the cameras and the already found points to try to interpolate and find what lies in between those points so it's going to densify the point cloud and create many many more points that we are then going to be able to use to create a mesh and here we are the dense cloud has been successfully created and as you can see now the shape of the castle really starts to come to life especially if I de-zoo it's easier to see because the points are bigger so like I said the holes are still present I mean it didn't create everything out of nothing but roughly the shape is there and this is quite satisfactory I would say given the limited amount of pictures that we have so uh we're gonna go ahead and launch the 3D model generation from the dense Point cloud I'm just going to keep the default values here because it's just a test of course I encourage you to experiment with different settings and see what works best in terms of computation time and result it really depends on the use that you have of it okay so it is already done it took about three minutes of course with the whole data set it took about 40 minutes more than three but here obviously the quality is not going to be as good we can switch to the wireframe view you could see that this Tower over here is slightly better refined because I did keep a bunch of the pictures around here and you can see that it lines up pretty well with the pictures uh the roof as well is pretty good this whole part is actually pretty good because I've just kept a lot of the photos if you can double click here you see that it's perfectly well aligned uh and same thing goes for the next picture so again the more photos you have the better refined it's going to be here you could see where we didn't have much data well the mesh is much rougher you have fewer points and the shape also is mostly straight but not quite as good as the real thing here as well we can see that we didn't have any photo in the data set from that point as you can see and so it did manage to resolve the fact that this is a wall that is 90 degree with this one but on that side it doesn't have the information because it never appeared in the data set that I used here be mindful of that because you're going to have some surprises and the last step after the mesh creation is going to be the texture generation same process just keep everything default it usually works relatively well let it think for a little while and I'll come back when it's done and here we are with the final textured mesh so as you can see there are a little bit of blobs here and there this is because it managed to correlate some parts of the picture here but it's actually the background it's the other Valley so we don't actually want this in the final render but it's relatively easy to clean it up in another software or even here you do have a lot of tools mesh filters and you can just fill holes or cut with a bounding box to remove all the parts that you don't need so this is relatively good for just a a test that took about less than 10 minutes but of course if I use the full data set it's going to take much longer but the results are going to be much better but there is something that this software allows us to do just with the data set that we already have and that's called photo consistency optimization so basically when you have the mesh like this without the texture you can actually refine it using the photos that you already have and what it does is that it tries to optimize the location of the vertices and the faces so that it matches better the different photos that you have this is a costly process it takes some time but when it's done it's usually much much much better so let me just demonstrate how much better it's going to get so here we have the photo consistency optimized mesh and I am actually right now in wireframe mode and if I just zoom in you can start seeing the little triangles that comprise the mesh but if I de-zoom you could see that the mesh is actually much much much denser than what we have before and this is basically what we wanted to do if I try to compare this is the mesh before this is the mesh after now the problem of this is that because it created so many more points it's going to be much heavier on your computer and much more complicated to deal with the model but it usually resolves a little details much better for example let's just focus on these stairs right now this is the optimized mesh and this is the unoptimized mesh this is the unoptimized mesh and this is the optimized mesh and remember that here I did not use any additional photos it's just with the photos that we had already in the software so all 50 of them and it just managed to recreate the steps pretty well I would say compared to the initial mesh that was just a suggestion really so now we have the photo consistency optimized mesh we can go through the process of creating the texture again because the texture has to be recomputed and I'm just gonna show you what it does in the end and here we are the textured mesh is finally done and this is the result we got remember once again that we didn't use all the photos that we had so this is a partial result but as you can see it looks pretty good overall we do have a good idea of the layout of the place we do have the texture of the walls of the ground with the grass where there is no grass the little footpath over here we even have this great to collect rainwater that is relatively well resolved remember that we took all of that from a drone we never set foot there and the Drone was relatively high it was always higher than the highest point of the castle so this is what we have with the reduced data set so in the final step for 3DF stuff here is to export the textured mesh so you can just select what kind of file type you want to use object and material is going to be fine for what I want to do texture type PNG is fine and you just click export and all done now we're going to compare the results with mesh room okay so here we are in mesh room this is a free and open source software that you can download right now and start using free of charge forever and that's amazing so we're going to basically follow the same kind of principle first load the photos of course here is my selection drop it in it extracts the metadata now one thing that it allows you to do is to kind of tweak the pictures right you can change the gains that are just properly exposed you can change the gamma let's go back to one you can choose the gamma to maybe have more Shadows or less Shadows usually you're going to want to have something that is as neutral as possible here I'm just going to keep the photos as they were when they came out of the Drone same jpeg inputs so that is actually a comparison that we can make now here it uses a node-based system so the steps are basically the same right you feature extraction from all the photos then you match the features between the photos here and then this is the main block structure from motion it's going to try to correlate all the Matched points in between the photos and try to create the 3D scene and reconstruct where the camera was so all that is basically exactly the same as what we have in the 3DF Zephyr it's just spelled out a little more precisely and at every step you could just change the parameters here it can get a little involved because you need to read the documentation or look up on the Internet what would work better for your scene right now I'm just going to keep everything by default and I'm just gonna save the project and now the only thing that you need to do is right click on the last node that you want to execute and click compute and now everything is in the queue as you can see the Scion a little bar means that it's in the queue this one has been done already this one is in progress and you can check the progress here by looking at the log or sometimes the status and here we are the process was quite a bit slower with mushroom than it was with 3DF zeph here but we have a final textured mesh with about three million vertices so that is a little less than the 4 million vertices that we had after photo consistency optimization but let's just take a look at the final mesh here so we have both the structure from motion points and dimesh let me just hide that for a little bit now here because we didn't use any bounding box or anything we can have some other parts of the surroundings that have been model see we have a part of the valley right here on the bottom of the valley is not modeled but the sides of the valley are modeled and we have part of the church over there and some of the houses in the village so I would say the quality is relatively good all things considered keep in mind I just use the exact same set of data as an input the 50 photos I didn't touch anything here for the Gamma or the gain I use the exact same stuff as an input and I'm gonna have to put them side by side so that you can actually check out the difference but I would say that for a free and open source solution this is really good although it's quite a bit slower but it's it's a very very good solution [Music] [Music] so here we are in 3DF zeph here this is the mesh that was generated using all 287 pictures plus some photo consistency optimization and we're just quickly gonna go to the different interesting parts of the mesh so first of all as you can see the global mesh is relatively good I did cut it out uh using a bounding box so that we wouldn't get all the stuff around that we don't necessarily care for and we didn't really optimize the captures for uh you do see here some artifacts so it picked up some of the clouds as actual geometry it's not so much of a big deal it's very easy to clean up in any 3D software 3D manipulation software this is what I did for example for the intro using blender I just removed these little ugly patches because they're obviously not part of the model that we're interested in but it's globally a very good model I would say so you can zoom in and get really close and personal uh to Parts even like inside of the castle where remember it was mostly just top down views and I think that overall the quality is pretty good here we can see that the window has been resolved partially we do have some some artifacts here and the texture is all stretched because we didn't really get any good data here you know the pictures we're taking from that angle so from that angle it looks okay but obviously from here it doesn't look as good and same thing happens here at the bottom of the tower we can see that from above this looks good this doesn't look as good but it's going to be the same in mesh room it's going to be the same in any photogrammetry software this is more to do with the technique the capture technique rather than the software used now here as well I really like this Tower I like the shape of it it also looks pretty good of course if you zoom in way too close everything is going to be blurry but globally it's a very dense mesh overall for all the parts of the castle even here the ground the surrounding walls even the trees got resolved really well trees are typically very hard to do in photogrammetry it never looks good like a tree in photogrammetry never looks good it always looks like a little blob but this I think is acceptable for some visualization and globally I'd say there's no real bad part maybe this part is slightly lower resolution but this is because we didn't have many photos here and the photos were taken from afar so it does explain the lower quality now we're just going to jump into mesh room and compare the two and I'm going to put them side by side so here we have the mushroom mesh so you can tell right away that it's a very good quality as well you could also tell right away that if I zoom out you do have indeed the whole valley and a little bit of the village and the church over there but let's just not focus on that because this wasn't the purpose so if we zoom on the same Parts as before I would say globally it does look very good it might be a little less high quality and a little less High detail you can see here for example at the very top of the tower there's some reconstruction artifacts and some holes that can be seen these were not in 3DS Zephyrs model here as well you have a little bit of a hole in the wall that wasn't in the actual wall and wasn't in the other model the trees have been reconstructed in a similar fashion I don't think there's that much of a difference again trees are super hard to do the texture I would say globally is slightly lower quality but this might have to do with the fact that this is a single 8K texture and also you have all the background that has to be fit into the texture while 3DF Zephyr exported multi-texture I think it's three or four 8K textures so you're obviously going to have a better quality with multiple textures than just one so that could be it and could probably be fixed by matching the two settings so yeah let me know what you think in the comments down below which one looks better which one would you pick I would recommend if you want to start and you've never done anything before like this just use mushroom it's free it's open source it's fantastic you also have a free demo version of 3DF Zephyr which is limited to 50 pictures but it's a good start to see whether you like it what kind of workflow you prefer I haven't had any problems with feature matching and Camera alignments in mesh room with the default values it tends to be more robust in mesh room I would say but then while tweaking the parameters with three of zephyr I did manage to include all my cameras it is obviously tunable and you should tune the parameters to get the best result possible but this is what you get out of the box remember this was completely out of the box in mesh room I just imported my pictures clicked on the button and waited a couple hours and this is what I got and I think it's a very good quality model although slightly lower quality like I said so there it is this is how you do photogrammetry for buildings outside uh using the DJI Mini 3 Pro so uh stay tuned for more because I'm gonna do more videos like these probably not 30 minutes long this time so um stay tuned subscribe give a like and don't forget to comment if you have any questions or remarks I'll be happy to answer all your questions see you [Laughter] [Music] thank you foreign

Channel: Jérôme Tabeaud

Views: 143,248

Keywords:

Id: 1_brrH3tywk

Length: 38min 43sec (2323 seconds)

Published: Wed Apr 26 2023