Is Reality Scan any good?

because of Apple's extensive 3D and AR sdks the iPhone has been the platform for 3D scanning apps polycam scanniverse turnio clone these are just a few of the apps available in the App Store there's a ton more there it's insane and just a couple of days ago a new player entered the space reality scan this one though is a little bit different it comes from epic games a company with a ton of experience in 3D and photogrammetry for those who might not follow the market closely epic games has a very popular desktop photogram duplication reality capture so when I saw reality scan finally coming out of beta I was very excited to try it out so let's find out how capable reality scan is [Music] foreign [Music] feature is something only a company with billions of dollars in Revenue could do and I'm not talking about the completely free processing of our 3D models or even the completely free upgrade to sketchfab Pro or the free 8 000 credits we get to use in reality capture no all these things are just the icing on the cake in reality scan's killer feature is the incredibly helpful AR preview that appears as we capture the object for this thing to work seamlessly the application needs to use a ton of resources let me explain as we go around capturing the object the photos are uploaded to a server so they can be analyzed and aligned that happens in real time after some initial processing the server sends back a point Cloud that's overlaid on top of the real object this point cloud is incredibly useful because it tells us in a visual way how well our scan is progressing green means we have captured that area really well yellow means we have fewer data but still enough for a good capture and red means that the area needs more photos as we take more pictures the process repeats the photos are uploaded in the background they're aligned on the server and the point cloud is updated to reflect the changes as you can imagine this requires an infrastructure that cannot be easily copied by a startup but besides that reality scan has a mother helpful overlays and these ones take full advantage of iPhone's default AR capabilities all the photos previously captured are displayed right there on the Real Environment so it's super easy to visualize where we need to go next do we need to shoot higher do we need to shoot lower or is there a space in between the pictures that could use some more coverage this overlaid information is something I always wanted to have with a regular photogrammetry process it's easy to lose track of what you've captured if you get distracted even for just a few seconds you're kind of screwed you'll wonder that I capture this area already or do I need to shoot some more images here but with a point cloud and the photo markers this is not an issue anymore I think for that alone reality scan is totally worth it but there's another really really cool feature I would like to highlight and that's scanning an object in more than one sessions let's say for example that we have completed a scan but we later find out that the images we captured weren't enough with reality scan we can just go back to that location enable that photo session and continue taking pictures this system will automatically align the already existing Point cloud and photo markers to the object it's really cool to see but as amazing as these features might be reality scan has its own set of issues we'll come back to those in a bit for now let's see what kind of results we can get out of reality scan with reality scan we can shoot a maximum of 200 pictures per scan it might not sound like much but it's good enough for models with moderate detail this model uses 102 pictures so there's still some more room there but definitely don't expect insane details especially if we have to cover a fairly big object 200 iPhone pictures can only take us so far for this tree trunk though this amount of pictures is absolutely fine especially if we're going to use the object in the background the scene has 1 million polygons in total so if we target our photos very closely to the subject we can get some nice details in this case some of the detail is wasted on the ground but that's my fault I could have cropped the scan a little bit tighter [Music] here's another scan that's not using the full 200 pictures limit this one uses 114 images and I would say the application did a really good job it even managed to resolve some of the thinner parts of the bench like the metal Parts on the base and the sides not all parts are fully resolved but that has to do with the fact that I didn't take enough pictures around these areas either way though the bench turned out great especially considering the fact that I didn't spend more than three or four minutes taking the pictures reality scan always exports a color and a normal map and it does so at 8K resolution which is perfect I wouldn't mind having more than one 8K textures but I think we need to keep in mind that the service needs to cater to a lot of people and it's also free so I can understand the limitations foreign uses 163 photos the wall has adequate detail especially considering that this is not going to be the hero object the size of the wall are quite a bit softer but that's because I didn't shoot as many images from left and right I mainly focused on the front part of the wall overall the scanning results are good I wonder how closely related reality scan is to reality capture my guess is that they probably do share some of the same attributes but that's just me guessing either way reality scans results are good but I don't think it can compete with other apps like polycam which brings us to the drawbacks of the app the first one has to do with reality scan's killer feature the point Cloud overlays in order for the feature to work there's a lot of data that has to go back and forth we need to send several images to reality scan servers and we also need to get back the point Cloud information let's use as an example the bench object the 114 images that make up the object make it a middle of the road scan before going through the scanning process I noted down the data I started with and I did the same once this scan was done in total I used 1 230 megabytes that's more than a gigabyte for this middle of the road object of course this is not unexpected we're sending 114 images so there's not much the app can do about that but it's something to keep in mind especially if you don't have a big data plan but even if data caps are no issue at all there's still the problem of cell coverage if it's not good enough you'll have a hard time seeing the point Cloud that's what happened to me with my first scan the connection was spotty so I had to wait a really long time for the images to upload and without any uploaded images there's no point Cloud I ended up abandoning the capture because it took way too long but in the process I wasted more than 500 megabytes of data so if you're in a remote location or a location with a spotty service you probably won't be able to use the point Cloud overlay at all but to reality scans defense there are ways to circumvent these issues first off there's an option to enable offline mode this will disable both Wi-Fi and cellular data or we can choose to limit data usage to Wi-Fi of course this means that we cannot use Point clouds or exporting to sketchfab at all but we definitely save up on data usage losing the point Cloud might sound like a big deal but it turns out it's not we still have the very valuable photo marker overlays because that part happens on device so we still get all the information we need it's just visualized in a different way after a couple of scans I ended up disabling the cellular option I still have all the visual feedback I need and I also don't go over my data plan so we kind of get to have our cake and eat it too I think my bigger issue has to do with this scanning limitations of the app for example reality scan doesn't seem to support the turntable method if there's no way for us to turn the object to its side and continue with a scanning process so we will always end up with an object that has one side completely unscanned for Landscapes of course that doesn't matter but for smaller objects we kind of want to have the whole thing there so with that limitation in mind I would say reality scan is a great app for big objects rock formations tree trunks stuff like that which also happens to play two iPhone's strengths as a camera for smaller objects though it's not that great here for example I try to unsuccessfully scan a brick it has a relatively small size 18 centimeters in height because I'm shooting inside the lighting is a bit more challenging it mostly comes from One Direction and with the apps lack of exposure controls notice what happens when the iPhone is hit with direct lighting it tries to maintain a balanced exposure for the whole image and as a result we get an underexposed object as we turn around the exposure improves and we get some good shots of the brick that's until we get closer to the original under extra closed position as a result the final scan has a lot of texturing issues one side is brighter than the other and the colors are completely wrong this area should be gray but it's more of a brownish color notice also the lack of resolution that's the best we can expect out of an object of this size this is because of a combination of things mostly it's due to iPhone's limitations but also because of some limit set by the app reality scan uses the iPhone's default lens which is great because that lens has a better quality CCD but that also limits how close we can get to the object let's take this can as an example this is as close as we can go if we go any closer everything will go out of focus and since we can't capture any more detail the final 3D model will be a lot simpler than the real life object that's why I think that reality scan is only good for big exterior objects there the default iPhone lens is good enough and because the outdoor lighting can produce more evenly lit results the app's lack of exposure controls won't cause an issue the last annoying thing I would like to mention has to do with the fact that we're forced to use sketchpad there's no other way for us to get the object and even then we have to jump through several Hoops in order for us to download the object for sketchfab we have to first make it public once that's done then we can go to the edit property section and download the fbx file so we always have to do this weird dance make the object public download the fbx file and then make the object private again it's kind of ridiculous I don't want to be too harsh after all reality scan is a completely free app at least up until now so it's kind of pointless to complain about something that we get to use for free but the reason I'm pointing out these flaws is because there's so much potential there if reality scan was a paid product and it was developed a little bit more I think it would have been a really amazing app I would also love to use the photo markers and the point cloud data on a non-mobile environment for example while using a local computer and a regular camera the camera could send the image data straight to the computer process and that way we could have a faster more capable setup currently the 3D objects we can get out of reality scan are constrained by the iPhone's camera and the app's lack of exposure controls so the results are not as good as they could be if I had to choose between reality scan and polycam I would have to go with pull cam it's a much more capable service overall we can upload photos from a camera we can use the turntable method there's no hard cap of 200 images per scan and as a result we can get better looking objects we might not have reality scans photo markers or the point cloud data but in the end none of these features matter if we cannot get a good looking scan out of the app and if polycam implements the photo markers feature which is possible since everything happens on the phone I think the gap between these two apps would be completely gone so I'm kind of torn on one hand I'm excited that epic games has entered the mobile slash Mac field but on the other hand the final product needs a little bit more work but what do you think have you tried reality scan are you satisfied with the results let me know in the comments below take care and I'll see you in the next one [Music]

Channel: Dimitris Katsafouros

Views: 16,774

Keywords: polycam, epic games, photogrammetry, 3D scanning

Id: nq3tzNOkN1g

Length: 13min 27sec (807 seconds)

Published: Tue Dec 13 2022