World's MOST DIFFICULT 3D Scan | 3D Gaussian Splatting

we are going to pull off the world's most difficult 3D scan we're going to scan Boral Arab with an iPhone but how it's all surrounded with [Music] water we chose Borel Arab because it is one of the most magnificent and iconic buildings in the world all we need to do is record the video while going around it and then use a technology called 3D gaussian splatting which takes videos and turns them into highly Pho realistic 3D scenes first we need to assemble the right crew introducing Alex a professional jet barder who's going to scan while riding his jet board Mahan an adventure junkie who's known for his Precision in driving Vehicles he's going to be riding our boats and you guys already know these two goof bags we're going to be at the back of the boat scan in this video we're going to show you how we 3D scan Borel AR all the settings we used our workflow and we're also going to share with you how we brought our scan to unal Engine 5 and recreated the entire scene to use for a cinematic so let's go we started by meeting at Alex's place we had some coffee to get zooted and discuss the game plan in order to achieve the best quality scan we need to orbit around our subject so we meticulously planned the path that we needed to take around the BGE with the boats and the Jet board Alex would be on his jet board recording with an insta 360 X2 at 4K 50 frames per second me and fire hod will be using our iPhone 13 Pro recording at 4K 60 frames per second attempting both a horizontal and vertical orientation so this is where we're going to say farewell me and Alex are going to go with the car to the beach that is closer to Bor Al Arab and farad Mah are actually going to get to the two boats and bring them to that beach where we going now we are going to Sunset Beach to help Alex set up his jetboard while farad and Mahar will pick up the boats from Dubai Marina since the jetboard can only carry one person far and Mard are going to pick me up from here so that we can all go to the Birch together F the boats are not allowed on the beach no we didn't have any time to waste so I got an Uber and went all the way to Du by Marina to get on the boats with everybody else and then meet Alex in the middle of the sea in front of Borel ARA I'm late nck is our savior today yeah let's go we finally got on the boat and just when we thought everything was okay we faced our Second Challenge earlier that day we checked the sea conditions online and the promise was that the water will be calm okay brother possible dude I did one run I four five times how are you feeling right now Alex super tired yeah the waves apparently were one of the most intense that Alex had surfed on but luckily since he's a pro he managed to do a few orbits right before we reached so to be safe we grabbed the insta 360 from him and started doing our orbits on the boat it was so difficult trying to hold on to dear life with one hand holding the ropes of the boat and the other holding the iPhone and the Ina 360 gosh we spent nearly 2 hours in the water not knowing if any of the footages we're recording will yield a decent result but as the wise man David gogin says stay hard when we got back home it was time to train the footages that we have recorded we turned them into image sequences and trained them manually using GitHub codes and our own GPU to be honest with you when we saw the results we were absolutely amazed the iPhone shots turned out so good specifically the horizontal shot think about it the C is was in constant motion and it was making up half of our screen the 360 results are also pretty okay but they don't come even close to what our iPhone achieved to be able to create a cinematic from our results we need to bring the gaussian Splats into Unreal Engine 5 we're going to do that using an Unreal Engine Marketplace plug-in that converts a Ply export into Niagara particles now we can just show you a cool cinematic and end it right there but let's dive a little deeper once you have your scene in unreal you can go ahead and crop the gaussians to keep only your area of interest and thus we can extract the Borge and just like that we have our subject in all its Glory now we can go ahead and change the Skylight mess with the clouds even add our own ocean to recreate the same exact scenery how about some Dynamic elements like a flying dragon or a floating astronaut why not and ultimately we can change multiple camera angles and render some beautiful beautiful [Music] [Music] shots but let's not ignore the fact that this technology has its own limitations for starters cleaning up the floaters and cropping out your subject can be a long and tedious task and with the current tools it is almost impossible to end up with a clean result free of artifacts also keep in mind that we only have the angles that were seen by our camera lens so technically we can't Elevate the camera too high off the ground what we hope will happen in the future is that AI will have a significant contribution to the Reconstruction of angles and frames that are missing from the raw input provided and that is it for our video today we hope you guys enjoyed that challenge honestly it was extremely difficult but we pulled it off so we have done the challenge in the desert we you've done it in the water what's next no no no no no no no no no no no

Channel: Bad Decisions Studio

Views: 9,238

Keywords: radiance field, gaussian splatting, neural radiance fields, Nerf, AI, 3d Scan, 3d

Id: z4HUzt_b8RA

Length: 6min 6sec (366 seconds)

Published: Tue Nov 07 2023