3D Gaussian Splatting: Changing 3D Forever

what dude dude I can't believe this is 3D yep it's here a literal reconstruction of our world in 3D with extremely high quality and guess what from just one single video we are talking about 3D gaussian splatting and if you thought Nerfs and photogrammetry were cool wait until you see this so last month at sigrap a group of geniuses released a paper introducing 3D gaussian splatting now they claim that it is much faster to render in real time and also comes with much higher Fidelity compared to Nerfs and photogrammetry and we're about to find out because Farhad is doing a gaussian splatting right now and you thought we're just gonna sit and watch hell no we're gonna do three tests a day scene a night scene and a drone shot let's go we should find a location to scan for the 3D gaussian splatting do you think we're the first people doing Guardian Splats in Dubai oh yeah 200 it's not gonna be easy though because unlike new photogrammetry that have Solutions online there is there is nothing for gaussian splatting so I had to spend the whole night yesterday cloning their git repository so we can view these 3D renders but what are you doing training so as you can see as you can see you know what this reminds me of Nerf days yes Nerf days we did exactly the same thing when we were testing out Luma ai's nerves and now it just feels like a Deja Vu because we're testing gaussian sliding but this time apparently the quality that's higher let's go and find out are you ready I'm ready let's go drop it in there drop it in there are you kidding bro this is your school what you know it feels like this is a photo do you see how close I can guess what can you go closer to the tree can you go closer to the tree dude I I've been almost behind the bench dude wait hold on hold on hold on hold on look at the plants look at the plants look at the plants are you you can never get this quality within their special vegetation it's not possible it's because look at the amount of detail that is required to create that can we look at the original videos 4K 30 frames how long and it's just it's not long let's just go one round so you didn't even go up close I didn't go different heights this is it this is exactly it you know what's funny our mom can take yeah without knowing it she's going to help us with photogram and you can never do it unintentionally oh no you have to take a thousands with nerves you have to be extremely careful with how you shoot your shot this is such a random shot but let me show you something in The Vibes of Matrix are you ready I'm ready so I'm going to show you all the splats boom this is what it looks like actually right look these are all the gaussians okay so for anybody that doesn't know how this works we're gonna make a proper explainer video gaussian Splash it's like a brush a Photoshop brush where it's dense in the middle and it's fading out towards the end and you can see that can you go to 50 of the Splat look at the bottom here look at the bottom look at all these points now being extended so when these Splats are representing light and color they're now being extended to shape into the form of the scene can you see the reflection yes have you noticed something what is it dude this is running at a crisp 60 frames per second wait a second I can increase it what no higher than that now we are going at 200 what what look at that wait don't move don't move don't move don't move don't move don't move that's 365 frames real-time 3D scene render this is crazy imagine taking this to Android you know what it means when you're rendering that entire scene at 300 frames on a 40 90 that means that this is something that can eventually not yet but eventually be used in video games dude I can't believe this is three pieces Crystal Clear look at your ground 560 FPS running this entire scene in real time is it possible to run 600 he went above 600. I never seen 600 FPS virtual production yes virtual [ __ ] production are you kidding me are you freaking kidding me this came from one video can you show the original footage do you see how shaky the original footage is we did not use even a handle let alone a gimbal this is a horrible raw footage can easily out of it look nothing you don't see the changing quality everybody knows low light conditions are not optimal for any sort of Photography videography unless you know exactly what you're doing and that's the mood you're going for but if you're scanning a scene you want light because you need to see the detail but does it matter when you're taking gaussian Splats let's find out dude no way that's we move around it okay I'm gonna increase my camera speeds up dude this cannot be a night scan are you kidding me this is so realistic do you see do you see this thing no dude okay don't go too far off this is literally my orbit around look around you just look around you yes look around you are you serious okay let me try this Vlad That's my favorite thing if you realize okay hold on so this was the night shot Splat and it's turning to a scene giraffe footage was really horrible look [Music] dude this is the raw footage mind you guys that everything at night was taken by our DSLR Sony camera no it was thinking about the iPhone so yes this was taken with the iPhone as well what's a flight pass green time you feel for that oh my God dude hold up hold on look at the Yachts look at the Yachts we got the freaking yards in this shot if we can go past this freaking Tower thingy this is meant to see something I'm gonna be just amazing can I just can you go through it can you fly through it do you see the depth between this tree and the other tree like this is freaking real dude let me go back to my home positions oh my freaking God I'm telling you okay all right amazing do you know what time it is Jonah Clark drone o'clock we had one of the craziest drone shots that we took in Vancouver off one of the largest the biggest stadium in BC we took that footage we color graded it because it was raw and we upscaled it all right so before we get started we're gonna do something for fun we took this footage originally to train it on Luma ai's Nerf and we wanted to do that because we were creating a cinematic and testing how big of a Nerf we can get in Unreal Engine 5. we train the Nerf and this is what the Nerf looks like now we haven't seen what the gaussian Splat looks like but we just want to show you the Nerf first so you see the difference in quality okay this is the Nerf and I'm pretty impressed with this look at the skyline at the back yeah look at this color at the back the fact that we can get all of this with two minutes of footage no photos just two minutes of footage and train it online for free is already crazy and we can take this one engine five super easy the training takes about 30 minutes or 45 minutes and then taking it to Unreal Engine 5 is extremely simple because they got the Unreal Engine plug-in you're gonna all the details here too you can go closer to the edges though that's where I think if you were to do close-up shots of this you might have some trouble yeah right because you can see the artifacts around there let's try we've been promised by the guys who released the paper of gaussian Splats the quality is higher than nerves are you ready one two three move around it dude man look at it dude I have every single angle I feel like Superman look at that dude I can't I think we already achieved perfection dude dude how is this possible do you know what this means look at that do you know what this means details do you know what the virtual production baby look at the edges where we had artifacts in our Nerf zero nothing no artifact noon nothing look at the windows do you know how big those windows are down here just kid can you imagine look at the size of humans this is the size of a human do you see the human here that's a human reference look at the size of these windows and look at the details go over here go over here fly fly all the way there are you no way we can't even get a capture this after this angle dude look at this card look at this buildings man this is two minute footage this can be used for virtual production 100 I can't [ __ ] leave this do you believe that we have here look at the car part look at Rogers Arena look look at that the fact that he can even read that [ __ ] dude this is crazy can you okay can we tell what car it is by just going close to it let's do a test I'm not good with cars but these are that's a Tesla that's a Tesla yeah bro you cannot do this with any other technology can you go to the max frame rates for this 330 frames per second going up to 500 go closer go closer so what you're looking at right now is the Splats that blend together with the alpha to make up the scene but if you were to take away the blending you would end up with a scene like this before the processing right you can see all the Splats look at what happened to the FPS so go close to it go close to it now do you see how it's laggy yeah because it doesn't have the alpha so you're seeing all the unnecessary information that you don't need kudos to whoever it is do we have a link from their paper yes we'll leave the paper link Down Below in the comment section plus shout out to Jonathan for his tutorial without his tutorial it was impossible I'm gonna put his tutorial link also and also Aldi has been doing a bunch of different videos on different situations like low light check him out as well he has a YouTube channel where he's experimenting with a insta 360. before we go let's do the outro in the room okay guys these were our very first experiments with gaussian splatting and the results were amazing you tried so hard not to say gaussian right yeah yeah we've been messing up saying gaussian every single time it's sin you can Google it now imagine a possibility is for gaussian splatting you can use it in entertainment industry real estate properties and anything a lot of something we're gonna find out what all these use cases are but before we do that we have to do some more experiments first of all I want to test out upscaling the footages and also test out with different camera settings to see which one yields the best results and of course we are going to take it into Unreal Engine 5 to create short cinematic like we did with Nerfs yeah there's literally one plugin that allows you to do that we found it it belongs to this Japanese guy and we're going to test it out and let you guys know how it goes so if you guys like this video okay subscribe they haven't I don't know if you guys like this video and you still haven't subscribed I don't know in the world are you doing what are you doing for God's sake there's just one button all you have to do is click it and after that we'll make more money and yes life will be good

Channel: Bad Decisions Studio

Views: 89,466

Keywords: radiance field, gaussian splatting, neural radiance fields, Nerf, AI, 3d Scan, 3d

Id: juRMRej2d5c

Length: 11min 51sec (711 seconds)

Published: Fri Sep 22 2023