Gaussian Splatting Has Never Been Easier! #gaussiansplatting #vfx #unrealengine

gausian splatting if you've been keeping up with the vfix world recently you've probably heard this term before in fact right after I recorded the first version of this video cordor crew actually released a video where they used the exact methods I'm going to show you how to use today but what actually is gausian splatting well just like how photogrametry creates a point cloud and turns that into a mesh gausian splatting makes a point cloud and turns those into smooth colorful ellipsoids or Splats combined these splats create 3D reconstructions that are second to none in terms of photo realism but until now if you actually wanted to make a gaan flat you really were only left with two options use a thirdparty app like Luma AI or become a whole lot more familiar with the command prompt than I think any of us want to be that's where post shot comes in post shot is basically the reality capture for gausian Splats it's a UI so no need to go copying command after command and pasting it into the Powershell or uploading your photo to who knows what server now you actually have full control over making your gaan Splat and it's all done locally and today I'm going to show you exactly how to use this program and how to export those Splats and bring them into unreal so as always I am Alex from level up plus v effects get subscribed and this is how to use poot to create gausian Splats let's jump right into it so now I'm going to go basically over every step of the process and how we set up train and use post shot so the very first thing of course is you need to go go ahead and download it and install it that's always pretty easy go ahead and download it install it you may need to create an account I'm not sure uh I don't remember if I did but once you're done you can open up the post shot app and you should be greeted with a scene that looks somewhat like this now once you've actually opened up post shot you should be viewing a window that looks something like this minus the gorgeous Porsche Splat of course uh this is what we're going to be making today you'll have a window that has basically these these options on the side you'll have this little scene thing here in your parameters so I'm going to go ahead and create a new scene and we're actually going to train this Splat from scratch to start with I want to go ahead and show you the images that I took of this Porsche so I just took my camera I set it to an extremely high shutter speed and I just walked around at a couple times from different angles um there are obviously going to be some dead spots based on the number of images that I exported out but they're all pretty good they're all high quality they're sharp enough to uh retain the detail that is needed on our final Splat here it's a little too blurry down here but this is definitely good enough for me for today so I'm pretty happy with it now let's go ahead and just jump straight into actually making the Splat and this is where I'm so excited to share this with you guys it's so easy so I'm just going to go to file new and then we're left with a scene like this this should be what you're seeing so I'm going to go ahead and select import and I'm going to go ahead and go over to images I'm going to select them all and I'm going click click open and it's going to open up this little dialogue box for me now here's two options that you just don't get in a lot of other programs you get an option to use the best images which basically it tries to figure out images that have motion blur in it and just ignore those um or you can just set it to use all images and for this training I'm actually going to use all I'm going to see what happens when I use all obviously because I had a shutter speed of a thousand none of my images were in motion blur so it should be good and then finally there's this other option that says down sample Imes the default is a resolution of 1,600 now on their website they say that this Default Resolution is kind of what you want to stick with going higher doesn't necessarily produce better results and it can actually just increase your render times for no additional benefit so they recommend sticking to 1,600 but if you want to get a little experimental you can always bump it up for me I'm actually going to get a little experimental I'm going to set it to a whopping 2000 finally we have the camera poses option which is just one thing compute from images and then we have a Max features for frame if you take the Max features and you increase them it'll create more points per frame um pretty self-explanatory uh they say the default of eight is good but you can go up or you can go down if you go down it'll take less time to calculate but you might have more inaccurate results if you go up it'll take more time to calculate but you should have more accurate results I'm going to set this to 12 for my scene finally you actually get some cool options you could use a Splat or you could create a Nerf obviously I want to create a gausian Splat so I'm going to go ahead and leave it at Splat and then the Splat density is the last thing that you're really going to want to mess with so what this Splat density option is is basically it'll determine how many Splats it makes in those you know areas where it's not exactly sure what something is if you increase the value above one it'll create more small Splats in those areas if you decrease it it will just leave it with bigger larger Splats but it will be more sure that those are about I found that for this scene a value of like 1.25 to 1.5 works pretty well now you can set it to stop training if you want after 30 or you can just let it train forever from some other resources I found online they say that a value of 300 is like the final point of no return where you really stop seeing any benefits you stop seeing a lot of benefits after 30 so we can just leave it at 30 and that is basically all the settings that you need to know about I have not messed with this create Sky model setting and I haven't really messed with anything else so once we click import it's just going to start the training right away first it will it'll basically do the same process that reality capture does it'll try to figure out where the cameras are then it will create a point cloud and then finally it will create your gausian Splat and the cool thing is you get to see this all in real time now here is the first big caveat when you train in post shot I've noticed that it is noticeably slower than just training it in command prompt when you're using things like anaconda and the gaan splatting actual repo from GitHub it is faster to train to 30,000 than it is in post shot however this is just too easy not to use the amount of time I save by just dropping stuff in and setting my settings and clicking go compared to copying one command and copying another command in run col map run the actual training finally running the viewer you know this just makes it so much easier I probably save time in the long run even though this takes a little bit longer to train so I'm going to go ahead and click import and I'm going to record my screen while it does all of its processing and I'll see you guys when it's done all righty and now it is done so this version of the Splat looks amazing Wheels actually have more detail than my previous one did just because of some slight changes I did let this actually end up cooking all the way up to the 3,000 but all my other settings were the same and really for most things you won't need that but just look at the detail look at the just the road across the street I that looks amazing now there's one last thing we can do to make this even better and that's clean it up right cuz a lot of the time you have all these weird Splats like the stuff up here floating in the air you know we have some random stuff we have a bunch of gibberish out in the middle of nowhere so we want to clean this thing up and again post shot makes this super easy so what you're going to want to do is you're going to want to go over to your left hand side of the screen and you'll see these two options one's a selection Circle and one's this little box Circle so we're going to go ahead and click on the selection Circle and now we basically just have this paintbrush here now something I've noticed is it's a little bit harder to pan but you can you still just need to basically just hold alt whenever you're rotating or doing anything because if you just click now we'll start painting what we're going to do is we're going to go ahead and just paint the Porsche we're going to paint it as much as we can and we're going to you know of course there will be some areas where you can't paint something I don't know how to do is how to zoom in I guess it's I guess it's right click when you're holding alt you can right click and hold alt to zoom in now but let's go ahead and we're just going to rotate around and we're going to keep painting the Porsche you don't need to hold like controller or anything to add more paint Strokes it just automatically adds them so we're just going to paint everything we can here something like this make sure everything's covered and we're going to zoom out that looks pretty good I know we have a bunch of stuff over here but that's fine but now once this is all painted I'm going to go ahead and and click this invert selection option then I'm going to click delete selected Splats and that's going to delete everything except what we selected and that cleans that right up now I'm going to go ahead and clean up some of the excess that we still have left here so I'm going to go ahead and go through grab a bunch of these points here I don't think there's really anything else out here so let's go ahead hit delete selected again we can just kind of refine this in more now that is awesome but there is one thing we want want to do is we want to probably bring this into a program of our choice and today I'm going to show you how to bring it into unrealed so let's go ahead and Export that and that is super easy I'm going to go ahead and just click the camera view again so I can just move around it and we're going to go ahead and do file export and now I have my driveway folder here I'm just going to call this tutorial and it saved as a p format so what we're going to do is we're going to go ahead and click save and that's all you need to do now it is saved as a gajin Splat Point Cloud so the next thing we need to do is open up unreal and bring this thing in all righty before we jump into unreal there is one thing we're going to want to pick up from the epic games Marketplace and that is the Luma AI plugin so if we go ahead and type in Luma we'll see it right here this is a free plugin that you can install into your engine and it's how you'll be able to view your gausian Splats in Unreal Engine now I should note that there are a lot of limitations with using goian Splats in real engine still but I have high hopes that in the future this is going to become much more stable and have a lot more flexibility than it does today you have to remember that this is just the beginning of this kind of stuff so let's go ahead and pop into an Unreal Engine scene and I'll show you what it looks like all righty so here we are in an Unreal Engine scene and I'm going to show you exactly how to import your model this is just a very basic scene I have a little floor texture some lights and let's go ahead head and bring our Luma AI guy in here now the first thing you're going to do now that you installed that plugin to the engine is you're going to need to enable it go ahead and head up to edit plugins and then search for Luma once you found the Luma AI plugin make sure to enable it it'll ask you to restart and once you've done that you'll be able to bring your Splats in and now this is why it's so easy I pull up my folder where I exported that Splat in the ply format if I go ahead and just drag it into my content browser it will automatically import with four blueprints now these four blueprints mean something so the two baked ones are ones that can't be affected by the world's lighting and the two Dynamic ones can so let me go ahead and show you that I'm going to drag my baked Splat into the scene it'll take a moment to load but then there it goes here is our baked gausian Splat of our Porsche and it looks awesome so something you'll probably Notice off the bat is that it looks worse than it does in the gaan splatting app that's because this is based off of a Niagara particle system and while it is impressive it does have some limitations but I'm going to go ahead and delete this for now we'll also talk about these coal boxes in a second and I'm going to delete those as well and I'm going to bring in my Dynamic Splat so let's go ahead and do that and instantly we can see that it's actually being affected by the lighting if I pull out one of my lights here and I move it around we can see that we're lighting up our Splat if I move it away and we just have this wreck light here we can see that that is also lighting our scene up which is really awesome now here you'll notice another limitation this thing can't cast any Shadows now it can receive Shadows but unfortunately right now gaan Splats cannot cast Shadows which is why this is probably going to be much more useful for visual effect shots where you want to do something cool using gaan splatting like cdor digital has but it is still awesome to have something like a background prop or even a background environment that's going to have no interactivity but let's go ahead and talk about these coal boxes here so as we can see if we go ahead and grab our Splat and drag it up we can see we brought in some of the ground let's go ahead and quickly remove that in the blueprint I'm going to go over down to the cbox one option and I'm going to check it once I've checked it I can select kbox one and we can see exactly what's happening it's doing exactly what you would think it is coling the Splats that are in its location so I'm going to go ahead and bring it down right until it's above the ground and then I'm going to scale it to so we can actually just get rid of that ground and now we have a much cleaner looking floor it is worth noting if we just click on the blueprint and drag it down everything will move with it including our Co box so that makes it easy for us now we have these little toughs of grass here that look a little weird so I'm going going to go ahead and enable cbox 2 I'm going to select that guy and we're going to move that over and just get rid of those I'm really excited to use this technology more in the future I'm so glad that cordor gave it a little bit of a highlight even if it does feel a little weird that they come out with a video the same day I do but I hope this was informative for you guys and I hope you guys can create some awesome stuff using this technology with bot it really has never been easier either way I've been Alex from level up plus v effect if you found this video useful Please Subscribe and I will see you in the next one peace

Channel: LevelUp+ VFX

Views: 2,033

Keywords: vfx, unreal engine, NeRF

Id: Z7W1PZuXpzo

Length: 14min 45sec (885 seconds)

Published: Mon Jun 10 2024