Gaussian Splatting | Luma AI plugin for UE5

[Music] [Music] what's up everybody guys welcome back to me Maron tutorial today we're tackling a something that might just blow your mind gaussian splatting in un engine five pered by Luma AI when I stumble upon this post on X I was like Wow Gan splatting that sounds really cool but the post was made by unity so seems uh complex to use it in Unreal Engine 5 if we can uh it used in the highend v v Loop or C effects but but guess what it's not as complicated as it looks in fact it's super easy and the high quality before gaussian splatting came onto the them I gave Nerf photogrametry a shot but let's be honest the sweede scans had their fair share of the noise kind of like I wanted the trash in the picture now gaan splatting has changed the game so let's dive in and implement this awesomeness I'm going to walk you through scanning and importing into Andre Engine 5 step by step these are only three step step one scanning we are going to use Luma AI because it got this sweet U fling they even offers sample scenes so you can practice without scanning anything you yourself I'm going to talk about it after the explanation of the scanning so I go about diy5 so let's do that the real deal pop up the Luma a up hit the capture button and choose scenes for environments or object mode for well like object you know like statue or apples like flu or something like that the tricks for staler scans is to move slowly and the snack as as many unique angles as you can when I did mine I moved this way and change the angle and move it back slowly like a spy movie sneaking around the scenes changing the perspectives and after uh the capture the scene uh hit the finish and upload your masterpie and glob a coffee because it takes about 20 to 30 minute to uh get done in the meantime let's chat about uh those sample scenes let's head to the notion link and the global sample project so if you uh scroll down so you can find the sample project download and we can uh download it so I'm going to download this one so I'm going to extract all and I'm going to open the this project so let's open up if you can uh open sample file so you can see that the beautiful like a scene that is captured by Luma AI so it's like a 360 uh environment map so that's really cool and then like it is not like a heavy file like we can learn the pro project uh with this 60 uh frame rate so that's really cool right okay and then also you can get another uh sample SC uh if you go down to the um scroll down to the uh this website you can find More Sample scenes and uh let's click on that so they have a five uh sample SC here so for example so let's uh download this one so uh if you uh open up uh these are two type of the uh goian splatting file for engine one is a Luma interactive scene Luma interactive scene uh second one is the Luma field download so uh I'm going to show you what is the difference of it so I'm going to uh download both of them so now uh I'm going to extract the pile of the uh interactive scene so importing is so easy so I'm going to make the new uh file for example a new folder uh field and another one is a uh interactive so first uh I'm going to uh import the uh Pei one so this is this one drawn to loops. Luma file so I'm going to import it okay they have a Luma file and for example I'm going to at the new level and I'm going to delete this one and I'm going to bring the uh the dynamic two RP so uh if you go uh inside of the this one so like this one which is really cool right num files let you tweak the exposure and the clop the scenes you know like this one you can crop the scenes but it's very heavy uh that's why uh both Luma Ai and I recommend the second one so which is a py file so uh I'm going to uh drop the P file uh from the folder I already uh extract so and I'm going to drop the uh this one okay and this is also uh allow us to uh crop crop the scene like if you if I change the CL box to the five yeah see we can clop it as well if you change the more smaller smaller yeah uh this is so light uh file uh it's easier uh on your system and also they have another one like a uh Dural Bridge uh physics reading uh Capital playground so you can uh use it like without uh capture the your scene okay I'm assuming our upload is done so we move to the step two is importing the scan so uh jump on the website of the Luma AI so let's go to the Luma Ai and login and let go to the my captures so now uh I have a this scene so uh hit let's hit the download but here is the inside scope go straight for the uh gussian Splat which is the py file uh optimized for u5 I've tested them all and this one is the gold standard don't download other file because you're going to waste your time so trust me so I'm going to download it now step three creating the project for Luma AI gum plus splatting let's fire up uh Unreal Engine five and uh select the C++ this will save you h of the errors when you want to package the project later if you want so I'm going to call it the see and then yeah BR pile and create it and then to work with the uh Luma AI files you going to install their plugins so let's go to the uh folder uh of the sample file this is a sample file and you can find a plugins folder and I'm going to copy it and let's go to the uh your project file so which is the uh Gan platin u5 and then here and I'm going to paste it the plag file and now uh I'm going to go to the uh R prag file myi but we can't find it so we need to uh restart this project so I'm going to uh close this uh project and I'm going to open up okay yeah which is a really bit little bit hustle uh to you know like make the own project uh with clean feed but this is the only way to do that okay so let's go to plugins file I'm going to search the Luma Ai and then you need to uh enable uh this plugin and then list uh your project I already enabled uh I don't have to uh restart this file so uh I'm going to uh create a new level and the basic and create it and then I'm going to uh create the new folder I'm going to goe GS then then is we already download the uh your scans so I think this one and I need to extract all so I'm going to open up and then you can see the uh p file so we can uh Dr and drop to the unre engine so let's uh go to the g s shadow. py and then I'm going to drop it to the content fold yes so you can see the uh for blueprint the first one is a baked baked uh keeps your Splats immute to uh lighting changed so that it mean like uh it it isn't be affected by lighting because it's already baked and also you can see the uh no TAA uh the TAA uh option is a lighter on your system I recommend to drop all of the file to see what is the difference so this is the not TAA which is a little bit lighter uh than the other file and this is Dynamic so it affect the uh lighting so let's see if I uh drop the light yeah you can see the uh the reflection of the light and they also have a a dynamic no TAA file so which is a little bit lighter and the TAA uh means temporary anti-os so if you have a problem uh with the anti in it's really good to use the not uh it makes you handle the file uh lighter so you can uh speed it up to work with the file so uh that's a wrap for this one folks next video we are going Dynamic lighting I'm talking about the dynamic lighting for Bridges and share the pl fire I scanned that means no need for you to scan just download and enjoy uh gaussian spatting catch you in the next video and let's keep creating

Channel: miwa maroon

Views: 8,055

Keywords: ai, atrificialintelligence, luma ai, 3dgaussiansplatting, unrealengine, vj, vjloop

Id: lhaEV5XewhQ

Length: 12min 36sec (756 seconds)

Published: Wed Nov 15 2023