Photogrammetry vs Gaussian Splatting for Virtual Reality

my favorite thing to do in virtual reality is to is to look and be in Virtual photogrametry environments and the new tech on the Block is Goan splining or Nerfs a separate Tech but similar so is that is that going to be the new medium that people do this virtual tourism with I'm going to do a direct comparison today of my own Creations in photogrametry and gorian splatting work and we'll we'll compare the pair so we'll go to my bridge underpass in Melbourne Australia this this photo scan was mainly done using a lar scanner done very quickly and cheaply but it's uh somewhere that I could get to to recreate engor and splitting to do a direct comparison of so with photogrametry or traditional 3D modeling it's all done with polygons which you know has its advantages and disadvantages we'll talk about that more when I compare this to the the Goran Splat of this same location but today I went through with the Luma Labs application and I walked along here and I recorded on my smartphone and made a gan Splat of this same location and I'm going to load that up now in an application called Gracia VR um links to all of that in the description and we're going to do a direct comparison and then talk about how I feel about the difference between standing in polygons in VR and standing in in Splats and just a a renderer so we'll uh we'll try and do a nice transition here and here we are viewing our Splats in the Goan VR viewer now I didn't do a perfect job of of this I don't think but what you're looking at here for people who may not may not know uh there's there's a lot of mathematics that I don't fully understand going on with this viewer but it's a Point Cloud that has been converted to a bunch of Splats which are these ellipses that use a a Goran wave in order to determine their shape their opacity and their color based on The View angle of the camera and their size um and they're able to do a few extra things like if they're too uh too transparent they just disappear altogether um and when you've got enough of that you you get this sense of being in 3D now again this is this is my favorite thing to do in Virtual Reality well I used to say my favorite thing to do was to be in photogrametry environments now I have to change that wording my favorite thing to do is just to be in photorealistic environments and if I look over at this bridge at the end of princess's Bridge here this is better this this looks so much better than my little scan of from a smartphone um which I I guess so I guess it was always going to look better than that but it even know I'm not rendering it is difficult it is difficult because my little my little polygonal topology brain Can't Wrap its head around this entirely but this is really neat this is really neat is the main thing so I did two Gan Splats of this you'll notice that this is not dense in some places um and I believe there might be a size limit on Luma Labs so I've done two different versions of this I've done this version where I've done the whole side and now we see out here and my gosh this is this is really good and then I've done uh another version where it's just facing this angle so it's a little bit High a little bit higher density and looks a bit better so we'll look at that next but the advantages to this is that I'm looking out here and I've got a sense of depth out there and everything I mean it's a little bit blurry but don't mind that someone who's better at making Splats would make this better everything looks accurate all the especially the leaves of the palm trees down there I can see the leaves of the palm trees I ain't doing that in photographing not that far away from the camera um so there's there's that Advantage now there is some disadvantages to Gorge and splitting I think the most common uh thing that people talk about is if you were to use this for a video game then there's the example of what if I open a door in the video game now this gorian Splat all these all these Splats they are are changing their gradients of of various different parameters based on my viewing position but not based on what is happening in the world if I open a if I'm in a dark room in a video game and I open a door and the place is made out of Splats it it has to retrain every moment that this door is opening in order to uh change the lighting coming in through the door because that's not what it trained on and so that's that's one way that that doesn't help um Goran splitting but I don't know people might work a way around this I don't know it's very new technology as for the time being I I think I would prefer looking at 3D stuff in this app than any other VR app now we'll go and have a look at the higher density version of this just to give a nice nicer example one moment okay now you can see I've just done this small portion here not sure why the background uh buildings didn't work here but especially this part this is just a thousand times better than my photogrametry version thousand times better we'll we'll try and do a um a little side to side here I look out over there even up at the bridge this is so much bloody better than than the photogrametry it's wild it's absolutely wild now I'm not sure that the Luma Labs app or really any gorian splatting uh software right now um is is designed for huge large Open Spaces a lot of people are just doing a scan of one thing a single uh single object so we'll have a look at that so here is my friend's fireplace as you can see it's all the detail is put into this one thing and this is still this is still using the Splats this is still the mathematics it's just a lot more dense and if you could have this level of detail in a large environment I mean that's kind of what a Nerf is but you're not really seeing that real time just yet though there is some Google stuff um this is not a very Evergreen video everything I'm saying is is changing at such a fast rate but my main comparison is is this better in VR now the answer is yes in in many many different ways so for example look at this little this feather here you're not doing you're not you're not rendering this bloody feather or the uh or the little bits of this fan any other way at the moment with this Fidelity in in lifetime in VR I'm running this on the index at 120 uh Hertz and it is smooth as a whistle inside my headset so yes I'm just wrapping my head around the fact that I'm looking at not something without structure and yet boy does it have it boy do I I feel like I could grab this there's no there's no controller in this application um but I feel like I could touch these candles wow wow uh it will be very interesting when when VR chat starts implementing something like this I mean there's already um a Unity plugin for viewing Goran Splats so would would VR chat users want to be in an environment like this I don't know maybe I guess it depends on where it's been scanned probably not just my friend's fireplace but other places like um for example this Splat that I made in Sydney looking at the Sydney haror bridge now it was it was actually just for this tree that's why the Syd Bridge is a bit broken but this is such a cheap way to make a great representation of a location if I could just import this to VR chat and say hey everybody come come have a look at where I live this this is Australia this is where I'm from not from Sydney but just being able to show people this oh my gosh virtual tourism amazing absolutely incredible so there's still a lot of work to be done with like animating it if you wanted trees rustling you can do that with polygon just fine animating things I've seen some work uh I'll link it in the description below of animating this stuff but right now no you're going to be looking at still things you'll add music um in unity of course you could you could have these trees be polygonal and waving in the wind and just have the the background being engorge and splat if you wanted that's always an option but yeah as far as viewing environments in VR this is so wildly better than anything else anything else at the moment is crazy so widly easier I'm working on this this location in VR at the moment here's some little Snippets of it that I'm working on and it is taking weeks and weeks and weeks for a huge data set in order to compile this and render it out and add all these things this tree I walked around it three times and and this is what I got if I spent more uh attention on it then it would look a lot better but I just walked around this tree three times and now I'm standing here again wow okay this was supposed to be like a comparison video but it's just turned into me yeah oh my gosh um I mean look at this this is this is a friend of mine that again I just I just walked around him three times and now and now is perfectly preserved like this and I'm looking at him in virtual reality this has to go somewhere people people have to get being able to render Gan Splats in virtual reality now there's there's always going to be uh better things that that polygons can do right now it's not going to entirely change the industry such as the door scenario that I mentioned it's not going to be perfect for most video games but if you have I don't know like a video game where you're looking at a still room and then you walk up to the door and then there's a loading screen you're at the next door that type of small thing this would be perfect this would be absolutely perfect but right now there's still just so much more um so much more customization in topology in in structure rather than in uh this rasterization of mathematical uh Point Cloud nonsense that is aan Splat that polygons will still will still be the main thing but my gosh this is going pleas places so in conclusion most everything in the tech World goes the path of cheaper and faster and if it is taking me more than a month to complete this where I could walk around a tree three times and give you an okay experience people are going to go with the okay experience than waiting a month so let me know what you think is the future of viewing things in virtual reality just rendering Splats or will there be a a healthy balance between that and topology or are you just scared of the whole thing if so that's that's fine I'm also I'm also scared I'm always scared but like on the edge of a precipice that little gut feeling keeps me excited about about tomorrow

Channel: Puffy

Views: 8,639

Keywords:

Id: zLH7z7tF3Wc

Length: 17min 13sec (1033 seconds)

Published: Wed Dec 20 2023