What is Better? Polycam Gaussian Splats vs Original Gaussian Splats

hi I'm Jonathan Stevens and today we're going to jump into an exciting development in 3D God and splatting that's polycam and they have added it as a beta feature to their platform and I think that's going to be something that's really intriguing to you if you want to get into 3D G and splatting and you don't have the right PC or you don't want to go over that overhead of installing and running this complicated GitHub project so we're going to dive into this I'm going to show the differences between what you get with polycam and what you get running the original project and then go into some pros and cons of why would you use one or the other hope this is helpful in the end I find that there'll both be something that you want to look into let's jump into it okay first off we're going to jump into poly Cam's New 3D gajin splatting interface here and as you can see it's it's polycam is simple I started a new account just to get fresh on this and I tried only five data sets but I'm just going to jump into one and then jump into the results for an original one that I did with original project so we can compare some quality so um of course if you live under a rock and haven't seen my social media I did my cat Cloud who kind of looks like a cloud and I thought this turned out really good um he you know he got a lot of of great details here and I would say you see some like jumping of his fur some blurriness in his face but that's actually nothing to do with the gajan splatting from polycam I think that's to do with the fact that as I moved the camera around I didn't get all the angles but also the the white balance was was kind of jumping around but I mean this is pretty cool I mean it's it's running fast it's running real time this is on a nice computer but again I've ran this on my phone I've ran this on an iPad shown this to my kids it's fast um and and that is something that's a huge Pro that we're going to talk about at the end um so there's my cat here in polycam I also ran this at 30,000 iterations on the original project so let's jump into that really quick and take a look okay so here's the model with my cat I'm just going to change the way I move around to um track ball so I can get hopefully a little bit more um and as you can see let me just make all these little out of the way you can see I did definitely pick out some more d detail but again it's still fuzzy and I don't think that has again anything to do about the splatting that was used but more along the lines of the the quality of the imagery I didn't get enough but you can get a lot of these really fine strands on his fur which I thought looked really cool um so this turned out really good what I don't like is these floaters right in the camera fresh room so you have to get like beyond that to really get a rid of that and you have to get really tight so that's something I think 3D G and Splats in general needs to be fixed um but some way to crop that out could make a huge difference but this looks really good so you do get some more level detail like look at his fur and his hair you're going to get a higher level detail um on these like high detailed surfaces now will this translate something like a room where the walls are pretty blank I don't think so like here my computer screen things like that it gives enough detail I don't I don't need to be perfect my desk look looks really good it's not crazy detailed but again I don't I don't need that um but one thing I did notice is that these tend to be like three times as many Splats as that I see um using the the the polycam version so we're going to jump back to polygam and just do a quick double check on the cat to see the difference okay so here's my cat one more time just so you remember um got quite a bit of detail I'm just going to drag in my other cat right on top and you can see that detail just kind of bumps up so uh this is the original project this ran for about 50 minutes on my 3090 TI and then there is poly cam so definitely getting more detail but still to me this one looks really cool I think this impressive this would be cool to put on a holographic display or something all right so let's jump into another one really quickly okay so now this is an outdoor scene uh captur with a drone in uh some of these northern islands in Scandinavia it looks really cool um the Drone path was this like kind of long arcing linear drone path to all the way over here and one thing I noticed with poly cam is I don't have like a yah so I can't I can't get this Horizon straight no matter what I do so that one downside cuz the Drone was definitely over here when it took it was actually came as far as over here when it was taken a shot and I and it was horizontal but I can't really change that so the viewer is definitely Limited but again it's running real time on my browser I don't have any hardware it's just right here in polyam uh there poly. cam website which is a big Pro for being to show off this stuff so you know it's kind of like do I care um I will eventually but but not a ton uh one thing I did notice is again you got you can see the large Splats at the top and then the water is kind of got this grain texture and it does really cool stuff with the reflections but we're going to compare that to the original project real soon um it did great around these rocks as well but everything kind of has a general fuzziness about it and I think that's just because the details in here are so small and so sharp that it's hard to represent it in um the number of Splats I ended up in this scene so I'm going to quickly jump over to the original version and you can see what that looks like as well so here's the original version uh one thing I noticed is the colors look more sharp which I think is fantastic and the water looks looks more glassy like I would expect in the original version and if I go to the track ball here I can get a little bit more movement um this definitely looks more I don't know lifelike but I still get some weird things like this this pink that's that's nowhere to be seen in the scene but that's because that's probably too far off the original frustrum I do notice though that things still are kind of fuzzy if I'm not perfectly aware the camera is kind of where taken but if I get over here you'll see that these rocks are a lot sharper the grass looks a little bit more vibrant and sharper so I'm going to bring this in really quick and try to get to that same area again navigation is kind of tough so okay so here we go so things are looking pretty good but you just see a little bit more high frequency detail here but again this isn't bad I mean this was this was free it was fast I didn't have to do anything um but you can definitely see some of the detail so the water here too is very clear and crisp and in this you kind of get this kind of I don't know it looks like little scratches in the water and again I think that's just cuz it can't represent it with the limited number of Splats and again so okay when I come over here everything's all tilted uh when I come over here everything is all tilted as well this viewer by the way is not our friend but I can then come to the side and adjust the yah so that's something as much as I I say is a a con to polycam it really isn't because that's something that they can easily add I know that the crew over there is um very capable of adding that so um this scene is incredible when you look at it on the original one but it's not bad here as well I mean if if I'm trying just to visualize this as well um turned out pretty good okay so I'm going to look at uh a couple more examples here just going to close this up if I go back to my projects I did did run two experiments uh this one is my other one of my three cats and it did not work out so well with poly Cam and I have no idea no idea why because it worked in the original one um I'm not even going to compare it because there's no reason to but I thought that was interesting that like whole map must have failed or whatever they use in the background and same one with this which if you've watched like my uh other tutorial videos you I did one of me at my college on the modder standing on a lawn just trying even get it to display correctly um this is all I got so I don't know what's going on there it could be that maybe their pipeline had a problem because there were so many people hitting it all at once I don't know I'm not going to ding them for that um no idea why this didn't work I thought this would be a great comparison but it didn't maybe there was there was too many Splats that it capped out something like that so um those were interesting so but it was free so I didn't have to like worry about oh did it work I have to pay for that um and then here's my last one so this one with 99 photos this is one of the very first Nerfs I ever made it's this bridge I captured about a year ago um and uh the water was moving underneath it it was kind of a cloudy day it wasn't the best images but it did a really good job I thought um but what I notic is is again we kind of get down to details so here's kind of my favorite shot and let me pull in the other version to do like a back and forth comparison okay so here we go we have the original one or the original project here and then if I just quickly jump to this you see there's a drastic difference um you just got so much more detail in this wood you got more detail in the trees in the background and I think that's just because the limitation of what polycam has to do they did say that they're someone has mentioned that they they're capping it to 10,000 iterations and certain amount of Splats 5 million and that would make sense cuz you know that needs I don't know if you had um 5 million Splats would it be able to be perform it on a desktop browser or a phone but um it did do really well one thing I noticed in both that I would docket points for is the floaters and those do not show up like that inside of the Nerf you all these these floaters especially around like these these pieces of side hor horizontal slats tons of floaters um I didn't get that another interesting thing to look at is I can see like this park bench and if I get this original version you just see that it's um that tree it's just like way more detailed a little bit further on but again my focus is on the bridge and the bridge is really nice um if we look at this sign here says Bridge may be Slippery When Wet or when in or icy that's kind of hard to read um looks like almost like an AI generated sign I get into it again here it's way more sharp it's not perfect still but let's just do a little comparison there so you know these little high frequency details uh the colors are also a little bit more washed out I don't know what that's about but um might be something about the web render I I I don't really care so much about that um so there we at so there's a good comparison between the two okay so now you've seen both of the different versions and you can see visually there is a striking difference uh there is still a strong to use one or the other and I'm going to quickly just kind of outline the pros and cons for each so as you're trying to figure out show is it worth using polycam is it worth setting up the original 3D gin Splat project um that's up for you to decide so first we're going to look at the original project talk about the pros and cons so of course the biggest Pro is the detail level you can get way more detail using the original project and with that you can also tune the parameters I can decide all kinds of of things in there to get the optimal output for what you need so again it's like I get more control but at the expense that I need a really expensive GPU I'm running it on a 3090 TI 490 is great or one of those a6000 but those are kind of getting hard to get a hold of still uh in today's age and they're expensive versus poam which is free another big benefit of using the original project is I'm able then to get that data into Unity I can get it into Unreal Engine 5 using a plugin on the marketplace and I can get it into the Nerf Studio viewer and get really great flyth through animations and that's pretty easy to run uh you have have tutorials I think on unity and the Nerf studio if you want to look into that now cons again as I said you need a really expensive GPU that might just be out of your budget that might be something you never ever get to do but also time it takes a lot more time to run out on my PC versus upload the images to polycam and on your way it's super easy with polycam all you have to do is drag your images into polycam and you're Off to the Races that's it just hit start another big con to using the original one is that you you have to have Windows or Linux you can't just run it on a Mac um so there's some limitations of how you can use it if you're a Mac User it's not even an option for you and unless you want to use like a collab but it's still not The Full Experience so now let's jump over to polycam and kind of talk about their pros and cons so with polycam again it's free it's easy and I think the biggest Pro about that even though it doesn't have nearly the detail is that you can test and play with data sets and learn to make these yourself without having to have the investment of the hardware and the time to process it that's huge so it's a great way to sandbox play with things like this and I can imagine that polycam will and as they have mentioned increase the quality of the output over time I think they did some of this knowing that we're going to get hammered in our servers we might as well make it look good or be somewhat fast and look good versus try to do Max quality and this probably opens up some room for them to have a paid tier in the future another thing is you don't need to be an expert running code pulling from GitHub following my 40 minute video you can just get up and running this by opening a browser dragging images in and hitting start and there you go super easy and I think that might be the biggest barrier for most people who want to use 3D gge and splatting and perhaps one of my favorite things is now I can ined my shares on like social media and let people see the model themselves that has integrated web viewer I don't have to build that myself U there is options for the original project but it's it's not just out of the box you would have to like do a bunch of server work and all that you can just share these gaj and splat scenes with your friends families co-workers things like that and I don't have to go set it up it's just part of the polycam experience along with your other models that you did using just the the the regular polyam uh functionality so that really compelling so for a sharability this is this is what you want to use so getting into the cons we kind of cover those the cons are the quality is not as high the navigation is not as good so those things are something to think about but for again for sandboxing I think this is where you want to go and you can't export things the same so you can export a py file but I haven't quite figured out how to get that in unity or really use it and I think maybe something's coming with that but there's not as much usability Downstream so that is a big con for the original project if you want to try to use something else uh with this data is kind of locked into the polycam um ecosystem but for most people I think that's fine I don't think people really care about that so I hope this helped you decide which version you would want to use I think if you're new to the whole 3D Reconstruction from images and you want to learn to make God and Splats without all the setup and everything go use poyam it's great but if you want to learn how to use gajin Splats in unity and doing really cool things and Concepts go use original project at that point you probably already have all the hardware you need need and you'll know what you're doing so I hope you found this helpful please put in the comments if there's anything I miss that you want me to dive into in a future video and please like And subscribe this channel I'm trying to grow the information here so people can learn more about 3D go Splats and all of this and Nerfs and 3D Reconstruction from images I think this is important info so I hope you guys follow along and I'll see you in the next video

Channel: The NeRF Guru

Views: 8,556

Keywords:

Id: RRLpjKXc1WU

Length: 16min 54sec (1014 seconds)

Published: Fri Oct 06 2023