Gaussian Splatting reconstructions with Android & iPhone

Hi everyone, in this video we'll go over how to  create Gaussian Splatting reconstructions that   run real time in your browser you'll need an  Android or iOS phone to capture the data any   newer phone will do you don't need LIDARs or depth  sensors if your phone has those they'll be used   and they can help in more difficult cases. Go to  App or Play Store and download free Spectacular   Rec app once you found the object or you want to  reconstruct launch the app and start recording While recording try to move slowly to avoid  motion blur in the video if possible you should   also have as bright lighting as possible  this produces the sharpest reconstructions Once the recording is done upload it to  cloud and then download it to your computer Extract the zip file and rename it to something   easier to remember you can also inspect  that the video turned out good and sharp now we'll install some tools required for  next steps you can find the links to these   instructions below now we need to install  Nerf Studio this is a rather long process   if you haven't done it yet and we'll skip it  here to save time there are good instructions   how to do this in the links below you  should also install FFMPEG using your   favorite package manager we still need to  install spectacular AI command line tools once the tools are installed we'll use process  tool to compute the camera trajectory for   recording you'll get a preview of the trajectory  as it's calculated once the preview closes it will   do one final optimiz step that can take some  time with Nerf Studio installed we are ready   to start the training process use NS train  in Gan splatting mode give it the folder   of processed recording as an argument during  the training you can monitor its progress in   web browser you can pretty quickly see early  results to confirm that it's turning out good once the training is complete you can  use ns-viewer to see the final results to make your reconstruction easily  sharable you can render a video out of it just add some camera set the frame rate to 60  to get really smooth results and render away now we have a video that plays on any device  however the really cool thing about Gaussian   Splatting is that you can actually  render them in real time even on a phone to do this we'll first use  ns-export to convert them into a ply file we can import this file into Super  Splat tool for some fine tuning first you'll want to rotate the Splats  so they are Level this makes VI them   much easier you should also move  the origin to center of your scene you can also cut away the part Parts you  don't want in the final reconstruction once you're happy with the results export  it back into py file then we'll check out   spectacular AI Point Cloud tools git repo with  it we can convert the scene into a web page double clicking the HTML file should open it in   a web browser this should run on  any newer device with a browser we also support conversion  to Splat format which you   can drag and drop directly into gplat JS editor and that's it we've gone from  recording to final go and spats   reconstruction on a web page thank  you for watching and have a great day

Channel: Spectacular AI

Views: 2,121

Keywords:

Id: OgXOUwZZ2VQ

Length: 6min 44sec (404 seconds)

Published: Wed Dec 20 2023