CVPR 2024 - SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction

hi I'm otan and I will present our paper sugar surface align GA and splitting for efficient 3D mesh reconstruction and high quality mesh rendering using RGB images captured with a camera the goal of no view synthesis is to create a representation of the scene that is able to generate new images from any novel point of view to solve this problem 3D gon splitting proposes to use millions of tiny 3D GA and Primitives to represent the geometry of the scene as well as a neurer strategy tailored for 3D GA to render images much faster than concurent methods relying on vric rendering even though R Fields allow for photorealistic rendering for many tasks th artist need an explicit representation of the surfaces typically what we call a triangle mesh indeed the mesh is still the default representation used in computer graphics for many reasin we also believe and show with our method that meshes can be excellent proxies for Ting sculpting rigging or animating 3dgs representations using traditional software the standard practice for extracting meshes from Ren's Fields is to apply margin cubes however 3dgs produces millions of tiny and unorganized Guss when reconstructing real scenes Marin cubes does not scale well to such tiny Primitives and outputs meshes like this one with a lot of holes and artifacts more however even though some methods proposed to extract ises from nerves these approaches can be very slow or inaccurate with sugar we introduce a method that extracts accurate and editable meshes from 3D GA splatting representations within minutes on a single GPU the meshes can be edited sculpted animated composited and rendered with very realistic gon spling [Music] rendering our approach involves the following three contribut ions first a regularization term that encourages 3D gaussin to align with the surface during optimization then a mesh extraction method that is scalable fast and preserves in details unlike marching cubes finally a refinement method that binds 3D gaussin through the triangles of the mesh resulting in a hybrid representation 3D gaon splitting reconstructs the geometry of the scene as a combination of many 3D gaussin thus it can be represented by the following density function which is proportional to the PDF of a gaussian mixture model we show that in an ideal scenario where gaussin would have limited LEL up with each other would be opaque flat and align with the surface the density function could be simplified as the following D Bar in this ideal scenario we can also Define a distance function f to regularize our 3D gaussin we simply minimize the distance distance between the actual density function D and the simplified density function D that should hold through if gaussin were aligned with a surface this regularization already works pretty well but we show that we can push it even further by instead using the distance function f for extracting a mesh from our surface align gaussin we do not rely on margin cubes instead we sample points and their corresponding normals on the surface of the scene then use pressor Construction recover a triangle mesh specifically with sample points on the visible part of an ISO surface of the density function D corresponding to the mixture of gaussin because the parameters of the gaussin are explicit sampling points on the iso surface of the density can be done very efficiently compared to marching cubes our method is able to extract a mesh with much less holes more however thanks to our regularization our food method greatly reduces the noise on the surface and produces a much smoother mesh after extracting a mesh we can bind new gaussin to it and render the meshes using very realistic gausson spling rering we simply texture the triangles of the mesh with very thin gaussin the parameters of these gaussin are refined in the coordinate space of the triangle so that they are automatically adjusted when editing or animating the mesh [Music] oh [Music]

Channel: Antoine Guédon

Views: 744

Keywords:

Id: Ci0mgiQ8JgU

Length: 5min 0sec (300 seconds)

Published: Sun Jun 02 2024