A new technique cleans up 3D scenes built from shaky, handheld phone video without needing a single perfectly clean frame to work from.
Researchers describe ClearGS, a method for 3D Gaussian Splatting - the fast-rendering technique that turns video into navigable 3D scenes - built specifically for uneven, mixed-quality handheld footage. Instead of a binary keep-or-discard call on each frame, it uses Reliability-aware View Allocation to grade every frame on appearance quality, degradation risk, and geometric usefulness, then weights its contribution to the reconstruction accordingly, while pulling back in weak-but-useful frames to preserve camera-path coverage. For damage that weighting alone can't fix, like motion blur, it runs Render-Guided In-Video Restoration: it compares the current 3D render, a restored version of the raw footage, and a high-frequency blend of both, then keeps whichever scores best on no-reference perceptual metrics. A final pass, Full-Trajectory Repair Consolidation, revisits earlier fixes so early-frame detail doesn't get lost later in the process. On the GS2E and GSOTM benchmarks, ClearGS posted state-of-the-art results on CLIP-IQA and MUSIQ image-quality scores and lower LPIPS error in most degradation cases, without ever using paired sharp footage or matched clean reference images.
Most 3DGS research still assumes tripod-steady, well-lit capture, which is not how anyone actually films a room on their phone. ClearGS targets the real failure mode - shaky, uneven, partly blurry footage - rather than a lab-clean input, which matters for any product trying to turn a walkthrough video into a usable 3D model.
The gains are measured against curated benchmark datasets, not a pile of random phone clips, and the paper is silent on how much extra compute all this reliability-grading and restoration costs per frame.