Irawati Nurmala Sari, Weiwei Du
In 3D reconstruction, creating accurate models from a single 2D image remains a complex challenge in computer vision and virtual environments. Acknowledging this complexity, our study presents a research approach that aims to advance the field beyond conventional reconstruction methods. We introduce a novel method designed to reconstruct 3D models from single damaged 2D images while tackling the added challenge of restoring missing details. Our method employs an expanded-scale stable diffusion model for inpainting the input 2D image, restoring missing information and enhancing depth estimation for sparse-view 3D reconstruction. By integrating inpainting within a scalable diffusion framework, we achieve improved fidelity to the original structure, even with limited sparse mesh data. Additionally, our framework incorporates multi-view stereo processing to optimize available viewpoint information, enhancing reconstruction accuracy while maintaining computational efficiency. Experimental results indicate a significant advancement in transforming single 2D image inpainting into effective 3D object reconstruction. By bridging inpainting and 3D reconstruction, our approach recovers missing or occluded details in the 2D image and enhances the accuracy of 3D modeling from sparse viewpoints. The inpainting technique is important here, as it restores essential details within the 2D input, directly improving the quality and fidelity of the 3D reconstruction output. Our integrated method of inpainting and sparse-view reconstruction outperforms existing approaches, achieving higher accuracy in 3D object representation from limited data. ©2025 IEEE.
Department of Informatics Engineering, University of Brawijaya, Malang, Indonesia; Kyoto Institute of Technology, Kyoto, Japan
