Multi-View 3D Scene Abstraction From Drone-Captured RGB Images

Abstract:

3D map reconstruction plays a critical role in fields such as autonomous driving, urban planning, and virtual reality. Using 2D images from real-world settings, multi-view image-based 3D map reconstruction offers a practical solution across these sectors. Specifically, generating 3D maps from realistic drone-view images has broad applicability. However, direct reconstruction from drone-view images is challenging due to extreme viewing angles and unstable image conditions like motion blur or varying light. Additionally, simultaneously generating and utilizing 3D maps from drone-captured images presents significant challenges, including fragmented sub-tasks such as 3D point cloud reconstruction, surface reconstruction, and shape fitting. To address these challenges, we introduce the 3D Scene Abstraction, which seamlessly integrates image matching, 3D reconstruction, 3D semantic segmentation, clustering, and shape fitting to produce 3D maps from RGB images. Our approach enhances the robustness of 3D map generation by merging 2D semantic information with 3D point clouds, ensuring reliable results under diverse drone-captured image conditions. For efficient data representation, we employ multiple primitives based on prior knowledge of various building shapes, improving both map formation performance and computational efficiency. Our approach not only enhances the effectiveness of 3D map reconstruction but also reduces the amount of data required to represent the scene, thereby improving overall efficiency and practical usability. These contributions address practical challenges in the field and lay the foundation for future advancements in 3D map reconstruction technologies.