SphereSR: 360◦ Image Super-Resolution with Arbitrary Projection via Continuous Spherical Image Representation
– Published Date : TBD
– Category : 360 Imaging, Super-Resolution
– Place of publication : IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2022
The 360° imaging has recently gained great attention; however, its angular resolution is relatively lower than that of a narrow field-of-view (FOV) perspective image as itis captured by using fisheye lenses with the same sensor size. Therefore, it is beneficial to super-resolve a 360° image. Some attempts have been made but mostly considered the equirectangular projection (ERP) as one of the way for 360° image representation despite of latitude-dependent distortions. In that case, as the output high-resolution (HR) image is always in the same ERP format as the low resolution (LR) input, another information loss may occur when transforming the HR image to other projection types. In this paper, we propose SphereSR, a novel framework to generate a continuous spherical image representation from an LR 360° image, aiming at predicting the RGB values at given spherical coordinates for super-resolution with an arbitrary 360° image projection. Specifically, we first propose a feature extraction module that represents the spherical data based on icosahedron and efficiently extracts features on the spherical surface. We then propose a spherical local implicit image function (SLIIF) to predict RGB values at the spherical coordinates. As such, SphereSR flexibly reconstructs an HR image under an arbitrary projection type. Experiments on various benchmark datasets show that our method significantly surpasses existing methods.
Acknowledgement : This work was supported by the National Research Foundation of Korea(NRF) grant funded by the Korea government(MSIT) (NRF-2022R1A2B5B03002636).