No More Ambiguity in 360° Room Layout via Bi-Layout Estimation
Abstract: Inherent ambiguity in layout annotations poses significant challenges to developing accurate 360{\deg} room layout estimation models. To address this issue, we propose a novel Bi-Layout model capable of predicting two distinct layout types. One stops at ambiguous regions, while the other extends to encompass all visible areas. Our model employs two global context embeddings, where each embedding is designed to capture specific contextual information for each layout type. With our novel feature guidance module, the image feature retrieves relevant context from these embeddings, generating layout-aware features for precise bi-layout predictions. A unique property of our Bi-Layout model is its ability to inherently detect ambiguous regions by comparing the two predictions. To circumvent the need for manual correction of ambiguous annotations during testing, we also introduce a new metric for disambiguating ground truth layouts. Our method demonstrates superior performance on benchmark datasets, notably outperforming leading approaches. Specifically, on the MatterportLayout dataset, it improves 3DIoU from 81.70% to 82.57% across the full test set and notably from 54.80% to 59.97% in subsets with significant ambiguity. Project page: https://liagm.github.io/Bi_Layout/
- Attention augmented convolutional networks. In ICCV, 2019.
- End-to-end object detection with transformers. In ECCV, 2020.
- Pq-transformer: Jointly parsing 3d objects and layouts from point clouds. IEEE Robotics and Automation Letters, 2022.
- Per-pixel classification is not all you need for semantic segmentation. NeurIPS, 2021.
- Masked-attention mask transformer for universal image segmentation. In CVPR, 2022.
- Manhattan world: Compass direction from a single image by bayesian inference. In ICCV, 1999.
- Zillow indoor dataset: Annotated floor plans with 360deg panoramas and 3d room layouts. In CVPR, 2021.
- Corners for layout: End-to-end layout recovery from 360 images. IEEE Robotics and Automation Letters, 2020.
- Ned Greene. Environment mapping and other applications of world projections. IEEE Computer Graphics and Applications, 1986.
- Estimating spatial layout of rooms using volumetric reasoning about objects and surfaces. NeurIPS, 2010.
- Deep residual learning for image recognition. In CVPR, 2016.
- Recovering the spatial layout of cluttered rooms. In ICCV, 2009.
- Smart hypothesis generation for efficient and robust room layout estimation. In WACV, 2020.
- Long short-term memory. Neural Computation, 1997.
- Holistic 3d scene parsing and reconstruction from a single rgb image. In ECCV, 2018.
- Arbitrary style transfer in real-time with adaptive instance normalization. In ICCV, 2017.
- Lgt-net: Indoor panoramic room layout estimation with geometry-aware transformer network. In CVPR, 2022.
- Semi-supervised classification with graph convolutional networks. In ICLR, 2017.
- Learning informative edge maps for indoor scene layout prediction. In ICCV, 2015.
- Trackformer: Multi-object tracking with transformers. In CVPR, 2022.
- Image transformer. In ICML, 2018.
- Pytorch: An imperative style, high-performance deep learning library. NeurIPS, 2019.
- Film: Visual reasoning with a general conditioning layer. In AAAI, 2018.
- Atlantanet: inferring the 3d indoor layout from a single 360∘\circ∘ image beyond the manhattan world assumption. In ECCV, 2020.
- Manhattan junction catalogue for spatial reasoning of indoor scenes. In CVPR, 2013.
- A coarse-to-fine indoor layout estimation (cfile) method. In ACCV, 2017.
- Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing, 1997.
- Efficient exact inference for 3d indoor scene understanding. In ECCV, 2012.
- Disentangling orthogonal planes for indoor panoramic room layout estimation with cross-scale distortion awareness. In CVPR, 2023.
- Slibo-net: Floorplan reconstruction via slicing box representation with local geometry regularization. In NeurIPS, 2023.
- Horizonnet: Learning room layout with 1d representation and pano stretch data augmentation. In CVPR, 2019.
- Hohonet: 360 indoor holistic understanding with latent horizontal features. In CVPR, 2021.
- Attention is all you need. NeurIPS, 2017.
- Led2-net: Monocular 360deg layout estimation via differentiable depth rendering. In CVPR, 2021.
- Discriminative learning with latent variables for cluttered indoor scene understanding. Communications of the ACM, 2013.
- Dula-net: A dual-projection network for estimating room layouts from a single rgb panorama. In CVPR, 2019.
- Connecting the dots: Floorplan reconstruction using two-level queries. In CVPR, 2023.
- Holistic 3d scene understanding from a single image with implicit representation. In CVPR, 2021.
- Physics inspired optimization on semantic transfer features: An alternative method for room layout estimation. In CVPR, 2017.
- Deep hough transform for semantic line detection. TPAMI, 2021.
- 3d room layout estimation from a cubemap of panorama image via deep manhattan hough transform. In ECCV, 2022.
- Global tracking transformers. In CVPR, 2022.
- Deformable detr: Deformable transformers for end-to-end object detection. In ICLR, 2021.
- Layoutnet: Reconstructing the 3d room layout from a single rgb image. In CVPR, 2018.
- Manhattan room layout reconstruction from a single 360^ ∘\circ∘ 360∘\circ∘ image: A comparative study of state-of-the-art methods. IJCV, 2021.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.