Matterport allows you to turn a cluttered room into an empty space without moving a single piece of furniture!
Imagine being able to completely redecorate your living room without moving a single piece of furniture. That's what Matterport is working on.
Having recently launched the AI pillar of Matterport 3, we are now looking at the introduction of 3D image augmentation for our digital twins.
Matterport initially focused on creating realistic, static reconstructions, creating excellent virtual twins of real spaces. But to effectively assess the potential uses of the spaces, their operation and maintenance needs, a static approach is not enough.
Matterport has developed this semantic approach to solve this problem by providing deeper insights and valuable information about real estate.
The result of Matterport's decades of machine learning and AI experience, you can now create new design and furnishing ideas at the touch of a button on Project Genesis.
What is "celarance"?
'Emptying' is a key technique in digital image processing and 3D modelling that involves removing furniture and movable objects from images of a space to create empty space.
The steps in the "celarance" process:
- Reconstruction: first the scanned space is captured and reconstructed to create the digital twin.
- Analysis: then Matterport analyses the space, the surfaces that belong to the furniture to be removed.
- Synthesis: since the areas covered by furniture were never directly fixed, after removing the furniture, you are left with blank pixels in the images and holes in the mesh. For the images, authentic "empty space" content is needed to fill in the gaps, while the holes in the mesh need to be filled and textured.
Semantic segmentation
Semantic segmentation, a critical computer vision task, divides an image into distinct regions and assigns a specific category to each. It aims to assign each pixel to a class (e.g. "floor", "wall", "window", "table"), thereby providing a comprehensive understanding of the scene by identifying objects and delineating their boundaries.
Semantic segmentation is an essential technique in computer vision, used in self-driving vehicles, medical imaging, robotics and many other fields.
It has recently become a prominent feature in virtual interior design. When initially capturing a space, the primary data available outline the overall structure and aesthetics of the space. Semantic segmentation plays a crucial role in providing a richer understanding of the content of the Matterport space, allowing for precise manipulation - whether it is moving, editing, indexing or removing elements.
Inpainting
To remove the furniture from the images and 3D structures of the digital twins, we first need to define the pixels or mesh surfaces that belong to the furniture. After removing the furniture, often incomplete information remains because the area behind/below the furniture was not visible during the digital twin capture.
Therefore, it is necessary to generate authentic visual/3D content that fills in these holes after the furniture is removed. This process is called "inpainting".