Home GitHub Rss

Building Scanning Patents

At a previous startup, I invented an Augmented Reality (AR) system for scanning buildings, and automatically extracting floor plans from scans. Recently a few of the patents I wrote were published, and I wanted to share those, with a brief summary.

Capturing Environmental Features Using 2D and 3D Scans


ar scan

This describes an augmented reality workflow for capturing a 3D mesh of an environment, and recording points of interest. A key idea is to capture all the user interactions and raw 3D data first. Later in a post-process step, we determine user intent and relative positioning using complete global information.

Floor Plan Extraction



Once we have a 3D scan of an environment, how do we go about turning this into a 2D floor plan? Floor plans are conceptual, rather than objective, and so are guided by user input. This invention describes both an augmented reality workflow and an algorithm for processing that input. It turns out, doing this perfectly is likely NP-complete, but this proper understanding guides effective heuristics.

Surface Animation During Dynamic Floor Plan Generation



It’s important to give feedback to the user during the scanning process. One way we do that is through the augmented reality visualization that shows which parts of a room are captured, and is programmed through shaders. The biggest challenge is to show accurate estimates quickly, despite the scan being incomplete.

Aligning Polygon-like Representations With Inaccuracies


align shapes

Scan measurements contain error, and the real world is imperfect. How can we correct the alignment of complex floor plan shapes with minimal distortion? One answer is to simultaneously move all points of the floor plan in a multi-variable optimization approach.

Door and Window Detection in an AR Environment


wall 3D

wall 2D

Along with the shape of the building, we are also interested in the apertures of each room. The polygonal shape tells us where the walls are in 3D. We can then render the portion of the 3D mesh corresponding to each wall, and detect cutouts in the rendered image. This transforms a complex 3D detection problem into a traditional 2D computer vision problem. These 2D detections are then projected back into 3D features on the wall.