RealityRogue | Room-Scale AR with RoomPlan: Scan, Parse, Furnish

RoomPlan is the part of Apple's AR stack that most people outside the industry have never heard of and most interior, real-estate, and facilities apps should be built on. Point a LiDAR iPhone or iPad around a room and in under a minute you get a parametric model — walls, doors, windows, openings, and classified furniture — as structured data, not just a mesh. This tutorial scans a room, parses the result, exports it, and then places a USDZ sofa against a real wall using RealityKit.

Requirements: a device with LiDAR (iPhone 12 Pro or later Pro models, iPad Pro 2020 or later), iOS 17 or later for the APIs used here, Xcode 16 or later. RoomPlan shipped in iOS 16; the multi-room and structured-export features used below arrived in iOS 17.

1. Scan: RoomCaptureView for the quickest start

RoomPlan offers two levels. RoomCaptureView is a drop-in UIKit view with Apple's coaching UI and live 3D preview; RoomCaptureSession is the lower-level session if you want to draw your own UI. Start with the view.

Add NSCameraUsageDescription to Info.plist, then wrap the view for SwiftUI:

import SwiftUI
import RoomPlan

struct RoomScannerView: UIViewRepresentable {
    @Binding var capturedRoom: CapturedRoom?

    func makeCoordinator() -> Coordinator { Coordinator(parent: self) }

    func makeUIView(context: Context) -> RoomCaptureView {
        let view = RoomCaptureView(frame: .zero)
        view.delegate = context.coordinator
        var config = RoomCaptureSession.Configuration()
        config.isCoachingEnabled = true
        view.captureSession.run(configuration: config)
        return view
    }

    func updateUIView(_ uiView: RoomCaptureView, context: Context) {}

    static func dismantleUIView(_ uiView: RoomCaptureView, coordinator: Coordinator) {
        uiView.captureSession.stop()
    }

    final class Coordinator: NSObject, RoomCaptureViewDelegate {
        let parent: RoomScannerView
        init(parent: RoomScannerView) { self.parent = parent }

        func captureView(shouldPresent roomDataForProcessing: CapturedRoomData,
                         error: Error?) -> Bool {
            true   // let RoomPlan run its post-processing and show the result
        }

        func captureView(didPresent processedResult: CapturedRoom, error: Error?) {
            if let error { print("RoomPlan error: \(error)"); return }
            parent.capturedRoom = processedResult
        }
    }
}

RoomCaptureViewDelegate must be NSObjectProtocol, hence the NSObject subclass. Call captureSession.stop() when the user taps Done; the view then runs its post-processing pass and calls didPresent.

Before showing the scanner, check RoomCaptureSession.isSupported — it is false on non-LiDAR devices and you should fall back to manual entry or a photo flow.

2. Parse the CapturedRoom

CapturedRoom is the payoff. It is a value type containing arrays of Surface (walls, doors, windows, openings, floors) and Object (furniture, classified). Each has a transform in the room's coordinate space and dimensions in meters.

import RoomPlan
import simd

struct RoomSummary {
    let wallCount: Int
    let floorArea: Float
    let longestWallTransform: simd_float4x4?
    let longestWallWidth: Float
    let furniture: [(CapturedRoom.Object.Category, SIMD3<Float>)]
}

func summarize(_ room: CapturedRoom) -> RoomSummary {
    let walls = room.walls
    let longest = walls.max { $0.dimensions.x < $1.dimensions.x }

    // Floor area: RoomPlan gives a polygon per floor surface on iOS 17+.
    let area = room.floors.reduce(Float(0)) { partial, floor in
        partial + polygonArea(floor.polygonCorners)
    }

    let furniture = room.objects.map { object in
        (object.category, object.dimensions)
    }

    return RoomSummary(wallCount: walls.count,
                       floorArea: area,
                       longestWallTransform: longest?.transform,
                       longestWallWidth: longest?.dimensions.x ?? 0,
                       furniture: furniture)
}

/// Shoelace formula on the XZ plane (floor corners are 3D points with y ≈ 0).
func polygonArea(_ corners: [SIMD3<Float>]) -> Float {
    guard corners.count >= 3 else { return 0 }
    var sum: Float = 0
    for i in 0..<corners.count {
        let a = corners[i], b = corners[(i + 1) % corners.count]
        sum += a.x * b.z - b.x * a.z
    }
    return abs(sum) / 2
}

Object.category is an enum with cases such as .sofa, .table, .bed, .refrigerator, .television, .storage, and so on — about 16 in current releases. Surface.category distinguishes .wall, .door(isOpen:), .window, .opening, and .floor. Doors and windows carry a parentIdentifier pointing at the wall they sit in, which matters for the furniture placement step: do not put a sofa in front of a door.

3. Export

For hand-off to a design tool, export USDZ. For your own backend, export the structured model as JSON by encoding CapturedRoom directly — it conforms to Codable.

func export(_ room: CapturedRoom, to directory: URL) throws {
    let usdz = directory.appendingPathComponent("room.usdz")
    try room.export(to: usdz, exportOptions: .parametric)

    let json = directory.appendingPathComponent("room.json")
    let encoder = JSONEncoder()
    encoder.outputFormatting = [.prettyPrinted, .sortedKeys]
    try encoder.encode(room).write(to: json)
}

.parametric gives clean box geometry per element, .mesh gives the raw scanned mesh, and .model substitutes Apple's generic furniture models for detected objects. Most downstream tools want .parametric.

Scanning several rooms? StructureBuilder (iOS 17+) merges multiple CapturedRooms into a CapturedStructure, which is how you produce a whole-apartment floor plan.

4. Furnish: place a sofa against the longest wall

Now the fun part. We take the longest wall from the summary and drop a USDZ sofa in front of it with RealityKit, in the same world coordinates RoomPlan used. RoomPlan and ARKit share a world origin when the scan and the placement happen in the same session; if you place content in a later session, relocalize with a saved ARWorldMap first, or anchor relative to a detected wall again.

import SwiftUI
import RealityKit
import RoomPlan

struct FurnishView: View {
    let room: CapturedRoom

    var body: some View {
        RealityView { content in
            content.camera = .spatialTracking

            guard let wall = room.walls.max(by: { $0.dimensions.x < $1.dimensions.x }),
                  let sofa = try? await Entity(named: "sofa") else { return }

            // Wall transform: x runs along the wall, y is up, z points out of the wall face.
            let wallTransform = Transform(matrix: wall.transform)
            let wallHalfHeight = wall.dimensions.y / 2

            // Start at the wall's origin (its center), drop to the floor,
            // and push the sofa out by half its depth plus a small gap.
            let sofaDepth = sofa.visualBounds(relativeTo: nil).extents.z
            var offset = SIMD3<Float>(0, -wallHalfHeight, sofaDepth / 2 + 0.05)
            offset = wallTransform.rotation.act(offset)

            sofa.transform.rotation = wallTransform.rotation
            sofa.position = wallTransform.translation + offset

            let root = AnchorEntity(world: .zero)
            root.addChild(sofa)
            content.add(root)
        }
        .ignoresSafeArea()
    }
}

Two details are easy to get wrong. RoomPlan wall transforms are centered on the wall, not at floor level, so you subtract half the wall height to reach the floor. And the wall's local z axis points into the room, so a positive z offset moves the sofa away from the wall — rotate the offset by the wall's rotation before adding it, or the sofa ends up in the neighbor's apartment.

For a product app you would then let the user slide the sofa along the wall with a DragGesture().targetedToAnyEntity() constrained to the wall's x axis, and check the new position against room.doors and room.windows to avoid blocking them.

5. Checking fit

Because CapturedRoom is data, fit checks are ordinary arithmetic:

func fits(_ itemWidth: Float, alongWall wall: CapturedRoom.Surface,
          avoiding doors: [CapturedRoom.Surface]) -> Bool {
    let doorsOnWall = doors.filter { $0.parentIdentifier == wall.identifier }
    let blocked = doorsOnWall.reduce(Float(0)) { $0 + $1.dimensions.x }
    return wall.dimensions.x - blocked >= itemWidth
}

That single function is the heart of "will this fit?" features in retail apps, and it needs no 3D rendering at all.

Limits and gotchas

RoomPlan is LiDAR-only. Plan a fallback for non-Pro devices.
Scans degrade in rooms with mirrors, large glass walls, or very dark surfaces. Coach the user to scan slowly and close blinds.
Object classification is good but not perfect; let users relabel.
Each RoomCaptureSession must run on the main thread and stops automatically after a few minutes; long scans should use multi-room capture.
visionOS does not include RoomPlan. On Apple Vision Pro, use ARKit's SceneReconstructionProvider and PlaneDetectionProvider (wall and floor classification) instead; the placement logic above ports with minor changes.