RealityRogue | iPhone Photogrammetry with Object Capture: From Product Photos to a USDZ AR Quick Look Viewer

The cheapest AR feature an e-commerce team can ship is "View in your space," and the expensive part of it has never been the AR code — it is producing a good 3D model of every SKU. Apple's Object Capture pipeline changes that economics: a LiDAR iPhone captures the product, PhotogrammetrySession reconstructs it on the phone or a Mac, and the resulting USDZ drops straight into AR Quick Look, which every iPhone and iPad can open from Safari or Messages with no app install.

This tutorial builds that pipeline end to end. Requirements: an iPhone with LiDAR and an A14 chip or later (iPhone 12 Pro and later Pro models) running iOS 17 or later, Xcode 16 or later, and for the optional Mac reconstruction path, an Apple silicon Mac.

1. Capture on iPhone with ObjectCaptureSession

iOS 17 introduced ObjectCaptureSession and ObjectCaptureView, which give you Apple's guided capture UI — the same one used in the Reality Composer iOS app. The session runs through a state machine: initializing → ready → detecting → capturing → finishing → completed.

import SwiftUI
import RealityKit

@MainActor
final class CaptureModel: ObservableObject {
    @Published var session = ObjectCaptureSession()
    let imagesDirectory: URL
    let checkpointDirectory: URL

    init() {
        let base = FileManager.default.temporaryDirectory.appendingPathComponent("capture", isDirectory: true)
        imagesDirectory = base.appendingPathComponent("images", isDirectory: true)
        checkpointDirectory = base.appendingPathComponent("checkpoints", isDirectory: true)
        try? FileManager.default.createDirectory(at: imagesDirectory, withIntermediateDirectories: true)
        try? FileManager.default.createDirectory(at: checkpointDirectory, withIntermediateDirectories: true)
    }

    func start() {
        var config = ObjectCaptureSession.Configuration()
        config.checkpointDirectory = checkpointDirectory
        config.isOverCaptureEnabled = true     // lets the user keep shooting past the guided orbits
        session.start(imagesDirectory: imagesDirectory, configuration: config)
    }
}

struct CaptureView: View {
    @StateObject private var model = CaptureModel()

    var body: some View {
        ZStack(alignment: .bottom) {
            ObjectCaptureView(session: model.session)
                .ignoresSafeArea()

            controls
        }
        .onAppear { model.start() }
    }

    @ViewBuilder
    private var controls: some View {
        switch model.session.state {
        case .ready:
            Button("Continue") { _ = model.session.startDetecting() }
        case .detecting:
            Button("Start Capture") { model.session.startCapturing() }
        case .capturing:
            if model.session.userCompletedScanPass {
                Button("Finish") { model.session.finish() }
            } else {
                Text("Keep moving around the object")
            }
        case .completed:
            Text("Capture complete")
        default:
            EmptyView()
        }
    }
}

ObjectCaptureSession.isSupported must be checked first; on unsupported devices, fall back to a plain camera capture and do the reconstruction on a Mac (section 3 handles both). The checkpoint directory lets reconstruction resume from the session's own intermediate data, which significantly speeds up on-device processing.

Capture tips that matter more than any code: a matte turntable or a plain surface, diffuse lighting with no hard shadows, 60 to 120 photos across the guided orbits, and a flip pass for products whose underside is visible in AR (shoes, bags).

2. Reconstruct with PhotogrammetrySession

Reconstruction is the same API on iPhone and Mac. On-device iOS supports .preview and .reduced detail; the Mac adds .medium, .full, and .raw.

import RealityKit

func reconstruct(from images: URL, checkpoints: URL?, to output: URL) async throws {
    var config = PhotogrammetrySession.Configuration()
    config.checkpointDirectory = checkpoints
    config.featureSensitivity = .normal
    config.sampleOrdering = .sequential        // guided captures are in orbit order
    #if os(macOS)
    let detail: PhotogrammetrySession.Request.Detail = .medium
    #else
    let detail: PhotogrammetrySession.Request.Detail = .reduced
    #endif

    let session = try PhotogrammetrySession(input: images, configuration: config)
    let request = PhotogrammetrySession.Request.modelFile(url: output, detail: detail)

    try session.process(requests: [request])

    for try await output in session.outputs {
        switch output {
        case .requestProgress(_, let fraction):
            print("Progress: \(Int(fraction * 100))%")
        case .requestComplete(_, let result):
            if case .modelFile(let url) = result { print("Wrote \(url.path)") }
        case .requestError(_, let error):
            throw error
        case .processingComplete:
            return
        default:
            break
        }
    }
}

.reduced on an iPhone 15 Pro produces a web-appropriate model (roughly 25k triangles, 1k textures) in a couple of minutes. For hero product shots, ship the captured images plus checkpoints to a Mac or a Mac-based build server and request .medium or .full; the same PhotogrammetrySession code runs unchanged, and you can request several detail levels in one process call to get a low-poly web asset and a high-poly archive asset from one reconstruction.

3. Batch reconstruction on a Mac

For a catalog of hundreds of SKUs, a command-line tool on an Apple silicon Mac is the workhorse. Apple ships a sample (HelloPhotogrammetry) that is essentially the function above wrapped in ArgumentParser. The folder convention is simple: one input folder of HEIC/JPEG images per SKU, one output <sku>.usdz. Photos captured with ObjectCaptureSession carry depth and gravity metadata that improves scale and orientation; photos from a DSLR still work but the model may need a scale correction.

Run reconstructions sequentially rather than in parallel; each job already saturates the GPU.

4. Inspect and clean the USDZ

Before publishing, open the USDZ in Reality Composer Pro or Preview on the Mac and check:

Scale. AR Quick Look shows the model at real size. A 30 cm shoe that reconstructs at 3 m will be noticed. Object Capture models from LiDAR devices are typically accurate to a few percent; fix outliers by setting the root prim's scale in Reality Composer Pro.
Orientation. The model should sit on the ground plane with +Y up; Quick Look places it on the detected surface at its bounding-box bottom.
Floating debris. Turntable or background fragments are common; delete stray meshes in Reality Composer Pro.
File size. Aim for under 10 MB for web delivery. Use the .reduced level or decimate in Reality Composer Pro.

5. Ship it: AR Quick Look on the web and in-app

AR Quick Look needs no ARKit code. On the web, a plain anchor with rel="ar" wrapping an image is the entire integration:

<a rel="ar" href="/models/sneaker-042.usdz">
  <img src="/images/sneaker-042.jpg" alt="Sneaker 042">
</a>

Safari on iOS and iPadOS shows the AR badge and opens the model in Quick Look; on a Mac it opens in Preview; on other browsers the link simply downloads. You can append a few display hints as a URL fragment, for example #allowsContentScaling=0 to stop users pinch-scaling a product that must appear at true size, and #canonicalWebPageURL=... plus #callToAction=Buy to show a banner with a purchase button inside Quick Look.

In a native app, present the same file with QLPreviewController:

import QuickLook
import SwiftUI

struct ARQuickLookView: UIViewControllerRepresentable {
    let fileURL: URL

    func makeCoordinator() -> Coordinator { Coordinator(url: fileURL) }

    func makeUIViewController(context: Context) -> QLPreviewController {
        let controller = QLPreviewController()
        controller.dataSource = context.coordinator
        return controller
    }

    func updateUIViewController(_ uiViewController: QLPreviewController, context: Context) {}

    final class Coordinator: NSObject, QLPreviewControllerDataSource {
        let url: URL
        init(url: URL) { self.url = url }
        func numberOfPreviewItems(in controller: QLPreviewController) -> Int { 1 }
        func previewController(_ controller: QLPreviewController, previewItemAt index: Int) -> QLPreviewItem {
            url as QLPreviewItem
        }
    }
}

The same USDZ also opens natively on Apple Vision Pro, where Quick Look shows it as a volume the user can place in the room.

6. When you outgrow Quick Look

Quick Look gives you placement, scaling, and a share sheet, and nothing else: no configurator, no analytics beyond the link click, no custom UI. When the product page needs a color picker that swaps materials, a size comparison against the user's own furniture, or a shared session, that is the point to move the viewer into a RealityView in your own app while keeping Quick Look as the zero-install web path. The asset pipeline you built in sections 1 through 4 stays exactly the same.