Tracking States and Placement UX: Coaching Overlays, Reticles, and Failing Gracefully in ARKit

Written against iOS 18–26 with RealityKit and ARKitSession, re-checked against the iOS 27 betas. Swift, RealityKit-first — SceneKit is deprecated and none of this is worth rebuilding there.

Every AR app we audit has the same bug, and it is never in the rendering code.

The tracking works. The model looks great. And yet the first thirty seconds are a mess: the user holds the phone still like they are taking a photo, no planes are found, nothing appears, and they conclude the app is broken. Or they walk into a dark stairwell, ARKit loses the world, the content teleports into a wall, and they never open the app again.

That is not a tracking problem. It is a tracking-state UX problem, and it is the highest-leverage week of work available in most AR products. This tutorial covers how to run onboarding with ARCoachingOverlayView (and what to do on the modern ARKitSession path where you build your own), how to read tracking state and interruptions honestly, how to design placement so users succeed on the first try, and how to fail gracefully instead of silently.

Why users fail at AR onboarding

ARKit's world tracking is visual-inertial: it needs parallax (you moving) and texture (things to look at). Users, left to themselves, do neither. The three classic failure modes:

1. They stand still. No translation means no depth, so no planes and a drifting origin.
2. They point at a blank surface. A white conference table or a glossy floor gives the tracker nothing to lock onto.
3. They point at the ceiling, or at nothing. The app is waiting for a horizontal plane in a scene that contains none.

None of these are recoverable by better code. They are recoverable by telling the user, in the moment, what to do — which is exactly what a coaching flow is for.

The cheap win: ARCoachingOverlayView

If you are still on ARView/ARSCNView with ARSession, Apple ships the whole onboarding experience for free, localized into every language you ship. Do not build your own.

import ARKit
import RealityKit

final class ARViewController: UIViewController, ARCoachingOverlayViewDelegate {
    let arView = ARView(frame: .zero)
    private let coachingOverlay = ARCoachingOverlayView()

    override func viewDidLoad() {
        super.viewDidLoad()
        view.addSubview(arView)
        arView.frame = view.bounds

        coachingOverlay.session = arView.session
        coachingOverlay.delegate = self
        coachingOverlay.goal = .horizontalPlane      // or .anyPlane / .tracking / .geoTracking
        coachingOverlay.activatesAutomatically = true
        coachingOverlay.translatesAutoresizingMaskIntoConstraints = false
        arView.addSubview(coachingOverlay)
        NSLayoutConstraint.activate([
            coachingOverlay.centerXAnchor.constraint(equalTo: arView.centerXAnchor),
            coachingOverlay.centerYAnchor.constraint(equalTo: arView.centerYAnchor),
            coachingOverlay.widthAnchor.constraint(equalTo: arView.widthAnchor),
            coachingOverlay.heightAnchor.constraint(equalTo: arView.heightAnchor)
        ])
    }

    func coachingOverlayViewWillActivate(_ view: ARCoachingOverlayView) {
        hidePlacementUI()          // get your own chrome out of the way
    }

    func coachingOverlayViewDidDeactivate(_ view: ARCoachingOverlayView) {
        showPlacementUI()          // now, and only now, invite placement
    }
}

Three rules that people break:

• Pick the goal that matches what you actually need. .horizontalPlane when you place furniture on floors, .anyPlane for wall-or-floor, .tracking when you use raycasts against scene geometry rather than planes, .geoTracking for ARGeoAnchor wayfinding. Asking for a horizontal plane you do not need makes users wave the phone at the floor for no reason.
• Leave activatesAutomatically = true. The overlay will come back on its own after an interruption or a relocalization, which is precisely when you want it.
• Hide your own UI while it is up. Two competing sets of instructions is worse than none.

For relocalization specifically, call setActive(true, animated: true) yourself when the session reports .limited(.relocalizing) and you have decided to wait rather than reset.

The modern path: your own coaching on ARKitSession

The newer ARKitSession + data-provider API (the one shared with visionOS, and the one you use with RealityView in SwiftUI) does not hand you a coaching overlay. You build it. The good news is that it is a small state machine, and building it means you can put your brand and your language on it.

enum ARReadiness: Equatable {
    case starting              // session authorized, no data yet
    case findingSurfaces       // tracking, but no usable plane/mesh yet
    case ready                 // safe to invite placement
    case degraded(String)      // tracking limited: reason for the user
    case failed(String)        // unrecoverable: needs a reset or permission
}

Drive it from what the providers tell you rather than from a timer:

@Observable
final class SessionCoach {
    private(set) var readiness: ARReadiness = .starting

    func observe(session: ARKitSession,
                 planes: PlaneDetectionProvider) async {
        Task {
            for await event in session.events {
                switch event {
                case .dataProviderStateChanged(_, let newState, let error):
                    if newState == .stopped {
                        readiness = .failed(error?.localizedDescription
                                            ?? "AR stopped unexpectedly.")
                    }
                case .authorizationChanged(_, let status) where status != .allowed:
                    readiness = .failed("Camera access is needed to place content.")
                default:
                    break
                }
            }
        }

        for await update in planes.anchorUpdates {
            if case .added = update.event, readiness != .ready {
                readiness = .ready
            }
        }
    }
}

Then render the coaching copy from readiness in SwiftUI, over the RealityView. Keep the copy imperative and short: "Move your phone slowly side to side." Not "Initializing AR session." Users do not have a mental model of a session.

Reading tracking state honestly

On the ARSession path, ARCamera.TrackingState is the signal, and each limited reason maps to a different sentence for the user:

State	What actually happened	What to tell the user
.limited(.initializing)	Session warming up	"Getting started — move your phone a little."
.limited(.excessiveMotion)	Phone moving too fast	"Slow down."
.limited(.insufficientFeatures)	Blank or dark surroundings	"Point at something with more detail, or turn on a light."
.limited(.relocalizing)	Returning from an interruption	"Point back at where you were."
.notAvailable	No tracking at all	Hide content, show the coach.
.normal	Healthy	Show everything.

func session(_ session: ARSession, cameraDidChangeTrackingState camera: ARCamera) {
    switch camera.trackingState {
    case .normal:
        statusLabel.isHidden = true
        contentRoot.isEnabled = true
    case .limited(let reason):
        statusLabel.isHidden = false
        statusLabel.text = message(for: reason)
        contentRoot.isEnabled = (reason == .relocalizing) ? false : true
    case .notAvailable:
        statusLabel.text = "AR unavailable"
        contentRoot.isEnabled = false
    }
}

The important line is contentRoot.isEnabled = false during relocalization. Content whose anchor is currently wrong is worse than no content: it is what produces the "the sofa is inside the wall" screenshot that ends up in your App Store reviews. Hide it, then bring it back when tracking returns to .normal.

Handle interruptions in the same spirit:

func sessionWasInterrupted(_ session: ARSession) {
    contentRoot.isEnabled = false
    showBanner("Paused")
}

func sessionInterruptionEnded(_ session: ARSession) {
    // Either wait for relocalization, or reset and re-place.
    coachingOverlay.setActive(true, animated: true)
}

For long interruptions — a phone call, backgrounding for a few minutes, or moving to a different room — relocalization usually will not succeed. Give it a bounded window (five to ten seconds is generous), then offer an explicit "Start over" instead of leaving the user staring at a coaching overlay forever. If your content genuinely must survive across sessions, that is a persistent anchor problem, not a coaching problem.

Placement UX: the reticle earns its keep

Once tracking is healthy, the second failure point is placement. The pattern that works, in order:

1. Show a reticle — a projected ring or shadow that snaps to the surface under the screen center via a raycast. It teaches the user, without words, that surfaces exist and that the app knows where they are.
2. Distinguish tracked from estimated. Use .existingPlaneGeometry first, fall back to .estimatedPlane, and style the reticle differently for each. A dashed reticle honestly says "I am guessing."
3. Place on tap, not on release-and-hope. Then keep the object draggable and rotatable — nobody nails the position first time, and the ability to nudge removes all the pressure from the initial tap.
4. Confirm with a sound and a haptic. A UIImpactFeedbackGenerator tick plus a short spatial audio thud reads as "it landed" far better than an animation.

func updateReticle() {
    let center = CGPoint(x: arView.bounds.midX, y: arView.bounds.midY)
    if let result = arView.raycast(from: center,
                                   allowing: .existingPlaneGeometry,
                                   alignment: .horizontal).first {
        reticle.isEnabled = true
        reticle.setTransformMatrix(result.worldTransform, relativeTo: nil)
        reticle.style = .solid
    } else if let est = arView.raycast(from: center,
                                       allowing: .estimatedPlane,
                                       alignment: .horizontal).first {
        reticle.isEnabled = true
        reticle.setTransformMatrix(est.worldTransform, relativeTo: nil)
        reticle.style = .dashed
    } else {
        reticle.isEnabled = false     // and let the coach explain why
    }
}

Use a tracked raycast (trackedRaycast) rather than a one-shot raycast for the reticle: ARKit refines plane estimates continuously, and a tracked raycast keeps the reticle glued to the surface as those estimates improve instead of jittering a centimetre every frame.

On LiDAR devices, raycast against scene geometry instead of planes where you can — see scene reconstruction and occlusion. It places correctly on cluttered desks and sofas, where plane detection produces one giant fictional tabletop.

Permission and capability, before anything else

Two checks belong before your first frame, not after:

guard ARWorldTrackingConfiguration.isSupported else {
    presentUnsupportedDeviceScreen(); return
}

And camera authorization. If the user has denied it, show a screen with a real explanation and a button that deep-links to Settings via UIApplication.openSettingsURLString. Silently showing a black view is the single most common one-star review cause in AR apps. Your NSCameraUsageDescription string should say what the camera is for in your app ("to place furniture in your room"), not "This app uses the camera" — App Review reads it, and so do users.

Also decide, deliberately, what your app does on a device that cannot do AR at all: a 2D fallback, a photo mode, or an honest "this feature needs a newer iPhone." Anything is better than a disabled button with no explanation.

Accessibility is part of coaching

Coaching copy is visual by default. Two additions cover most of the gap:

• Post UIAccessibility.post(notification: .announcement, argument:) on state changes so VoiceOver users hear "surface found" and "tracking lost."
• Provide a non-spatial path to the core task where you can — a form, a photo, a list — because holding a phone up and walking around a room is not available to everyone.

Public-sector and large-enterprise procurement increasingly asks about this in writing. It is cheaper to build in now than to retrofit during a security review.

Test the states, not just the happy path

You cannot wave a phone in CI, but tracking-state UX is unusually testable because it is a state machine over an event stream. Record real ARKit sessions — including a bad one: dark room, fast motion, a backgrounding event — and replay them against your coach, asserting the sequence of states and the copy shown. Our post on testing ARKit apps covers the harness.

Then do the manual pass everyone skips: launch the app in a dim room, in a room with a plain white floor, and while walking. Those three take four minutes and find more real defects than an afternoon of unit tests.

A one-week checklist

If your app has none of this today:

1. Add ARCoachingOverlayView with the correct goal (or the readiness state machine on ARKitSession).
2. Hide content during .limited(.relocalizing) and interruptions.
3. Map every tracking state to one short imperative sentence.
4. Ship a tracked-raycast reticle that distinguishes tracked from estimated surfaces.
5. Add haptic plus audio placement confirmation and post-placement nudging.
6. Handle denied camera permission and unsupported devices with real screens.
7. Add VoiceOver announcements for state changes.
8. Bound relocalization with a timeout and an explicit "Start over."

None of that renders a single new polygon, and in our experience it moves first-session completion more than any visual work you could do in the same week.

---

Need a hand? RealityRogue's senior ARKit and RealityKit developers do exactly this work on client codebases — as a full build, or as contract developers embedded in your iOS team. Get in touch with your app concept, scope, or timeline and we will tell you honestly what it takes.