
realitykit
Build iOS augmented reality and 3D experiences with RealityKit and ARKit. Use when adding RealityView content, loading entities or USDZ models, anchoring objects to planes or world positions, distinguishing entity hit tests from ARKit real-world raycasts, handling AR camera availability, world tracking, scene updates, or RealityKit entity gestures and interactions.
- Installs
- 3.2K
- Last 24h
- +17
- Trending rank
- #6527
- Security
- Not audited
Install
Works with every agent the skills CLI supports.
npx skills add https://github.com/dpearson2699/swift-ios-skills --skill realitykit- 01
Install
Run the command in your project. The CLI asks which agents to install for.
- 02
Describe the task
Ask your agent for something this skill covers. It loads realitykit automatically when relevant.
- 03
Review the result
The agent follows the SKILL.md steps, scripts and references bundled with the skill.
SKILL.md
The instructions your agent reads when the skill is triggered.
RealityKit
Build AR experiences on iOS using RealityKit for rendering and ARKit for world
tracking. Covers RealityView, entity management, raycasting, scene
understanding, and gesture-based interactions. Targets Swift 6.3 / iOS 26+.
Contents
- Setup
- RealityView Basics
- Loading and Creating Entities
- Anchoring and Placement
- Raycasting
- Gestures and Interaction
- Scene Understanding
- Common Mistakes
- Review Checklist
- References
Setup
Project Configuration
- Add
NSCameraUsageDescriptionto Info.plist - On iOS,
RealityViewCameraContentdisplays an AR camera view by default (iOS 18+, macOS 15+); use.virtualcamera mode for explicit non-AR fallback - No entitlement is required for basic AR. If AR is core to the app, add the
arkitrequired-device capability; otherwise gate AR UI withisSupported.
Device Requirements
Check the exact AR configuration's isSupported value before presenting AR UI.
import ARKit
guard ARWorldTrackingConfiguration.isSupported else {
showUnsupportedDeviceMessage()
return
}
Key Types
| Type | Platform | Role |
|---|---|---|
RealityView | iOS 18+, visionOS 1+ | SwiftUI view that hosts RealityKit content |
RealityViewCameraContent | iOS 18+, macOS 15+ | Content displayed through an AR camera view on iOS, non-AR on macOS |
Entity | All | Base class for all scene objects |
ModelEntity | All | Entity with a visible 3D model |
AnchorEntity | All | Tethers entities to a real-world anchor |
RealityView Basics
RealityView is the SwiftUI entry point for RealityKit.
RealityViewCameraContent is the iOS/macOS content type. On iOS, it uses an AR
camera view by default and can use content.camera = .virtual for non-AR mode
when requested or when AR/camera access is unavailable.
import ARKit
import SwiftUI
import RealityKit
struct ARExperienceView: View {
var body: some View {
RealityView { (content: RealityViewCameraContent) in
if !ARWorldTrackingConfiguration.isSupported {
content.camera = .virtual
}
let sphere = ModelEntity(
mesh: .generateSphere(radius: 0.05),
materials: [SimpleMaterial(
color: .blue,
isMetallic: true
)]
)
sphere.position = [0, 0, -0.5] // 50cm in front of camera
content.add(sphere)
}
}
}
Make and Update Pattern
Use the update closure to respond to SwiftUI state changes:
struct PlacementView: View {
@State private var modelColor: UIColor = .red
var body: some View {
RealityView { content in
let box = ModelEntity(
mesh: .generateBox(size: 0.1),
materials: [SimpleMaterial(
color: .red,
isMetallic: false
)]
)
box.name = "colorBox"
box.position = [0, 0, -0.5]
content.add(box)
} update: { content in
if let box = content.entities.first(
where: { $0.name == "colorBox" }
) as? ModelEntity {
box.model?.materials = [SimpleMaterial(
color: modelColor,
isMetallic: false
)]
}
}
Button("Change Color") {
modelColor = modelColor == .red ? .green : .red
}
}
}
Loading and Creating Entities
Loading from USDZ Files
Load 3D models asynchronously to avoid blocking the main thread:
RealityView { content in
if let robot = try? await ModelEntity(named: "robot") {
robot.position = [0, -0.2, -0.8]
robot.scale = [0.01, 0.01, 0.01]
content.add(robot)
}
}
Adding Components
Entities use an ECS (Entity Component System) architecture. Add components to give entities behavior:
let box = ModelEntity(
mesh: .generateBox(size: 0.1),
materials: [SimpleMaterial(color: .red, isMetallic: false)]
)
// Make it respond to physics
box.components.set(PhysicsBodyComponent(
massProperties: .default,
material: .default,
mode: .dynamic
))
// Add collision shape for interaction
box.components.set(CollisionComponent(
shapes: [.generateBox(size: [0.1, 0.1, 0.1])]
))
// Enable input targeting for gestures
box.components.set(InputTargetComponent())
Anchoring and Placement
AnchorEntity
Use AnchorEntity to anchor content to detected surfaces or world positions:
RealityView { content in
// Anchor to a horizontal surface
let floorAnchor = AnchorEntity(.plane(
.horizontal,
classification: .floor,
minimumBounds: [0.2, 0.2]
))
let model = ModelEntity(
mesh: .generateBox(size: 0.1),
materials: [SimpleMaterial(color: .orange, isMetallic: false)]
)
floorAnchor.addChild(model)
content.add(floorAnchor)
}
Anchor Targets
| Target | Description |
|---|---|
.plane(.horizontal, ...) | Horizontal surfaces (floors, tables) |
.plane(.vertical, ...) | Vertical surfaces (walls) |
.plane(.any, ...) | Any detected plane |
.world(transform:) | Fixed world-space position |
Raycasting
Keep RealityKit scene queries separate from ARKit real-world raycasts:
RealityViewCameraContent.ray(through:in:to:)returns a camera ray in RealityKit coordinate spaces. It projects a screen point into the virtual scene; it is not proof of a detected physical surface.RealityViewCameraContent.hitTest(point:in:query:mask:)hits virtual entities made hittable byCollisionComponentshapes. Use those shapes for entity picking and targeted gestures, not ARKit plane detection.- Use
AnchorEntity(.plane(...))for simple placement on detected planes. - Use ARKit
ARRaycastQueryplusARSession.raycast(_:)when the task needs a one-shot intersection with real-world surfaces, then anchor withAnchorEntity(raycastResult:).
let results = session.raycast(query)
if let result = results.first {
let anchor = AnchorEntity(raycastResult: result)
anchor.addChild(model)
content.add(anchor)
}
Do not treat entity hit tests as substitutes for ARKit surface raycasts.
Gestures and Interaction
For gesture-based entity interaction, add CollisionComponent for the hittable
shape and InputTargetComponent for input targeting.
Drag Gesture on Entities
struct DraggableARView: View {
var body: some View {
RealityView { content in
let box = ModelEntity(
mesh: .generateBox(size: 0.1),
materials: [SimpleMaterial(color: .blue, isMetallic: true)]
)
box.position = [0, 0, -0.5]
box.components.set(CollisionComponent(
shapes: [.generateBox(size: [0.1, 0.1, 0.1])]
))
box.components.set(InputTargetComponent())
box.name = "draggable"
content.add(box)
}
.gesture(
DragGesture()
.targetedToAnyEntity()
.onChanged { value in
let entity = value.entity
guard let parent = entity.parent else { return }
entity.position = value.convert(
value.location3D,
from: .local,
to: parent
)
}
)
}
}
Scene Understanding
Per-Frame Updates
Subscribe to SceneEvents.Update for continuous scene work rather than driving
RealityKit with a SwiftUI timer. Keep the subscription alive and use
event.deltaTime; see Entity Animations.
Platform Boundaries
On visionOS, ARKit provides a different API surface with ARKitSession,
WorldTrackingProvider, and PlaneDetectionProvider. These visionOS-specific
types are not available on iOS. On iOS, RealityKit handles world tracking
automatically through RealityViewCameraContent.
For iOS architecture or migration notes, gate AR with
ARWorldTrackingConfiguration.isSupported, host content with
RealityViewCameraContent, and build scenes from Entity/ModelEntity placed
with AnchorEntity.
Handoffs: CollisionComponent + InputTargetComponent handle RealityKit
interaction; AccessibilityComponent handles entity accessibility metadata;
detailed SwiftUI gestures and VoiceOver/Switch Control policy belong to siblings.
Treat existing SCNView/SCNNode work as either a separate SceneKit path or an
explicit migration to RealityKit, not a mixed scene graph.
Common Mistakes
DON'T: Skip AR capability checks
Use the configuration-specific support check from Setup before presenting AR. Show non-AR content or an explicit unavailable state when it fails.
DON'T: Load heavy models synchronously
Loading large USDZ files on the main thread causes frame drops and hangs.
The make closure of RealityView is async -- use it.
// WRONG -- synchronous load blocks the main thread
RealityView { content in
let model = try! Entity.load(named: "large-scene")
content.add(model)
}
// CORRECT -- async load
RealityView { content in
if let model = try? await ModelEntity(named: "large-scene") {
content.add(model)
}
}
DON'T: Forget collision and input target components for interactive entities
Interactive entities need both components shown in Gestures and Interaction; without them, taps and drags pass through.
DON'T: Create new entities in the update closure
The update closure runs on every SwiftUI state change. Creating entities
there duplicates content on each render pass.
// WRONG -- duplicates entities on every state change
RealityView { content in
// empty
} update: { content in
let sphere = ModelEntity(mesh: .generateSphere(radius: 0.05))
content.add(sphere) // Added again on every update
}
// CORRECT -- create in make, modify in update
RealityView { content in
let sphere = ModelEntity(mesh: .generateSphere(radius: 0.05))
sphere.name = "mySphere"
content.add(sphere)
} update: { content in
if let sphere = content.entities.first(
where: { $0.name == "mySphere" }
) as? ModelEntity {
// Modify existing entity
sphere.position.y = newYPosition
}
}
DON'T: Ignore camera permission
RealityKit on iOS needs camera access. If the user denies permission, the view shows a black screen with no explanation.
// WRONG -- no permission handling
RealityView { content in
// Black screen if camera denied
}
// CORRECT -- check and request permission
struct ARContainerView: View {
@State private var cameraAuthorized = false
var body: some View {
Group {
if cameraAuthorized {
RealityView { content in
// AR content
}
} else {
ContentUnavailableView(
"Camera Access Required",
systemImage: "camera.fill",
description: Text("Enable camera in Settings to use AR.")
)
}
}
.task {
let status = AVCaptureDevice.authorizationStatus(for: .video)
if status == .authorized {
cameraAuthorized = true
} else if status == .notDetermined {
cameraAuthorized = await AVCaptureDevice
.requestAccess(for: .video)
}
}
}
}
Review Checklist
-
NSCameraUsageDescriptionset in Info.plist - AR device capability checked before presenting AR views
- Camera permission requested and denial handled with a fallback UI
-
arkitrequired-device capability added when AR is the app's core purpose - 3D models loaded asynchronously in the
makeclosure - Entities created in
make, modified inupdate(not created inupdate) - Interaction prerequisites and sibling handoffs follow the Gestures section
- Entity hit tests, camera rays, and ARKit surface raycasts follow the Raycasting distinctions
-
SceneEvents.Updatesubscriptions used for per-frame logic (not SwiftUI timers) - Large scenes use
ModelEntity(named:)async loading, notEntity.load(named:) - Anchor entities target appropriate surface types for the use case
- Entity names set for lookup in the
updateclosure
References
- Read references/realitykit-patterns.md for physics, animations, lighting, ECS, accessibility, and performance patterns.
- RealityKit framework
- RealityView
- RealityViewCameraContent
- RealityViewCamera
- Entity
- ModelEntity
- AnchorEntity
- ARKit framework
- ARKit in iOS
- Verifying Device Support and User Permission
- ARWorldTrackingConfiguration
- ARRaycastQuery
- ARSession.raycast(_:)
- Loading entities from a file
FAQ
What does the realitykit skill do?
Build iOS augmented reality and 3D experiences with RealityKit and ARKit. Use when adding RealityView content, loading entities or USDZ models, anchoring objects to planes or world positions, distinguishing entity hit tests from ARKit real-world raycasts, handling AR camera availability, world tracking, scene updates, or RealityKit entity gestures and interactions.
How do I install realitykit?
Run npx skills add https://github.com/dpearson2699/swift-ios-skills --skill realitykit in your project. The skills CLI detects the agents you use and copies the skill into the right folder. For Claude Code you can also place the skill folder at ~/.claude/skills/realitykit/ (personal) or .claude/skills/realitykit/ (project).
Which AI agents can use realitykit?
Any agent that supports the open Agent Skills format (a folder with a SKILL.md file), including Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, Windsurf, Cline, OpenCode and more.
Is realitykit safe to install?
No partner security audit is available yet. Skills can run commands and scripts, so read the SKILL.md and any bundled files before installing.
Who maintains realitykit?
It is published by dpearson2699 in dpearson2699/swift-ios-skills. It has 3,227 recorded installs.
More from dpearson2699/swift-ios-skills
swiftui-animation
dpearson2699/swift-ios-skills
Implement, diagnose, or review SwiftUI motion using explicit and scoped implicit animations, springs, transitions, PhaseAnimator, KeyframeAnimator, matched geometry or navigation zoom, SF Symbol effects, and custom Animation types. Use when views should animate on state changes, insertion, removal, navigation, or multi-step choreography, or when motion must respect Reduce Motion and Swift concurrency.
ios-accessibility
dpearson2699/swift-ios-skills
Build and audit SwiftUI, UIKit, and AppKit accessibility for VoiceOver, Voice Control, Switch Control, Full Keyboard Access, Dynamic Type, focus restoration, labels/traits/actions, traversal, custom rotors, NSAccessibility, XCTest checks, adaptive system preferences, and App Store accessibility declarations. Use when implementing accessible UI, fixing an accessibility audit, testing assistive-technology behavior, or substantiating App Store Accessibility Nutrition Labels.
swiftui-performance
dpearson2699/swift-ios-skills
Profile, diagnose, and remediate SwiftUI runtime performance using code review, Instruments, and repeatable measurements. Use when a SwiftUI screen renders slowly, scrolling or animations hitch, view bodies update excessively, list identity churns, layout work spikes, or broad Observation dependencies raise CPU cost. Covers evidence-based triage, SwiftUI Instruments lanes, lazy-container guardrails, state lifetime, and before/after verification.
swiftui-patterns
dpearson2699/swift-ios-skills
Builds and reviews SwiftUI views with modern MV architecture, state, composition, isolated previews, and migration guidance. Covers @Observable ownership, @State/@Bindable/@Environment wiring, view decomposition, ViewModifiers, environment values, .task loading, iOS 26+ handoffs, Writing Tools, clipboard availability, and performance. Use when structuring SwiftUI state, managing @Observable, composing views, previewing meaningful UI states, or correcting SwiftUI patterns.
app-store-review
dpearson2699/swift-ios-skills
Audits App Store submission readiness and rejection risk across current review guidelines, PrivacyInfo.xcprivacy and required-reason APIs, privacy labels, ATT, StoreKit payments, metadata, entitlements, widgets, and Live Activities. Use when preparing a submission, responding to rejection, reconciling privacy evidence, or separating upload blockers from cleanup.
ios-networking
dpearson2699/swift-ios-skills
Build, review, or improve networking code in iOS/macOS apps using URLSession with async/await, structured concurrency, and modern Swift patterns. Use when working with REST APIs, downloading files, uploading data, WebSocket connections, pagination, retry logic, request middleware, caching, background transfers, or network reachability monitoring. Also use when handling HTTP requests, API clients, network error handling, or data fetching in Swift apps.
Popular mobile skills
design-mobile-apps
designed-by-ai/skills
Design mobile app screens with Sleek, edit Sleek projects, and implement their designs in React Native or HTML.
ios-design
uizze.sh
Design and review iOS UI/UX for iPhone apps in SwiftUI, UIKit, or React Native. Use for mobile app design, screen redesigns, navigation flows, typography, forms, keyboard and safe-area layout, and accessibility checks. Apply Apple Human Interface Guidelines and consult real product references through the optional Uizze connection.
vercel-react-native-skills
vercel-labs/agent-skills
React Native and Expo best practices for building performant mobile apps. Use when building React Native components, optimizing list performance, implementing animations, or working with native modules. Triggers on tasks involving React Native, Expo, mobile performance, or native platform APIs.
animate-expo
emilkowalski/skills
Build animations in React Native and Expo, making the decisions in the order that determines whether they feel right — should it animate, which thread it runs on, which properties, spring or timing, how the gesture hands off, how it degrades. Writes the implementation with Reanimated, Gesture Handler, Expo Router and expo-haptics. Use when animating anything in an Expo app, adding gestures, sheets, screen transitions, press feedback or haptics, or fixing motion that stutters on device. For web animation use animate.
xcode-project-setup
firebase/agent-skills
Safely modifies Xcode projects (.pbxproj) to add Swift Packages and link files. Use this skill whenever an iOS project needs dependencies installed (e.g. Firebase, Alamofire).
firebase-remote-config-basics
firebase/agent-skills
Manages Firebase Remote Config templates, feature flags, loading strategies, and SDKs (Android, iOS). Use when downloading/deploying remoteconfig JSON templates, managing version history/feature flags, setting in-app defaults, fetchAndActivate(), real-time listeners, or SDK setup. Don't use for Firebase Hosting, Auth, Firestore, Data Connect, Crashlytics, or App Hosting.