QA Testing (iOS)

High-signal iOS test execution and flake control for XCTest, XCUITest, Swift Testing, xcodebuild, xcresult, and simctl.

Pair this skill with software-ios-native when the work is part of a native iOS rewrite or a Codex / Claude Code implementation loop.

Core docs:

• https://developer.apple.com/documentation/xctest
• https://developer.apple.com/documentation/testing
• https://developer.apple.com/documentation/xcode/testing-your-apps-in-xcode
• https://developer.apple.com/documentation/xcode/simctl
• https://developer.apple.com/documentation/xcode-release-notes/xcode-27-release-notes

Quick Reference

Release Validation Boundary

• Do not treat simulator green or archive green as full iOS release proof when the feature depends on distribution signing, real hardware, APNs, camera, biometrics, purchases, or background execution.
• For those cases, require the release path that matches production behavior: signed archive, beta/distribution channel validation where relevant, and a real-device user-visible outcome.
• Keep project-specific rollout procedures, backend endpoints, tester-account cleanup, and internal runbooks out of this portable core. Put them in project docs or scoped references.
• Use references/ios-ci-general.md for release-CI and archive-path checks, and qa-testing-mobile/references/release-and-rollout.md for distribution-channel planning.

Defaults

• New unit and integration tests: prefer Swift Testing unless the project is already standardized on XCTest.
• UI and performance tests: keep using XCTest/XCUITest.
• PR gate: thin simulator smoke coverage with xcresult artifacts always enabled.
• Release confidence: add a real-device pass only where hardware behavior matters.
• Flake posture: prove the flake first, then fix isolation, waits, or environment drift; retries are a debugging aid, not a success criterion.
• Locale, region, timezone, permissions, and app state must be explicit in automation.
• Cross-framework assertions can lose failures depending on interop mode. Never assume test discovery implies assertion compatibility; use the version-specific modes and CI checks in references/swift-testing.md.
• After adopting the iOS 26 SDK, compare existing snapshot baselines against Liquid Glass rendering and regenerate affected baselines. Review each diff before accepting.

Inputs to Gather

• Xcode entrypoint: -workspace or -project
• -scheme and optional -testPlan
• Destination strategy: simulator, real device, or both
• Required hooks: launch arguments, launch environment, test data, auth bypass, animation toggles
• Artifact needs: xcresult, coverage, screenshots, logs, diagnostics
• CI environment: local, GitHub Actions, Xcode Cloud, or self-hosted macOS
• Whether the task first needs fresh uninstall/install/launch proof before trusting UI observations
• Whether the task includes distribution-channel or hardware-specific behavior that requires a real-device validation pass

Quick Start

Workflow

• Resolve the build inputs first: workspace or project, scheme, test plan, destination, and required launch hooks.
• Make the environment repeatable: simulator boot, permissions, locale, region, and app state reset.
• If the task depends on whether the current binary is really on screen, do a fresh uninstall/install/launch smoke pass before interpreting screenshots or UI-test failures.
• If a simulator screenshot path is missing or expired, treat it as a tooling artifact, not as no evidence. Re-capture from the current simulator or use the reported visible symptom plus source inspection to choose the next focused check.
• If the task includes push, purchases, deep links, or other distribution-channel behavior, split transport proof from user-visible outcome proof and use a real-device pass when required.
• Keep compile, test, archive, install/launch, TestFlight, and user-journey evidence distinct. Record source revision, scheme/configuration, archive or app identity, destination/OS, exact test plan or selector, and result-bundle path; stop the release claim at the last observed stage.
• Run with artifacts enabled: -resultBundlePath, and add coverage or diagnostics only when they serve the task.
• Triage from xcresult first, then reproduce a single failing test with -only-testing.
• If CoreSimulatorService returns “Connection refused”, stop retrying the same destination; use generic/platform=iOS for compile-only proof and route service recovery to software-ios-runtime-debugging.
• For backend-coupled flows, exercise the account recreation, cache isolation and web/API parity checklist in references/e2e-harness-and-selectors.md.
• Treat rerun-pass as a flake that needs ownership and a root-cause fix.

Runtime Proof Boundary

• Use this skill for test execution, xcresult, destinations, and flake control after the app is buildable and installable.
• If the core problem is stale installs, simulator drift, malformed .app bundles, missing executables, or install/launch failures, route to software-ios-runtime-debugging.
• If the app cannot be installed or launched reliably, that is a runtime-debugging problem first and a test problem second.
• A simulator or explicitly unsigned build does not exercise device signing. A development-signed device build can prove development signing for that destination, but does not prove distribution signing, export, TestFlight processing, or release-artifact installability. A green archive alone does not prove export, installation, or launch, and a simulator pass does not prove APNs, StoreKit, entitlements, background modes, camera, biometrics, or other device/channel behavior.

xcodebuild Patterns

# Enumerate before an expensive run
xcodebuild test \
  -scheme MyApp \
  -testPlan Smoke \
  -destination 'platform=iOS Simulator,name=<simulator-name>,OS=latest' \
  -enumerate-tests \
  -test-enumeration-format json

# Target one test
xcodebuild test \
  -scheme MyApp \
  -destination 'platform=iOS Simulator,name=<simulator-name>,OS=latest' \
  -only-testing:MyAppUITests/LoginFlowTests/testHappyPath \
  -resultBundlePath TestResults.xcresult

# Parallelize only when the suite is isolation-safe
xcodebuild test \
  -scheme MyApp \
  -destination 'platform=iOS Simulator,name=<simulator-name>,OS=latest' \
  -parallel-testing-enabled YES \
  -maximum-parallel-testing-workers 4 \
  -resultBundlePath TestResults.xcresult

# Controlled retry for CI triage
xcodebuild test \
  -scheme MyApp \
  -destination 'platform=iOS Simulator,name=<simulator-name>,OS=latest' \
  -retry-tests-on-failure \
  -test-iterations 2 \
  -test-repetition-relaunch-enabled YES \
  -collect-test-diagnostics on-failure \
  -resultBundlePath TestResults.xcresult

# Prove a flake locally
xcodebuild test \
  -scheme MyApp \
  -destination 'platform=iOS Simulator,name=<simulator-name>,OS=latest' \
  -only-testing:MyAppUITests/LoginFlowTests/testHappyPath \
  -run-tests-until-failure \
  -test-iterations 25

Flake Control

• Prefer waitForExistence, expectations, and state-based assertions over sleeps.
• Disable or reduce animations in UI-test runs where the app allows it.
• Stub or redirect third-party boundaries; do not depend on live external services in UI tests.
• Reset permissions and app state between tests.
• Pin -testLanguage and -testRegion when locale affects assertions.
• Use test plans for matrix-style coverage across device classes, locales, and environments.
• Keep UI suites thin. Put most business logic coverage in lower layers.
• For multi-locale apps, use qa-testing-mobile/references/localization-testing.md for layered coverage.

Localization and Visual Regression Proof

• Missing-key crashes in string-catalog or generated-accessor lookups are testable defects, not acceptable runtime assertions. Add or run catalog coverage before returning to UI polish.
• Verify both key presence and translated value quality. A locale file containing the English fallback is still a failed localization gate for user-visible copy.
• When new UI copy is added for a feature, run a focused key/value check for the new keys across every shipped locale, then run the broader static-key coverage suite.
• Pair locale coverage with narrow-width visual smoke for high-risk locales such as German, Russian, Japanese, and Arabic. Container width, wrapping, and overlay occlusion are part of the localization test, not a separate design nicety.
• For dense data-visualization screens, include a targeted smoke pass that proves controls, help/info cards, and detail affordances do not cover the primary diagram and remain usable after zoom/filter changes.

Deterministic E2E Harness

For auth, onboarding, billing, and other backend-coupled flows, prefer a fixture-backed launch-environment harness over live credentials. Keep user variants explicit, keep the fixture branch in the same function as the real path, and treat .accessibilityIdentifier() strings as part of the test contract.

Load references/e2e-harness-and-selectors.md for:

• the canonical launch-environment fixture pattern
• harness rules and reset-hook discipline
• selector rules for identifiers vs labels
• the atomic-commit pattern for landing a new E2E suite
• multi-account and backend-parity edge cases

Test Plan Organization

Xcode test plans (.xctestplan) control which tests run, with what configuration, and in which environment. Use them to manage matrix-style test execution:

• Smoke plan: thin critical-path tests for PR gates. Fast, reliable, minimal device matrix.
• Full plan: complete unit + integration + UI suite for nightly or release-candidate runs.
• Locale plan: same UI tests with different -testLanguage / -testRegion overrides per configuration.
• Keep test plans in the project directory alongside the scheme. Reference via -testPlan PlanName in xcodebuild.
• Each configuration within a plan can override launch arguments, environment variables, and enabled tests independently.
• Prefer separate plans over complex multi-configuration single plans — easier to run, triage, and maintain.

AI-Agent Testing for iOS

AI-native test tools (Maestro MCP and others) complement XCUITest; keep deterministic PR-gate smoke in XCUITest. For tool positioning and the decision framework, load qa-testing-mobile/references/ai-native-testing.md. Vendor capability claims change quickly — verify them against the vendor's current docs before recommending one.

When NOT To Use

Resources

Scripts

Templates

Navigation

• ## Workflow, ## xcodebuild Patterns, and ## Flake Control for the baseline sequence
• ## Resources and ## Templates for deeper materials

Learnings Loop

When prior decisions or pitfalls are relevant, consult learnings.consolidated.md if present; use learnings.md only for needed history or as the available fallback. Otherwise skip both.

After applying it, if you encountered a pattern worth remembering, a mistake worth preventing, or a domain fact that surprised you, append one dated bullet to learnings.md via agents-skills-feedback-loop/scripts/append_learning.py. Do not modify SKILL.md itself.