Skipping overhead for --without-building selective testing phase

RFCs
1

Summary

Extend tuist test --build-only to produce a self-contained .xctestproducts bundle that embeds the selective testing graph and all metadata needed to run tests. The corresponding tuist test --without-building stage then consumes this bundle directly, skipping project generation, graph computation, and rehashing entirely. This removes the requirement for a source checkout in the test-execution step and eliminates the overhead that currently makes two-stage CI workflows incompatible/impractical with selective testing.

Motivation

Teams with large iOS codebases often split their CI into two stages for parallelism:

  1. Build stage: tuist test --build-only compiles everything and produces derived data + xctestrun files.
  2. Test stage(s): Multiple parallel runners execute tuist test --without-building against the build artifacts. e.g., one runner for unit tests, one for snapshot tests, one for UI tests.

This pattern is well-established and works without selective testing. However, enabling selective testing breaks it because the --without-building step currently:

  • Requires a full source checkout. The selective testing graph (target-to-hash mappings) is computed from source files. Without the source tree, hashes cannot be computed and selective testing results cannot be reported.
  • Regenerates the Xcode project. Even though no building occurs, tuist test runs project generation to resolve the graph — adding 30-60+ seconds of overhead per runner.
  • Recomputes target hashes. Each parallel runner independently hashes every target, duplicating work already done in the build stage.

The sharding feature (PR #9796) already solved this problem for sharded test runs by embedding selective-testing-graph.json into the .xctestproducts bundle. The execute phase reads the graph from the bundle and skips generation and rehashing entirely. This RFC proposes bringing the same optimization to the standard (non-sharded) two-stage flow.

Prior Art

Sharding (Tuist, PR #9796)

The sharding feature already implements the core mechanism this RFC builds on. During tuist test --build-only with shard flags, the CLI:

  1. Computes the SelectiveTestingGraph (target name → content hash mappings) from the mapper environment.
  2. Writes selective-testing-graph.json into the .xctestproducts bundle.
  3. Uploads the bundle to the server for distribution to shard runners.

Each shard runner then downloads the bundle, reads the embedded graph, runs its assigned tests, and reports selective testing results — all without a source checkout or project generation.

This RFC generalizes that mechanism to non-sharded two-stage workflows.

Xcode’s native two-stage flow

Xcode’s xcodebuild build-for-testing produces derived data and an .xctestrun plist. xcodebuild test-without-building consumes them. This is a pure Xcode mechanism with no awareness of Tuist’s graph, hashing, or selective testing. Teams using this flow today must layer Tuist’s logic on top, which is where the overhead comes from.

Xcode’s .xctestproducts

xcodebuild build-for-testing always produces a .xctestproducts bundle in derived data, containing all test binaries, frameworks, and the .xctestrun plist. When -testProductsPath is passed, the bundle is placed at a custom location instead. The bundle is self-contained and portable — it can be copied to another machine and executed with xcodebuild test-without-building -testProductsPath. The sharding PR already uses this as the transport format.

Proposed Solution

Overview

The change is minimal because the sharding PR already built the infrastructure. The proposal is to:

  1. Embed selective-testing-graph.json in the .xctestproducts bundle (already produced by Xcode in derived data) when selective testing is active during --build-only (currently only done in the shard plan flow).
  2. Accept a .xctestproducts path during --without-building and read the embedded graph instead of computing it from source.
  3. Skip project generation and rehashing when a .xctestproducts path is provided and contains an embedded graph.

CLI Changes

tuist test --build-only

When --build-only is used (with or without sharding):

  1. Generate project and compute target hashes as today.
  2. Invoke xcodebuild build-for-testing, which produces a .xctestproducts bundle. If the user passes -testProductsPath as a passthrough xcodebuild argument, respect that custom path; otherwise use the default location in derived data.
  3. If selective testing is active, write selective-testing-graph.json into the resolved .xctestproducts bundle.
  4. Print the .xctestproducts path to stdout for downstream consumption.

tuist test --without-building

When --without-building is used with -testProductsPath (passed through to xcodebuild):

  1. Read the .xctestproducts bundle from the provided path.
  2. Check for selective-testing-graph.json inside the bundle.
  3. If the graph is present: skip project generation and rehashing. Use the embedded graph for selective testing result reporting.
  4. Invoke xcodebuild test-without-building -testProductsPath <path>.
  5. After tests complete, upload results and store successful test hashes using the embedded graph.

If -testProductsPath is not passed, fall back to the current behavior (project generation + rehashing) for backward compatibility. No new Tuist-specific flags are introduced. -testProductsPath is Xcode’s native flag, passed through as a standard xcodebuild argument.

Workflow Examples

GitHub Actions

jobs:
  build:
    runs-on: macos-15
    steps:
      - uses: actions/checkout@v4
      - name: Build for testing
        run: tuist test --build-only -- -testProductsPath ./MyApp.xctestproducts
      - uses: actions/upload-artifact@v4
        with:
          name: test-products
          path: ./MyApp.xctestproducts

  test-unit:
    needs: build
    runs-on: macos-15
    steps:
      # No checkout needed!
      - uses: actions/download-artifact@v4
        with:
          name: test-products
      - name: Install Tuist
        run: curl -Ls https://install.tuist.io | bash
      - name: Run unit tests
        run: tuist test --without-building --scheme UnitTests -- -testProductsPath ./MyApp.xctestproducts

  test-snapshots:
    needs: build
    runs-on: macos-15
    steps:
      - uses: actions/download-artifact@v4
        with:
          name: test-products
      - name: Install Tuist
        run: curl -Ls https://install.tuist.io | bash
      - name: Run snapshot tests
        run: tuist test --without-building --scheme SnapshotTests -- -testProductsPath ./MyApp.xctestproducts

What Gets Skipped

Step --build-only --without-building (current) --without-building + -testProductsPath (proposed)
Source checkout Required Required Not required
Project generation Yes Yes Skipped
Target hashing Yes Yes Skipped (read from bundle)
Xcode build Yes No No
Test execution No Yes Yes
Result upload Hashes only Results + hashes Results + hashes

Bundle Contents

The .xctestproducts bundle produced by --build-only contains:

MyApp.xctestproducts/
├── MyApp.xctestrun              # Xcode's test run configuration
├── __xctestproducts_path__/     # Test binaries and frameworks (Xcode-managed)
└── selective-testing-graph.json # Tuist's target-to-hash mappings (new)

The selective-testing-graph.json file uses the existing SelectiveTestingGraph struct:

{
  "testTargetHashes": {
    "MyAppTests": "abc123def456",
    "MyLibraryTests": "789ghi012jkl",
    "SnapshotTests": "345mno678pqr"
  }
}

Alternatives Considered

Pass selective testing data via environment variables or CI artifacts

Instead of embedding the graph in .xctestproducts, pass it as a separate file or environment variable between CI jobs.

  • Adds coordination complexity — teams must manage an additional artifact alongside the test products.
  • Error-prone: the graph and the test products can get out of sync if jobs are re-run independently.
  • The .xctestproducts bundle is already the natural transport unit; embedding the graph keeps everything atomic.
1 Like
2

Very onboard. I don’t see why we wouldn’t want to loosen the coupling of this solution to sharding.