How to let `tuist test` generate test coverage data?

General
1

I am trying to use Slather to generate test coverage reports for Xcode projects and hook it into CI.

Experiment 1 - Using xcodebuild test (Succeed)

> xcodebuild test \
    -workspace mobile-ios.xcworkspace \
    -scheme mobile-ios \
    -destination "platform=iOS Simulator,name=iPhone 17,OS=26.0"

generates DerivedData folder with test coverage info inside.

Next Slather succeed as well.

> bundle exec slather
Slathering...
Slathered

Experiment 2 - Using tuist test (Failed)

Here is my Project.swift

import Foundation
import ProjectDescription

let swiftVersion = try! String(
  contentsOfFile: ".swift-version",
  encoding: .utf8
).trimmingCharacters(in: .whitespacesAndNewlines)

let project = Project(
  name: "mobile-ios",
  settings: .settings(
    base: [
      "SWIFT_VERSION": .string(swiftVersion),
    ]
  ),
  targets: [
    .target(
      name: "mobile-ios",
      destinations: .iOS,
      product: .app,
      bundleId: "dev.tuist.mobile-ios",
      infoPlist: .extendingDefault(
        with: [
          "UILaunchScreen": [
            "UIColorName": "",
            "UIImageName": "",
          ]
        ]
      ),
      buildableFolders: [
        "mobile-ios/Sources",
        "mobile-ios/Resources",
      ],
      dependencies: [
        .external(name: "WhisperKit"),
        .external(name: "MLXLLM"),
        .external(name: "MLXLMCommon"),
      ]
    ),
    .target(
      name: "mobile-iosTests",
      destinations: .iOS,
      product: .unitTests,
      bundleId: "dev.tuist.mobile-iosTests",
      infoPlist: .default,
      buildableFolders: [
        "mobile-ios/Tests"
      ],
      dependencies: [.target(name: "mobile-ios")]
    ),
  ],
  schemes: [
    .scheme(
      name: "mobile-ios",
      shared: true,
      buildAction: .buildAction(
        targets: [
          .target("mobile-ios")
        ]
      ),
      testAction: .targets(
        [
          .testableTarget(target: "mobile-iosTests")
        ],
        options: .options(
          coverage: true,
          codeCoverageTargets: [
            .target("mobile-ios")
          ]
        )
      )
    )
  ]
)

Currently,

> tuist test --device "iPhone 17"

generated DerivedData folder, however, inside it is missing many test coverage related info, for example ~/Library/Developer/Xcode/DerivedData/mobile-ios-xxx/Build/ProfileData folder.

This caused Slather failed

> bundle exec slather
Slathering...
No coverage directory found.

        Are you sure your project is generating coverage? Make sure you enable code coverage in the Test section of your Xcode scheme.
        Did you specify your Xcode scheme? (--scheme or 'scheme' in .slather.yml)
        If you're using a workspace, did you specify it? (--workspace or 'workspace' in .slather.yml)
        If you use a different Xcode configuration, did you specify it? (--configuration or 'configuration' in .slather.yml)

bundler: failed to load command: slather (/Users/hongbo-miao/.rbenv/versions/3.4.7/bin/slather)
/Users/hongbo-miao/.rbenv/versions/3.4.7/lib/ruby/gems/3.4.0/gems/slather-2.8.5/lib/slather/project.rb:244:in 'Slather::Project#profdata_coverage_dir': No coverage directory found. (StandardError)
        from /Users/hongbo-miao/.rbenv/versions/3.4.7/lib/ruby/gems/3.4.0/gems/slather-2.8.5/lib/slather/project.rb:542:in 'Slather::Project#find_binary_files'
        from /Users/hongbo-miao/.rbenv/versions/3.4.7/lib/ruby/gems/3.4.0/gems/slather-2.8.5/lib/slather/project.rb:472:in 'Slather::Project#configure_binary_file'
        from /Users/hongbo-miao/.rbenv/versions/3.4.7/lib/ruby/gems/3.4.0/gems/slather-2.8.5/lib/slather/project.rb:347:in 'Slather::Project#configure'
        from /Users/hongbo-miao/.rbenv/versions/3.4.7/lib/ruby/gems/3.4.0/gems/slather-2.8.5/lib/slather/command/coverage_command.rb:62:in 'CoverageCommand#execute'
        from /Users/hongbo-miao/.rbenv/versions/3.4.7/lib/ruby/gems/3.4.0/gems/clamp-1.3.3/lib/clamp/command.rb:66:in 'Clamp::Command#run'
        from /Users/hongbo-miao/.rbenv/versions/3.4.7/lib/ruby/gems/3.4.0/gems/clamp-1.3.3/lib/clamp/subcommand/execution.rb:18:in 'Clamp::Subcommand::Execution#execute'
        from /Users/hongbo-miao/.rbenv/versions/3.4.7/lib/ruby/gems/3.4.0/gems/clamp-1.3.3/lib/clamp/command.rb:66:in 'Clamp::Command#run'
        from /Users/hongbo-miao/.rbenv/versions/3.4.7/lib/ruby/gems/3.4.0/gems/clamp-1.3.3/lib/clamp/command.rb:140:in 'Clamp::Command.run'
        from /Users/hongbo-miao/.rbenv/versions/3.4.7/lib/ruby/gems/3.4.0/gems/slather-2.8.5/bin/slather:17:in '<top (required)>'
        from /Users/hongbo-miao/.rbenv/versions/3.4.7/bin/slather:25:in 'Kernel#load'
        from /Users/hongbo-miao/.rbenv/versions/3.4.7/bin/slather:25:in '<top (required)>'
        from /Users/hongbo-miao/.rbenv/versions/3.4.7/lib/ruby/gems/3.4.0/gems/bundler-2.5.11/lib/bundler/cli/exec.rb:58:in 'Kernel.load'
        from /Users/hongbo-miao/.rbenv/versions/3.4.7/lib/ruby/gems/3.4.0/gems/bundler-2.5.11/lib/bundler/cli/exec.rb:58:in 'Bundler::CLI::Exec#kernel_load'
        from /Users/hongbo-miao/.rbenv/versions/3.4.7/lib/ruby/gems/3.4.0/gems/bundler-2.5.11/lib/bundler/cli/exec.rb:23:in 'Bundler::CLI::Exec#run'
        from /Users/hongbo-miao/.rbenv/versions/3.4.7/lib/ruby/gems/3.4.0/gems/bundler-2.5.11/lib/bundler/cli.rb:455:in 'Bundler::CLI#exec'
        from /Users/hongbo-miao/.rbenv/versions/3.4.7/lib/ruby/gems/3.4.0/gems/bundler-2.5.11/lib/bundler/vendor/thor/lib/thor/command.rb:28:in 'Bundler::Thor::Command#run'
        from /Users/hongbo-miao/.rbenv/versions/3.4.7/lib/ruby/gems/3.4.0/gems/bundler-2.5.11/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in 'Bundler::Thor::Invocation#invoke_command'
        from /Users/hongbo-miao/.rbenv/versions/3.4.7/lib/ruby/gems/3.4.0/gems/bundler-2.5.11/lib/bundler/vendor/thor/lib/thor.rb:527:in 'Bundler::Thor.dispatch'
        from /Users/hongbo-miao/.rbenv/versions/3.4.7/lib/ruby/gems/3.4.0/gems/bundler-2.5.11/lib/bundler/cli.rb:35:in 'Bundler::CLI.dispatch'
        from /Users/hongbo-miao/.rbenv/versions/3.4.7/lib/ruby/gems/3.4.0/gems/bundler-2.5.11/lib/bundler/vendor/thor/lib/thor/base.rb:584:in 'Bundler::Thor::Base::ClassMethods#start'
        from /Users/hongbo-miao/.rbenv/versions/3.4.7/lib/ruby/gems/3.4.0/gems/bundler-2.5.11/lib/bundler/cli.rb:29:in 'Bundler::CLI.start'
        from /Users/hongbo-miao/.rbenv/versions/3.4.7/lib/ruby/gems/3.4.0/gems/bundler-2.5.11/exe/bundle:28:in 'block in <top (required)>'
        from /Users/hongbo-miao/.rbenv/versions/3.4.7/lib/ruby/gems/3.4.0/gems/bundler-2.5.11/lib/bundler/friendly_errors.rb:117:in 'Bundler.with_friendly_errors'
        from /Users/hongbo-miao/.rbenv/versions/3.4.7/lib/ruby/gems/3.4.0/gems/bundler-2.5.11/exe/bundle:20:in '<top (required)>'
        from /Users/hongbo-miao/.rbenv/versions/3.4.7/bin/bundle:25:in 'Kernel#load'
        from /Users/hongbo-miao/.rbenv/versions/3.4.7/bin/bundle:25:in '<main>'
error: Recipe `test-coverage` failed on line 28 with exit code 1

Here is the difference

image
image1072×1097 99.3 KB

How to let tuist test generate test coverage data properly? Thanks! :smiley:

2

Hey @hongbo-miao :waving_hand:

You can run tuist test with –-verbose to see the exact xcodebuild command that we’re running. I’d be surprised if you can’t get the test coverage report if you’re not using selective testing.

If you are, then the report will incomplete if tests are skipped. This is a known limitation of selective testing – but we believe the trade-off of missing code coverage is worth the improved test times.