Indicate that internal targets prefer replacement with binary cache

Ideas
1

When I run tuist generate, all my internal targets are generated, and all my external targets are replaced by cached binaries (when available).

If I run tuist generate MyModule, I get internal target MyModule generated, and all internal targets it depends on replaced by cached binaries (when available).

I’d like to support something in-between. I have a few modules in my project which are expensive to build, but rarely change. I’d like for them to be treated the same external targets (without having to actually make them external targets). This would allow me to run tuist generate and get all my internal targets, but have these specific internal targets replaced by cached binaries (when available).

I was thinking we could add something to Target.target which lets me configure the targets for this different treatment. Something like:

Target.target(
    name: "MyModule",
    ...
    metadata: .metadata(treatment: .preferBinaryCache)
)

Alternatively we could add more options to the tuist generate command, but that would require more environment configuration for my team so I would prefer something like above.

1 Like
2

I was thinking we can have something like caching profiles. There are two profiles at the moment, only-dependencies (default in generate) and as-many-as-possible (implicitly used by tuist build and test commands). So we can make those explicit via a flag, and expose an API to configure those:

tuist generate —cache-profile only-external
tuist generate —cache-profile all-possible
tuist generate —cache-profile custom

Would that work? The grouping of targets under a profile can be done using target names and tags.

3

I think that would work. Could we also configure a default cache profile in Config’s generationOptions?

4

I’m trying something new here.

I wrote the following spec with a lot of help from Cursor Max in the Tuist codebase. I tried work through as many edge cases as I could think of to really flesh it out.

Posting here for discussion if anyone wants to weigh in:

Cache Profiles Spec

Overview

Cache profiles provide a flexible system for controlling binary replacement behavior in Tuist generation. This feature allows users to define named strategies for replacing targets with cached binaries that can be configured in Config manifests and selected via CLI flags.

Motivation

Currently, Tuist treats internal and external targets differently for binary replacement:

  • External targets are replaced by cached binaries by default
  • Internal targets are always built from source, unless one or more targets are specified at generate time

However, some internal targets share characteristics with external dependencies: they are expensive to build and rarely change. Users want these targets treated like external dependencies for binary replacement, without actually making them external.

Cache profiles solve this by providing:

  1. Selective internal replacement - Choose which internal targets get replaced by cached binaries
  2. Multiple replacement strategies - Different profiles for development, CI, debugging
  3. Team-wide configuration - Shared replacement rules via Config manifests
  4. Flexible overrides - CLI flags to change replacement behavior per-session

Design Overview

The cache profiles system introduces:

  • Built-in profiles for common scenarios
  • Custom profiles with target and tag-based rules
  • Priority-based resolution from CLI flags, config defaults, and system defaults

Core Types

CacheProfileType

extension Tuist {
    public enum CacheProfileType: Codable, Equatable, Sendable {
        /// Replace external dependencies only (system default)
        case onlyExternal

        /// Replace as many targets as possible with cached binaries
        case allPossible

        /// No binary replacement, build everything from source
        case none

        /// Use named custom profile from `cacheProfiles` dictionary
        case custom(String)
    }
}

CacheProfile

extension Tuist {
    public struct CacheProfile: Codable, Equatable, Sendable {
        /// Target names to replace with cached binaries
        public let targets: [String]

        /// Tags for targets to replace with cached binaries
        public let tags: [String]

        /// Creates a cache profile specifying targets to replace with cached binaries
        /// - Parameters:
        ///   - targets: Target names to replace with cached binaries
        ///   - tags: Tags for targets to replace with cached binaries
        /// - Returns: A configured cache profile
        public static func profile(
            targets: [String] = [],
            tags: [String] = []
        ) -> Self {
            CacheProfile(
                targets: targets,
                tags: tags
            )
        }
    }
}

GenerationOptions Extension

extension Tuist.GenerationOptions {
    /// Default cache profile to use when none is specified via CLI
    public var defaultCacheProfile: CacheProfileType

    /// Named cache profiles specifying targets to replace with cached binaries
    public var cacheProfiles: [String: CacheProfile]

    /// Creates generation options with cache profile configuration
    /// - Parameters:
    ///   - defaultCacheProfile: Default cache profile to use when none is specified via CLI
    ///   - cacheProfiles: Named cache profiles specifying targets to replace with cached binaries
    /// - Returns: Configured generation options
    public static func options(
        // ... existing parameters
        defaultCacheProfile: CacheProfileType = .onlyExternal,
        cacheProfiles: [String: CacheProfile] = [:]
    ) -> Self {
        // ... implementation with new parameters
    }
}

Built-in Profiles

onlyExternal (System Default)

  • Behavior: Replace external dependencies only
  • Use case: Conservative replacement, current default behavior
  • Internal targets: Built from source

allPossible

  • Behavior: Replace as many targets as possible with cached binaries, including internal targets
  • Use case: Fast iteration when focusing on specific targets
  • Internal targets: Replaced by cached binaries when possible

none

  • Behavior: No binary replacement, build everything from source
  • Use case: Debugging build issues, clean builds
  • Internal targets: Always built from source

Configuration

Basic Configuration

import ProjectDescription

let config = Config(project: .tuist(generationOptions: .options(
    // Use built-in profile as default
    defaultCacheProfile: .allPossible
)))

Custom Profiles Configuration

import ProjectDescription

let config = Config(project: .tuist(generationOptions: .options(
    // Set custom profile as default
    defaultCacheProfile: .custom("development"),

    // Define custom profiles
    cacheProfiles: [
        "development": .profile(
            targets: ["ExpensiveFramework", "SlowToCompileModule"],
            tags: ["expensive", "rarely-changed"]
        ),
        "ci": .profile(
            tags: ["all"]
        ),
        "debug": .profile(
            targets: ["UtilityFramework"],
            tags: ["stable"]
        )
    ]
)))

CLI Interface

New Flag

@Option(
    name: .long,
    help: "Cache profile to use: 'only-external', 'all-possible', 'none', or custom profile name"
)
var cacheProfile: String?

Usage Examples

# Built-in profiles
tuist generate --cache-profile only-external
tuist generate --cache-profile all-possible
tuist generate --cache-profile none

# Custom profiles (defined in Config)
tuist generate --cache-profile development
tuist generate --cache-profile ci
tuist generate --cache-profile debug

# Use config default (no flag)
tuist generate

# Focus on specific targets (implies --cache-profile all-possible)
tuist generate MyModule AnotherTarget

# Disable binary replacement entirely (backwards compatible)
tuist generate --no-binary-cache  # equivalent to --cache-profile none

Priority Resolution System

The cache profile is resolved using the following priority order (highest to lowest):

1. --no-binary-cache Flag → none

tuist generate --no-binary-cache
# → Equivalent to --cache-profile none

2. Target Focus → allPossible

tuist generate MyModule
# → Uses allPossible (ignores any --cache-profile)

3. --cache-profile Flag → Explicit Profile

tuist generate --cache-profile development
# → Uses "development" profile from Config

4. Config Default → defaultCacheProfile

tuist generate
# → Uses config.generationOptions.defaultCacheProfile

5. System Default → onlyExternal

tuist generate
# → Falls back to .onlyExternal if no config default

Resolution Logic

enum CacheProfileError: Error, LocalizedError {
    case profileNotFound(String)

    var errorDescription: String? {
        switch self {
        case .profileNotFound(let profile):
            return "Cache profile '\(profile)' not found. Available profiles: only-external, all-possible, none, or custom profiles defined in cacheProfiles."
        }
    }
}

func resolveFinalCacheProfile(
    binaryCache: Bool,                    // --no-binary-cache flag
    explicitCacheProfile: String?,        // --cache-profile flag
    includedTargets: Set<TargetQuery>,    // Target arguments
    config: Tuist                         // Loaded configuration
) throws -> CacheProfileType {

    // 1. --no-binary-cache takes highest precedence
    if !binaryCache {
        return .none
    }

    // 2. If targets specified, use allPossible (overrides any profile)
    if !includedTargets.isEmpty {
        return .allPossible
    }

    // 3. --cache-profile comes next (only when no targets specified)
    if let explicitProfile = explicitCacheProfile {
        return try parseCacheProfileString(explicitProfile, config: config)
    }

    // 4. Config default
    if let configDefault = config.project.generatedProject?.generationOptions.defaultCacheProfile {
        return configDefault
    }

    // 5. System default
    return .onlyExternal
}

func parseCacheProfileString(_ profile: String, config: Tuist) throws -> CacheProfileType {
    switch profile {
    case "only-external": return .onlyExternal
    case "all-possible": return .allPossible
    case "none": return .none
    default:
        // Validate custom profile exists
        guard config.project.generatedProject?.generationOptions.cacheProfiles[profile] != nil else {
            throw CacheProfileError.profileNotFound(profile)
        }
        return .custom(profile)
    }
}

Implementation Details

Cache Profile Resolution

When a CacheProfileType.custom(name) is resolved:

  1. Lookup: Find name in config.cacheProfiles dictionary
  2. Validation: If profile doesn’t exist, throw CacheProfileError.profileNotFound
  3. Application: Apply profile’s target and tag rules to graph mapping

Graph Mapping Integration

The cache profile affects the GraphMapperFactory pipeline:

// In GraphMapperFactory
func generation(
    config: Tuist,
    cacheProfile: CacheProfileType,
    includedTargets: Set<TargetQuery>,
    configuration: String?,
    cacheStorage: CacheStoring
) -> [GraphMapping] {

    var mappers: [GraphMapping] = []

    // ... existing mappers

    // Apply cache profile logic
    switch cacheProfile {
    case .none:
        // Skip all binary replacement mappers
        break

    case .onlyExternal:
        // Use conservative binary replacement (current behavior)
        mappers.append(ExternalDependenciesCacheMapper(...))

    case .allPossible:
        // Use binary replacement for as many targets as possible
        mappers.append(AllPossibleCacheMapper(...))

    case .custom(let profileName):
        // Profile existence already validated during resolution
        if let profile = config.cacheProfiles[profileName] {
            mappers.append(CustomProfileCacheMapper(profile: profile, ...))
        } else {
            // This should never happen due to validation, but handle gracefully
            fatalError("Cache profile '\(profileName)' should have been validated during resolution")
        }
    }

    return mappers
}

Custom Profile Target Resolution

For CacheProfile target resolution:

  1. Target Names: Targets listed in targets array are replaced with cached binaries
  2. Tags: Targets with tags listed in tags array are replaced with cached binaries
  3. Fallback: Targets not explicitly listed fall back to onlyExternal

Backwards Compatibility

Existing CLI Flags

  • :white_check_mark: --no-binary-cache continues to work (maps to --cache-profile none)
  • :white_check_mark: --binary-cache continues to work (uses configured profile)
  • :white_check_mark: No flags continues to work (uses system default)

Existing Behavior

  • :white_check_mark: Default generation caches external dependencies only
  • :white_check_mark: Target focus replaces non-focused targets with cached binaries when possible
  • :white_check_mark: All existing workflows remain unchanged

Migration Path

  • :white_check_mark: No breaking changes to existing configurations
  • :white_check_mark: New features are opt-in via new configuration properties

Error Handling

Invalid Profile Names

tuist generate --cache-profile non-existent
# Error: Cache profile 'non-existent' not found. Available profiles: only-external, all-possible, none, or custom profiles defined in cacheProfiles.
# Exit code: 1

Invalid Default Profile in Config

Config(project: .tuist(generationOptions: .options(
    defaultCacheProfile: .custom("non-existent"),
    cacheProfiles: [
        "development": .profile(targets: ["MyModule"])
        // "non-existent" profile missing!
    ]
)))

// Error during config validation (at load time):
// Error: Cache profile 'non-existent' not found. Available profiles: only-external, all-possible, none, or custom profiles defined in cacheProfiles.

Empty Custom Profiles

cacheProfiles: [
    "empty": .profile()  // No targets or tags specified
]
// Behaves like 'only-external' (no internal targets replaced by cached binaries)

Conflicting Flags

# Example 1: Target focus overrides custom profile
tuist generate MyModule --cache-profile development
# Priority: Target focus (2) > --cache-profile (3)
# Result: CacheProfileType.allPossible (focus on MyModule, cache everything else)

# Example 2: --no-binary-cache overrides everything
tuist generate MyModule --cache-profile development --no-binary-cache
# Priority: --no-binary-cache (1) > Target focus (2) > --cache-profile (3)
# Result: CacheProfileType.none (no binary replacement)

# Example 3: Redundant but consistent flags
tuist generate MyModule --cache-profile none --no-binary-cache
# Priority: --no-binary-cache (1) > Target focus (2) > --cache-profile (3)
# Result: CacheProfileType.none (both flags want the same behavior)

Testing Strategy

Unit Tests

  • Cache profile resolution priority
  • CLI flag parsing and validation
  • Custom profile target matching
  • Error handling for missing profiles
  • Config validation during loading
  • Built-in profile validation

Integration Tests

  • End-to-end generation with different profiles
  • Graph mapper pipeline with cache profiles
  • Backwards compatibility scenarios
  • Error cases with descriptive messages
  • Config validation during project loading

Fixtures

  • Projects with various cache profile configurations
  • Custom profiles with different target/tag combinations
5

It’s a lot, I know, I did actually go through it line-by-line and remove much of the Ai cruft. If there aren’t any concerns with it I can implement and open a PR.

6

Thanks a lot @hiltonc for putting this together. Few thoughts:

Configuration API

What about the following changes to your API design:

  • Hoist cacheOptions to the root.
  • Merge default and profiles under a single attribute
let tuist = Tuist(
  cacheOptions: .options(
    profiles: .profiles([...], default: .onlyExternal)
  )
)

Profile API

What if instead of having .profile(targets: [], tags: []) we have an array of strings using the tag: prefix convention for tags: `[“TargetA”, “TargetB”, “tag:Foo”, “tag:Bar”]``

Deprecation of current defaults

There are some defaults that I wonder if we should consider moving away from long-term:

  • tuist build defaulting to all-possible.
  • Focused projects defaulting to all-possible.
  • --no-binary-cache defaulting to none

The last one is something that we can warn people about so that they get used to the new concept of profiles. However, the first two will be a bit tricky without a breaking change, which I’d be against since the release of Tuist 4 confirmed doing major changes is very undesirable.

Priorities

I’m ongoard with this. We’ll need to make sure we document it well to avoid confusions.

7

Also before we move to code, let’s wait to see what @core thinks about the proposal.

8

I like your profile API better.

Regarding deprecation of current defaults, are you suggesting we have tuist build and focused projects default to only-external? I’m not sure what you mean for --no-binary-cache, I don’t think anything other than none would make sense there.

9

I generally like the direction and the proposed API. Flexible caching like this would give some new tools for larger teams to help manage more layers of caching with more complex projects.

10

I was thinking that long-term it might make sense for any behaviour to be explicit because we often get asked about the implicit behaviours. However, changing the state of things today would imply breaking changes, and we can’t afford that. We can add a note somewhere flagging this a design debt that we can address in the future if we ever have the opportunity to release a major version.

1 Like