Tuist Workflows - Swift for Automation

RFCs
1

Need/problem

Developers in the Swift ecosystem want to write their automation in Swift.

Motivation

Automation has always been possible within Xcode, even in Objective-C times. Automation can be materialized as an executable (or multiple) whose execution can be controlled through flags and arguments. Executables have existed for a long time.

However, an important piece was missing: the ability to encapsulate units of automation and reuse them across workflows. This is something that Swift Package Manager addressed, but while required, it was not sufficient for people to make the switch from Fastlane to a solution that uses the language that they are familiar with. In my opinion, three elements are necessary for that to happen:

  • An extensive foundation of packages and utilities upon which to build workflows. Swift is behind there compared to other ecosystems, but it’s slowly catching up with the community investing more in server-side Swift, and also organizations like ours investing in utilities like Command or FileSystem.
  • A directory of resources, packages, and examples where people can learn how to build their automation. It needs to be something oriented towards automation and not that much towards how to use Swift in the context of servers or building apps.
  • And last, a solution to ensure workflows run as instantly as they can to align with the expectations that developers have when they run Ruby or bash scripts. They launch instantly! This is where Tuist can bring a lot to the table by leveraging years of investment in a caching solution that works with external packages.

I propose that Tuist lead a change there that goes beyond the technical work in this document. I think it’s time for a change and to meet developers eager to use Tuist for automation.

Detailed design

I propose extending our current project directory structure model, to account for a new type of manifest file, workflows.

A workflow represents a unit of automation (i.e. a lane). They live under the Tuist/Workflows directory:

Tuist/
  Workflows/
    Build.swift
    Test.swift
MyProject.xcodeproj

Workflows can be invoked through tuist workflows run {name} where {name} is the dash-cased version of the workflow name. Workflows are editable with tuist edit and are included by default in the generated projects if developers are using Tuist Projects. Every workflow is represented by an Xcode scheme named Workflow - {name} so developers can run them directly from Xcode and debug them using tools like LLDB.

Developers can depend on third-party package dependencies. Those are declared in either a Package.swift or a Tuist/Package.swift file, and reference to from a Tusit/Workflows.swift file with the following structure:

import WorkflowDescription

let workflows = Workflows(dependencies: [
  "FileSystem",
  "AppStoreConnect"
])

By doing the above, we can bind workflows to the external dependencies they depend on, and not only that, thanks to our binary caching functionality, developers don’t need to compile their external dependencies, just their workflow files, which we expect to be compilable in a matter of seconds.

We’ll migrate our existing scripts to workflows for dogfooding purposes and work on putting resources (examples, blog posts, and packages) to nudge the community toward adopting workflows.

A note on the workflow DSL
This is a bit TDB at the moment. I’d like to create something tangible first so that I can get a better grasp of what a good interface would be.

Drawbacks

  • It’s an extra surface to maintain, which would stretch us. However, I don’t expect much maintenance once the feature is built.
  • Some organizations have started using Bash as a substitute for Fastlane. However, bash is not straightforward to write and doesn’t support re-using units of business logic like developers would be able to do through SPM.
  • Developers could use a Swift Package to model workflows (it’s just an executable). However, going through the compilation cycles, especially once the automation logic grows and the workflows depend on a handful of dependencies, can add significant time to every CI build.

Alternatives

  • Bash scripts that Tuist can show and invoke. However, developers can’t share or reuse units of automation business logic built by other developers.
  • Swift Package. As mentioned above, we think it makes more sense to have an Xcode-native graph that we can cache and compile right after opening it in Xcode without depending on some asynchronous activity within Xcode.

Adoption strategy

New Tuist projects

They’ll come with a default workflow:

Tuist/
  Workflows.swift
  Workflows/
    Example.swift
Tuist.swift
Project.swift

Existing Tuist projects

They can run tuist workflows setup, and we’ll create the workflow scaffold for them.

Xcode projects

They can also run tuist workflows setup, and we’ll create a Tuist.swift file at the project’s root.

How we teach this

We should leverage the following tools for education:

  • Documentation: We’ll include content documenting the feature and also get started guidelines for people who have heard about “Tuist Workflows” and want to give it a shot without getting distracted by other Tuist capabilities.
  • Blog posts: We can write blog posts and help some community projects transition to this new automation model. We can do the work for them and publish case studies at the end of the work.
1 Like
2

I’m building a small prototype on this PR for anyone interested in following it.

1 Like
3

Thanks for writing this up :clap:

I’m excited about what this exploration will bring.

One thing omitted from the proposal is how we will foster reusing workflows, similar to Fastlane plugins. Arguably, the huge number of Fastlane plugins and their ease of reuse is one of the biggest reasons why Fastlane is still so popular.

A Tuist Workflow can be reused as an external dependency and imported as a regular Swift library. The CLI and the workflow itself would be still fully defined in the Workflows directory.

However, I’d argue we also want to make it easy to integrate full CLIs, similar to actions such as match. CLIs can be kept out-of-scope as their distribution shouldn’t have to rely on Tuist at all. But I do think that at the very least we should put out a guidance for how to build the CLIs and what conventions should be followed (e.g., having a CLI and library product, so the code can be reused in workflows but some actions can also be run directly via the CLI – primarily those that would not be run on the CI).

  1. CLI:

We can cache vanilla packages as well. We could consider replacing Workflows.swift with Package.swift instead and then rely on conventions.

The configuration for workflows could be done in Tuist.swift.

4

That’s a good point. I wonder what the naming should be for each of those blocks:

  • Library: It represents the lowest-level primitive, and it embodies a reusable unit of business logic, not necessarily tied to Tuist Workflows (e.g. Tuist’s system library). Packages are the unit of distribution.
  • Workflow: It represents a unit of automation that can be shared across projects (i.e., like lanes in Fastlane) in packages.
  • Runnable workflow: A workflow that can be directly invoked from the command line.

What do you think? I wonder if all the workflows should be runnable from the CLI. I don’t see a reason why not. I wonder if we can leverage a Swift Macro to turn a workflow into an executable at compile-time (i.e. add the @main dynamically.

We can cache vanilla packages as well. We could consider replacing Workflows.swift with Package.swift instead and then rely on conventions.

This could work too. Would you reuse the root’s Package.swift to avoid having two sources of external packages, one for the project and one fo the workflows?

5

Hi :wave:

Thanks for putting this proposal together — I’m really excited about Tuist Workflows!

The idea of writing automation in Swift, having it run instantly, and doing all that without needing tools like Ruby is super appealing.

Here are a few thoughts I wanted to share.

Disclaimer: I’ll reference Fastlane quite a bit, but that doesn’t mean Tuist Workflows should follow the same path exactly. It’s just a well-tested, long-standing tool with a lot of lessons we can learn from.

Workflow structure

The proposed structure looks good at first glance.

In Fastlane, a Fastfile can contain multiple lanes (aka workflows). I’m curious whether Tuist will allow a similar approach — multiple workflows in one file — or if each workflow will live in its own file.

If workflows are file-independent, Swift macros could be an elegant solution. Similar to the @Test macro, they could help with auto-discovery. I explored something like this in swift-fastlane (POC):

// Build.swift

@Lane
struct Build { ... }

@Lane
func build(...) { ... }

Passing arguments

Fastlane lets you pass arguments to actions like this:

fastlane run scan parameter1:"value1" parameter2:"value2"

Will Tuist Workflows support something similar?

Developer experience

The proposed DX sounds great. I like that we can edit workflows using tuist edit.

You mentioned an Xcode scheme named Workflow - {name} will be added when using Tuist Projects. Does that mean we can run workflows directly when opening a project via tuist generate or tuist edit?

That’d be a big win — running and iterating on a workflow as you build it is super valuable.

Defining workflows

A few questions that might be worth clarifying in a follow-up RFC:
• Are workflows defined per file, or can we have multiple in one file?
• Will workflows support arguments?

Answering these will help shape the public API for defining workflows.

Built-in primitives & reuse

You mentioned plugins and reusability — totally agree, that’s important.

Fastlane’s ecosystem benefits a lot from the wide range of available plugins, but just as important are the built-in primitives for things like building, testing, and deploying.

Starting with only the bare minimum might leave people unsure how to begin. Clear documentation and some out-of-the-box actions could really help developers get started faster.

Thanks again! Looking forward to seeing this evolve.

6

Hi @mollyIV :wave:

I’m still going back and forth with what the right path would be here. Mostly for a couple of reasons:

  • Bash is not as hard to write as it used to be thanks to LLMs, and the portability of bash scripts is unbeatable.
  • Declaring the interface using comments in the script is an amazing experience, which eliminates the need for any kind of argument parser and validation logic.
  • And if you don’t like Bash, you can use Ruby, Python or Node, because the problem of installing the runtime and the dependencies is already solved by Mise, which we are huge advocates of.

As you noted, a huge piece of value of Fastlane is its plugins ecosystem, and leveraging RubyGems and Gems as the channel and distribution format. This is something you’d lose if you use bash, but also, once again, if you have LLMs that can write, let’s say Match’s core workflow with a single prompt, how important it is to reuse abstractions, if you can get the code written making you the owner of that code instead of navigating layers of abstraction? I don’t know… these are just questions that I ask myself.

I feel “we need a Fastlane in Swift” is an understandable appetite, but things are changing quite rapidly, and I start to believe that there’s a bit of sunk cost fallacy in this vision. Oh! And let’s not forget the magic of invoking an script and seeing it running right away with no compilation.

I don’t want to spoil the plan, but I’m not sure building an automation layer in Swift, or something that allows people to writhe their pipelines in Swift is the way to go. Forking ecosystems is expensive. We did that with project generation, and it’d be too costly for us to develop and maintain.

1 Like