Turning `tuist/tuist` into a monorepo

[pepicrft]

Need/problem

With our plans to make the server source code available, and also provide an SDK that developers can add to their apps to bring runtime capabilities, like runtime intelligence (like Tidewave), I think this presents us with an opportunity to start taking steps towards a monorepo setup under tuist/tuist.

Motivation

Working across repositories worsens the developer experience. In cases where dependencies like FileSystem or Noora are standalone, mature projects, it makes sense for them to be in their own repository for discoverability reasons. Still, everything else should be in a monorepo, in my opinion.

Detailed design

I propose that we make some changes to GitHub - tuist/tuist: A virtual platform team for mobile devs who ship directory structure to accommodate new projects in the repo:

• Move the CLI code under cli/. That includes Sources, Templates, Tests, fixtures, Project.swift, .xcode-version files, .swiftlint.yml and .swiftformat.yml.
• Rename .github/workflows/tuist.yml to cli.yml.
• Adjust the README.md to reflect the new structure.
• Namespace the CLI Mise tasks under cli (e.g. mise run cli:build).

Long-term I expect the structure to be something along the lines of:

/
  app/
  cli/
  docs/
  server/
  sdk/
  README.md
  LICENSE.md
  ...

Drawbacks

A monorepo comes with investment from our side in the tooling and pipelines to ensure we run the right tasks based on file changes. However, tooling is what we excel at, so we have the expertise to handle this kind of work, and we can iterate as we go.

Alternatives

The alternative is to stay in a multi-repo setup, but as I stated earlier, the developer experience is not ideal. Moreover, changes span across repositories, making it harder to reason about feature contributions.

Adoption strategy

With PRs opened, this scan will be a bit painful. We can focus on closing/merging all the existing PRs, but I don’t think that’s very realistic. We can at least try to close as many as we can, and for the remaining, there will have to be a rebase work that needs to happen. I’d say that we support there.

[marekfort]

Generally, definitely aligned in making tuist/tuist a monorepo, we’re technically there already.

@pepicrft

• Move the CLI code under cli/. That includes Sources, Templates, Tests, fixtures, Project.swift, .xcode-version files, .swiftlint.yml and .swiftformat.yml.

Where would you put code shared between different end products, like the CLI and the app sharing the TuistServer module? Would those modules stay in the root or would they be moved in the cli that is currently the “primary” consumer?

Additionally, configuration files like .swiftlint.yml are also shared across the app and the CLI – I’d consider keeping those in the root.

I would also add that there are some gotchas we ran into when making Noora into a monorepo:

• The repository must have a Package.swift in the root to ensure you can still consume tuist via SwiftPM
• You can’t have multiple Package.swift manifests in a single repository that you want to use for distribution
• You can’t have prefixed tags for code distributed via SwiftPM. Again, that means we’re limited to having a single package that we can distribute from the monorepo.

This is generally fine – for example, server can be distributed under its own prefixed tag – but I don’t think we can incorporate something like sdk in the monorepo. The SDK needs to be a lightweight dependency and due to SwiftPM limitations, I believe we’ll be forced to have it in its own repository. fwiw, the SDK would probably fall into the same bucket as FileSystem where it eventually matures and having it in a different repository won’t cause that much friction anymore.

@pepicrft

With PRs opened, this scan will be a bit painful. We can focus on closing/merging all the existing PRs, but I don’t think that’s very realistic. We can at least try to close as many as we can, and for the remaining, there will have to be a rebase work that needs to happen. I’d say that we support there.

Git usually deals with files that were only moved quite gracefully, so I don’t think we need to rush to merge all opened PRs.

[pepicrft]

@marekfort

Where would you put code shared between different end products, like the CLI and the app sharing the TuistServer module? Would those modules stay in the root or would they be moved in the cli that is currently the “primary” consumer?

I’d say we can place these under /swift, /swift-shared, or just /shared

@marekfort

Additionally, configuration files like .swiftlint.yml are also shared across the app and the CLI – I’d consider keeping those in the root.

We can keep them at the root, or have a config/ directory where we place those to keep the root as lean as possible, such that users coming across the GitHub repository don’t have to scroll a lot to start reading the README.

@marekfort

I would also add that there are some gotchas we ran into when making Noora into a monorepo:

This is a bit annoying. I think having the SDK elsewhere makes sense. Alternatively, we can distribute an .xcframework for this one? Since this one will be integrated by Xcode through SwiftPM, I think it’s a nice distribution bundle.

[marekfort]

@pepicrft

I’d say we can place these under /swift, /swift-shared, or just /shared

swift-shared or swift-common sounds like the most descriptive, but I’m good with either of the suggestions.

@pepicrft

We can keep them at the root, or have a config/ directory where we place those to keep the root as lean as possible, such that users coming across the GitHub repository don’t have to scroll a lot to start reading the README.

Not sure I would optimize for that and having config files in the root is quite common – but no strong opinion on that.

@pepicrft

This is a bit annoying. I think having the SDK elsewhere makes sense. Alternatively, we can distribute an .xcframework for this one? Since this one will be integrated by Xcode through SwiftPM, I think it’s a nice distribution bundle.

Let’s make the decision on that when we get closer to publishing the SDK.