Revising modularization to cater for the iOS app

[marekfort]

Need/problem

We’ve had two final products for a while now – a CLI and a macOS menu bar app. The menu bar app reuses some CLI modules, like TuistSupport or TuistServer. Since the menu bar app has the same destination as the CLI, we haven’t run into any issues.

However, we are now bringing an iOS app to the mix and not all Tuist modules are compilable for iOS. One example is XcodeProj that only compiles on macOS.

Motivation

We need to align on how to update our modularization strategy, so we can:

• Reuse common modules across destinations, like interacting with the server (TuistServer)
• Modules imported by the iOS app should fully compile on iOS – without those modules having to do too many of destination-specific compiler directives or build rules.

Detailed design

There are two major changes we need to do.

Splitting monolith modules

We need to split monolith modules like TuistSupport, so products like iOS app can only depend on the part of the graph that it actually needs – and doesn’t end up depending on parts of TuistSupport that don’t compile on macOS.

How do we split it? I think the most sensible approach is for each utility in TuistSupport to be its own module. It does mean we’ll end up with a lot of small modules, but I believe that’s fine.

So, TuistSupport/GitController.swift would be in TuistGit or TuistGitController (slightly leaning to go without the Controller suffix). TuistSupport/Constants.swift would be in TuistConstants, TuistSupport/Environment.swift would be in Environment. You get the idea.

In a sense, we’ll split TuistSupport into a new layer of Support modules. Similarly, we’ll need to split TuistCore into a new layer of Core modules. Support modules shouldn’t have dependencies between themselves. Core modules can depend on Support modules, but again, they should not have dependencies between themselves.

This might be a great opportunity for us to dogfood tuist graph --json and build a custom linter that ensures the mentioned conditions are not broken.

Another convention we need to establish is for modules that are already scoped to a single domain, but still need to be split to make them compilable for the desired destination. Such an example is TuistServer that we split into:

• TuistServerCore – compilable on all destinations
• TuistServerCLI- CLI-specific components, such as for interacting with the binary cache
• TuistServerApp – scoped to the iOS app

We expect to need this rarely. Splitting the monolith TuistSupport and TuistCore modules should be enough in most scenarios.

Drawbacks

We’ll end up with a lot more modules and imports. That’s not necessarily a con, but it does mean some extra complexity. Additionally, splitting the modules will take some effort. We should aim to do this iteratively.

Alternatives

The Tuist app doesn’t share any modules with the CLI. We’ll end up duplicating some efforts, such as communicating with the server – but we would end up with a simplified module hierarchy. However, I think the proposed modularization adjustments would be beneficial regardless.

[pepicrft]

I agree we should split up support. Not only do we allow dependent targets to pick at a more granular level what they need, but incremental builds should be faster too since we are working with smaller portions of the graph when making changes.

I wonder if we should revisit the grouping “core” & “support”. In the early days, we agreed that the difference between one and the other would be knowing Tuist (i.e., support is Tuist-agnostic), but time has proved that this is tricky to hold. Many data structures and business logic have some knowledge of Tuist.

If we blur that distinction, I wonder if inter-dependencies are acceptable in that support/core group. For example, if we have a utility Environment (that lives in TuistEnvironment) to interface with the outside world, and TuistXcode needs to access that interface, why not through TuistXcode > TuistEnvironment dependency? However, we want to have that restriction at the feature level.

I think restricting inter-dependencies between implementations makes sense at the feature level, such that we get the most out of the caching capabilities, and we can also dog-food our proposed modular architecture, but it’ll require us to design features with an interface that other features can depend on. For example, if we have a module TuistXcodeBuild, which needs to depend on TuistServer for storing test results. Do we come up with a protocol that TuistXcodeBuild can depend on? Or… we consider TusitServer a core feature, and allow the other features to depend on this one. For example, I’d say server or generation are features that are core across all the others. Something like:

• High-level features: Xcodebuild, Graph
• Core features: Generator, server
• Support: Environment, Git…

Another thing that comes to my mind is what happens in cases where a feature has a different set of transitive dependencies based on the destination. For example, let’s say TuistServer encapsulates the logic to interface with the server in services, and some of those services only make sense in the context of macOS, for example uploading the result of a run, which requires depending on TuistGit to get the current Git branch to include it in the payload. How would you model that? Would you then have TuistServermacOS that adds the service that only makes sense in a platform? Would you place it in TuistServer and declare a conditional dependency with TuistGit that’s only available when you compile against macOS?

[waltflanagan]

I think we should also be sure to consider the approachability of the graph for new contributors. One issue I had starting was no knowing what should be in support or core or wherever. It was unclear where things should live. From being in the codebase for a bit, I feel this has also been a struggle for others given how some additions have created tight coupling across modules over time.

Generally I wouldn’t shy away from using swift compiler directives to allow a given library to still have platform specific features if it can keep help manage the complexity of the dependency graph.

[marekfort]

@pepicrft

I wonder if we should revisit the grouping “core” & “support”. In the early days, we agreed that the difference between one and the other would be knowing Tuist (i.e., support is Tuist-agnostic), but time has proved that this is tricky to hold. Many data structures and business logic have some knowledge of Tuist.

I’d be for removing that distinction and we can combine the “core” & “support” into a single layer. It’s true the distinction has never been clear and has been rather confusing than helpful.

Not sure how I would name that layer – “Utilities” or “Core”? – but that layer should not depend on any feature level modules.

Speaking of feature modules, I think we’ve discussed this in the past, but I think we should consider splitting TuistKit as well into individual features.

Soo, the final structure could be something like:

Features should not have dependencies themselves, whereas we’d be more benevolent for the Core layer.

@pepicrft

For example, let’s say TuistServer encapsulates the logic to interface with the server in services, and some of those services only make sense in the context of macOS, for example uploading the result of a run, which requires depending on TuistGit to get the current Git branch to include it in the payload. How would you model that? Would you then have TuistServermacOS that adds the service that only makes sense in a platform? Would you place it in TuistServer and declare a conditional dependency with TuistGit that’s only available when you compile against macOS?

If a module only has a fraction of platform-specific features, that I would use compiler directives to keep the module structure as simple as possible as raised by @waltflanagan. Otherwise, I’d do the split with TuistServerCLI and TuistServerApp as outlined above. There might be a fraction of features that are useful in the macOS app and the CLI, but not in the iOS app – but I think that should be rare and we should fall back to compiler directives there.

@waltflanagan

From being in the codebase for a bit, I feel this has also been a struggle for others given how some additions have created tight coupling across modules over time.

We’re certainly missing module guidance and I think we should remedy that as an output of this RFC over at the contributors section of the documentation. Documentation and clear guidelines could go a long way here to help contributors make decisions – even if we do end up with more modules than we have now.

@waltflanagan

Generally I wouldn’t shy away from using swift compiler directives to allow a given library to still have platform specific features if it can keep help manage the complexity of the dependency graph.

Agree, we should be pragmatic and use compiler directives unless the module would get full of them. Even in those cases, I wonder if compiler directives are a better option. While the TuistServerCLI module has quite a bit of files, maybe the structure would have been easier to understand if we just kept them in TuistServer and used compiler directives.

[pepicrft]

Looking at the graph above made me think about the “TuistKit” which sits between the features and the CLI. I believe this target absorbed too much business logic because it was where we could “glue” multiple features together. I wonder if we can come up with some dependency-injection pattern with dependency-inversion, such that we can get rid of that layer, and then end up with:

• Products layer: CLI / macOS App / iOS App
• Features layer: Generate / Build / Graph
• Support layer: Git / Xcode / Environment

And regarding the dependencies between features… would you have interface targets for each feature such that they don’t need to implement in implementations? (e.g. Build depending on GenerateInterface).

@marekfort

Agree, we should be pragmatic and use compiler directives unless the module would get full of them. Even in those cases, I wonder if compiler directives are a better option. While the TuistServerCLI module has quite a bit of files, maybe the structure would have been easier to understand if we just kept them in TuistServer and used compiler directives.

I think this is a good idea. Having too many targets makes the contributors’ learning curve quite steep.

[marekfort]

@pepicrft

I wonder if we can come up with some dependency-injection pattern with dependency-inversion, such that we can get rid of that layer, and then end up with

Do we even need a dependency injection pattern when each feature module should not depend on other features? And common code being hoisted into the Support layer.

@pepicrft

I think this is a good idea. Having too many targets makes the contributors’ learning curve quite steep.

I’ll revise that piece then and bring back all the server-related code into a single TuistServer module.

[pepicrft]

@marekfort

Do we even need a dependency injection pattern when each feature module should not depend on other features? And common code being hoisted into the Support layer.

Won’t features depend on each other? For example generation needing to interact with cache to mutate the graph with binaries, or build with the server to report results on completion. How would you address those scenarios? Would you have a delegate pattern where generate calls a delegate?

[marekfort]

@pepicrft

For example generation needing to interact with cache to mutate the graph with binaries, or build with the server to report results on completion

In which layer would you put a module like TuistGenerator? Is it a feature, does it belong to a support layer? Or is there one extra layer hiding here, a core layer, for modeling the core functionality of Tuist that then individual features/commands build upon?

My thinking around the feature layer was for those to be more-or-less scoped to individual commands or groups of commands – for those, we really shouldn’t have any interdependencies. If we consider modules like TuistGenerator a “feature”, then we absolutely need to allow for interdependencies.

I would try to stay away from splitting modules into their interface + implementation. I don’t think the benefits outweigh the quite some additional complexity for contributors.

[pepicrft]

@marekfort

In which layer would you put a module like TuistGenerator? Is it a feature, does it belong to a support layer? Or is there one extra layer hiding here, a core layer, for modeling the core functionality of Tuist that then individual features/commands build upon?

I think “generation” and “server” are core to Tuist’s business domain. They are equivalent to things like “customers” & “products” in an e-commerce domain. So in that sense, they should be dependent upon any feature. Then there are features like “MCP” or “Graph” (visualization), which are higher-level. At that level, I don’t expect MCP to know about the Graph or vice versa.

@marekfort

My thinking around the feature layer was for those to be more-or-less scoped to individual commands or groups of commands – for those, we really shouldn’t have any interdependencies. If we consider modules like TuistGenerator a “feature”, then we absolutely need to allow for interdependencies.

I think we are on the same page. So something like:

• Feature layer: Graph, Xcodebuild… They cannot depend on features in the same layer, but can depend on lower layers
• Core layer: Generation, Server… They can depend on other core layer modules and modules under them, but not on features above them.
• Support layer: Environment, Git, Xcode. They have no dependencies.

Thoughts?

[marekfort]

@pepicrft

I think we are on the same page. So something like:

• Feature layer: Graph, Xcodebuild… They cannot depend on features in the same layer, but can depend on lower layers
• Core layer: Generation, Server… They can depend on other core layer modules and modules under them, but not on features above them.
• Support layer: Environment, Git, Xcode. They have no dependencies.

Thoughts?

I’m aligned with that structure.

@waltflanagan wdyt?