I’ve been investigating using Tuist for our companies’ repo and was looking for buildable folder support.
We’re currently using SwiftPM which works similar to buildable folders in that all sources from the folder are automatically added to the build phases (and resources too, if needed). This has proved helpful when adding files in interactive sessions using external tools / VS Code / Cursor for iOS development, since it doesn’t require a manual step of, say, either adding the file to the group in Xcode manually, or running tuist generate each time to get syntax errors / SwiftUI preview updates working, which is important for iteration speed.
If there’s some other solution for this scenario I’d love to know.
I’ve already contributed a small bug fix to XcodeProj for PBXFileSystemSynchronizedRootGroup to help explore the feasibility of implementing this, but it still seems like we’d need to implement PBXFileSystemSynchronizedGroupBuildPhaseMembershipExceptionSet / PBXFileSystemSynchronizedBuildFileExceptionSet support before we could seriously consider adding it to Tuist, I believe.
While synchronized groups are less useful in the context of Tuist, I can definitely see them being marginal improvements, primarily by making it necessary to run tuist generate less often.
There are two approaches for the ProjectDescription interface we can take here:
Keep the current glob API and instead of expanding the glob when generating a project, we could instead generate the glob as the synchronized group.
Provide a dedicated API to opt-in.
Personally, I’m leaning to the former as long as the transition from globs to groups is seamless.
Would be curious to get @pepicrft thoughts on this, too.
These might be necessary to support exclusions based on the specified globs.
I’m on board with the idea, but I leaned on a new API. If I’m not mistaken (it’s been a while since I implemented support for them in XcodeProj), they reference a directory and then you can add “exceptions” to files inside the directory. How would you imagine the API @bryansum ? Since you have more context on the available APIs from your recent contributions, I suggest that you take the lead on the API proposal and we iterate together.
I don’t have a clear picture of what an API could be at this point, but I’m willing to iterate on a solution and come up with a proposal given there’s some interest / approval from you to do so.
Regarding how the new object (PBXFileSystemSynchronizedRootGroup) works:
Yes, from what I’ve observed once you add a synchronized group to the fileSystemSynchronizedGroups property for a target it includes any source files / resources in their associated build phases (Compile Sources / Copy Bundle Resources) without explicitly specifying them for that target.
More complicated is the behavior of PBXFileSystemSynchronizedBuildFileExceptionSet exceptions, which comprise membershipExceptions (a list of file paths) and an associated target. From what I can tell, if the exception set’s target property matches the parent target of its parent, the PBXFileSystemSynchronizedRootGroup, the exceptions act to exclude the listed files from that folder (which happens when manually clicking “-” in the Compile Sources / Copy Bundle Resources build phases). If the target property is different, the membershipExceptions act to include those listed files (which happens when manually adding additional paths from another buildable folder to either build phases).
When envisioning the API, I’d recommend aligning with Xcode’s terminology. Maybe something along the lines of:
let target = Target(
/..../
sources: .synchronized("Sources/MyTarget"),
resources: .synchronized("Resources/MyTarget"),
)
I’d start with the simplest API, and then iterate on that to add exceptions later.
Let me know if you need any guidance to contribute to the projects. I’d be happy to give you a rundown of the surfaces of the code bases that you’ll have to touch.
Note that the schema of the feature in Xcode projects is based on a reference to a folder (with defaults applied by Xcode) and then a list of exceptions, which in many cases I’d say it’s more of an “override of defaults” than exceptions.
I haven’t formed an idea around what the API might look like, but I’d start trying to answer:
Can we reuse existing ProjectDescription APIs? Or adding this capability would make things confusing?
Would it make sense to have a new API to indicate the relationship between a target and buildable folders that mimics the internals of Xcode projects? Or should we take the opportunity to rethink the API since those exceptions are at the end of the day implementation details for users?
I suggest that we align on the API first, and then kick off the implementation work.
I wonder if we should turn globs such as Sources/** by default into buildable groups. I feel that would result across the board in a better experience – no need to re-generate a project after switching a branch that just adds new source files without changing the graph itself.
The excluding option can be implemented with buildable groups as well, so I don’t really see a reason why not to do this.
I also don’t see a reason to opt-out, but we could enable that in the glob static factory if this is raised as a need by someone.
I think a glob and a buildable folder which is a reference to a folder are different things. We can enable it though for strings that don’t represent globs, like Sources/
If it only works for Sources/ but not for Sources/**, then this implicit behavior might confuse me as a Tuist user.
I like @marekfort’s idea. Perhaps it’s worth asking a more general question:
Is it really necessary to keep virtual group support in Tuist?
If the project structure is completely under Tuist’s control and cannot be (undesirably) modified by individual user actions, then a full switch to Buildable folder without implicit logic on the path string format should not be a hindrance.
The only reason why it makes sense to implement buldable folder as a separate type is to preserve backwards compatibility in case Xcode and the build system have any bugs with buildable folder.
I agree with @Ernest0N on this one – @pepicrft would you mind expanding on why you’d default to buildable groups for folder references, but not for globs?
A buildable folder is a “pointer” to a file-system folder with a list of exceptions. If you take something like sources: "App/*/Sources/**/*.swift", how would you translate that to that other model? We could say “App” is the buildable folder, then we resolve the glob at generation time, then diff that against the complete list of files, and include them in the exceptions list.
I wouldn’t make App the buildable folder in that scenario. That would make the buildable folder way too complex and I don’t think it maps well conceptually to the specified glob. Imho, what should be turned into buildable folders is a suffix of the glob, in this case Sources/**/*.swift where Sources would be the buildable folder.
In your example where you have the extra * glob in the middle of the path, we can either turn that into multiple build folders (App/A/Sources, App/B/Sources) or keep the current behavior – and only turn simple globs like Sources/**/*.swift into buildable groups.
I have mixed feelings with the implicitness of “we decide based on some heuristics”, but we can try and see how people find it. I guess as long as it works people will be ok with that?
Long-term, any glob that groups multiple source files should be turned into a buildable folder(s) as that does result in a better experience imho. But I’d start with picking the easier globs initially before taking on the more complex globs.
Can you tell me if the API refinement is extensive? I could try to contribute if the time needed for integration and the number of edge scenarios are minimal does not require much time.
Since we plan the API to remain the same, then no, the API refinement should not be extensive. But this change will have ripple effects across the codebase as all globs will need to remain as globs instead of expanding them as we do know – so we can convert them to buildable folders when actually generating the project.