Trick to speed up `tuist install` on self-hosted CI runners

General
1

We are currently moving our external dependencies from the regular SPM setup to the XcodeProj-based approach in Tuist. One of the drawbacks we saw from this was a very slow execution of tuist install which would take more than a minute on our self-hosted Bitbucket Pipelines runner. All our CI builds are cloned in a new folder, so the Tuist/.build folder is empty on every build.

Our first attempt was to use the cache feature on Bitbucket Pipelines, but that requires the cached folders to be uploaded to Bitbucket and downloaded again on a new build. This is also a bit slow, and if you have enough dependencies, it take up more space than what Bitbucket will allow (1GB).

Our second attempt was to make the Tuist/.build as a symlink to a persisted folder in the home folder on the machine. This seems to work pretty well and it has taken the tuist install duration down to 15 seconds (from over one minute before).

These are the steps we run to make the symlink:

  1. mkdir -p ~/project-name-tuist-cache
  2. ln -s ~/project-name-tuist-cache Tuist/.build
  3. tuist install
1 Like
2

It’s good that you brought this up because we just started working on a solution for this. For context, at the end of the day the issue is that SPM pulls for every dependency the entire repository history for dependency resolution. So even if you copy and restore that directory, it’ll take that time because it can be quite large.

The workaround that you mention is the one that we’ve seen some companies adopting, but that requires owning the machines in which your builds run. It moves some complexity to your side, and might potentially arise some errors if the resolution algorithm is unable to incrementally deal with the state of the symlinked directory.

@marekfort will be sharing more about the work that we are doing here soon.