We run a lot of Tuist builds and tests on CI, so we tend to get a bunch of failures whenever there’s a dip in availability. We’re thinking of enhancing our CI script to run tests a second time without the binary cache if it fails with it. But to do that effectively we would need to be able to distinguish between a tuist test failure due to downloading cached binaries or selective test results, vs. an actual test failure.
One idea would be to support different return codes from Tuist. Would that work, is there another option we could use, or is this already supported and I missed it?
I believe the better option here instead of using specific exit codes is to morph cache unavailability errors into warnings. What errors triggered this post?
We can either:
Turn any cache-related error into a warning
Turn errors that we know are due to unavailability into warnings
I’d rather try to do the latter – there are errors such as when a TUIST_PROJECT_TOKEN is missing on the CI that I’d like to still keep as errors.
I’d propose that you flag any unavailability error that caused your pipelines to fail and we’ll make sure to handle it.