Distinguish errors and test failures when using tuist test

Hey @apps4everyone

How would you handle the errors if you were not using tuist test at all and you were using xcodebuild directly?

We use Fastlane (read xcodebuild) to run tests and used to have a need to distinguish build and test errors. The goal was to post a report message on pull requests pages. I wrote a bash script to parse xcodebuild log and extract build errors if they are presented. As a result, when a developer sees that their “Test” CI job has failed, they go to the pull request page and check the report post comment. If there are build errors on it, then is was the build has failed. If not, then it’s the tests have failed. This approach speeds up the check and eliminates the need to dive into CI logs and look for errors there.

2 Likes

Here is the script:

#! /usr/bin/env bash
set -e

BUILD_LOG_FILE="$1"
METRICS_FILE="${2}/xcodebuild_report.txt"
CURRENT_DIR=$(pwd)
# Remove if found
rm -rf "$METRICS_FILE"

# Extract errors
echo Extracting xcodebuild errors
ERRORS=`egrep '^(/.+:[0-9+:[0-9]+:.(error):|fatal|===)' "$BUILD_LOG_FILE" | sed "s|^$CURRENT_DIR/||"`
if [ -n "$ERRORS" ]; then
  echo "ERRORS:" >> "$METRICS_FILE"
  echo "$ERRORS" | awk '{print}' | sort -u >> "$METRICS_FILE"
fi

# Extract warnings
echo Extracting xcodebuild warnings
WARNINGS=`egrep '^(/.+:[0-9+:[0-9]+:.(warning):|fatal|===)' "$BUILD_LOG_FILE" | sed "s|^$CURRENT_DIR/||" | grep -v 'deprecated' | grep -v 'Tuist/.build'`
if [ -n "$WARNINGS" ]; then
  echo "WARNINGS:" >> "$METRICS_FILE"
  echo "$WARNINGS" | sort -u | awk '{print NR ". " $0}' >> "$METRICS_FILE"
fi

We have recently rewrote the script on Ruby just to make formatting easier and this is how it looks like on GitLab:

2 Likes

We are using fastlane as a wrapper