There is a known issue with Xcode 15, mentioned in Apple’s release notes that results in slow test execution.
Bitrise is aware of the problem. Our internal benchmarking shows a roughly 25% slowdown in test execution speed compared to Xcode 14.3. If you are running a test suite on the Xcode 15 stack and observe slow performance, you may be experiencing this problem.
While we are hoping that Apple includes fixes for the problem in Xcode 15.1, in the meantime we can offer the following measures to speed test execution:
Open Source Tool: yeetd
An open-source tool, yeetd, was recently published that attempts to improve Xcode 15 simulator performance by killing unnecessary and resource-hungry processes. Our benchmarking has shown some marginal speed improvements enabling yeetd prior to test execution.
Confirm your company’s supply chain policies before using open-source code in your CI/CD pipelines. If you decide to use yeetd, it can be added to your Bitrise workflow with a script step that runs:
wget
https://github.com/biscuitehh/yeetd/releases/download/1.0/yeetd-normal.pkg
sudo installer -pkg yeetd-normal.pkg -target /
yeetd &This script will run yeetd in the background while your tests execute.
Use Faster Hardware
Bitrise Velocity customers have access to M1 Max machines with substantial CPU and memory resources. Experiment with running your test workflow on the M1 Max Large for faster execution. Test execution on our internal benchmark app using Xcode 15 was approximately 30% faster using M1 Max Large (10 CPU cores, 54 GB RAM) compared to M1 Large (8 CPU cores, 12 GB RAM).
Stack Updates
Keep an eye on stacks.bitrise.io, where we post the latest information about our MacOS and Linux stacks. As mentioned above, we are hoping Apple will remedy this problem in Xcode 15.1. As soon as the first Xcode 15.1 beta version becomes available, Bitrise will make it available to our customers, and an update will be posted to stacks.bitrise.io.
