In this guest article, John Sundell will share some of his top tips and techniques for making iOS UI tests easier to work with when using Apple’s built-in XCTest framework, as well as how to easily visualize the results of such tests using Bitrise’s Test Reporting add-on.
UI testing can be an incredibly useful tool for verifying that an app’s key features and user interactions remain fully functional while quickly iterating on its code base.
Especially when it comes to screens and user flows that are easy to miss when doing manual testing — such as an app’s onboarding flow, certain settings screens, or features that only appear under a given set of conditions — UI testing can act as a neat “insurance policy” for when those flows are accidentally broken by a seemingly unrelated change.
However, over the years, Xcode’s built-in UI testing system has developed somewhat of a reputation for being slow, unstable and generally hard to work with. While it’s true that UI tests are inherently slower to run than something like unit tests (given that they actually have to run and interact with the UI as a user would), there are a few tricks and techniques that we can employ to make the tests that we write much more stable — and in the end, easier to both write and maintain.
Waiting for elements to appear
Timing is arguably one of the most common sources of flakiness when it comes to UI tests (meaning that a test occasionally fails, often seemingly at random), as elements and controls sometimes take variable amounts of time to appear on the screen, which our tests might not account for.
Let’s start by taking a look at a relatively simple example, in which we’re writing a UI test for an app used to keep track of various meetings. In this case, we want to verify that our app displays a screen showing details about a meeting when it was selected within a list — which is currently done like this:
class MeetingListUITests: XCTestCase {
...
func testSelectingMeeting() {
// Launch the app:
let app = XCUIApplication()
app.launch()
// Select the first meeting in the list:
app.tables.cells.firstMatch.tap()
// Assert that the meeting screen is shown by verifying
// that its title is displayed in the navigation bar:
XCTAssertTrue(app.navigationBars["Meeting details"].exists)
}
}While there are a number of ways in which the above test can be improved, perhaps the biggest issue is that it currently makes a quite a strong assumption that the meeting list will be immediately available as soon as the app is launched.
Even though that list might always be the first screen shown, chances are that the amount of time it’ll take to actually load its data might depend on a number of factors — especially if we need to load parts of that data over the network.
Initially, this might actually not be an issue, since Xcode’s UI test runner will automatically account for different kinds of delays, and automatically insert a bit of waiting time in between each of our test operations. However, sometimes those automatic waiting mechanisms might not be enough, resulting in our test becoming somewhat unpredictable and — in other words — flaky.
So how can we fix that kind of flakiness? One idea might be to simply call sleep() within our test, which will effectively make the test runner pause for a given number of seconds. While that might actually solve the problem, it also comes with a quite significant downside — it makes our tests unnecessarily slow to run, as the test runner will always be paused for the given number of seconds, even if our UI ends up being rendered much faster than that.
Thankfully, there’s a better way, as XCTest includes a dedicated API that lets us wait for any element to appear — without having to pause our tests for a fixed amount of time. Here’s how we could use that API to make sure that our test won’t continue before our meeting list’s UITableView has been rendered:
class MeetingListUITests: XCTestCase {
...
func testSelectingMeeting() {
let app = XCUIApplication()
app.launch()
// Wait for the table view cell to appear before tapping it:
let cell = app.tables.cells.firstMatch
XCTAssertTrue(cell.waitForExistence(timeout: 10))
cell.tap()
XCTAssertTrue(app.navigationBars["Meeting details"].exists)
}
}Note how we use a quite generous timeout above in order to make sure that our app always has enough time to finish loading its data, and since the test runner won’t actually wait for all of those 10 seconds (unless it has to), we’re not making our test any slower to run. However, we wouldn’t want to use a much larger timeout, since we want legitimate errors to be reported as soon as possible.
Writing robust element queries
Next, let’s take a look at how we can often make our tests more stable and easier to maintain by slightly tweaking the way we construct our element queries.
This again comes back to avoiding making hard assumptions about how our app behaves, as that’s something that’s bound to change over time. After all, the point of writing UI tests is not to verify the exact structure of our UI, but rather to make sure that our core interactions keep behaving as we’d expect as we iterate on our code base.
Looking at our above test, however, our queries are currently quite heavily tied to the exact way that our UI has currently been implemented. For example, we’re retrieving the table view cell to tap using app.tables.cells.firstMatch, which will match the first cell that the test runner encounters anywhere on the entire screen — meaning that if we ever add another table view to our UI, our test will most likely become unstable, as the wrong cell could end up being selected.
To address that problem, let’s make our query use an accessibility identifier, rather than simply selecting the first element that was matched. What’s interesting about UI tests in general is that they are very closely related to accessibility, since Xcode’s UI test runner actually uses the accessibility system to navigate and perform other actions within our app.
Accessibility identifiers are simply arbitrary strings that we can programmatically assign to our views, which won’t be used for any actual accessibility purposes (so they won’t be used by VoiceOver and other assistive technologies). Instead, they’re there to enable us to query our elements in a much more predictable fashion.
To get started, let’s assign an accessibility identifier to the UITableView that we’re using to render our meeting list, for example within its presenting view controller’s viewDidLoad() method:
class MeetingListViewController: UIViewController {
private lazy var tableView = UITableView()
...
override func viewDidLoad() {
super.viewDidLoad()
// An accessibility identifier can be any string that
// we'd like, so we can use whichever naming convention
// that we prefer:
tableView.accessibilityIdentifier = "meeting-list"
...
}
...
}Then, let’s update our test to use the above accessibility identifier, which is done by subscripting the app.tables part of our query to explicitly select our meeting list table view — like this:
// Using an accessibility identifier-based query:
let cell = app.tables["meeting-list"].cells.firstMatch
XCTAssertTrue(cell.waitForExistence(timeout: 10))
cell.tap()Simple as that! Next, let’s also do something about the way we verify that the correct screen is shown once one of our cells has been tapped.
Currently, that’s done by assuming that the meeting details screen will always display the string ”Meeting details” in the navigation bar, which isn’t very robust — both because we might change that string to something more dynamic in the future, and because we’re only verifying that we’re displaying any meeting details screen — which might not be the correct one.
Capturing UI content
For the most part, it’s arguably a good idea to only use static content for verification within our UI tests — as doing so tends to give us the highest degree of predictability. However, in this case, we’re looking to match the cell that was tapped against its corresponding details screen, which could be done in a deterministic way by capturing content from our UI.
Just like before, our first task will be to make sure that the view that we’re looking to capture content from has an accessibility identifier assigned to it — so that we’ll be able to predictably query it within our tests. In this case, let’s use the current cell’s titleLabel, so a good place to assign our accessibility identifier would be within our cell configuration code — for example like this:
extension MeetingListViewController: UITableViewDataSource {
...
func tableView(_ tableView: UITableView,
cellForRowAt indexPath: IndexPath) -> UITableViewCell {
let cell = tableView.dequeueReusableCell(
withIdentifier: cellReuseIdentifier,
for: indexPath
)
cell.textLabel?.accessibilityIdentifier = "meeting-cell-title"
// Cell configuration code
...
return cell
}
}With the above in place, we can now go back to our test and heavily improve the way we verify that the right screen is displayed when a meeting cell gets tapped — by first capturing the cell’s title, and then verifying that a corresponding text is displayed somewhere on the next screen:
class MeetingListUITests: XCTestCase {
func testSelectingMeeting() {
let app = XCUIApplication()
app.launch()
// Using an accessibility identifier-based query:
let cell = app.tables["meeting-list"].cells.firstMatch
XCTAssertTrue(cell.waitForExistence(timeout: 10))
// Capture the cell's title text:
let titleLabel = cell.staticTexts["meeting-cell-title"]
let meetingTitle = titleLabel.label
XCTAssertFalse(meetingTitle.isEmpty)
// Tap the cell to navigate to the next screen:
cell.tap()
// Assert that the meeting's title is displayed on the new
// screen (note how we also now query the new screen's
// main view using an accessibility identifier, rather
// than using the navigation bar):
let nextScreenLabel = app
.otherElements["meeting-details"]
.staticTexts[meetingTitle]
XCTAssertTrue(nextScreenLabel.exists)
}
}So when writing element queries, it’s recommended to either use static content (when possible), or to capture the content that we’re looking to use for verification from our UI itself — as to make our tests as predictable and deterministic as possible.
Setting up predictable app states
However, writing really robust queries and handling all sorts of delays is not enough, because one final source of potential flakiness (and cause of slowness) still remains: the app that we’re UI testing is highly likely to have quite a lot of states.
Statefulness is, in general, something that can make any kind of testing more difficult — especially if we have no way of controlling that state, as that’d essentially make the app that we’re testing somewhat of a moving target.
As an example, let’s say that our app includes an onboarding feature, which gets displayed when the user first installs and launches the app. To keep track of whether the user has already gone through that onboarding flow, we’re currently using a simple boolean value stored within the app’s UserDefaults — like this:
func shouldShowOnboarding() -> Bool {
!UserDefaults.standard.bool(forKey: "onboarding-shown")
}Now, unless we take control over the above kind of persisted state, our tests will likely start succeeding or failing completely based on what state the app was in when we last launched it — which isn’t great.
Thankfully, when it comes to UserDefaults in particular, we can actually override any persisted value simply by using launch arguments — we don’t need to change our app’s code in any way. All we have to do is to prefix the key we’re looking to override with a dash (-), and assign that (along with our override) to XCUIApplication before launching it:
class OnboardingUITests: XCTestCase {
func testShowingOnboardingOnAppLaunch() {
// Assigning the launch arguments to send to our app when
// it's launched:
let app = XCUIApplication()
app.launchArguments = ["-onboarding-shown", "NO"]
app.launch()
// Again we're using an accessibility identifier to make
// sure that our custom onboarding view gets displayed:
let welcomeView = app.otherElements["onboarding-welcome"]
XCTAssertTrue(welcomeView.waitForExistence(timeout: 10))
}
}The cool thing is that the above technique can be used to override all sorts of values, even if they’re not actually being stored using UserDefaults. For example, here’s how we could also pass the user ID of a specific account that we wish to perform our tests against:
app.launchArguments = ["-onboarding-shown", "YES", "-user-id", "12345"]Then, if the value we’re looking to override isn’t stored within UserDefaults itself, we can still use it to retrieve our launch argument, and then perform that override manually — for example like this:
func loadStoredUserID() -> User.ID? {
// If our app is running using its DEBUG configuration,
// we check if a launch argument was passed for overriding
// the current user's ID, and if so, we return it:
#if DEBUG
let override = UserDefaults.standard.string(forKey: "user-id")
if let override = override {
return User.ID(rawValue: override)
}
#endif
// Load our user ID as we normally would, for example
// by querying our database or using the key chain API:
...
}Using this technique can not only help us put our app in a completely predictable state, it could also let us speed up our tests — for example by providing mocked data, rather than having to fetch it over the network — or by telling our app to navigate to a specific screen immediately, rather than having to navigate there by interacting with our UI.
Visualizing test results using Bitrise’s Test Reporting add-on
Whenever we’re looking to improve anything about our development or testing setup — whether that’s our UI testing suite or something else — having access to clear, reliable data and reports can make a huge difference. After all, if we don’t measure and observe the way something is currently performing, how can we know if we’re actually improving things over time?
When it comes to both unit and UI tests, Bitrise has an incredibly useful feature that lets us visualize this kind of data for our entire test suite — appropriately named “Test Reports”. Simply make sure that your Bitrise workflow has both a “Xcode Test for iOS” step (to run your tests) and a “Deploy to Bitrise.io” step (to deploy your test data and artifacts) and you’re good to go.
To learn more about the Test Reports feature, check out its documentation on the Bitrise Dev Center.
Conclusion
While UI tests will most likely always remain both slower and more complicated compared to things like unit tests, they can be an incredibly useful tool to strategically deploy in order to verify an app’s key flows and user interactions. Should you cover your entire app with UI tests? I personally don’t think so. But I don’t think they should be completely ignored either.
Hopefully, using the tips and techniques from this article, and by running your tests on a rock-solid CI/CD platform like Bitrise, you’ll be able to make your UI tests run faster and with a higher degree of predictability — which in turn should make the investment of writing and maintaining those UI tests pay off more easily.
For more on UI testing, check out this category page on Swift by Sundell, which contains articles, videos, and podcast episodes all about that topic. You can also find me on Twitter @johnsundell, if you have any questions, comments, or feedback.
Thanks for reading! 🚀
.jpg)
.jpg)
