2015-09-20

Automated UI testing in Xcode 7

I’ve tried many UI testing tools for iOS apps, but never found anything that completely satisfied me. Recent versions of KIF have come closest, but I suspect the new XCUITest in Xcode 7 will probably win now that Apple seem to be investing in making the tooling genuinely practical. It certainly lowers the barrier to entry, which given the state of testing in many iOS teams, is probably the most important thing.

Last week I helped set up a team with Xcode 7 UI testing, and thought I should share some observations from the process. In no particular order…

Update (20 March 2016)

Quite surprisingly this post remains very popular, and astonishingly is the top hit on Google for xcuitest at the time of writing. Perhaps this is an indication that there’s a lot of interest in using XCUITest, but not quite the level of official documentation needed to support it…

Anyway, I’d be doing the reader a disservice to not point out an excellent and more comprehensive set of guidance from Joe Masilotti. If you don’t find what you’re looking for in my post, you’ll most likely find it in Joe’s:

UI Element selectors can be made DRY

For anyone with sufficient battle scars from Selenium testing, this looks like an obvious anti-pattern:

XCUIApplication().buttons["Checkout"].tap()
... later ...
XCUIApplication().buttons["Checkout"].tap()   // NOOOO!!!

There are significant maintainability challenges associated with hardcoding and repeating element identifiers, and experience has taught us to isolate identifiers into a constant or (even better) use the page object pattern.

This should actually be quite achievable for XCUITest too - the key discovery is that XCUIElements are lazily evaluated. XCUITest will evaluate the element selector when you use it, rather than when you declare it. This means that you can define an instance variable in your test class (or test base class, or page object class) like this:

class PurchaseUITest: XCTestCase {
    let checkoutButton = XCUIApplication().buttons["Checkout"]
    ...

and now, whenever you need to tap on a button with this label, you can just use checkoutButton as follows:

    checkoutButton.tap()

This means no need to repeat the selector everywhere the element is used, and substantially less effort when it comes to dealing with changes to accessibility labels later.

Waiting for an element to appear

You’ll find yourself doing this more often than you’d think, so this is a useful thing to put into a utility class:

func waitFor(element:XCUIElement, seconds waitSeconds:Double) {
    let exists = NSPredicate(format: "exists == 1")
    expectationForPredicate(exists, evaluatedWithObject: element, handler: nil)
    waitForExpectationsWithTimeout(waitSeconds, handler: nil)
}

Secure field input is broken (Xcode 7 GM/release)

For password fields, calling typeText('...') currently seems to do nothing. A forum post indicates that this is a bug, and includes a workaround: pasting into the field rather than using the device keyboard. This obviously feels like a bit of a hack, but seems to be the only way to do it for now.

Given that I’m working on a non-English application, I’ve revised the workaround to be agnostic of language (the original workaround looks for a popup menu item named Paste’):

// Get the password into the pasteboard buffer
UIPasteboard.generalPasteboard().string = "the password"
// Bring up the popup menu on the password field
passwordField.doubleTap()
// Tap the Paste/ペースト button to input the password
app.menuItems.elementBoundByIndex(0).tap()

There’s possibly some incompatibility with exclusiveTouch

It seems as though taps can get lost when buttons have exclusiveTouch set. I suspect that successive taps are being fired by XCUITest before previous buttons have stopped blocking. I need to look into this further, but it seems like a bug, only solvable right now by the insidious evil of calls to sleep().

Use code generated by the recorder for inspiration, but establish a proper structure for your real test code

Just like with Selenium for web projects, the test recorder generates reasonable code for getting started. However, a suite of tests built solely with the recorder is going to be a maintainability nightmare: repetition of element selectors and common sequences of actions will lead to tests becoming a drag on updating the code rather than an accelerator.

Enforce DRY and other coding standards on test code, just as you would for production app code.

This is by no means a perfect structure, but a good-enough structure I’m using right now is:

class MyProjectNameUITestBase : XCTestCase {
    let app:XCUIApplication = XCUIApplication()
    // common UI elements that appear in all tests
    let usernameField = XCUIApplication().textFields["Username"]
    let passwordField = XCUIApplication().secureTextFields["Password"]

    ...

    // setup/teardown
    override func setUp() {
        super.setUp()
        continueAfterFailure = false
        XCUIApplication().launch()
    }
    override func tearDown() {
        super.tearDown()
    }

    ...

    // common sequences of steps that are used in many test cases
    func doLogin() {
        ...
    }

    // utility methods (e.g. waitFor)
}

class SpecificFunctionalAreaUITest : MyProjectNameUITestBase {
    // specific UI elements, e.g.:
    let checkoutButton = XCUIApplication().buttons["Checkout"]

    func testHappyPath() {
        doLogin()

        ...
    }

    func testSadPath() {
        doLogin()

        ...
    }

    func testAnotherSadPath() {
        doLogin()

        ...
    }
}

You can use XCUITest with pre-iOS 9 projects

For some reason I originally thought the new UI test tooling would only work for projects that targetted iOS 9+. Actually, this works on at least projects built for iOS 8+ SDK; albeit the automated UI tests can only run in an iOS 9 simulator. That is to say, you can’t run tests against iOS 8 - but for teams just getting started in testing, having good coverage only for iOS 9 is probably better than mediocre coverage for both iOS 8 and 9.


testing iOS


Previous post
Duct Tape is a little Java 8 library providing fault tolerance support for code that calls external APIs and unreliable components, including those
Next post
GPG and SSH with Yubikey for Mac Yubikey Neo The Yubikey Neo [affiliate] is a great, inexpensive security device that supports Universal 2nd Factor authentication to web services