Automated UI test­ing in Xcode 7 20 Sep 2015

I’ve tried many UI test­ing tools for iOS apps, but never found any­thing that com­pletely sat­is­fied me. Recent ver­sions of KIF have come clos­est, but I sus­pect the new XCUITest in Xcode 7 will prob­a­bly win now that Apple seem to be in­vest­ing in mak­ing the tool­ing gen­uinely prac­ti­cal. It cer­tainly low­ers the bar­rier to en­try, which given the state of test­ing in many iOS teams, is prob­a­bly the most im­por­tant thing.

Last week I helped set up a team with Xcode 7 UI test­ing, and thought I should share some ob­ser­va­tions from the process. In no par­tic­u­lar or­der…

Update (20 March 2016) #

Quite sur­pris­ingly this post re­mains very pop­u­lar, and as­ton­ish­ingly is the top hit on Google for xcuitest at the time of writ­ing. Perhaps this is an in­di­ca­tion that there’s a lot of in­ter­est in us­ing XCUITest, but not quite the level of of­fi­cial doc­u­men­ta­tion needed to sup­port it…

Anyway, I’d be do­ing the reader a dis­ser­vice to not point out an ex­cel­lent and more com­pre­hen­sive set of guid­ance from Joe Masilotti. If you don’t find what you’re look­ing for in my post, you’ll most likely find it in Joe’s:

• UI Testing Cheat Sheet and Examples
• github.com/​joe­masilotti/​UI-Test­ing-Cheat-Sheet

UI Element se­lec­tors can be made DRY #

For any­one with suf­fi­cient bat­tle scars from Selenium test­ing, this looks like an ob­vi­ous anti-pat­tern:

XCUIApplication().buttons["Checkout"].tap()
... later ...
XCUIApplication().buttons["Checkout"].tap()   // NOOOO!!!

There are sig­nif­i­cant main­tain­abil­ity chal­lenges as­so­ci­ated with hard­cod­ing and re­peat­ing el­e­ment iden­ti­fiers, and ex­pe­ri­ence has taught us to iso­late iden­ti­fiers into a con­stant or (even bet­ter) use the page ob­ject pat­tern.

This should ac­tu­ally be quite achiev­able for XCUITest too - the key dis­cov­ery is that XCUIElements are lazily eval­u­ated. XCUITest will eval­u­ate the el­e­ment se­lec­tor when you use it, rather than when you de­clare it. This means that you can de­fine an in­stance vari­able in your test class (or test base class, or page ob­ject class) like this:

class PurchaseUITest: XCTestCase {
	let checkoutButton = XCUIApplication().buttons["Checkout"]
	...

and now, when­ever you need to tap on a but­ton with this la­bel, you can just use checkoutButton as fol­lows:

	checkoutButton.tap()

This means no need to re­peat the se­lec­tor every­where the el­e­ment is used, and sub­stan­tially less ef­fort when it comes to deal­ing with changes to ac­ces­si­bil­ity la­bels later.

Waiting for an el­e­ment to ap­pear #

You’ll find your­self do­ing this more of­ten than you’d think, so this is a use­ful thing to put into a util­ity class:

func waitFor(element:XCUIElement, seconds waitSeconds:Double) {
    let exists = NSPredicate(format: "exists == 1")
    expectationForPredicate(exists, evaluatedWithObject: element, handler: nil)
    waitForExpectationsWithTimeout(waitSeconds, handler: nil)
}

Secure field in­put is bro­ken (Xcode 7 GM/release) #

For pass­word fields, call­ing typeText('...') cur­rently seems to do noth­ing. A fo­rum post in­di­cates that this is a bug, and in­cludes a workaround: past­ing into the field rather than us­ing the de­vice key­board. This ob­vi­ously feels like a bit of a hack, but seems to be the only way to do it for now.

Given that I’m work­ing on a non-Eng­lish ap­pli­ca­tion, I’ve re­vised the workaround to be ag­nos­tic of lan­guage (the orig­i­nal workaround looks for a popup menu item named ‘Paste’):

// Get the password into the pasteboard buffer
UIPasteboard.generalPasteboard().string = "the password"
// Bring up the popup menu on the password field
passwordField.doubleTap()
// Tap the Paste/ペースト button to input the password
app.menuItems.elementBoundByIndex(0).tap()

There’s pos­si­bly some in­com­pat­i­bil­ity with exclusiveTouch #

It seems as though taps can get lost when but­tons have exclusiveTouch set. I sus­pect that suc­ces­sive taps are be­ing fired by XCUITest be­fore pre­vi­ous but­tons have stopped block­ing. I need to look into this fur­ther, but it seems like a bug, only solv­able right now by the in­sid­i­ous evil of calls to sleep().

Use code gen­er­ated by the recorder for in­spi­ra­tion, but es­tab­lish a proper struc­ture for your real test code #

Just like with Selenium for web pro­jects, the test recorder gen­er­ates rea­son­able code for get­ting started. However, a suite of tests built solely with the recorder is go­ing to be a main­tain­abil­ity night­mare: rep­e­ti­tion of el­e­ment se­lec­tors and com­mon se­quences of ac­tions will lead to tests be­com­ing a drag on up­dat­ing the code rather than an ac­cel­er­a­tor.

Enforce DRY and other cod­ing stan­dards on test code, just as you would for pro­duc­tion app code.

This is by no means a per­fect struc­ture, but a good-enough struc­ture I’m us­ing right now is:

class MyProjectNameUITestBase : XCTestCase {
	let app:XCUIApplication = XCUIApplication()
	// common UI elements that appear in all tests
    let usernameField = XCUIApplication().textFields["Username"]
    let passwordField = XCUIApplication().secureTextFields["Password"]

    ...

    // setup/teardown
    override func setUp() {
        super.setUp()
        continueAfterFailure = false
        XCUIApplication().launch()
    }
    override func tearDown() {
        super.tearDown()
    }

    ...

    // common sequences of steps that are used in many test cases
    func doLogin() {
        ...
    }

    // utility methods (e.g. waitFor)
}

class SpecificFunctionalAreaUITest : MyProjectNameUITestBase {
	// specific UI elements, e.g.:
	let checkoutButton = XCUIApplication().buttons["Checkout"]

	func testHappyPath() {
		doLogin()

		...
	}

	func testSadPath() {
		doLogin()

		...
	}

	func testAnotherSadPath() {
		doLogin()

		...
	}
}

You can use XCUITest with pre-iOS 9 pro­jects #

For some rea­son I orig­i­nally thought the new UI test tool­ing would only work for pro­jects that tar­get­ted iOS 9+. Actually, this works on at least pro­jects built for iOS 8+ SDK; al­beit the au­to­mated UI tests can only run in an iOS 9 sim­u­la­tor. That is to say, you can’t run tests against iOS 8 - but for teams just get­ting started in test­ing, hav­ing good cov­er­age only for iOS 9 is prob­a­bly bet­ter than mediocre cov­er­age for both iOS 8 and 9.

← Home