2016-01-12

Better JUnit Selenium testing with Docker and Testcontainers

This is the third article in a series (1, 2) that explores integrated JUnit testing involving external components, supported by Testcontainers.

This post explores how Testcontainers can directly help improve our Selenium testing ruggedness and reduce some of the problems that I’ve observed on previous projects.

Some common problems and solutions

  • HTMLUnit and PhantomJS are perhaps the most portable browser options for Selenium, but lack feature parity with modern browsers, and are hard to debug. We really need real browsers
  • Every build machine needs to have the right web browser installed - this includes CI environments, and developers’ machines. This isn’t a hard problem, but it adds scope for things to go wrong. We need as few dependencies as possible, as ubiquituous as possible. Also:
  • Every build machine needs the right version of the browser installed, otherwise compatibility with the selenium API may be broken. This is particularly challenging now that browsers automatically upgrade. I’ve lost track of the number of times I’ve seen a project team lose time to resolving ill-timed versioning breakages.
  • Headless operation is a must to allow developers to run tests in the background, in a degree of isolation. Solutions like xvfb are great in linux environments, but a bit of a chore for developers on OS X. Headless operation is important
  • We’d like to be able to step in to a running test to inspect and diagnose why it might be failing unexpectedly. We need to be able to break out of a headless mode sometimes, for interactive sessions
  • We need to know why tests failed, with as much information as possible. Screenshots are useful, but in modern webapps with highly interactive UIs, videos would be better.

A simple solution with Testcontainers selenium module

Testcontainers allows fixed versions of Chrome and Firefox to be run inside of Docker containers, fully wired up to Selenium, VNC, and with automated video recording of tests.

The only dependency outside of the JUnit test suite is Docker.

Let’s try a simple example - here’s a very simple JUnit test that uses selenium:

Note: this example does not follow good practices such as use of page objects, but it’s here for simple illustration using the raw Selenium API.

public class SeleniumContainerTest {

    @Test
    public void simplePlainSeleniumTest() {
        RemoteWebDriver driver = // obtain a Webdriver instance somehow

        driver.get("https://wikipedia.org");
        WebElement searchInput = driver.findElementByName("search");

        searchInput.sendKeys("Rick Astley");
        searchInput.submit();

        WebElement otherPage = driver.findElementByLinkText("Rickrolling");
        otherPage.click();

        boolean expectedTextFound = driver.findElementsByCssSelector("p")
                .stream()
                .anyMatch(element -> element.getText().contains("meme"));

        assertTrue("The word 'meme' is found on a page about rickrolling", expectedTextFound);
    }
}

Assuming we have the Testcontainers Selenium module on the classpath, the next step is to add a @Rule field to our test class as follows:

@Rule
public BrowserWebDriverContainer chrome = new BrowserWebDriverContainer()
                    .withDesiredCapabilities(DesiredCapabilities.chrome())
                    .withRecordingMode(RECORD_ALL, new File("target"));

When we do this, Testcontainers will instantiate a new Docker container for each test method. We tell it to use Chrome, and Testcontainers will select a specific version-locked docker image (in this case selenium/standalone-chrome-debug:2.45.0).

Additionally, notice that we set a recording mode and a location for recordings to be placed. The mode can be RECORD_ALL or RECORD_FAILING - useful if we only want to inspect videos of failing test methods. Videos are recorded automatically for us, and saved as FLV files that can be played with VLC and other video players.

Next, in our test method (or perhaps in a @Before method) we obtain an instance of RemoteWebDriver, bound to our browser container:

RemoteWebDriver driver = chrome.getWebDriver();

This driver instance allows us to conduct our tests as we would normally! The following video was captured during execution of the tests, automatically:

Our successful log output look a bit like:

21:22:23.392 [main] INFO  org.testcontainers.DockerClientFactory - Checking environment for docker settings
21:22:26.178 [main] INFO  org.testcontainers.DockerClientFactory - Could not initialize docker settings using environment variables: org.apache.http.conn.UnsupportedSchemeException: https protocol is not supported
21:22:26.180 [main] INFO  org.testcontainers.DockerClientFactory - Checking for presence of docker-machine
21:22:26.197 [main] INFO  org.testcontainers.utility.CommandLine - Executing shell command: `docker-machine ls -q`
21:22:26.538 [main] INFO  org.testcontainers.DockerClientFactory - Found docker-machine, and will use first machine defined (dev)
21:22:26.538 [main] INFO  org.testcontainers.utility.CommandLine - Executing shell command: `docker-machine status dev`
21:22:26.741 [main] INFO  org.testcontainers.utility.CommandLine - Executing shell command: `docker-machine ip dev`
21:22:27.437 [main] INFO  org.testcontainers.DockerClientFactory - Docker daemon IP address for docker machine dev is 192.168.99.100
21:22:30.480 [main] INFO  org.testcontainers.DockerClientFactory - Disk utilization in Docker environment is 22%21:22:31.147 [main] INFO  🐳 [selenium/standalone-chrome-debug:2.45.0] - Creating container for image: selenium/standalone-chrome-debug:2.45.0
21:22:31.340 [main] INFO  🐳 [selenium/standalone-chrome-debug:2.45.0] - Starting container with ID: b23b210ab508ed5714dd7ea22e09af696e303519a709ba4d09ac9a50b4e73242
21:22:31.587 [main] INFO  🐳 [selenium/standalone-chrome-debug:2.45.0] - Container selenium/standalone-chrome-debug:2.45.0 is starting: b23b210ab508ed5714dd7ea22e09af696e303519a709ba4d09ac9a50b4e73242
21:22:38.851 [pool-1-thread-1] INFO  🐳 [selenium/standalone-chrome-debug:2.45.0] - Obtained a connection to container (http://192.168.99.100:33215/wd/hub)
21:22:39.069 [main] INFO  🐳 [richnorth/vnc-recorder:latest] - Creating container for image: richnorth/vnc-recorder:latest
21:22:39.385 [main] INFO  🐳 [richnorth/vnc-recorder:latest] - Starting container with ID: 7067d9fecebad921308609e6416059219d25e9ba9a2aab53abf031d9b0a0ab1f
21:22:39.786 [main] INFO  🐳 [richnorth/vnc-recorder:latest] - Container richnorth/vnc-recorder:latest is starting: 7067d9fecebad921308609e6416059219d25e9ba9a2aab53abf031d9b0a0ab1f
21:22:40.100 [main] INFO  🐳 [richnorth/vnc-recorder:latest] - Container richnorth/vnc-recorder:latest started
21:22:40.101 [main] INFO  🐳 [selenium/standalone-chrome-debug:2.45.0] - Container selenium/standalone-chrome-debug:2.45.0 started
        ✔ The word 'meme' is found on a page about rickrolling 
21:23:02.780 [main] INFO  org.testcontainers.containers.BrowserWebDriverContainer - Screen recordings for test simplePlainSeleniumTest(SeleniumContainerTest) will be stored at: target/recording-20160112-212302.flv
21:23:03.007 [main] INFO  🐳 [richnorth/vnc-recorder:latest] - Stopped container: richnorth/vnc-recorder:latest
21:23:03.096 [main] INFO  🐳 [richnorth/vnc-recorder:latest] - Removed container and associated volume(s): richnorth/vnc-recorder:latest
21:23:03.907 [main] INFO  🐳 [selenium/standalone-chrome-debug:2.45.0] - Stopped container: selenium/standalone-chrome-debug:2.45.0
21:23:03.965 [main] INFO  🐳 [selenium/standalone-chrome-debug:2.45.0] - Removed container and associated volume(s): selenium/standalone-chrome-debug:2.45.0

You can see the complete source code here.

Other useful methods on BrowserWebDriverContainer that we can use as we see fit:

  • getHostIpAddress(): enables us to find the IP address of the machine where the JUnit suite is running relative to the Dockerized browser. This is useful for the common scenario of testing a webapp that is running on the local machine (e.g. at http://localhost:8080).
  • getVncAddress(): provides a full VNC URL that may be used to connect to our browser container - for viewing or interacting with the browser. On OS X, one can quickly open a VNC session at the terminal with the open command, e.g.:

    open vnc://vnc:secret@192.168.99.100:33212

Outcomes

  • We can use a real web browser, either Chrome or Firefox
  • Our dependencies are reduced to one thing: Docker. I believe that this will increasingly be available as standard on developer machines and CI environments.
  • Testcontainers will always use a specific fixed version of the browser, regardless of auto-updates or what version developers use locally.
  • Test runs are inherently headless, but we can jump into a test session interactively via VNC when the need arises.
  • We have full videos of testing sessions, for free!

Conclusion

I hope this post is informative about one potential technique for making selenium testing more rugged at the infrastructure’ level. If it interests you, please give Testcontainers are try in your own projects.

A future post will hopefully look another awesome tool that addresses some of the problems of writing UI tests for modern webapps: Selenide.


testcontainers testing selenium docker java


Previous post
Fun with Disque, Java and Spinach This example continues similarly to a previous post, in which I used my Testcontainers tool to aid integration testing of Java code against a
Next post
Testcontainers 1.0.0 release I’ve just released v1.0.0 of Testcontainers, the little library for using Docker containers in JUnit tests. It’s been great to get feedback and