Better JUnit Selenium test­ing with Docker and Testcontainers 12 Jan 2016

This is the third ar­ti­cle in a se­ries (1, 2) that ex­plores in­te­grated JUnit test­ing in­volv­ing ex­ter­nal com­po­nents, sup­ported by Testcontainers.

This post ex­plores how Testcontainers can di­rectly help im­prove our Selenium test­ing rugged­ness and re­duce some of the prob­lems that I’ve ob­served on pre­vi­ous pro­jects.

Some com­mon prob­lems and so­lu­tions #

• HTMLUnit and PhantomJS are per­haps the most portable browser op­tions for Selenium, but lack fea­ture par­ity with mod­ern browsers, and are hard to de­bug. We re­ally need real browsers
• Every build ma­chine needs to have the right web browser in­stalled - this in­cludes CI en­vi­ron­ments, and de­vel­op­ers’ ma­chines. This is­n’t a hard prob­lem, but it adds scope for things to go wrong. We need as few de­pen­den­cies as pos­si­ble, as ubiq­ui­tu­ous as pos­si­ble. Also:
• Every build ma­chine needs the right ver­sion of the browser in­stalled, oth­er­wise com­pat­i­bil­ity with the se­le­nium API may be bro­ken. This is par­tic­u­larly chal­leng­ing now that browsers au­to­mat­i­cally up­grade. I’ve lost track of the num­ber of times I’ve seen a pro­ject team lose time to re­solv­ing ill-timed ver­sion­ing break­ages.
• Headless op­er­a­tion is a must to al­low de­vel­op­ers to run tests in the back­ground, in a de­gree of iso­la­tion. Solutions like xvfb are great in linux en­vi­ron­ments, but a bit of a chore for de­vel­op­ers on OS X. Headless op­er­a­tion is im­por­tant
• We’d like to be able to step in to a run­ning test to in­spect and di­ag­nose why it might be fail­ing un­ex­pect­edly. We need to be able to break out of a head­less mode some­times, for in­ter­ac­tive ses­sions
• We need to know why tests failed, with as much in­for­ma­tion as pos­si­ble. Screenshots are use­ful, but in mod­ern we­bapps with highly in­ter­ac­tive UIs, videos would be bet­ter.

A sim­ple so­lu­tion with Testcontainers selenium mod­ule #

Testcontainers al­lows fixed ver­sions of Chrome and Firefox to be run in­side of Docker con­tain­ers, fully wired up to Selenium, VNC, and with au­to­mated video record­ing of tests.

The only de­pen­dency out­side of the JUnit test suite is Docker.

Let’s try a sim­ple ex­am­ple - here’s a very sim­ple JUnit test that uses se­le­nium:

Note: this ex­am­ple does not fol­low good prac­tices such as use of page ob­jects, but it’s here for sim­ple il­lus­tra­tion us­ing the raw Selenium API.

public class SeleniumContainerTest {

    @Test
    public void simplePlainSeleniumTest() {
        RemoteWebDriver driver = // obtain a Webdriver instance somehow

        driver.get("https://wikipedia.org");
        WebElement searchInput = driver.findElementByName("search");

        searchInput.sendKeys("Rick Astley");
        searchInput.submit();

        WebElement otherPage = driver.findElementByLinkText("Rickrolling");
        otherPage.click();

        boolean expectedTextFound = driver.findElementsByCssSelector("p")
                .stream()
                .anyMatch(element -> element.getText().contains("meme"));

        assertTrue("The word 'meme' is found on a page about rickrolling", expectedTextFound);
    }
}

Assuming we have the Testcontainers Selenium mod­ule on the class­path, the next step is to add a @Rule field to our test class as fol­lows:

@Rule
public BrowserWebDriverContainer chrome = new BrowserWebDriverContainer()
                    .withDesiredCapabilities(DesiredCapabilities.chrome())
                    .withRecordingMode(RECORD_ALL, new File("target"));

When we do this, Testcontainers will in­stan­ti­ate a new Docker con­tainer for each test method. We tell it to use Chrome, and Testcontainers will se­lect a spe­cific ver­sion-locked docker im­age (in this case se­le­nium/​stand­alone-chrome-de­bug:2.45.0).

Additionally, no­tice that we set a record­ing mode and a lo­ca­tion for record­ings to be placed. The mode can be RECORD_ALL or RECORD_FAILING - use­ful if we only want to in­spect videos of fail­ing test meth­ods. Videos are recorded au­to­mat­i­cally for us, and saved as FLV files that can be played with VLC and other video play­ers.

Next, in our test method (or per­haps in a @Before method) we ob­tain an in­stance of RemoteWebDriver, bound to our browser con­tainer:

RemoteWebDriver driver = chrome.getWebDriver();

This driver in­stance al­lows us to con­duct our tests as we would nor­mally! The fol­low­ing video was cap­tured dur­ing ex­e­cu­tion of the tests, au­to­mat­i­cally:

https://​www.youtube.com/​watch?v=cX_iT2df­bq0

Our suc­cess­ful log out­put look a bit like:

21:22:23.392 [main] INFO  org.testcontainers.DockerClientFactory - Checking environment for docker settings
21:22:26.178 [main] INFO  org.testcontainers.DockerClientFactory - Could not initialize docker settings using environment variables: org.apache.http.conn.UnsupportedSchemeException: https protocol is not supported
21:22:26.180 [main] INFO  org.testcontainers.DockerClientFactory - Checking for presence of docker-machine
21:22:26.197 [main] INFO  org.testcontainers.utility.CommandLine - Executing shell command: `docker-machine ls -q`
21:22:26.538 [main] INFO  org.testcontainers.DockerClientFactory - Found docker-machine, and will use first machine defined (dev)
21:22:26.538 [main] INFO  org.testcontainers.utility.CommandLine - Executing shell command: `docker-machine status dev`
21:22:26.741 [main] INFO  org.testcontainers.utility.CommandLine - Executing shell command: `docker-machine ip dev`
21:22:27.437 [main] INFO  org.testcontainers.DockerClientFactory - Docker daemon IP address for docker machine dev is 192.168.99.100
21:22:30.480 [main] INFO  org.testcontainers.DockerClientFactory - Disk utilization in Docker environment is 22%21:22:31.147 [main] INFO  🐳 [selenium/standalone-chrome-debug:2.45.0] - Creating container for image: selenium/standalone-chrome-debug:2.45.0
21:22:31.340 [main] INFO  🐳 [selenium/standalone-chrome-debug:2.45.0] - Starting container with ID: b23b210ab508ed5714dd7ea22e09af696e303519a709ba4d09ac9a50b4e73242
21:22:31.587 [main] INFO  🐳 [selenium/standalone-chrome-debug:2.45.0] - Container selenium/standalone-chrome-debug:2.45.0 is starting: b23b210ab508ed5714dd7ea22e09af696e303519a709ba4d09ac9a50b4e73242
21:22:38.851 [pool-1-thread-1] INFO  🐳 [selenium/standalone-chrome-debug:2.45.0] - Obtained a connection to container (http://192.168.99.100:33215/wd/hub)
21:22:39.069 [main] INFO  🐳 [richnorth/vnc-recorder:latest] - Creating container for image: richnorth/vnc-recorder:latest
21:22:39.385 [main] INFO  🐳 [richnorth/vnc-recorder:latest] - Starting container with ID: 7067d9fecebad921308609e6416059219d25e9ba9a2aab53abf031d9b0a0ab1f
21:22:39.786 [main] INFO  🐳 [richnorth/vnc-recorder:latest] - Container richnorth/vnc-recorder:latest is starting: 7067d9fecebad921308609e6416059219d25e9ba9a2aab53abf031d9b0a0ab1f
21:22:40.100 [main] INFO  🐳 [richnorth/vnc-recorder:latest] - Container richnorth/vnc-recorder:latest started
21:22:40.101 [main] INFO  🐳 [selenium/standalone-chrome-debug:2.45.0] - Container selenium/standalone-chrome-debug:2.45.0 started
        ✔ The word 'meme' is found on a page about rickrolling
21:23:02.780 [main] INFO  org.testcontainers.containers.BrowserWebDriverContainer - Screen recordings for test simplePlainSeleniumTest(SeleniumContainerTest) will be stored at: target/recording-20160112-212302.flv
21:23:03.007 [main] INFO  🐳 [richnorth/vnc-recorder:latest] - Stopped container: richnorth/vnc-recorder:latest
21:23:03.096 [main] INFO  🐳 [richnorth/vnc-recorder:latest] - Removed container and associated volume(s): richnorth/vnc-recorder:latest
21:23:03.907 [main] INFO  🐳 [selenium/standalone-chrome-debug:2.45.0] - Stopped container: selenium/standalone-chrome-debug:2.45.0
21:23:03.965 [main] INFO  🐳 [selenium/standalone-chrome-debug:2.45.0] - Removed container and associated volume(s): selenium/standalone-chrome-debug:2.45.0

You can see the com­plete source code here.

Other use­ful meth­ods on BrowserWebDriverContainer that we can use as we see fit:

• getHostIpAddress(): en­ables us to find the IP ad­dress of the ma­chine where the JUnit suite is run­ning rel­a­tive to the Dockerized browser. This is use­ful for the com­mon sce­nario of test­ing a we­bapp that is run­ning on the lo­cal ma­chine (e.g. at http://localhost:8080).
• getVncAddress(): pro­vides a full VNC URL that may be used to con­nect to our browser con­tainer - for view­ing or in­ter­act­ing with the browser. On OS X, one can quickly open a VNC ses­sion at the ter­mi­nal with the open com­mand, e.g.:

open vnc://vnc:[email protected]:33212

Outcomes #

• We can use a real web browser, ei­ther Chrome or Firefox
• Our de­pen­den­cies are re­duced to one thing: Docker. I be­lieve that this will in­creas­ingly be avail­able as stan­dard on de­vel­oper ma­chines and CI en­vi­ron­ments.
• Testcontainers will al­ways use a spe­cific fixed ver­sion of the browser, re­gard­less of auto-up­dates or what ver­sion de­vel­op­ers use lo­cally.
• Test runs are in­her­ently head­less, but we can jump into a test ses­sion in­ter­ac­tively via VNC when the need arises.
• We have full videos of test­ing ses­sions, for free!

Conclusion #

I hope this post is in­for­ma­tive about one po­ten­tial tech­nique for mak­ing se­le­nium test­ing more rugged at the ‘infrastructure’ level. If it in­ter­ests you, please give Testcontainers are try in your own pro­jects.

A fu­ture post will hope­fully look an­other awe­some tool that ad­dresses some of the prob­lems of writ­ing UI tests for mod­ern we­bapps: Selenide.

← Home