Monthly Archives: October 2016

Aggregate Test Coverage Report for Gradle Multi-Module Project

twittergoogle_plusrss

Facebooktwittergoogle_plusredditlinkedinmail

I am going to explain how to aggregate test coverage report for Gradle multi-module project. For measuring test coverage, we will use JaCoCo. Example project will use TravisCI build server and will submit coverage report to Coveralls.io.

Multi-Module project is project which creates various modules in single build, typically JARs in Java world. Such project structure is handy for splitting monolithic projects into decoupled pieces. But wait a second. Didn’t Microservices fever put monoliths into architecture anti-pattern position? No, it didn’t.

Problem

We are also building monolithic application at Dotsub. Build system of our choice is Gradle and build server is TravisCI. We use Coveralls.io to store our test coverage reports. When our code-base grew to the point when we needed to separate concerns into separate modules, I ran into problems around gathering test coverage. On Coveralls.io, you can submit only one coverage report per TravisCI build. As we wanted to have whole project built with single build, our only option was merging test coverage reports for each module and submit such aggregated report.

Example Project Structure

Our example project will have 4 modules. It is hosted in this Github repository.

.
├── build.gradle
├── .coveralls.yml
├── gradlecoverage
│   ├── build.gradle
│   └── src
│       ├── main
│       │   └── java
│       │       └── net
│       │           └── lkrnac
│       │               └── blog
│       │                   └── gradlecoverage
│       │                       └── GradleCoverageApplication.java
│       └── test
│           └── java
│               └── net
│                   └── lkrnac
│                       └── blog
│                           └── gradlecoverage
│                               └── GradleCoverageApplicationTest.java
├── module1
│   ├── build.gradle
│   └── src
│       ├── main
│       │   └── java
│       │       └── net
│       │           └── lkrnac
│       │               └── blog
│       │                   └── gradlecoverage
│       │                       └── Module1Service.java
│       └── test
│           └── java
│               └── net
│                   └── lkrnac
│                       └── blog
│                           └── gradlecoverage
│                               └── Module1ServiceTest.java
├── module2
│   ├── build.gradle
│   └── src
│       ├── main
│       │   └── java
│       │       └── net
│       │           └── lkrnac
│       │               └── blog
│       │                   └── gradlecoverage
│       │                       └── Module2Service.java
│       └── test
│           └── java
│               └── net
│                   └── lkrnac
│                       └── blog
│                           └── gradlecoverage
│                               └── Module2ServiceTest.java
├── moduleCommon
│   ├── build.gradle
│   └── src
│       ├── main
│       │   └── java
│       │       └── net
│       │           └── lkrnac
│       │               └── blog
│       │                   └── gradlecoverage
│       │                       └── CommonService.java
│       └── test
│           └── java
│               └── net
│                   └── lkrnac
│                       └── blog
│                           └── gradlecoverage
│                               └── CommonServiceTest.java
├── settings.gradle
└── .travis.yml

There is main module gradlecoverage, which is dependent on module1 and module2. Last moduleCommon is shared dependency for module1 and module2. Each module contains simple string concatenation logic with test. The code is very basic, so we are not going to explain it here. You can inspect it on Github.

Each test intentionally doesn’t cover some test cases. This way we will prove that test coverage reports are accurate.

Module Build Script

Each module has very simple build script. Example of module1/gradle.properties follows:

jar {
    baseName = 'module1'
}

dependencies {
    compile project(':moduleCommon')
}

The script just defines JAR module name and imports other module as dependency. Other scripts have very similar build scripts.

Main Build Script

First of all we need to import all sub-modules in settings.properties:

include 'moduleCommon'
include 'module1'
include 'module2'
include 'gradlecoverage'

In main script we will apply jacoco and coveralls plugins. JaCoCo plugin will be needed for aggregating coverage reports form sub-modules. We will create custom task for aggregation. Coveralls plugin will submit aggregated report to Coveralls.io.

plugins {
    id 'jacoco'
    id 'com.github.kt3k.coveralls' version '2.6.3'
}

repositories {
    mavenCentral()
}

Next we configure Java and import dependencies for sub-modules in subprojects section:

subprojects {
    repositories {
        mavenCentral()
    }

    apply plugin: 'java'
    apply plugin: "jacoco"

    group = 'net.lkrnac.blog'
    version = '1.0-SNAPSHOT'

    sourceCompatibility = JavaVersion.VERSION_1_8
    targetCompatibility = JavaVersion.VERSION_1_8

    dependencies {
        testCompile("junit:junit:4.12")
    }
}

For each sub-module, Java plugin will build JAR artifact. Each module is tested by JUnit framework. Test coverage is measured by JaCoCo plugin.

Aggregate Test Coverage Reports

Last piece of build configuration will aggregate test coverage reports:

def publishedProjects = subprojects.findAll()

task jacocoRootReport(type: JacocoReport, group: 'Coverage reports') {
    description = 'Generates an aggregate report from all subprojects'

    dependsOn(publishedProjects.test)

    additionalSourceDirs = files(publishedProjects.sourceSets.main.allSource.srcDirs)
    sourceDirectories = files(publishedProjects.sourceSets.main.allSource.srcDirs)
    classDirectories = files(publishedProjects.sourceSets.main.output)
    executionData = files(publishedProjects.jacocoTestReport.executionData)

    doFirst {
        executionData = files(executionData.findAll { it.exists() })
    }

    reports {
        html.enabled = true // human readable
        xml.enabled = true // required by coveralls
    }
}

coveralls {
    sourceDirs = publishedProjects.sourceSets.main.allSource.srcDirs.flatten()
    jacocoReportPath = "${buildDir}/reports/jacoco/jacocoRootReport/jacocoRootReport.xml"
}

tasks.coveralls {
    dependsOn jacocoRootReport
}

First we read all sub-modules into publishedProjects variable. After that we define custom aggregation jacocoRootReport task. It is inherited from JacocoReport task type. This task will be dependent on test task of each module from publishedProjects. This makes sure that test coverage reports for sub-modules are gathered for aggregation before jacocoRootReport is executed. Next we configure necessary directories for JaCoCo engine and gather all execution data from sub-modules.

Lastly we configure what kind of output we want to generate. Configuration xml.enabled = true will create aggregated report which will be submitted to Coveralls.io via coveralls task.

TravisCI Manifest

TravisCI Manifest (.travis.yml) file configure Java environment and runs single command to build and submit aggregated test coverage report by executing coveralls task:

language: java
jdk:
  - oraclejdk8

install: echo "skip './gradlew assemble' step"

script: ./gradlew build coveralls --continue

Coveralls Manifest

To be able to submit report to coveralls, we need to define Coveralls.io repository token in file .coveralls.yml:

repo_token: JQofR7TNqCMebvHgE8wHwF6rznjvEc0Fr

Final Report

Final report can be found here:

aggregate test coverage report

As you can see, there are all classes from separate modules. The only problem is that we can’t fond which module class belong to from such coverage report. But it’s not a big deal to me.

Credits

Custom aggregation task was inspired by build script for Caffeine project. It is caching library.

twittergoogle_plusrss

Component Object pattern example

twittergoogle_plusrss

Facebooktwittergoogle_plusredditlinkedinmail

Previously I showed how we at Dotsub use Page Object pattern for Selenium testing. But we use one more abstraction to make end-to-end test more maintainable. We call it Component Object pattern. This blog post will explain it on example.

Web development is shifting more and more towards reusable components. Frameworks like React, Polymer, Angular, Ember and others provide various component friendly abstractions to make front-end code-bases more maintainable. So our web applications are now full of “widgets” that have same behavior. We can use component various times on single web page or re-use it on various web pages.

Therefore it is logical to create abstraction which covers functionality of single component and reuse it across end-to-end tests. As I mentioned before, we going to call it Component Object pattern. So when we have various same components on single web page, we are going to use various Component Objects of same type per Page Object.

Example project for testing

Now we need to application we are going to test. Example application is hosted in this Github repository. You can run it by executing command from root directory:

./gradlew bootRun

From this command, you can find that it is Spring Boot application based on Gradle build system. In Dotsub, we are using Spring Boot + Java on back-end, but in fact there is no back-end mentioned example web application. We don’t need back-end for our demonstration.

For UI, we going to use React + Redux combo. I chose famous Dan Abramov’s Todos example to demonstrate Component Object pattern on. But I needed to amend it a little bit to reuse components. Without re-usable components we couldn’t demonstrate Component Object pattern.

After visiting URL http://localhost:8080, we can see following page:

Component Object pattern

There are two input components with buttons. One creates item in Todo list and second creates item in Grocery list:

Component Object pattern

When you click on item, it will mark it as completed (strike-through). When you click on completed item, it will become active again. Last element on the page is filter. You can show only active items:

Component Object pattern

or only completed items:

Component Object pattern

UI code is hosted in this Github repository, located under folder src/main/ui. I will leave this code for self study because the implementation is not deeply relevant for end-to-end testing code. Important fact for us is that input components / list components for Todo and Grocery lists should have same behavior. Thus they can be covered by reusable Component Objects pattern. Interesting is that they doesn’t necessary need to be implemented as same component in UI code.

The only important for our testing are CSS classes of particular components:

  • AddTodo input component is using CSS class add-todo
  • TodoList is using CSS class todo-list
  • AddGroceryItem input component is using CSS class add-grocery-item
  • GroceryList is using CSS class grocery-list

Note that goal of this blog post isn’t to explain Selenium or its APIs. It is expected for reader to be slightly familiar with them already.

Component Object for adding the item

First explained component object will control adding the item:

package net.lkrnac.blog.pageobject.e2e;

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;

class AddItemComponent {
    private WebDriver driver;
    private String containerCssSelector;

    AddItemComponent(WebDriver driver, String containerCssSelector) {
        this.driver = driver;
        this.containerCssSelector = containerCssSelector;
    }

    AddItemComponent addItem(String todo) {
        WebElement input = driver.findElement(By.cssSelector(containerCssSelector + " input"));
        input.sendKeys(todo);
        WebElement button = driver.findElement(By.cssSelector(containerCssSelector + " button"));
        button.click();
        return this;
    }
}

Alongside Selenium web driver instance, this component object also accepts CSS selector of component it’s going to control. It has just one method addItem, which enters a text into input field and clicks Add button. It creates new item.

Component Object for item list

package net.lkrnac.blog.pageobject.e2e;

import org.apache.commons.lang3.StringUtils;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;

import java.util.List;

import static java.lang.String.format;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNotNull;
import static org.junit.Assert.assertTrue;

class ItemsListComponent {
    private final WebDriver driver;
    private final String containerCssSelector;

    ItemsListComponent(WebDriver driver, String containerCssSelector) {
        this.driver = driver;
        this.containerCssSelector = containerCssSelector;
    }

    ItemsListComponent clickOnItem(String todoItem) {
        findElementWithText(todoItem).click();
        return this;
    }

    ItemsListComponent verifyItemShown(String todoItem, boolean expectedStrikethrough) {
        WebElement todoElement = findElementWithText(todoItem);
        assertNotNull(todoElement);
        boolean actualStrikethrough = todoElement.getAttribute("style").contains("text-decoration: line-through;");
        assertEquals(expectedStrikethrough, actualStrikethrough);
        return this;
    }

    ItemsListComponent verifyItemNotShown(String todoItem) {
        assertTrue(findElementsWithText(todoItem).isEmpty());
        return this;
    }

    private WebElement findElementWithText(String text) {
        return driver.findElement(getConditionForText(text));
    }

    private List<WebElement> findElementsWithText(String text) {
        return driver.findElements(getConditionForText(text));
    }

    private By getConditionForText(String text) {
        String containerClassName = StringUtils.substring(containerCssSelector, 1);
        return By.xpath(format("//*[@class='" + containerClassName + "']//*[text()='%s']", text));
    }
}

Similar to AddItemComponent, ItemsListComponent also takes Selenium web driver instance and CSS selector of belonging component as constructor parameters. It exposes function clickOnItem for which clicks on particular item. Other two non-private methods are used to verify if particular item is shown (verifyItemShown) or hidden (verifyItemNotShown).

Page Object using Component Objects

Now it’s time to explain Page Object using mentioned Component Objects:

package net.lkrnac.blog.pageobject.e2e;

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;

import static java.lang.String.format;

class ItemsPageObject {
    private final WebDriver driver;
    private final WebDriverWait wait;
    private final ItemsListComponent todoItemsList;
    private final AddItemComponent addTodoItemComponent;
    private final ItemsListComponent groceryItemsList;
    private final AddItemComponent addGroceryItemComponent;

    ItemsPageObject(WebDriver driver) {
        this.driver = driver;
        this.wait = new WebDriverWait(driver, 10);
        todoItemsList = new ItemsListComponent(driver, ".todo-list");
        addTodoItemComponent = new AddItemComponent(driver, ".add-todo");
        groceryItemsList = new ItemsListComponent(driver, ".grocery-list");
        addGroceryItemComponent = new AddItemComponent(driver, ".add-grocery-item");
    }

    ItemsPageObject get() {
        driver.get("localhost:8080");
        wait.until(ExpectedConditions.elementToBeClickable(By.tagName("button")));
        return this;
    }

    ItemsPageObject selectAll() {
        findElementWithText("All").click();
        return this;
    }

    ItemsPageObject selectActive() {
        findElementWithText("Active").click();
        return this;
    }

    ItemsPageObject selectCompleted() {
        findElementWithText("Completed").click();
        return this;
    }

    ItemsPageObject addTodo(String todoName) {
        addTodoItemComponent.addItem(todoName);
        return this;
    }

    ItemsPageObject addGroceryItem(String todoName) {
        addGroceryItemComponent.addItem(todoName);
        return this;
    }

    ItemsListComponent getTodoList() {
        return todoItemsList;
    }

    ItemsListComponent getGroceryList() {
        return groceryItemsList;
    }

    private WebElement findElementWithText(String text) {
        return driver.findElement(getConditionForText(text));
    }

    private By getConditionForText(String text) {
        return By.xpath(format("//*[text()='%s']", text));
    }
}

In constructor it creates component objects with correct CSS selectors. As you can see various instances of same component object type are used to control similar components on the page. Method get opens the page and waits until it’s loaded. Methods selectAll, selectActive, selectCompleted are used to control filter component on the page. Methods addTodo and addGroceryItem are used to enter new item into particular list. Finally getters getTodoList and getGroceryList are useful to let test class enable control over list components. Exposing component object instances to directly seemed easier than wrapping all their functions in Page Object.

Test Cases using Page Object with Page Components

I believe final test cases are readable and doesn’t require comments (which is result of using Page Object + Page Component patterns):

package net.lkrnac.blog.pageobject.e2e;

import io.github.bonigarcia.wdm.ChromeDriverManager;
import net.lkrnac.blog.pageobject.TodoApplication;
import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.springframework.boot.test.SpringApplicationConfiguration;
import org.springframework.boot.test.WebIntegrationTest;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;

@RunWith(SpringJUnit4ClassRunner.class)
@SpringApplicationConfiguration(classes = TodoApplication.class)
@WebIntegrationTest
public class ItemsAppTest {
    private static WebDriver driver;

    @BeforeClass
    public static void setUp() {
        ChromeDriverManager.getInstance().setup();
        driver = new ChromeDriver();
    }

    @AfterClass
    public static void tearDown() {
        driver.quit();
    }

    @Test
    public void testCreateTodos() {
        // GIVEN
        new ItemsPageObject(driver).get()

            // WHEN
            .addTodo("Buy groceries")
            .addTodo("Tidy up")

            // THEN
            .getTodoList()
            .verifyItemShown("Buy groceries", false)
            .verifyItemShown("Tidy up", false);
    }

    @Test
    public void testCompleteTodo() {
        // GIVEN
        new ItemsPageObject(driver).get()
            .addTodo("Buy groceries")
            .addTodo("Tidy up")
            .getTodoList()

            // WHEN
            .clickOnItem("Buy groceries")

            // THEN
            .verifyItemShown("Buy groceries", true)
            .verifyItemShown("Tidy up", false);
    }

    @Test
    public void testSelectTodosActive() {
        // GIVEN
        ItemsPageObject todoPage = new ItemsPageObject(driver).get();

        todoPage
            .addTodo("Buy groceries")
            .addTodo("Tidy up")
            .getTodoList()
            .clickOnItem("Buy groceries");

        // WHEN
        todoPage
            .selectActive()

            // THEN
            .getTodoList()
            .verifyItemNotShown("Buy groceries")
            .verifyItemShown("Tidy up", false);
    }

    @Test
    public void testSelectTodosCompleted() {
        // GIVEN
        ItemsPageObject todoPage = new ItemsPageObject(driver).get();
        todoPage
            .addTodo("Buy groceries")
            .addTodo("Tidy up")
            .getTodoList()
            .clickOnItem("Buy groceries");

        // WHEN
        todoPage
            .selectCompleted()

            // THEN
            .getTodoList()
            .verifyItemShown("Buy groceries", true)
            .verifyItemNotShown("Tidy up");
    }

    @Test
    public void testSelectTodosAll() {
        // GIVEN
        ItemsPageObject todoPage = new ItemsPageObject(driver).get();
        todoPage
            .addTodo("Buy groceries")
            .addTodo("Tidy up")
            .getTodoList()
            .clickOnItem("Buy groceries");
        todoPage
            .selectCompleted()

            // WHEN
            .selectAll()

            // THEN
            .getTodoList()
            .verifyItemShown("Buy groceries", true)
            .verifyItemShown("Tidy up", false);
    }

    @Test
    public void testCreateGroceryItems() {
        // GIVEN
        new ItemsPageObject(driver).get()

            // WHEN
            .addGroceryItem("avocados")
            .addGroceryItem("tomatoes")

            // THEN
            .getGroceryList()
            .verifyItemShown("avocados", false)
            .verifyItemShown("tomatoes", false);
    }

    @Test
    public void testCompleteGroceryItem() {
        // GIVEN
        new ItemsPageObject(driver).get()
            .addGroceryItem("avocados")
            .addGroceryItem("tomatoes")
            .getGroceryList()

            // WHEN
            .clickOnItem("avocados")

            // THEN
            .verifyItemShown("avocados", true)
            .verifyItemShown("tomatoes", false);
    }

    @Test
    public void testSelectGroceryItemsActive() {
        // GIVEN
        ItemsPageObject todoPage = new ItemsPageObject(driver).get();

        todoPage
            .addGroceryItem("avocados")
            .addGroceryItem("tomatoes")
            .getGroceryList()
            .clickOnItem("avocados");

        // WHEN
        todoPage
            .selectActive()

            // THEN
            .getGroceryList()
            .verifyItemNotShown("avocados")
            .verifyItemShown("tomatoes", false);
    }

    @Test
    public void testSelectGroceryItemsCompleted() {
        // GIVEN
        ItemsPageObject todoPage = new ItemsPageObject(driver).get();
        todoPage
            .addGroceryItem("avocados")
            .addGroceryItem("tomatoes")
            .getGroceryList()
            .clickOnItem("avocados");

        // WHEN
        todoPage
            .selectCompleted()

            // THEN
            .getGroceryList()
            .verifyItemShown("avocados", true)
            .verifyItemNotShown("tomatoes");
    }

    @Test
    public void testSelectGroceryItemsAll() {
        // GIVEN
        ItemsPageObject todoPage = new ItemsPageObject(driver).get();
        todoPage
            .addGroceryItem("avocados")
            .addGroceryItem("tomatoes")
            .getGroceryList()
            .clickOnItem("avocados");
        todoPage
            .selectCompleted()

            // WHEN
            .selectAll()

            // THEN
            .getGroceryList()
            .verifyItemShown("avocados", true)
            .verifyItemShown("tomatoes", false);
    }

    @Test
    public void testSelectCombinedItemsActive() {
        // GIVEN
        ItemsPageObject todoPage = new ItemsPageObject(driver).get();

        todoPage
            .addTodo("Buy groceries")
            .addTodo("Tidy up")
            .addGroceryItem("avocados")
            .addGroceryItem("tomatoes");

        todoPage
            .getGroceryList()
            .clickOnItem("avocados");

        todoPage
            .getTodoList()
            .clickOnItem("Tidy up");

        // WHEN
        todoPage
            .selectActive();

        // THEN
        todoPage
            .getTodoList()
            .verifyItemShown("Buy groceries", false)
            .verifyItemNotShown("Tidy up");

        todoPage
            .getGroceryList()
            .verifyItemNotShown("avocados")
            .verifyItemShown("tomatoes", false);
    }
}
twittergoogle_plusrss