I was recently speeding up Gradle build on TravisCI for one Dotsub project. It builds Spring Boot based project (written in Java), but also uses Gulp sub-tasks on top of NodeJS to bundle front-end code and assets. This blog post is about sharing these small build optimization hints. They are also shared in this Github repository.
Default TravisCI configuration for Gradle
By default, when Travis detects Gradle as build system it configures these two phases by default:
install: gradle assemble build: gradle check
So as part of your build, Gradle is executed twice by default. And it also does make some sense when you take a look at how Gradle Java plugin works:
As you can see
check phases has only few initial phases in common. There isn’t much tasks redundancy if we run
There is also Gradle UP-TO-DATE feature. It means Gradle is smart enough to mark task as UP-TO-DATE after first execution. Redundant execution of this task can be than skipped if its input doesn’t change. Of course it works across various separate build executions. Let’s take a look at what happen when we build mentioned example project with this example configuration. Link to build is here.
$ ./gradlew assemble :compileJava :processResources :classes :findMainClass :jar :bootRepackage :assemble BUILD SUCCESSFUL Total time: 21.0 secs $ ./gradlew check :compileJava UP-TO-DATE :processResources UP-TO-DATE :classes UP-TO-DATE :compileTestJava :processTestResources UP-TO-DATE :testClasses :test 2016-02-06 11:26:06.883 INFO 2455 --- [ Thread-6] ationConfigEmbeddedWebApplicationContext : Closing org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext@295c442d: startup date [Sat Feb 06 11:25:47 UTC 2016]; root of context hierarchy :check BUILD SUCCESSFUL Total time: 55.216 secs
As you can see execution of check task didn’t have to run tasks
classes again, because they were executed previously as part of
So TravisCI default configuration may seem reasonable for Gradle.
Merge install and script phase
But even if Gradle makes best effort to reduce overhead, the overhead is there. As Gradle Java Plugin tasks visualization showed earlier in the blog post, we can cover
check tasks with build task. So why should we run two Gradle processes in single build when single process execution is enough?
Also mentioned two default tasks may be fine for Continuous Integration workflow, but modern developers and teams doesn’t stop there. Every company that takes software development seriously wants to incorporate Continuous Delivery or Continuous Deployment. So we want to assemble, check, build and deploy all our assets in one build pipeline. So why would we want to run two Gradle processes?
Let’s do this:
install: echo "skip 'gradle assemble' step" script: gradle build --continue
In Travis Install phase we override default
assemble task and run full Gradle
build as one process. Let take a look at timings of such build for very same example project. Build link is here.
$ echo "skip './gradlew assemble' step" skip './gradlew assemble' step $ ./gradlew build --continue :compileJava :processResources :classes :findMainClass :jar :bootRepackage :assemble :compileTestJava :processTestResources UP-TO-DATE :testClasses :test 2016-01-31 20:19:37.202 INFO 2353 --- [ Thread-6] ationConfigEmbeddedWebApplicationContext : Closing org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext@46441da4: startup date [Sun Jan 31 20:19:24 UTC 2016]; root of context hierarchy :check :build BUILD SUCCESSFUL Total time: 36.926 secs
So even if Gradle tries to optimize tasks as much as possible, there is noticeable overhead running two separate Gradle processed instead of two.
Cache Gradle and NodeJS dependencies
Second hint to speed up TravisCI build for Gradle can be found in TravisCI docs itself. It involves Gradle dependencies caching.
As I mentioned at the beginning, project I work on also involves Gulp/NodeJS build. Therefore it make sense to put also NPM dependencies into TravisCI cache. Gradle integration with Gulp can be done via Gradle Gulp plugin. This plugin locates NPM global dependencies into
$HOME/.gradle/nodejs and local NPM depndencies into
So full of Gradle caching configuration for our build looks like this:
before_cache: - rm -f $HOME/.gradle/caches/modules-2/modules-2.lock cache: directories: - $HOME/.gradle/caches/ - $HOME/.gradle/wrapper/ - $HOME/.gradle/nodejs/ - node_modules
Don’t ask me why we need to remove lock in
before_cache section. That is taken from mentioned TravisCI docs. Rest of the configuration caches all Gradle and NPM dependencies between builds.
Here is link to build without this caching: 1 min 56 seconds
Here is link to build with this caching: 59 seconds. This example build doesn’t contain Gradle Gulp plugin, so saving from NPM dependencies caching are not included.