Monthly Archives: May 2014

Continous Integration for JavaScript multi module project

JavaScript multi module project – Continuous Integration



JavaScript multi module project

Few days ago, I wrote blog post about JavaScript multi module project with Grunt. This approach allows you to split application into various modules. But at the same time it allows you to create one deployment out of these modules. It was inspired by Maven multi module concept (Maven is build system from Java world).

But configuration of the project is just a half of the puzzle. Testing is must for me. And tests have to be executed. Execution must be automatic. So I needed to figure out Continuous Integration story for this project configuration.

Project parameters to consider

Let me quickly summarize base attributes of project configuration:

  • Two Github repositories/sub-projects
  • One main project called primediser
    • builds sub-projects
    • gathers deployment
    • runs Protractor end-to-end tests against deployment
    • gathers code coverage stats for client side code

Choosing CI server

First of all I had to pick CI server. Obviously it has to support chaining of builds (one build would be able to kick off another build). I have Java background and experience with Jenkins. So it would be natural choice. I bet that Jenkins can handle it quite easily. But I wanted to try workflow closer to majority of JavaScript community. So I tried TravisCI. Straight away not suitable because it’s not possible to chain builds.

Next I tried Relatively new service. Initially seemed not very feature reach. But closer look actually showed that this is the right choice. It provides these features necessary for me:

  • Temporary Docker Linux instance (is deleted after build) with pre-installed Node.JS
  • Web hook for remote triggering builds
  • Web interface for specifying Bash build commands
  • Web interface for specifying private keys or credentials needed during build

This combination of features turned to be very powerful. There are few unpolished problems (e.g. visualization of build process makes Firefox unresponsive, missing option for following build progress – so need to scroll manually), but I can live with these.

I was also able to execute Protractor tests against Sauce Labs from and measure the code coverage. Measuring Protractor tests code coverage is described in separate blog. Any change against sub-project triggers also build of main project. Does this sound interesting? Let me describe this CI configuration in details.


I wouldn’t dive into, Sauce Labs or Protractor basic usage. They were new for me and I easily and quickly came through their docs. Here is list of docs I used to put together this CI configuration.

Protractor – Souce Labs integration

Important part of this setup is Protractor integration with Sauce Labs. Sauce Labs provide Selenium server with WebDiver API for testing. Protractor uses Sauce Labs by default when you specify their credentials. So credentials are the only special configuration in test/protractor/protractorConf.js (bottom of the snippet). Other configuration was taken from grunt-protractor-coverage example. I am using this grunt plug-in for running Protractor tests and measuring code coverage.

// A reference configuration file.
exports.config = {
  // ----- What tests to run -----
  // Spec patterns are relative to the location of this config.
  specs: [

  // ----- Capabilities to be passed to the webdriver instance ----
  // For a full list of available capabilities, see
  // and
  capabilities: {
    'browserName': 'chrome'
    //  'browserName': 'firefox'
    //  'browserName': 'phantomjs'
  params: {
  // ----- More information for your tests ----
  // A base URL for your application under test. Calls to protractor.get()
  // with relative paths will be prepended with this.
  baseUrl: 'https://localhost:3000/',

  // Options to be passed to Jasmine-node.
  jasmineNodeOpts: {
    showColors: true, // Use colors in the command line report.
    isVerbose: true, // List all tests in the console
    includeStackTrace: true,
    defaultTimeoutInterval: 90000

  sauceUser: process.env.SAUCE_USERNAME,
  sauceKey: process.env.SAUCE_ACCESS_KEY

You may ask now, how can I use localhost in the configuration, when remote selenium server is used for testing. Good question. Sauce Labs provide very useful feature called Sauce Connect. It is a tunnel that emulates access to your machine from Selenium server. This is super useful when you need to bypass company firewall. It will be used later in main project CI configuration.

Setting up CI for sub-projects

The idea is that each change in sub-project would trigger a build of main project. That is why we need to copy a Build Hook of main project from Settings -> Repository section.primediser-hook

Main project kicked off by hitting the Web hook via wget Linux command from sub-project. As you can see in following picture, sub-project informs main project about a change and process its own build afterwards. doesn’t provide concurrent builds (not sure if this is limitation of open source projects only), so main project will wait for sub-project’s build to finish. After that main project is build.

CI Configuration of Main Project

So build of main project is now triggered from sub-projects. Main project configuration is slightly more complicated. I uses various commands:

  • First of all we need to specify Sauce Labs credentials in Environment Variables section
export SAUCE_USERNAME=********
export SAUCE_ACCESS_KEY=****************************
  • Download, extract and execute Sauce Connect tunnel for Linux. This will make accessible some ports on build image for Selenium testing. Notice that execution of the tunnel is done with & (ampersand). This means that tunnel execution is done in forked process and current console can continue executing build.
tar -xzvf sc-latest-linux.tar.gz
cd sc-4.3-linux/bin
./sc &
cd ../..
  • Now standard Node.JS/Grunt build workflow is executed. Downloading and installing all NPM dependencies creates enough time for Sauce Connect tunnel to start.
npm install phantomjs
time npm install -g protractor
time npm install -g grunt-cli bower
time npm install
  • Last command is closing the Sauce Labs tunnel after build, so that Docker build image wouldn’t hang.
kill %1

EDIT: Source Labs in the meantime changed Sauce Connect tunnel address and Linux version. Please refer to scripts above, because picture contains old URL and Linux version.


Such configured job now takes 9 minutes. limit is 15 minutes. Hopefully there will be enough space for all end-to-end tests I am going to create. If you are curious about build output/progress, take a look at my jobs below.

Project Links

Github repositories: jobs:

Multi module JavaScript project

Multi module JavaScript project with Grunt



Writing blog post how I managed to configure multi module JavaScript project with Grunt for my spare time project. It is using Protractor for end-to-end testing, but I believe that this multi module approach would be easily portable onto non-Angular stack.

In my spare time I work on pet project based on EAN stack (Express, Angular, Node.JS). (Project doesn’t need DB, that’s why MongoDB is missing from famous MEAN stack). Initial draft of the project was scaffolded by Yeoman with usage of angular-fullstack generator. Build is based on Grunt. Apart from that generator was using Grunt, I chose it over Gulp, because it would be probably more mature. Also Grunt vs Gulp battle seem to me similar as Maven vs Gradle one in Java world. I never had a need to move away from Maven. Also don’t like idea of creating  some custom algorithms in build system (bad Bash and Ant experience in the past). Grunt is similar to Maven in terms of configuration approach. I can very easily understand any build in Maven and expect similar build consistency from Grunt.

Nearly immediately I started to feel that Node.JS and Express back-end build concerns (Mocha based test suite) are pretty different to concerns of Angular front-end build (minification, Require.js optimalization, Karma based test suite, …). There was clear distinction between these two.

My main problem was having separate test suites. Karma makes generation of unit test code coverage very easy. Slightly tricky was setting up generation of code coverage stats for Mocha based server unit test suite. I managed to do that with Instanbul. So far so good. But when I wanted to send my stats to Coveralls server I could do that only for one suite. Coveralls support one stat per project. Combining stat files didn’t work nicely for me.

Multi module JavaScript project

So I felt a need for splitting the projects. As I’m developer with Java background, this situation reminded me Maven multi module project. In this concept you can have various separated projects/sub-modules that can evolve independently. These can be grouped/integrated together via special multi module project. This way you can build large enterprise and also modular application.

So I said to myself, that I wouldn’t give a try to this stack until I figure out how to configure multi module project. I separated main repository called primediser into two:

(Notice I created branch blog-2014-05-19-multi-module-project to to have code consistent with blog post)

So now I am able to set up Continuous Integration for each project and submit coverage stats separately. But how to integrate these two together? I created umbrella project, that doesn’t contain any JavaScript production code (similar to multi module project in Maven world). It will contain only Protractor E2E tests and grunt file for integration two modules.  This project is located in separate Github repository called primediser.

It uses various Grunt plugins and one conditional trick to do the integration:


This plugin is used to clone mentioned sub-projects from Github:

    gitclone: {
      cloneServer: {
        options: {
          repository: '<%= dirs.server %>',
          directory: '<%= dirs.server %>'
      cloneClient: {
        options: {
          repository: '<%= dirs.client %>',
          directory: '<%= dirs.client %>'

I could use grunt-shell plugin for this (I am using it anyway if you read further), but this one seems to be platform independent. Grunt-shell obviously isn’t.

Conditional cloning

Git can clone repository only once. Second attempt fails. Therefore we need to clone sub-projects only when they don’t exist. It is obviously up to developer to

  var cloneIfMissing = function (subTask) {
    var directory = grunt.config.get('gitclone')[subTask];
    var exists = fs.existsSync(directory);
    if (!exists) {'gitclone:' + subTask);

  grunt.registerTask('cloneSubprojects', function () {

My setup expects that developer would update sub-projects as needed. Also expects that Continuous Integration system that throws away entire workspace after the build. If you would be using Jenkins, you could use similar conditional trick in conjunction with gitupdate maven task that grunt-git provides.


After cloning, we need to install dependencies for both sub-projects. Unfortunately I didn’t find any platform independent way of doing this (Have to be honest I didn’t look very deeply though).

    shell: {
      npmInstallServer: {
        options: {
          stdout: true,
          stderr: true
        command: 'cd <%= dirs.server %> && npm install && cd ..'
      npmInstallClient: {
        options: {
          stdout: true,
          stderr: true
        command: 'cd <%= dirs.client %> && npm install && bower install && cd ..'


Next step is to kick off builds of sub-projects via grunt-hub plugin:

    hub: {
      client: {
        src: ['<%= dirs.client %>/Gruntfile.js'],
        tasks: ['build'],
      server: {
        src: ['<%= dirs.server %>/Gruntfile.js'],
        tasks: ['build'],

Tasks configurations

  grunt.registerTask('npmInstallSubprojects', [

  grunt.registerTask('buildSubprojects', [

  grunt.registerTask('coverage', [

  grunt.registerTask('default', [

As you can see there is one task I didn’t mention called coverage:

  • This one gatheres builds of sub-projects into dedicated sub-direcotory
  • Instrument the files
  • Run the Express back-end
  • Kicks Protractor end to end tests
  • and measures front end test coverage

Main driver in this task is grunt-protractor-coverage plugin. I already wrote blog post about this plugin. That blog post was done at stage when there wasn’t multi module configuration in place, so you can expect differences (There is also branch dedicated for that blog post also). Backbone should be the same though.

Maybe you are asking now: What’s Continuous Integration story? It is very nice and include and Sauce Labs. But this is topic for another blog post.

That is pretty much it. If you have any suggestions how can I enhance/simplify this build process, please let me know. I would also love to hear if similar stuff is easily doable in Gulp.


Step filtering: Skip certain packages while debugging in Eclipse



As Java developer, you are probably using a lot of frameworks. Spring or Hibernate can significantly reduce amount of boilerplate code. But inversion of control, dependency injections and proxies can be confusing during debugging. It is often annoying when debugger steps into third party library code. You realize that your code is wired via proxies and you have to add breakpoint into code you expect to hit.

But there is better way in Eclipse. Feature called Step Filtering. Very few my former and current colleagues using Eclipse were aware of it. Just go into Window -> Preferences and search for Step Filtering. Check Use Step Filters and specify packages/classes you want to skip during debugging.



Load implementors of an interface into list with Spring



Last week I wrote a blog post how to load complete inheritance tree of Spring beans into list. Similar feature can be used for autowiring all implementors of certain interface.

Let’s have this structure of Spring beans. Notice that Bear is abstract class, therefore it’s not a Spring bean. So we have three beas: Wolf, PolarBear and Grizzly.


Now let’s load implementors into list with constructor injection:

public class Nature {
	List<Runner> runners;

	public Nature(List<Runner> runners) {
		this.runners = runners;

	public void showRunners() {

Method showRunners is using Java 8 forEach method that consumes method reference.  This construct outputs loaded beans into console. You would find a lot of reading about these new Java 8 features these days.

Spring context is loaded by this main class:

public class Main {
	public static void main(String[] args) {
		AnnotationConfigApplicationContext context =
				new AnnotationConfigApplicationContext(SpringContext.class);

		Nature nature = context.getBean(Nature.class);

Console output:

PolarBear []
Wolf []
Grizzly []

This feature can be handy sometimes. Source code of this short example is on Github.


Generate UML class diagrams from code with ObjectAid



I’ve used ObjectAid Eclipse plugin for years. But very few of my colleagues were aware of it. This fact surprises me and therefore I would like to highlight it. It is very handy for generation of UML class diagrams from your code. It is also invaluable for analyzing of existing design.

One of the top UML tools is Enterprise Architect. But I was never happy with Enterprise Architect usability. I find it much easier to sketch class structure in Java and generate  UML class diagrams by ObjectAid. Advantage of this approach is that you have basic skeleton of the module. You can check-it in, so that team can start working on implementation.

Have to mention that it’s commercial product, but Class Diagram version is free.  For my UML class diagrams needs (design analyze, generating UML diagrams for documentation, modules design sketching) it was enough so far.

They provide also paid Sequence diagram and Diagram Add-On versions. To be honest I have never tried these. For sequence diagrams I was using only Enterprise Architect so far. I would be definitely pushing towards buying ObjectAid licenses if we wouldn’t have Enterprise Architect licenses already.

There’s no point for me to provide detailed description of it, because ObjectAid site contains short and explanatory overview. I suggest to start with One-Minute Introduction.

Everyone understand how inaccurate can UML class diagrams become. Keeping them up to date manually doesn’t make sense at all. ObjectAid fills this gap, because you can maintain your class diagrams up to date. But I had problems in the past when diagrams couldn’t handle renaming or moving of Java classes. Not sure if this is still problem of recent versions, because my diagrams are short lived. I can easily generate new diagram by just dragging appropriate classes into Class diagram editor. This is beauty of ObjectAid plugin.