Category Archives: Automation

Measuring code coverage by Protractor end-to-end tests

Was just setting up new JavaScript project based on Grunt. I scaffolded the project template by Yeoman with usage of angular-fullstack generator. I decided to try MEAN stack without MongoDB for my new project (DB isn’t needed).

Next step was integrating Require.JS and configuring measurement of code coverage on client and server by Instanbul. When was this all done I was wondering if it is possible to measure code coverage by Protractor end-to-end testing.

After quick search I found that Ryan Bridges recently released grunt-protractor-coverage plugin. Interesting coincidence. So I decided to try it and can confirm that it’s working fine with mentioned stack. Configuration was smooth and Ryan fixed small issue very promptly. It’s based on grunt-protractor-runner plugin.

I created separate Grunt configuration file just for this purpose not to mess around with normal build. I had also problems to run ‘makeReport’ task of grunt-istanbul plugin for two different directories (Mocha server side code coverage measurement is using same task).

So here is the Grunt flow. First we need to copy non JavaScript files into target directory. It needs to be in separate directory because JavaScript files will need to be instrumented.

copy: {
  coverageE2E: {
    files: [{
      expand: true,
      dot: true,
      cwd: '<%= dirs.app %>',
      dest: '<%= dirs.instrumentedE2E %>/app',
      src: [
        '*.{ico,png,txt}',
        '.htaccess',
        'bower_components/**/*',
        'images/**/*',
        'fonts/**/*',
        'views/**/*',
        'styles/**/*',
      ]
    }]
  },
},

Next step is instrumentation of the code. It is needed for gathering coverage stats. Each line is decorated by special instructions that helps during measurement. Pay attention to fact that we are instrumenting server and client side code. Instrumented code is placed into target directory represented by placeholder <%= dirs.instrumentedE2E %>.

instrument: {
  files: ['server/**/*.js', 'app/scripts/**/*.js'],
  options: {
    lazy: true,
    basePath: '<%= dirs.instrumentedE2E %>/'
  }
},

Next we start Express from target directory.

express: {
  options: {
    port: process.env.PORT || 9000
  },
  coverageE2E: {
    options: {
      script: '<%= dirs.instrumentedE2E %>/lib/server.js',
      debug: true
    }
  },
},

And the protractor_coverage task of grunt-protractor-coverage plugin. Configuration should be the same as for grunt-protractor-runner plugin.

protractor_coverage: {
  options: {
    configFile: 'test/protractor/protractorConf.js', // Default config file
    keepAlive: true, // If false, the grunt process stops when the test fails.
    noColor: false, // If true, protractor will not use colors in its output.
    coverageDir: '<%= dirs.instrumentedE2E %>',
    args: {}
  },
  chrome: {
    options: {
      args: {
        baseUrl: 'https://localhost:3000/',
        // Arguments passed to the command
        'browser': 'chrome'
      }
    }
  },
},

Last step is generation  of coverage report.

makeReport: {
  src: '<%= dirs.instrumentedE2E %>/*.json',
  options: {
    type: 'html',
    dir: '<%= dirs.coverageE2E %>/reports',
    print: 'detail'
  }
},

Finally, this is grunt task gathering all steps.

grunt.registerTask('default', [
  'clean:coverageE2E',
  'copy:coverageE2E',
  'instrument',
  'express:coverageE2E',
  'protractor_coverage:chrome',
  'makeReport'
]);

EDIT: Notice that following Github link was changed to branch, because project structure was significantly changed:

Source code for this project can be found on GitHub.

For running end to end Protractor test you have to have webdriver-manager running. See  Protractor documentation.

To install and start Selenium Webdriver:

npm install -g protractor
webdriver-manager update
webdriver-manager start

To install project dependencies:

npm install
bower install

To run this end-to-end testing with coverage measurement (webdriver-manager has to be running):

grunt --gruntfile Gruntfile-e2e.js

Trigger Continuous Delivery every GitHub commit

Crucial piece of puzzle when developing web application is Continuous Delivery. Testers or users can by early access to alpha version contribute to development process. Design,  requirements, architecture or performance problems can be catched much sooner.

I am going to show how to set up this process with usage of Maven and Jenkins. Target environment is hosted on Tomcat7. Source code is hosted on GitHub. Because I am type of developer that tries to avoid polling as much as possible, I am going to show how to trigger this process by GitHub’s cool feature called WebHooks.

1. Create Continuous Delivery job

Creation of Jenkins job and integrating it with Maven is very easy. Will quickly cover this:

  • Create it with “New Item” -> “
  • Set up GitHub URL in section “Source Code Management” (Authentication is not needed in my case, because my GitHub repository is public)
  • Skip section “Build Triggers” for now, will come back to this later.
  • Configure “Build” section with POM path and goals you are using for building your WAR file
  • Set up “Build Settings” -> “E-Mail Notification”

Save and try to run the Jenkins job. This is very common and basic Jenkins job configuration. Now we are about to set up deployment of WAR file into Tomcat7. But here comes dilemma into play. There are two very mature ways for deployment. I will cover both and let reader pick one.

a) Continuous Delivery with usage of tomcat7-maven-plugin

  • First of all we need to enable access into Tomcat7. Edit $CATALINA_HOME/conf/tomcat-users.xml (CATALINA_HOME is Tomcat’s home directory) and configure role and user as follows
<role rolename="manager-script"/>
<user username="deployer" password="===PASSWORD===" roles="manager-script"/>
  • Configure Tomcat7 credentials for Maven in configuration file settings.xml. This file is usually located in <user_home>/.m2.
<server>
	<id>tomcat-server-alpha</id>
	<username>deployer</username>
	<password>===PASSWORD===</password>
</server>
  • Set up tomcat7-maven-plugin into pom.xml
<plugin>
	<groupId>org.apache.tomcat.maven</groupId>
	<artifactId>tomcat7-maven-plugin</artifactId>
	<version>2.2</version>
	<configuration>
		<url>https://[JENKINS-URL]/manager/text</url>
		<server>tomcat-server-alpha</server>
		<path>/[TOMCAT-DEPLOY-PATH]</path>
	</configuration>
</plugin>
  • Lastly add additional Maven goal “tomcat7:redeploy” into Jenkins job

b) Continuous Delivery with usage of Jenkins Deploy plugin

  • Install Jenkins Deploy plugin
  • In Jenkins job that builds WAR, configure “Add post-build action” -> “Deploy ear/war to container”
jenkins-tomcat-deploy

2. Jenkins – GitHub integration

  • Blocking requirement here is to have Jenkins server accessible from web. If you can’t for whatever reason, you must stick with polling the source control in Jenkins.
  • Install GitHub plugin into Jenkins
  • Generate Personal access token in GitHub for Jenkins. This can be found under “Edit Your Profile” -> “Applications”
github-generate-new-token
  • Set up GitHub plugin to use generated token in Jenkins. You can find this section in “Manage Jenkins” -> “Configure System” -> “GitHub Web Hook”. Note that you don’t need to use password. API URL is “https://api.github.com”
jenkins-github-web-hook
  • Create WebHook in Github. Open repository -> “Settings” -> “Webhooks & Services” -> “Create Webhook”
  • Use Jenkins URL with suffix “/github-webhook”. Jenkins will replace automatically as you configure jobs, so that it’s not needed to create GitHub hook for each Jenkins job
  • After creation you can test webhook via three dots in “Recent Deliveries”. HTML error code “302 Found” means that it’s working fine (even when GitHub highlights it with exclamation mark).
github-webhook-creation
  • Finally enable GitHub triggering in Jenkins job
jenkins-job-triggers

That’s it. Github commit should cause deploy into Tomcat now.

References

  1. Jenkins GitHub plugin documentation
  2. Tomcat Maven plugin documentation

Share Dependencies Between Tycho and Plain Maven project

It is well known that RCP bundle is not the Maven’s best friend. Peacemaker between them is Maven’s plug-in Tycho. I was recently migrating my free time RCP based project into Maven Tycho. My goal was to share dependencies between normal Maven project and Tycho driven project. Plain maven project is needed for running TestNG tests (TestNG is not supported by Tycho).

There are two approaches how to automatically handle dependencies of Eclipse 4 RCP based project driven by Tycho:

  • manifest-first – this approach follows standard RCP/OSGi dependency mechanisms. All dependencies has to be also assembled into bundles. This is preferred and nicely described in tutorial suggested by Tycho’s website. It is hosted on github. But there are few problems complicating my goal:
    • It is needed to find existing bundle or create one for every dependency. Unhandy.
    • If I would use manifest-first approach, bundle dependencies wouldn’t be visible by non Tycho maven projects. I would need to have two set of dependencies. Redundancy.
  • pom-first – this is alternative approach when dependencies are packed into bundle by maven-bundle-plugin and this is used as dependency to project’s bundle. This approach is described here. Problem of this approach:
    • It is working only for Maven build and dependencies are not picked up by Eclipse.

Finally got the clue here (in the last approach) and came up with solution that is maybe not that pretty, but the dependencies are specified at one place and even Maven as well as Eclipse can pick them up. It is using some aspects both mentioned approaches.

I will show here only configuration crucial for my workaround. Whole project can be found on google code page. Let me describe maven projects structure:

artefact-inheritance

discography-organizer.parent

Contains various properties used by child poms. Is also used to run build of all sub-modules.

discography-organizer.dependencies.parent

Shared dependencies are specified here

discography-organizer.dependencies.bundle

Is doing two important steps:

1. maven-bundle-plugin to pack dependencies

<plugin>
  <groupId>org.apache.felix</groupId>
  <artifactId>maven-bundle-plugin</artifactId>
  <version>2.1.0</version>
  <extensions>true</extensions>
  <configuration>
    <instructions>
      <Export-Package>*; -split-package:=merge-last</Export-Package>
    </instructions>
  </configuration>
</plugin>

2. maven-antrun-plugin to copy packed jar with all dependencies into location where it can be picked up by RCP project

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-antrun-plugin</artifactId>
  <version>1.7</version>
  <executions>
    <execution>
      <phase>package</phase>
      <configuration>
        <target>
          <copy tofile="${temp.dependencies.folder}/dependencies.jar">
            <fileset dir="${project.build.directory}" includes="*.jar" />
          </copy>
        </target>
      </configuration>
      <goals>
        <goal>run</goal>
      </goals>
    </execution>
  </executions>
</plugin>

discography-organizer.bundle

Main RCP bundle project driven by Tycho. Expect Tycho call, there are two important configurations done:

1. In plugin.xml configuration on tab “Runtime” choose dependency jar created by discography-organizer.dependencies.bundledependency-plugin-xml

2. Pack main project classes into jar. This will be used by testing project discography-organizer.tests

Packing of application classes into jar:

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-antrun-plugin</artifactId>
  <version>1.7</version>
  <executions>
    <execution>
      <phase>package</phase>
      <configuration>
        <target>
          <jar destfile="${temp.dependencies.folder}/${bundle.name}-TESTING.jar" 
             basedir="${project.basedir}/bin"/>
        </target>
      </configuration>
      <goals>
        <goal>run</goal>
      </goals>
    </execution>
  </executions>
</plugin>

Pom file for this test dependency:
files-in-bundle

<groupId>sk.lkrnac.discorg</groupId>
<artifactId>discography-organizer.bundle</artifactId>
<version>TESTING</version>
<packaging>jar</packaging>

discography-organizer.tests

Normal maven project with tests. It is reading testing classes from jar created by discography-organizer.bundle

<dependency>
  <groupId>sk.lkrnac.discorg</groupId>
  <artifactId>${bundle.name}</artifactId>
  <version>TESTING</version>
  <scope>system</scope>
  <systemPath>${temp.dependencies.folder}/${bundle.name}-TESTING.jar</systemPath>
</dependency>

And that’s it. Dependencies needed by main bundle or test artifacts are packed into jar files. This workaround has side effects that I really don’t like. Dependencies are ugly packed in one jar for main bundle. But I can live with that because this approach enables me to combine features of Tycho and non-Tycho maven plug-ins. I am planning to use another Tycho’s features (automatic building and JUnit plug-in tests) and I have good start point for it.

If you can think of some simpler or more elegant solution please don’t hesitate to comment it. Thanks