Running test in parallel with surefire and displaying them properly with the TestNG Jenkins plugin

I’m running tests with parallel execution using surefire, and that all seems to work fine.

However, there’s only one testng-results.xml generated in the target/surefire-reports/ folder which … only contains the results of the test that ran last.

  • Jenkins gerrit trigger not fetching my change while building
  • Jenkins Pipeline Job with file parameter
  • Jenkins Git pull request ${sha1}
  • Variable substitution in Jenkins plugin
  • How to set environment variables in Jenkins?
  • Android Git Tag in Jenkins
  • I found an issue exactly for that reported for an older version of surefire, and it says “won’t fix” here:

    • http://t161727.apache-maven-issues.apacheforum.info/surefire-1018-surefire-reports-overwrite-each-other-whenusing-forking-t161727.html

    However, I doubt that I’m the first person on the planet who is trying to run unit tests in parallel with Jenkins and wants the results displayed properly using the TestNG Jenkins plugin, so I’m pretty sure there must be a solution for this, right?

    Here’s my surefire plugin config:

        <plugin>
          <groupId>org.apache.maven.plugins</groupId>
          <artifactId>maven-surefire-plugin</artifactId>
          <version>${basepom.plugin.surefire.version}</version>
          <configuration>
            <forkCount>2.5C</forkCount>
            <reuseForks>false</reuseForks>
          </configuration>
        </plugin>
    

    Here’s the link to the surefire config btw.:

    • http://maven.apache.org/surefire/maven-surefire-plugin/examples/fork-options-and-parallel-execution.html

  • Jenkins Sonar plugin suddenly stops working
  • Configure Jenkins EC2-Plugin with Groovy
  • Changing github credentials for Jenkins
  • Where is the PHPUnit report schema used by Jenkins?
  • Get Jenkins upstream jobs
  • Use e-mail address from Git commit to send Jenkins notifications
  • One Solution collect form web for “Running test in parallel with surefire and displaying them properly with the TestNG Jenkins plugin”

    I actually ended up finding a solution that worked for me.

    I think the problem is trying to do it with forkCount / reuseForks, so I’ve set those back to the default (you could also just skip those properties if you’re not trying to overwrite something from a base pom etc.).

    Instead I’ve used parallel and threadCount. Those however only apply to TestNG, but then, I also need it for TestNG (not JUnit).

    This makes tests run in parallel, but the testng-results.xml is generated correctly (without being overwritten by each test running in parallel).

    More details here:

    Here’s the plugin configuration I have now:

    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-surefire-plugin</artifactId>
      <configuration>
        <!-- those are the two default values, you can probably skip them -->
        <forkCount>1</forkCount>
        <reuseForks>true</reuseForks>
        <!-- that's what made it work -->
        <parallel>classes</parallel>
        <threadCount>10</threadCount>
      </configuration>
    </plugin>
    

    Obviously, the thread count could be lower or higher depending on what you want to do / what specs your server has, and you could change the settings depending on whether you want to run classes, or some other “level” in parallel.

    Also, you could configure parallel and threadCount together with any other properties in the suite file, if you choose to use a suite file.

    Git Baby is a git and github fan, let's start git clone.