Running test in parallel with surefire and displaying them properly with the TestNG Jenkins plugin

I’m running tests with parallel execution using surefire, and that all seems to work fine.

However, there’s only one testng-results.xml generated in the target/surefire-reports/ folder which … only contains the results of the test that ran last.

  • Error 500 during Sonar runner execution on Jenkins
  • Export Talend items from talend studio put on SVN and build it from Jenkins
  • Intermittent Build Failure on Jenkins
  • Jenkins windows slave No tty allocated failure
  • Jenkins - add chart to the job interface
  • Clearcase UCM Configuration with Jenkins to get latest baseline Integration code
  • I found an issue exactly for that reported for an older version of surefire, and it says “won’t fix” here:

    • http://t161727.apache-maven-issues.apacheforum.info/surefire-1018-surefire-reports-overwrite-each-other-whenusing-forking-t161727.html

    However, I doubt that I’m the first person on the planet who is trying to run unit tests in parallel with Jenkins and wants the results displayed properly using the TestNG Jenkins plugin, so I’m pretty sure there must be a solution for this, right?

    Here’s my surefire plugin config:

        <plugin>
          <groupId>org.apache.maven.plugins</groupId>
          <artifactId>maven-surefire-plugin</artifactId>
          <version>${basepom.plugin.surefire.version}</version>
          <configuration>
            <forkCount>2.5C</forkCount>
            <reuseForks>false</reuseForks>
          </configuration>
        </plugin>
    

    Here’s the link to the surefire config btw.:

    • http://maven.apache.org/surefire/maven-surefire-plugin/examples/fork-options-and-parallel-execution.html

  • Building Docker images using Jenkins results in “Unsupported protocol scheme found”
  • No such file: ‘pom.xml’ error in jenkins
  • How to attach files to Jenkins Pipeline notification
  • Jenkins Workflow Build Information
  • github pull request builder not working
  • How to get CTest results in Hudson / Jenkins
  • One Solution collect form web for “Running test in parallel with surefire and displaying them properly with the TestNG Jenkins plugin”

    I actually ended up finding a solution that worked for me.

    I think the problem is trying to do it with forkCount / reuseForks, so I’ve set those back to the default (you could also just skip those properties if you’re not trying to overwrite something from a base pom etc.).

    Instead I’ve used parallel and threadCount. Those however only apply to TestNG, but then, I also need it for TestNG (not JUnit).

    This makes tests run in parallel, but the testng-results.xml is generated correctly (without being overwritten by each test running in parallel).

    More details here:

    Here’s the plugin configuration I have now:

    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-surefire-plugin</artifactId>
      <configuration>
        <!-- those are the two default values, you can probably skip them -->
        <forkCount>1</forkCount>
        <reuseForks>true</reuseForks>
        <!-- that's what made it work -->
        <parallel>classes</parallel>
        <threadCount>10</threadCount>
      </configuration>
    </plugin>
    

    Obviously, the thread count could be lower or higher depending on what you want to do / what specs your server has, and you could change the settings depending on whether you want to run classes, or some other “level” in parallel.

    Also, you could configure parallel and threadCount together with any other properties in the suite file, if you choose to use a suite file.

    Git Baby is a git and github fan, let's start git clone.