Running test in parallel with surefire and displaying them properly with the TestNG Jenkins plugin

I’m running tests with parallel execution using surefire, and that all seems to work fine.

However, there’s only one testng-results.xml generated in the target/surefire-reports/ folder which … only contains the results of the test that ran last.

  • Jenkins hanging at “Fetching upstream changes from origin”
  • Dynamic Parameter in Jenkinsfile?
  • Jenkins Google Login Plugin Redirect URIs error
  • chromedriver fails on windows jenkins slave running in foreground
  • How to upload an APK from Jenkins/Hudson to Play Store?
  • jenkins slave can't get started
  • I found an issue exactly for that reported for an older version of surefire, and it says “won’t fix” here:

    • http://t161727.apache-maven-issues.apacheforum.info/surefire-1018-surefire-reports-overwrite-each-other-whenusing-forking-t161727.html

    However, I doubt that I’m the first person on the planet who is trying to run unit tests in parallel with Jenkins and wants the results displayed properly using the TestNG Jenkins plugin, so I’m pretty sure there must be a solution for this, right?

    Here’s my surefire plugin config:

        <plugin>
          <groupId>org.apache.maven.plugins</groupId>
          <artifactId>maven-surefire-plugin</artifactId>
          <version>${basepom.plugin.surefire.version}</version>
          <configuration>
            <forkCount>2.5C</forkCount>
            <reuseForks>false</reuseForks>
          </configuration>
        </plugin>
    

    Here’s the link to the surefire config btw.:

    • http://maven.apache.org/surefire/maven-surefire-plugin/examples/fork-options-and-parallel-execution.html

  • Execute windows batch command from Jenkins fails but runs fine in cmd.exe
  • Jenkins parameterized build changing default value
  • How to suppress source script lines echo in jenkins pipeline console output?
  • Jenkins Slave Environment Variable Refresh
  • Jenkins Failed to connect to gerrit repository (Windows installation)
  • Repositories URL in Jenkins + Git
  • One Solution collect form web for “Running test in parallel with surefire and displaying them properly with the TestNG Jenkins plugin”

    I actually ended up finding a solution that worked for me.

    I think the problem is trying to do it with forkCount / reuseForks, so I’ve set those back to the default (you could also just skip those properties if you’re not trying to overwrite something from a base pom etc.).

    Instead I’ve used parallel and threadCount. Those however only apply to TestNG, but then, I also need it for TestNG (not JUnit).

    This makes tests run in parallel, but the testng-results.xml is generated correctly (without being overwritten by each test running in parallel).

    More details here:

    Here’s the plugin configuration I have now:

    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-surefire-plugin</artifactId>
      <configuration>
        <!-- those are the two default values, you can probably skip them -->
        <forkCount>1</forkCount>
        <reuseForks>true</reuseForks>
        <!-- that's what made it work -->
        <parallel>classes</parallel>
        <threadCount>10</threadCount>
      </configuration>
    </plugin>
    

    Obviously, the thread count could be lower or higher depending on what you want to do / what specs your server has, and you could change the settings depending on whether you want to run classes, or some other “level” in parallel.

    Also, you could configure parallel and threadCount together with any other properties in the suite file, if you choose to use a suite file.

    Git Baby is a git and github fan, let's start git clone.