Fork me on GitHub
Built by Maven

surefire:test

Full name:

org.apache.maven.plugins:maven-surefire-plugin:3.5.4:test

Description:

Run tests using Surefire.

Attributes:

  • Requires a Maven project to be executed.
  • Requires dependency resolution of artifacts in scope: test.
  • The goal is thread-safe and supports parallel builds.
  • Binds by default to the lifecycle phase: test.

Optional Parameters

Name Type Since Description
<additionalClasspathDependencies> List<Dependency> 3.2 Additional Maven dependencies to be added to the test classpath at runtime. Each element supports the parametrization like documented in POM Reference: Dependencies.

Those dependencies are automatically collected (i.e. have their full dependency tree calculated) and then all underlying artifacts are resolved from the repository (including their transitive dependencies). Afterwards the resolved artifacts are filtered to only contain compile and runtime scoped ones and appended to the test classpath at runtime (after the ones from additionalClasspathElements).

The following differences to regular project dependency resolving apply:

  • The dependency management from the project is not taken into account.
  • Conflicts between the different items and the project dependencies are not resolved.
  • Only external dependencies (outside the current Maven reactor) are supported.

User Property: maven.test.additionalClasspathDependencies
<additionalClasspathElements> String[] 2.4 Additional elements to be appended to the test classpath at runtime. Each element must be a file system path to a JAR file or a directory containing classes. No wildcards are allowed here.
User Property: maven.test.additionalClasspath
<argLine> String 2.1 Arbitrary JVM options to set on the command line. Only effective for forked executions.

Since the Version 2.17 using an alternate syntax for argLine, @{...} allows late replacement of properties when the plugin is executed, so properties that have been modified by other plugins will be picked up correctly. See the Frequently Asked Questions page with more details:
http://maven.apache.org/surefire/maven-surefire-plugin/faq.html
http://maven.apache.org/surefire/maven-failsafe-plugin/faq.html
See also: forkCount
User Property: argLine
<childDelegation> boolean 2.1 When false it makes tests run using the standard classloader delegation instead of the default Maven isolated classloader. Only used when forking (forkCount is greater than zero).
Setting it to false helps with some problems caused by conflicts between xml parsers in the classpath and the Java 5 provider parser.
Default: false
User Property: childDelegation
<classesDirectory> File - The directory containing generated classes of the project being tested. This will be included after the test classes in the test classpath.
Default: ${project.build.outputDirectory}
<classpathDependencyExcludes> String[] 2.6 List of dependencies to exclude from the test classpath at runtime. Each item is passed as pattern to org.apache.maven.shared.artifact.filter.PatternIncludesArtifactFilter. The pattern is matched against the following artifact ids:
  • groupId:artifactId (Short ID)
  • groupId:artifactId:type:classifier (Dependency Conflict ID)
  • groupId:artifactId:type:classifier:version (Full ID)
The matching algorithm is described in detail in Advanced Artifact-Matching for the maven-assembly-plugin. This parameter behaves the same as the excludes pattern described there. The dependency matching is applied to the project dependency IDs (including transitive ones) after resolving, i.e. excluding one dependency will not exclude its transitive dependencies!
User Property: maven.test.dependency.excludes
<classpathDependencyScopeExclude> String 2.6 A dependency scope to exclude from the test classpath at runtime. The scope should be one of the scopes defined by org.apache.maven.artifact.Artifact. This includes the following:
  • compile - system, provided, compile
  • runtime - compile, runtime
  • compile+runtime - system, provided, compile, runtime
  • runtime+system - system, compile, runtime
  • test - system, provided, compile, runtime, test
<consoleOutputReporter> SurefireConsoleOutputReporter - No description.
<debugForkedProcess> String 2.4 Attach a debugger to the forked JVM. If set to "true", the process will suspend and wait for a debugger to attach on port 5005. If set to some other string, that string will be appended to the argLine, allowing you to configure arbitrary debuggability options (without overwriting the other options specified through the argLine parameter).
User Property: maven.surefire.debug
<dependenciesToScan> String[] 2.15 List of dependencies to scan for test classes to include in the test run. The child elements of this element must be <dependency> elements, and the contents of each of these elements must be a string which follows the general form:

groupId[:artifactId[:type[:classifier][:version]]]

The wildcard character * can be used within the sub parts of those composite identifiers to do glob-like pattern matching. The classifier may be omitted when matching dependencies without a classifier.

Examples:

  • group or, equivalently, group:*
  • g*p:*rtifac*
  • group:*:jar
  • group:artifact:*:1.0.0 (no classifier)
  • group:*:test-jar:tests
  • *:artifact:*:*:1.0.0

Since version 2.22.0 you can scan for test classes from a project dependency of your multi-module project.

In versions before 3.0.0-M4, only groupId:artifactId is supported.


User Property: dependenciesToScan
<disableXmlReport> Boolean 2.2
Deprecated.
Instead use disable within statelessTestsetReporter since of 3.0.0-M6.

Flag to disable the generation of report files in xml format. Deprecated since 3.0.0-M4.
User Property: disableXmlReport
<enableAssertions> boolean 2.3.1 By default, Surefire enables JVM assertions for the execution of your test cases. To disable the assertions, set this flag to "false".
Default: true
User Property: enableAssertions
<enableOutErrElements> boolean 3.3.1 Flag for including/excluding <system-out /> and <system-err /> elements for successfully passed tests in XML reports. Note that the default value may change to false is a future version.
Default: true
User Property: enableOutErrElements
<enableProcessChecker> String 3.0.0-M4 Since 3.0.0-M4 the process checkers are disabled. You can enable them namely by setting ping and native or all in this parameter.
The checker is useful in situations when you kill the build on a CI system and you want the Surefire forked JVM to kill the tests asap and free all handlers on the file system been previously used by the JVM and by the tests.
The ping should be safely used together with ZGC or Shenandoah Garbage Collector. Due to the ping relies on timing of the PING (triggered every 30 seconds), slow GCs may pause the timers and pretend that the parent process of the forked JVM does not exist.
The native is very fast checker. It is useful mechanism on Unix based systems, Linux distributions and Alpine/BusyBox Linux. See the JIRA SUREFIRE-1631 for Windows issues.
Another useful configuration parameter is forkedProcessTimeoutInSeconds.
See the Frequently Asked Questions page with more details:
http://maven.apache.org/surefire/maven-surefire-plugin/faq.html#kill-jvm
http://maven.apache.org/surefire/maven-failsafe-plugin/faq.html#kill-jvm
Example of use:
mvn test -Dsurefire.enableProcessChecker=all
User Property: surefire.enableProcessChecker
<enablePropertiesElement> boolean 3.3.1 Flag for including/excluding <properties /> element for successfully passed tests in XML reports.
Default: true
User Property: enablePropertiesElement
<encoding> String 3.0.0-M1 The character encoding scheme to be applied while generating test report files (see target/surefire-reports/yourTestName.txt). The report output files (*-out.txt) are encoded in UTF-8 if not set otherwise.
Default: ${project.reporting.outputEncoding}
User Property: surefire.encoding
<environmentVariables> Map<String,String> 2.1.3 Additional environment variables to set on the command line.
<excludeJUnit5Engines> String[] 3.0.0-M6 Provide the ID/s of an JUnit engine to be excluded in the test run.
User Property: surefire.excludeJUnit5Engines
<excludedEnvironmentVariables> String[] 3.0.0-M4 You can selectively exclude individual environment variables by enumerating their keys.
The environment is a system-dependent mapping from keys to values which is inherited from the Maven process to the forked Surefire processes. The keys must literally (case sensitive) match in order to exclude their environment variable.
Example to exclude three environment variables:
mvn test -Dsurefire.excludedEnvironmentVariables=ACME1,ACME2,ACME3
User Property: surefire.excludedEnvironmentVariables
<excludedGroups> String 2.2 (TestNG/JUnit47 provider with JUnit4.8+ only and JUnit5+ provider since 2.22.0) Excluded groups/categories/tags. Any methods/classes/etc with one of the groups/categories/tags specified in this list will specifically not be run.
For JUnit4, this parameter forces the use of the 4.7 provider. For JUnit5, this parameter forces the use of the JUnit platform provider.
This parameter is ignored if the suiteXmlFiles parameter is specified.
Since version 2.18.1 and JUnit 4.12, the @Category annotation type is automatically inherited from superclasses, see @java.lang.annotation.Inherited. Make sure that test class inheritance still makes sense together with @Category annotation of the JUnit 4.12 or higher appeared in superclass.
User Property: excludedGroups
<excludes> List<String> - A list of <exclude> elements specifying the tests (by pattern) that should be excluded in testing. When not specified and when the test parameter is not specified, the default excludes will be 

<excludes>
    <exclude>**/*$*</exclude>
</excludes>
(which excludes all inner classes).
This parameter is ignored if the TestNG suiteXmlFiles parameter is specified.
Each exclude item may also contain a comma-separated sub-list of items, which will be treated as multiple <exclude> entries.
Since 2.19 a complex syntax is supported in one parameter (JUnit 4, JUnit 4.7+, TestNG): 

<exclude>%regex[pkg.*Slow.*.class], Unstable*</exclude>

Notice that these values are relative to the directory containing generated test classes of the project being tested. This directory is declared by the parameter testClassesDirectory which defaults to the POM property ${project.build.testOutputDirectory}, typically src/test/java unless overridden.
User Property: surefire.excludes
<excludesFile> File 2.13 A file containing exclude patterns. Blank lines, or lines starting with # are ignored. If excludes are also specified, these patterns are appended. Example with path, simple and regex excludes: 

*/test/*
**/DontRunTest.*
%regex[.*Test.*|.*Not.*]
Since 3.0.0-M6, method filtering support is provided in the exclusions file as well, example: 

pkg.SomeTest#testMethod

User Property: surefire.excludesFile
<failIfNoSpecifiedTests> boolean 2.12 Set this to "true" to cause a failure if none of the tests specified in -Dtest=... are run. Defaults to "true".
Default: true
User Property: surefire.failIfNoSpecifiedTests
<failIfNoTests> boolean 2.4 Set this to "true" to cause a failure if there are no tests to run. Defaults to "false".
Default: false
User Property: failIfNoTests
<failOnFlakeCount> int 3.0.0-M6 Set this to a value greater than 0 to fail the whole test set if the cumulative number of flakes reaches this threshold. Set to 0 to allow an unlimited number of flakes.
Default: 0
User Property: surefire.failOnFlakeCount
<forkCount> String 2.14 Option to specify the number of VMs to fork in parallel in order to execute the tests. When terminated with "C", the number part is multiplied with the number of CPU cores. Floating point value are only accepted together with "C". If set to "0", no VM is forked and all tests are executed within the main process.

Example values: "1.5C", "4"

The system properties and the argLine of the forked processes may contain the place holder string ${surefire.forkNumber}, which is replaced with a fixed number for each of the parallel forks, ranging from 1 to the effective value of forkCount times the maximum number of parallel Surefire executions in maven parallel builds, i.e. the effective value of the -T command line argument of maven core.
Default: 1
User Property: forkCount
<forkNode> ForkNodeFactory 3.0.0-M5 This parameter configures the forked node. Currently, you can select the communication protocol, i.e. process pipes or TCP/IP sockets. The plugin uses process pipes by default which will be turned to TCP/IP in the version 3.0.0. Alternatively, you can implement your own factory and SPI.
See the documentation for more details:
https://maven.apache.org/plugins/maven-surefire-plugin/examples/process-communication.html
User Property: surefire.forkNode
<forkedProcessExitTimeoutInSeconds> int 2.20 Forked process is normally terminated without any significant delay after given tests have completed. If the particular tests started non-daemon Thread(s), the process hangs instead of been properly terminated by System.exit(). Use this parameter in order to determine the timeout of terminating the process. see the documentation: http://maven.apache.org/surefire/maven-surefire-plugin/examples/shutdown.html Turns to default fallback value of 30 seconds if negative integer.
Default: 30
User Property: surefire.exitTimeout
<forkedProcessTimeoutInSeconds> int 2.4 Kill the forked test process after a certain number of seconds. If set to 0, wait forever for the process, never timing out.
User Property: surefire.timeout
<groups> String 2.2 (TestNG/JUnit47 provider with JUnit4.8+ only and JUnit5+ provider since 2.22.0) Groups/categories/tags for this test. Only classes/methods/etc decorated with one of the groups/categories/tags specified here will be included in test run, if specified.
For JUnit4 tests, this parameter forces the use of the 4.7 provider. For JUnit5 tests, this parameter forces the use of the JUnit platform provider.
This parameter is ignored if the suiteXmlFiles parameter is specified.
Since version 2.18.1 and JUnit 4.12, the @Category annotation type is automatically inherited from superclasses, see @java.lang.annotation.Inherited. Make sure that test class inheritance still makes sense together with @Category annotation of the JUnit 4.12 or higher appeared in superclass.
User Property: groups
<includeJUnit5Engines> String[] 3.0.0-M6 Provide the ID/s of an JUnit engine to be included in the test run.
User Property: surefire.includeJUnit5Engines
<includes> List<String> - A list of <include> elements specifying the tests (by pattern) that should be included in testing. When not specified and when the test parameter is not specified, the default includes will be 

<includes>
    <include>**/Test*.java</include>
    <include>**/*Test.java</include>
    <include>**/*Tests.java</include>
    <include>**/*TestCase.java</include>
</includes>
Each include item may also contain a comma-separated sub-list of items, which will be treated as multiple <include> entries.
Since 2.19 a complex syntax is supported in one parameter (JUnit 4, JUnit 4.7+, TestNG): 

<include>%regex[.*[Cat|Dog].*], Basic????, !Unstable*</include>
<include>%regex[.*[Cat|Dog].*], !%regex[pkg.*Slow.*.class], pkg/**/*Fast*.java</include>

This parameter is ignored if the TestNG suiteXmlFiles parameter is specified.

Notice that these values are relative to the directory containing generated test classes of the project being tested. This directory is declared by the parameter testClassesDirectory which defaults to the POM property ${project.build.testOutputDirectory}, typically src/test/java unless overridden.
User Property: surefire.includes