Managing Risky Tests in PHPUnit

Introduction

PHPUnit can be made to do some additional checks while it runs some tests on the background. Some of these additional checks are explored below

#Useless Tests

PHPUnit is by default strict about tests that do not test anything. This check can be disabled by using the `--don't-report-useless-tests` option on the command line or by setting `beStrictAboutTestsThatDoNotTestAnything="false"` in PHPUnit’s XML configuration file.

A test that does not perform an assertion will be marked as risky when this check is enabled. Expectations on mock objects or annotations such as @expectedException count as an assertion.

#Unintentionally Covered Code

PHPUnit can be strict about unintentionally covered code. This check can be enabled by using the `--strict-coverage` option on the command line or by setting `beStrictAboutCoversAnnotation="true"` in PHPUnit’s XML configuration file.

A test that is annotated with `@covers` and executes code that is not listed using a `@covers` or `@uses` annotation will be marked as risky when this check is enabled.

#Output During Test Execution

PHPUnit can be made to be strict about output during tests. This check can be enabled by using the `--disallow-test-output` option on the command line or by setting `beStrictAboutOutputDuringTests="true"` in PHPUnit’s XML configuration file.

A test that emits output, for instance by invoking print in either the test code or the tested code, will be marked as risky when this check is enabled.

#Test Execution Timeout

A time limit can be enforced for the execution of a test if the `PHP_Invoker` package is installed and the `pcntl` extension is available. The enforcing of this time limit can be enabled by using the `--enforce-time-limit` option on the command line or by setting `enforceTimeLimit="true"` in PHPUnit’s XML configuration file.

A test annotated with `@large` will fail if it takes longer than 60 seconds to execute. This timeout is configurable via the `timeoutForLargeTests` attribute in the XML configuration file.

A test annotated with @medium will fail if it takes longer than 10 seconds to execute. This timeout is configurable via the `timeoutForMediumTests` attribute in the XML configuration file.

A test annotated with @small will fail if it takes longer than 1 second to execute. This timeout is configurable via the `timeoutForSmallTests` attribute in the XML configuration file.

Tests need to be explicitly annotated by either `@small`, `@medium`, or `@large` to enable run time limits.

#Global State Manipulation

PHPUnit can be strict about tests that manipulate global state. This check can be enabled by using the `--strict-global-state` option on the command line or by setting `beStrictAboutChangesToGlobalState="true"` in PHPUnit’s XML configuration file.

Previous: Organizing Tests in PHPUnit: Best Practices for Test Suites.
Next: Handling Incomplete and Skipped Tests in PHPUnit.