Testing Output and Error Messages in PHPUnit

Introduction

#Testing Output

Sometimes you want to assert that the execution of a method, generates an expected output using either echo or print.

This functionality is provided by the PHPUnit\Framework\TestCase class using  PHP’s Output Buffering feature.

The example below illustrates the use  of  expectOutputString() method to set the expected output. If this expected output is not generated, the test will be counted as a failed test.

<?php
use PHPUnit\Framework\TestCase;

class OutputTest extends TestCase
{
    public function testExpectFooActualFoo()
    {
        $this->expectOutputString('foo');
        print 'foo';
    }

    public function testExpectBarActualBaz()
    {
        $this->expectOutputString('bar');
        print 'baz';
    }
}

?>

The test above, once executed, gives the following output.

$ phpunit OutputTest
PHPUnit |version|.0 by Sebastian Bergmann and contributors.

.F

Time: 0 seconds, Memory: 5.75Mb
There was 1 failure:

1) OutputTest::testExpectBarActualBaz
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'bar'
+'baz'

FAILURES!
Tests: 2, Assertions: 2, Failures: 1.

Some other methods can also be used to test outputs in PHPUnit tests, the table lists some of methods with their meanings.

It should be noted that in strict modes, a  test that emits output will fail.

#Error output

Whenever a test fails PHPUnit tries its best to provide you with as much context as possible that can help to identify what the problem was. The example below illustrates some of these error messages generated by PHPUnit in failed test instances.

Example illustrating an error output generated when an array comparison fails

<?php
use PHPUnit\Framework\TestCase;

class ArrayDiffTest extends TestCase
{
    public function testEquality()
    {
        $this->assertSame(
            [1, 2,  3, 4, 5, 6],
            [1, 2, 33, 4, 5, 6]
        );
    }
}
?>

When this test is executed, the message below given by PHPUnit tries to explain why the test failed.

$ phpunit ArrayDiffTest
PHPUnit |version|.0 by Sebastian Bergmann and contributors.

F

Time: 0 seconds, Memory: 5.25Mb

There was 1 failure:

1) ArrayDiffTest::testEquality
Failed asserting that two arrays are identical.
--- Expected
+++ Actual
@@ @@
 Array (
     0 => 1
     1 => 2
-    2 => 3
+    2 => 33
     3 => 4
     4 => 5
     5 => 6
 )

/home/sb/ArrayDiffTest.php:7

FAILURES!
Tests: 1, Assertions: 1, Failures: 1.

In this example only one of the array values differs and the other values are shown to provide context on where the error occurred.

If the generated output is long, PHPUnit will split it up and provide a few lines of context around every difference.

The example below illustrates an error output when an array comparison of a long array fails

<?php
use PHPUnit\Framework\TestCase;

class LongArrayDiffTest extends TestCase
{
    public function testEquality()
    {
        $this->assertSame(
            [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2,  3, 4, 5, 6],
            [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 33, 4, 5, 6]
        );
    }
}
?>

Below is the generated output for the above test

$ phpunit LongArrayDiffTest
PHPUnit |version|.0 by Sebastian Bergmann and contributors.

F

Time: 0 seconds, Memory: 5.25Mb

There was 1 failure:

1) LongArrayDiffTest::testEquality
Failed asserting that two arrays are identical.
--- Expected
+++ Actual
@@ @@
     11 => 0
     12 => 1
     13 => 2
-    14 => 3
+    14 => 33
     15 => 4
     16 => 5
     17 => 6
 )

/home/sb/LongArrayDiffTest.php:7

FAILURES!
Tests: 1, Assertions: 1, Failures: 1.

#Edge Cases

When a comparison fails PHPUnit creates textual representations of the input values and compares them. Due to that implementation a difference might show more problems than it actually exists.

This only happens when using assertEquals() or other ‘weak’ comparison functions on arrays or objects.

The example below illustrates an edge case in the using weak comparison.

<?php
use PHPUnit\Framework\TestCase;

class ArrayWeakComparisonTest extends TestCase
{
    public function testEquality()
    {
        $this->assertEquals(
            [1, 2, 3, 4, 5, 6],
            ['1', 2, 33, 4, 5, 6]
        );
    }
}
?>

The above test, illustrating weak comparison, gives the out below

$ phpunit ArrayWeakComparisonTest
PHPUnit |version|.0 by Sebastian Bergmann and contributors.

F

Time: 0 seconds, Memory: 5.25Mb

There was 1 failure:

1) ArrayWeakComparisonTest::testEquality
Failed asserting that two arrays are equal.
--- Expected
+++ Actual
@@ @@
 Array (
-    0 => 1
+    0 => '1'
     1 => 2
-    2 => 3
+    2 => 33
     3 => 4
     4 => 5
     5 => 6
 )

/home/sb/ArrayWeakComparisonTest.php:7

FAILURES!
Tests: 1, Assertions: 1, Failures: 1.

In this example the difference in the first index between 1 and '1' is reported even though assertEquals() considers the values to be equal.

Previous: Writing Tests for PHPUnit Using Data Providers.
Next: Writing Tests for Exceptions and Errors in PHPUnit.