A Conversation About Testing in PHP

We are proud to sponsor Chris Hartjes and Ed Finkler's Development Hell podcast series where they record their freewheeling, uncensored discussions on programming the web, so future generations can learn from their failures.

Read on to get the low down on different testing tools and their relative merits--check it out as Ed and Chris weep for the future, come to some interesting conclusions and get their hands dirty so you don't have to.

To hear more from Chris and Ed tune in to their podcast, /dev/hell Ed and Chris had a little chat about testing in PHP.

Chris: Okay, so today's topic is PHP testing

Ed: Word up

Chris: Now, Ed, I know that for the most part you are not a big fan of the mainstream PHP testing tools

Ed: Yes, that's true

Chris: So what is it that you don't like about them

Ed: I guess realistically my complaints are aimed at PHPUnit . It's very powerful and very complete from what I can tell, but I think it's difficult to pick up and I think that difficulty makes people less likely to use it. Because it's by far the best known testing tool, I think that tends to limit the use of unit testing, period, in PHP. That's not necessarily PHPUnit's fault per se. I just think it's the situation we're in. I think the documentation, the setup, and just obtaining PHPUnit is a challenge, particularly when compared to unit testing options I've seen in other languages. Python, for example, has a simple but effective unit testing library built into the core.

Chris: So, when you say "difficult to pick up", is it because tests look like this?

<?php 
class Labels
{
    public $db;

    /**
     * @param GrumpyDb $db
     */
    public function__construct($db)
    {
        $this->db = $db;
    }

    /**
     * Turns label values like codingStandardsSuck into
     * CODING_STANDARDS_SUCK
     */
    public function screamingSnakeLabels()
    {
        $results = $db->query("SELECT name FROM labels");
        $labels = array();
        foreach ($results as $result) {
            $labels[] = $this->_camelToScreamingSnake($result);
        }
        return $labels;
    }

    /**
     * Method that takes a camelCase string into SCREAMING_SNAKE_CASE
     *
     * @param string $value
     */
    protected function _camelToScreamingSnake($value)
    {
        $result = preg_replace_callback(
            '/[A-Z]/',
            function ($match) {
                return "_" . strtolower($match[0]);
            },
            $value
        );
        return strtoupper($result);
    }
}

class DevhellTest extends PHPUnit_Framework_TestCase
{
    public function testShowEdHow()
    {
        $db = $this->getMockBuilder('Foo')
            ->disableOriginalConstructor()
            ->setMethods(array('query'))
            ->getMock();
        $db->expects($this->once())
            ->method('query')
            ->will($this->returnValue(array('devHell', 'camelCase'));
        $label = new Label($db);
        $expectedResults = array('DEV_HELL', 'CAMEL_CASE');
        $testResults = $label->screamingSnakeLabels();
        $this->assertEquals(
            $expectedResults,
            $testResults,
            "Labels were not correctly coverted to screaming snake case"
        );
    }
}

Chris: Maybe it's because I've worked with it a lot, all I see is some boilerplate and then a few statements that seem pretty intuitive to me.

Ed: I think boilerplate is part of the issue. I think that's intimidating. Tools can mitigate that to some extent, but I don't think it eliminates the problem entirely. I just don't think writing a simple test should be anything more than a couple lines of code. Then you can build upon that iteratively as you need. I think that approach of starting simply and building up your set of tests really helps you understand what's going on, and I think it makes testing a lot more accessible to people who haven't done it before. A lot of testing framework docs I see throw a ton of nomenclature out at the reader. I think if you don't already understand that nomenclature, you won't understand what's up.

Chris: So when you say 'nomenclature', you're talking about things like what exactly? Assertions and mocks?

Ed: Knowing how to mock that stuff up is pretty complex. In my experience the majority of people who work with PHP don't have a lot of formal training and even if they do, it often doesn't cover testing concepts. Like, what's a "unit?" What's an assertion? What's a mock or a stub?

Chris: I weep for the future, Ed. A unit is a small amount of code that you're trying to test

In PHP, that's usually one object, An assertion is simply a statement that "I am saying that the following is true", whatever that assertion happens to be. I do agree that there is lots of confusion about what a mock or a stub is so in my book I devote a chapter to explaining those things.

Ed: So I know what that stuff is (although I get confused about the diff between a mock and stub). But the real problem is that in order to write tests, you have to already know how to program, and that in itself is super-intimidating for people. PHP has a very shallow learning curve: the time between learning and becoming productive in some way is very short. That's certainly one of the reasons PHP is so popular. We need, I think, to mirror that in how we present testing, and make it easy to get into. It shouldn't be something that is terribly complex to set-up and do.

Chris: In that light, I understand the motivation to develop your own testing tools, but I still think PHPUnit is the way to go. So many people use it and there are so many resources available to learn it, that picking it up isn't as difficult as I think you're making it out to be. Alternately, I think the Behavior-Driven Development (BDD) model that Behat offers is appealing, and easier to pick up than the xUnit style. Behat combined with Mink is a solid alternative to PHPUnit.

Ed: If you are doing acceptance testing (meaning that you only care that the application as a whole is working) I don't think you can go wrong with being able to write tests that look like this:

Feature:
    Scenario: Main page loads
    Given I am on "/index.php"
    Then I should see "Lies I Told My Kids"

    Scenario: Empty form fields trigger errors
    Given I am on "/index.php"
    When I press "submitButton"
    Then I should see "You submitted an invalid e-mail address"

    Scenario: Missing description triggers errors
    Given I am on "/index.php"
    When I fill in "email" with "test@domain.com"
    And I press "submitButton"
    Then I should see "You submitted a blank description"

Chris: The Behat and Mink combo can let you create some very interesting acceptance tests, and it even provides you with tools that will tell you when you when you will have to write your own helpers to supplement what they can provide you. It took me a few days to figure out Behat's own way of doing things but once I did I was able to create some very interesting tests, even ones where JavaScript (long the bane of automated acceptance testing) was being used.

If your mind doesn't align well with unit testing, then something like Behat is definitely the way to go. There's something neat about watching PHP run Behat which in turn opens up a browser and starts acting like a user and hopefully using your application correctly.

Ed: Ultimately, though, a lot of the problem with testing in PHP is that PHP's insane flexibility makes it super easy to write code that you cannot test. That and PHP is almost always working in concert with other systems, like a web server, so it can be tough to know what you can easily test inside the CLI and what you'd need to use a different approach.

To write testable code, you really have to be thinking about testing when you write your code. It takes a bit of time to get used to that, but I think it's very doable. In much the same way, it's taken us a long time to make security a first-order concern in PHP development, but I think we've done a decent job of that. We need to do that for testing as well.

Chris: If only, Ed. If only.