We are proud to sponsor Chris Hartjes and Ed Finkler's Development Hell podcast series where they record their freewheeling, uncensored discussions on programming the web, so future generations can learn from their failures.
Read on to get the low down on different testing tools and their relative merits--check it out as Ed and Chris weep for the future, come to some interesting conclusions and get their hands dirty so you don't have to.
Chris: Okay, so today's topic is PHP testing
Ed: Word up
Chris: Now, Ed, I know that for the most part you are not a big fan of the mainstream PHP testing tools
Ed: Yes, that's true
Chris: So what is it that you don't like about them
Ed: I guess realistically my complaints are aimed at PHPUnit . It's very powerful and very complete from what I can tell, but I think it's difficult to pick up and I think that difficulty makes people less likely to use it. Because it's by far the best known testing tool, I think that tends to limit the use of unit testing, period, in PHP. That's not necessarily PHPUnit's fault per se. I just think it's the situation we're in. I think the documentation, the setup, and just obtaining PHPUnit is a challenge, particularly when compared to unit testing options I've seen in other languages. Python, for example, has a simple but effective unit testing library built into the core.
Chris: So, when you say "difficult to pick up", is it because tests look like this?
Chris: Maybe it's because I've worked with it a lot, all I see is some boilerplate and then a few statements that seem pretty intuitive to me.
Ed: I think boilerplate is part of the issue. I think that's intimidating. Tools can mitigate that to some extent, but I don't think it eliminates the problem entirely. I just don't think writing a simple test should be anything more than a couple lines of code. Then you can build upon that iteratively as you need. I think that approach of starting simply and building up your set of tests really helps you understand what's going on, and I think it makes testing a lot more accessible to people who haven't done it before. A lot of testing framework docs I see throw a ton of nomenclature out at the reader. I think if you don't already understand that nomenclature, you won't understand what's up.
Chris: So when you say 'nomenclature', you're talking about things like what exactly? Assertions and mocks?
Ed: Knowing how to mock that stuff up is pretty complex. In my experience the majority of people who work with PHP don't have a lot of formal training and even if they do, it often doesn't cover testing concepts. Like, what's a "unit?" What's an assertion? What's a mock or a stub?
Chris: I weep for the future, Ed. A unit is a small amount of code that you're trying to test
In PHP, that's usually one object, An assertion is simply a statement that "I am saying that the following is true", whatever that assertion happens to be. I do agree that there is lots of confusion about what a mock or a stub is so in my book I devote a chapter to explaining those things.
Ed: So I know what that stuff is (although I get confused about the diff between a mock and stub). But the real problem is that in order to write tests, you have to already know how to program, and that in itself is super-intimidating for people. PHP has a very shallow learning curve: the time between learning and becoming productive in some way is very short. That's certainly one of the reasons PHP is so popular. We need, I think, to mirror that in how we present testing, and make it easy to get into. It shouldn't be something that is terribly complex to set-up and do.
Chris: In that light, I understand the motivation to develop your own testing tools, but I still think PHPUnit is the way to go. So many people use it and there are so many resources available to learn it, that picking it up isn't as difficult as I think you're making it out to be. Alternately, I think the Behavior-Driven Development (BDD) model that Behat offers is appealing, and easier to pick up than the xUnit style. Behat combined with Mink is a solid alternative to PHPUnit.
Ed: If you are doing acceptance testing (meaning that you only care that the application as a whole is working) I don't think you can go wrong with being able to write tests that look like this:
If your mind doesn't align well with unit testing, then something like Behat is definitely the way to go. There's something neat about watching PHP run Behat which in turn opens up a browser and starts acting like a user and hopefully using your application correctly.
Ed: Ultimately, though, a lot of the problem with testing in PHP is that PHP's insane flexibility makes it super easy to write code that you cannot test. That and PHP is almost always working in concert with other systems, like a web server, so it can be tough to know what you can easily test inside the CLI and what you'd need to use a different approach.
To write testable code, you really have to be thinking about testing when you write your code. It takes a bit of time to get used to that, but I think it's very doable. In much the same way, it's taken us a long time to make security a first-order concern in PHP development, but I think we've done a decent job of that. We need to do that for testing as well.
Chris: If only, Ed. If only.