Tuesday, June 21, 2011

Anatomy of a Horde test suite - III

Ready for the next item on the test suite agenda? This time the topic is Autoloading. We use a rather simple autoloading setup for most component test suites. It requires no additional setup and works out of the box if you run php AllTests.php.

That is already nice and allows running the complete test suite without further ado. But I must admit that I want more. My default work mode looks like this:

  • open test case (and modify it)
  • hit <f3> <f8> to run phpunit on this single test case
  • hit <f4> <f4> to to jump to the code line that produced the first error

This allows me to add a new test definition and immediately run it so that I can check for problems. And I don't need to execute the full AllTests.php to get feedback on the new test. So I'm annoyed every time I hit a test case that does not allow me to do that. A working autoloading setup is the key for that.

Luckily not only my own preferences make using an Autoload.php file in a test suite attractive. There are a number of reasons why such a file can be useful. The Wiki page for the Horde_Test component details them and this is a copy of the relevant section:

The Autoload.php file is not required in a test suite but it is strongly recommended that you use it. It's purpose is to setup PHP autoloading so that all tests in the test suite automatically have access to all the classes required for executing the tests. The reason why it is not mandatory is that Horde_Test_AllTests already loads a basic autoloading definition that works for most framework components.

This means that running php AllTests.php usually does not hit any autoloading problems. Running a single test case (e.g. phpunit Horde/Xyz/Unit/UnitTest.php) is a different matter though.

The *Test.php files do not extend Horde_Test_AllTests and thus there is nothing that would magically setup autoloading if you try to run such a test suite in isolation. And running single test cases can be quite convenient if the whole test suite would take a long time to execute. Using an Autoload.php file alongside the AllTests.php file is the recommended way to provide a single test case with autoloading and thus enable commands such as phpunit Horde/Xyz/Unit/UnitTest.php. In addition the file is helpful for any case where you need slightly more complex loading patterns or want to pull in special files manually.

Once you created an Autoload.php file for your test suite it will also be heeded by Horde/Test/AllTests.php. The latter will avoid the basic autoloading setup if it detects the presence of an Autoload.php file for the current test suite. That one will be loaded and is assumed to contain the required autoloading setup.

The content of Autoload.php

You should at least require the Autoload.php from Horde_Test in this file. This is also what Horde_Test_AllTests would do when choosing the simple autoloading setup.

require_once 'Horde/Test/Autoload.php';

It also makes sense to adapt the error reporting level to the same standards as required in the AllTests.php wrapper:

error_reporting(E_ALL | E_STRICT);

If you derive your test cases from a central test case definition you should load this one in Autoload.php as well:

/** Load the basic test definition */
require_once dirname(__FILE__) . '/TestCase.php';

Sometimes it makes sense to pull in the definition of test helpers that may be used throughout the test suite. They are usually not available via autoloading and need to be pulled in explicitely:

/** Load stub definitions */
require_once dirname(__FILE__) . '/Stub/ListQuery.php';
require_once dirname(__FILE__) . '/Stub/DataQuery.php';

Real world examples for Autoload.php helpers can be found in the Horde_Date and the Kolab_Storage components.

Within the test cases you only need to load the Autoload.php file which usually looks like this (and obviously depends on the position of the test case in the directory hierarchy of the test suite):

require_once dirname(__FILE__) . '/../Autoload.php';

You'll find additional background information on autoloading within test suite runs on the Wiki page for the Horde_Test component.

Monday, June 20, 2011

Horde4 debian packages - Second round

Nearly half a year passed since I started my first attempt at Debian packages for Horde4. Back then it was just about snapshots which I generated via a continuous integration setup. But Horde4 has been released now, there are packages and it is time for the real thing.

I did get a first set of packages installable today but this is just the initial draft. The main point about it is that the majority of it is automated.

If you want more details and the installation steps then I would suggest to follow my discussion with Mathieu Parent on pkg-php-pear@lists.alioth.debian.org. These were the mails exchanged so far (with the last two detailing my current status):

  1. On PEAR packaging (25.5., Mathieu)
  2. On PEAR packaging (7.6., Gunnar)
  3. On PEAR packaging (7.7., Mathieu)
  4. On PEAR packaging (17.7., Gunnar)
  5. On PEAR packaging (20.7., Gunnar)

There is not much more to say right now. Any testing and feedback is welcome and there is obviously still a lot of work ahead until this pops up in your default package channels. But it is at least on the horizon and the variant that I ship on files.pardus.de should become useable very soon.

Tuesday, June 14, 2011

Anatomy of a Horde test suite - II

This morning I completed the next step on the journey through Horde's test suites and added the description of the AllTests.php file to the wiki page. I am not going to copy the complete text here but instead focus on the use cases for this file as I still have a few question to the audience below.

AllTests.php is the only mandatory requirement for a Horde test suite. Everything else is optional but there has to be an AllTests.php file which serves as an entry point into the test suite.

This is the functionality expected from the file:

  1. It must collect all tests of the test suite.
  2. It must allow to retrieve all tests of the suite via Horde_Xyz_AllTests::suite().
  3. It must allow running the test suite via phpunit AllTests.php.
  4. It must allow running the test suite via php AllTests.php.

The Horde_Test package already delivers a boilerplate AllTests.php class in framework/Test/lib/Horde/Test/AllTests.php and deriving an AllTests.php for a standard test suite becomes rather simple. The full code for this is presented on the wiki page and you can also look at an example from our repository.

Now I wonder if the items listed above are in fact all the requirements we have for this file.

Requirements (1) and (2) are obvious as this is functionality needed for our horde/framework/bin/test_framework helper that runs all framework tests. Though I assume nobody uses this one on a regular basis at the moment.

But I noticed that (3) does not work out of the box with the current PHPUnit. This led to a pull request as it definitely should (and can) work.

I usually run the tests with a rather long command line that ultimately boils down to phpunit Horde_Xyz_AllTests AllTests.php which is tied to a shortcut in Emacs. As the Lisp code I use for that extracts the class name automatically I never noticed that a plain phpunit AllTests.php does not work.

So are most people using php AllTests.php? How do you run the test suites or would like to run them? Can I get some feedback on this (either here, on IRC or via tweet)?

Anything additional I missed about the requirements for the AllTests.php file?

Next in the series will be on autoloading which should allow me to also look at the problems we still have with that in the application components.

Thursday, June 09, 2011

The Horde newsletter again

Time flies and it is soon time for the next issue of the Horde newsletter. Which reminds me to post the link to the last one in case you missed it. You can subscribe to the monthly newsletter here.

As Jan already mentioned in his blog he gave Radio Tux an interview on Horde 4. I just checked by roughly translating half of it to English that I should be able to include a transcript of it in the next newsletter. It delivers a very good overview on all things Horde 4.

Anatomy of a Horde test suite - I

Just got issue 07/2011 of the the German Linux Magazine in the mailbox and on the final page there is this little abstract about 08/2011 saying...

"PHP Unit and Jenkins - There are two things guarding against programming errors: unit tests covering your code and continuous integration systems that automate the testing. The next issue will demonstrate this based on a real example from a PHP web project." [translated from German].

The "PHP web project" is actually named "Horde" and hm... I guess this means I have to write this thing - ;) . When agreeing to the article I immediately knew I wanted to combine it with an overview on how the Horde test suites are arranged. So far we have been lacking a summary in that area and it should help newcomers to the Horde project to get into testing mode as well.

My mind is currently fully tuned to unit testing and code quality and it is amazing how easy it is to write about this. The initial draft for the article already exceeded all limits when it comes to size. Though I got pretty positive feedback on it I will have to leave some stuff out. Those sections should make it to this blog instead so that I can link to it in the article.

Basically I will make this into a short series of blog entries on unit testing in Horde. I will include parts of the Horde_Test overview, personal musings, and stuff related to the article. Let's hope it is useful to some people out there.

Here we go with the introduction to the Horde_Test overview...


The Horde Project has always had high standards when it comes to code quality. Of course these standards evolved with time and also with the progress the PHP community made. The code from IMP-1.0.0 (1998) didn't come with unit tests. And somehow it lacked classes. And there is an awful lot of code mixed with HTML. Somehow this looks horribly like PHP3.

Oh, it was PHP3.

Of course PHP development changed over time and so did the Horde project. Nowadays each and every commit into our repository leads to the automatic execution of thousands of unit tests written by the Horde developers and they check our code for failures. Night and day our continuous integration server broadcasts the current test status to us in particular but also to anyone else interested.

With the release of Horde 4 the test suites of the Horde components available via our PEAR server all show some common patterns. There are certain Do's and Don'ts and a lot of playground in between. Often the Horde_Test component is involved. So it makes sense to associate the overview on the anatomy of Horde test suites with this particular module.

I must admit that I really like the way the Horde project approaches unit testing. There is no way we could be unit test purists which would be too extreme given the fact that the project already exists for more than a decade. But at the same time there was also no one complaining when testing entered the equation. It just felt like continuing to adhere to the quality standards that seem so familiar when it comes to Horde code.

So much for now. More technical stuff to follow soon...

Wednesday, June 01, 2011

Mixing stable Horde components with snapshots

One thing I completely love about Horde 4 being a component framework is the ability to have stable installations into which I can inject experimental snapshots of packages further developed, patched or hacked in some other way. Just an overview of one such installation:

PACKAGE                   VERSION              STATE
Horde_ActiveSync          1.0.0                stable
Horde_Alarm               1.0.1                stable
Horde_Argv                1.0.1                stable
Horde_Auth                1.0.3                stable
Horde_Autoloader          1.0.0                stable
Horde_Browser             1.0.0                stable
Horde_Cache               1.0.3                stable
Horde_Cli                 1.0.0                stable
Horde_Compress            1.0.1                stable
Horde_Constraint          1.0.0                stable
Horde_Controller          1.0.0                stable
Horde_Core                1.1.2dev201105300702 stable
Horde_Crypt               1.0.2                stable
Horde_Data                1.0.0                stable
Horde_DataTree            1.0.0                stable
Horde_Date                1.0.1                stable
Horde_Date_Parser         1.0.0                stable
Horde_Db                  1.0.1                stable
Horde_Editor              1.0.1                stable
Horde_Exception           1.0.1                stable
Horde_Feed                1.0.1dev201106011224 stable
Horde_Form                1.0.1                stable
Horde_Group               1.0.0                stable

How do you get these snapshots? Ensure you have the horde-component helper set up. Then enter the package you wish to snapshot (e.g. Horde_Core) and run:

horde-components snapshot

The snapshot package will be assembled in this directory, can be uploaded to your webserver and installed using:

pear upgrade --offline --force Horde_Core-1.1.2dev201105300702.tgz

What happens if you patched and deployed your package, sent the patch upstream, it gets accepted and a new package released?

pear upgrade horde/Horde_Core


Blogging to blogger.com via Horde_Feed

With Horde 4 being available as PEAR based components a lot of functionality that was hidden behind the Horde groupware applications have suddenly become available as building blocks for your own PHP based software. There is still a lot documentation that should get written. As one piece of the puzzle I will use my blog to post short tutorials on interesting things you might wish to do with the Horde PEAR packages.

Let's start with blog posts today...

Creating Atom blog posts becomes a rather simple task with the help of the Horde_Feed package. Install it via PEAR first:

pear channel-discover pear.horde.org
pear install horde/Horde_Feed

Start with creating an instance of Horde_Feed_Entry_Atom and populate it with content similar to the example below:

$entry = new Horde_Feed_Entry_Atom();
$entry->{'atom:title'} = 'Entry 1';
$entry->{'atom:title'}['type'] = 'text';
$entry->{'atom:content'} = '1.1';
$entry->{'atom:content'}['type'] = 'text';

The type could also be html or xhtml if the text in the corresponding field is not plain text. See the overview on the Atom format for that.

Now it is sufficient to post the entry with:

try {
} catch (Horde_Feed_Exception $e) {
    die('An error occurred posting: ' . $e->getMessage() . "\n");

In most situations this example will be somewhat too simplistic. Most sites will require you to authenticate before being able to add Atom entries. How such authentication information can be transmitted when posting the entry is detailed in the blogger.com example that comes with the Horde_Feed package. At the time I'm writing this the authentication is not yet available in the released package though - you will have to wait for the next release to hit pear.horde.org.