A number of Kolab_* releases on files.kolab.org

Yesterday and this morning I released Kolab_FreeBusy-0.5.2, Kolab_Server-0.5.1, and Kolab_Storage-0.5.1. All of these are bug fix releases for the Kolab server.

With Horde 4 being deployed via pear.horde.org I cannot release the Kolab_* packages from the Horde 3 branch via that channel anymore. So you can expect updates to hit files.kolab.org exclusively. I don't like that too much but as Horde 3 was not yet released via PEAR and the Kolab_* releases are only targeted at the Kolab server this compromise makes sense.

Blogging via Jonah with the Kolab backend

This should be the first blog entry using Horde 4 Jonah with a Kolab backend. This obviously still needs a lot of improvement but it means I can finally store one additional data element I work with in my favourite storage backend. And export it to the cloud on demand.

Another intermediate Horde 4 release for Kolab-Server-2.2.4

Two days later than promised in the revised roadmap there is finally an "intermediate" release of Horde 4 for Kolab ready. With "intermediate" being the euphemistic word for "while it contains the dynamic calender that one is still pretty broken".

To install the release on a Kolab-Server-2.2.4 system the following commands should suffice:

wget http://files.pardus.de/horde4-20110224.sh
sh horde4-20110224.sh 

The usual warnings that went with the first Horde 4 release for Kolab apply for this release as well: Do not consider doing this on a productive server. This is just an early preview. And if you want to be able to see anything useful in the added calendar application you should ensure the user you log in with already has a calendar folder and has some data in it.

The state visible in the calendar frontend does not do the changes that happened in the backend any justice. But it can't be helped at the moment: While the Kolab backend for Horde is now largely complete the connection to the various Horde applications still need to be adapted to the changes in the backend.

Goodbye Hudson, hello Jenkins!

The Horde continuous integration setup switched from Hudson to Jenkins today. The switch was nothing I fancied because it meant fixing a number of CI setups that I created in the past months. However,.. with the core developer Kohsuke on the Jenkins team it didn't make much sense to stick to Hudson.

In order to make this switch feel at least somewhat productive I threw some additional updates into the pot. Here is a rough changelog:

• Hudson got switched to Jenkins so the former "hudson-install" repository was renamed to "jenkins-install"
• The PHP toolset for the continuous integration setup changed its name from "php-hudson-tools" to "php-ci-tools"
• phploc was added to the toolbox and most other tools in the set were upgraded.
• In addition I imported the latest changes from Sebastian Bergmanns Jenkins template into the horde-support repository. Thus we have two new graphics and some new plots for the jobs on http://ci.horde.org
.

Horde 4/Kolab road map update

The roadmap for Horde 4/Kolab that was published in November needs some adjustments. This represents an updated version:

• [06.12.10] COMPLETED - Horde 4 Portal + Mail
• [22.02.11] Horde 4 Calendar (alpha)
• [22.03.11] Horde 4 Addressbook, Notes, Tasks (beta)
• [12.04.11] Horde 4/Kolab Final

There were several factors that made the corrections necessary:

• The Horde/Kolab integration machinery needs some more work to allow using the new Horde 4 IMAP capabilites to the full extent in order to benefit from a major performance boost. This delays the release of the calendar.
• The number of pre-releases initially planned was just too high and needed to be reduced. This joins the releases of the other applications.
• The Horde development team decided on a final release date for Horde 4 so it is now possible to add a release date for the first stable Horde 4/Kolab release.
• Releases were switched to Tuesdays.

You are encouraged to watch the Horde commit stream. There are a lot of Kolab related commits flying by at the moment. More on the changes soon.

Ubuntu based Kolab-Server on EC2

During the last two years p@rdus published the new Kolab Server versions as ready-to-go Amazon EC2 images. These images were always based on Gentoo.

This base platform has recently been exchanged with Ubuntu as the underlying distribution. Since yesterday there is now a Kolab-Server-2.2.4 available as an image to be started and used for quick testing. You can expect all upcoming server version to be also based on Ubuntu. This does of course include the Kolab-Server-2.3 that will hopefully be released soon.

Instructions on how to use the Kolab Server images can be found in the Kolab wiki.

Horde release cycle

• Horde 1.0.0: 1999
• Horde 2.0: 2002
• Horde 3.0: 2004

... and finally Horde4 in 2011.

The gap between 3.0 and 4.0 has been too large. The Horde team tries to keep backward compatibility within a major version. Probably nobody tried running a recent Kronolith from 2010 on Horde 3.0 from 2004. There'd probably be some issues and limitations but they should be minor.

While this kind of long term support may be useful for some edge cases it can also impede development progress. A good example are the Kolab drivers in the currently released version: Kolab support within Horde started to improve in 2006. At that time Horde3 was already the active branch and there was some very basic Kolab functionality in there. So the new code had to keep those interfaces stable. Which basically meant twisting and bending it into code that would do just that but otherwise be really problematic. That code is still a reality in the current stable release from Horde3.

This and similar problems did not pass unnoticed though and changes to the release cycle have been discussed internally for a while already. With the release of Horde4 approaching it now makes sense to discuss such changes with the Horde community. The envisioned target is a time based release cycle.

The discussion started today on the Horde development mailing list. Feel free to listen in or to add your own comments if you want to influence the direction of our future release mode.

The Horde4 package mill for Debian

As mentioned before p@rdus intends to provide Horde4 packages for Debian beside the Kolab specific OpenPKG based build. The first steps of this process have been taken now and another Hudson based package mill has been created for the task. The new system bundles Horde4 packages for Debian after each upstream commit and lives here.

This is just considered a first step on the way to a full Debian release as there will be a fair amount of quality control required to get the packages fit for wider distribution. In addition the exact policy on how to handle PEAR packages on Debian is apparently still under debate. p@rdus will join this discussion now with a set of more than 50 PEAR based packages in tow.

The timing for this seems just right as there are still a few weeks left until Horde4 will see its first release. So there should be enough time to ensure that there are no major conflicts between PEAR based and Debian packaging.

An early Horde4 release for Kolab-Server-2.2.4

A while back p@rdus promised an early alpha of Horde4 for the Kolab Server for today. Which would have been followed by the release of the Horde4 mail client and the new mobile mail view in the following weeks. But as the Horde developer team has been quick to provide the new mobile view there is no need to delay the delivery of the mail client and the mobile portal any further. So you get a little bit more than promised today but as it is St Nicholas' Day that seems to be just fair.

Of course all the usual warnings apply. This is an extremely early packaging attempt. It is not well prepared and the installation procedure is flawed. There will be no upgrade path. Doing the following on your server has a high chance of damaging the system. Do not even think about installing the packages on a productive machine. In addition the packages just install the mail part of Horde and won't provide a lot of new functionality. And since there are tons of issues the stuff that is new is likely to be broken.

But those of you who have a spare machine with a throw-away Kolab Server 2.2.4 can try to install the packages using the following steps (this assumes your server is up and running):

wget http://files.pardus.de/horde4-20101206.sh
chmod u+x horde4-20101206.sh
./horde4-20101206.sh

You should be able to access the Horde4 installation at http://your-domain.com/client4.

Any kind of feedback is of course appreciated and will be used to improve the packages during the months to come. p@rdus will also continue testing and bug fixing of the packages while the final phases of the Horde4 development continue.

Now there are no December items of the roadmap left. So p@rdus will continue the next weeks with highlighting the new elements you can expect to find in Horde4. This should be accompanied with a few bug fix releases. In addition it is planned to provide you with a demo server in case you do not have a spare machine to test the new packages. So stay tuned!

Screenshots of the new Horde4 mobile portal

As mentioned before p@rdus currently sponsors development of draft mobile mail and mobile calendar portals for Android, iPad and iPhone. These portals are based on Horde4 and the recently released jQuery mobile framework.

The Horde developers have been far quicker than they promised themselves and so without further ado here are some early screenshots of the projects:

• Login screen
• Portal
• List of folders
• Mailbox view
• Message view
• Month view
• Day view
• Event view

Now p@rdus still needs to ensure that these screens go live on a Kolab server near you. Packaging is progressing well and chances are high that you can try the alpha version very soon.

Installing the Kolab Server on Ubuntu 10.04

Just a quick note on the required steps to get the Kolab server installed on an Ubuntu 10.04 server. The first step pulls the required dependencies to be able to compile the Kolab sources:

sudo apt-get update
sudo apt-get install bison flex gcc automake libtool autoconf tcl

And subsequently the server code gets downloaded and installed via the standard commands from the Kolab wiki:

mkdir /tmp/kolabtemp
cd /tmp/kolabtemp
wget -r -l1 -nd --no-parent http://files.kolab.org/server/release/kolab-server-2.2.4/sources/
gpg --keyserver wwwkeys.pgp.net --recv-key 5816791A
gpg --verify SHA1SUMS.sig
sha1sum -c SHA1SUMS
sudo sh install-kolab.sh 2>&1 | tee /root/kolab-install.log

The Horde4/Kolab package mill

p@rdus established another Hudson based continuous integration setup in preparation of the first experimental Horde4 packages in two weeks. The new system bundles Horde4 packages for the Kolab-Server-2.2.4/OpenPKG and lives here.

Packaging the Horde4 source code for a distribution is nearly trivial as the basic Horde packages are already continously integrated and packaged as PEAR modules. Because of that it is likely that p@rdus will establish a similar process for the native Kolab-Server/Debian.

With Kolab currently planning to move away from OpenPKG it is also possible that p@rdus makes the native Debian port the primary supported platform for Horde4 on Kolab in the future.

Horde continuous integration

With the increasing complexity of a huge codebase such as provided by Horde there is a significant need for quality control. As p@rdus accumulated a decent amount of experience with a variety of PHP QA tools during the past years it was only natural to apply this knowledge to the Horde codebase.

Many modules - including the Kolab_* packages - are now under constant surveillance of a continous integration server. The system is available for your inspection at ci.horde.org.

The service itself is run by Hudson - the standard CI server these days. The tools deployed on the server are a collection of the latest and greatest in the arena of quality control for PHP. PHPUnit certainly being the most important one as it delivers the framework for unit testing and code coverage analysis. In addition Hudson runs PEAR CodeSniffer (style check), PHP Depend (structural analysis), PHP Mess Detector, and the PHP Copy Paste Detector. On top of this the brand new PHP Code Browser integrates the analysis results into a view of the code.

With this system in place Horde is probably the only PHP based groupware client that can claim to validate its code base against current commercial grade software standards for PHP with each and every commit.

Horde4/Kolab road map

The current version of the Kolab web client saw its first real release on the Kolab Server in December 2007 and is based on Horde3. In 2008 p@rdus promised the switch to Horde4 for 2011.

In the past months the Horde development team has steadily progressed towards the first release of Horde4. With that being in sight now it finally makes sense to provide a detailed roadmap for Horde4/Kolab.

• [06.12.10] Horde4 Portal
• [13.12.10] Horde4 Mail
• [20.12.10] Horde4 Mail on jQuery mobile
• [07.02.11] Horde4 Calendar
• [14.02.11] Horde4 Calendar on jQuery mobile
• [28.02.11] Horde4 Addressbook
• [07.03.11] Horde4 Tasks
• [14.03.11] Horde4 Notes
• [28.03.11] Horde4/Kolab Beta

Both jQuery mobile oriented projects are financed by p@rdus. These will not be able to compete with the full feature set of the default view but should be an interesting draft of mobile apps based on the Horde applications.

You are encouraged to measure the reliability of p@rdus on the basis of this timeline.

A server is a server is just a server

In the process of getting the Horde4 packages ready for the Kolab Server the p@rdus development environment needed some love. Besides a few minor updates it remained virtually unchanged during the last two years.

The process started with an update of the underlying Gentoo base. Which took a few hours to compile. And of course there were a few problems associated. Which also took some time.

Subsequently the configuration via puppet didn't really work out. Some additional obscure issues. More time.

And ultimately the bundling and upload of the updated image didn't work. Even Google wasn't able to help this time.

All that is fine if the required time is available and if the stuff learned while fixing the issues means progress for yourself. But it is a fact: there is neither the time nor does fixing obscure issues align with the targets at hand. The server setup and upgrade must complete in minutes and not hours. It is "just a server" after all.

Sigh... I love Gentoo. I have been a Gentoo developer. I wrote a book about this distribution. Giving up on my developer status for Gentoo was hard. Giving up on Gentoo... damn. Nevertheless: Farewell.

PHP Hudson tools

A while ago p@rdus published an easy to use toolset for PHP quality control purposes. Many PHP software tools left the good old hacky script era of PHP some years ago and nowadays quality control becomes increasingly important.

The toolset is primarily intended to provide a basis for the continuous integration setup of the upcoming Horde4 release. It helps to enforce commercial grade quality guidelines for the Horde4 code base and allows to publish this to the outside.

Since the Kolab Server bases a large part of it's functionality on PHP packages it is obvious that the toolset is likely to also play its role in the quality control of the server.

As the name implies the toolset is primarily oriented towards providing the standard PHP QA toolset for the Continuous Integration server Hudson.

The content of the package is a collection of PEAR based packages and an ant script for automated installation. Thus you can basically use the resulting toolset for any other purpose in the field of PHP QA, too.

The included software in detail:

• PHPUnit
• PHP Depend
• PHP Documentor
• PEAR CodeSniffer
• PHP Mess Detector
• PHP Copy Paste Detector
• PHP Code Browser
• Horde Components

The toolset hopefully helps to get you up and running with PHP QA in no time.

Horde_Kolab_Server-0.1.0 and Horde_Kolab_Format-0.1.1 have been released!

The Horde project released the second PHP PEAR package representing a small subpart of the Kolab functionality within the Horde framework: Horde_Kolab_Server.

The package allows to access the Kolab LDAP database. Some examples are being given on a page in the Kolab wiki.

The package is the second in a series of five packages that will be released over the next few months. The full set of packages will allow you to easily deal with data stored on a Kolab server within your own web applications.

In addition the Horde_Kolab_Format-0.1.1 has been released. This is a bug fix release.

Kolab on Gmail

In the recent weeks the Kolab specific code in Horde has been significantly restructured to make it more developer-friendly. This cleanup also made it easy to add a small hack that allows you to run Horde with a standard IMAP server that provides no support for folder annotations as a back end.

The Kolab concept is based on IMAP folder annotations but so far the feature is only being provided by the Cyrus IMAP server. In addition the Kolab Server uses some patches in that area which means that you always need a full Kolab Server as a basis for Kolab specific development.

For Horde this means that the other developers have no chance to test the Kolab specific code sections even if they sometimes need to touch these areas. But installing a Kolab server is too much of a hurdle.

So I always wanted to allow running the Kolab code on a plain IMAP server. And ever since Gmail started providing IMAP access I considered the idea of Horde/Kolab on Gmail as a back end a nice toy thing.

Today the code that allows this went into Horde CVS. It is far from finished but it is sufficient to provide you with a demo installation.

You can use standard Gmail credentials there. But please be aware that I could grab these credentials! So you should only use a dummy account in your own best interest.

You'll certainly find many bugs or things that are not working yet but it is of course just a demonstration.

This line of coding is something I won't invest too much time into. It will never get any support from Kolab (as using annotations is the better solution) and I don't guarantee you that the format I'm using stays the same. So if you'd start using the code in a production environment the next upgrade might prevent access to the old data.

The main intention of this is to ease access to the code and allow more people to play with it.

The way things work at the moment is a special Kolab XML format for storing annotations. Each folder gets a single message in this format that carries the UID "1". This message holds all annotation values you'd usually store as folder metadata.

If you want to configure a Horde CVS installation specifically for Gmail you will still need to patch some IMAP parts. For other IMAP servers this might not be necessary.

First part of SyncML concluded

p@rdus completed coding on SyncML for Kolab a while ago and after some serious testing Univention uses the code in production.

In addition a short press release has been issued on several channels.

Horde_Kolab_Format-0.1.0 has been released!

The Horde project released a first PHP PEAR package representing a small subpart of the Kolab functionality within the Horde framework: Horde_Kolab_Format.

The package allows you to read and write the Kolab XML format. The XML data is converted from or to a data array within PHP. Some examples are being given on a page in the Kolab wiki.

The package is the first in a series of five packages that will be released over the next few months. The full set of packages will allow you to easily deal with data stored on a Kolab server within your own web applications.