tag:blogger.com,1999:blog-9984187800280448612024-03-14T07:08:49.245+01:00p@rdusp@rdus blog.Anonymoushttp://www.blogger.com/profile/05751776969338923853noreply@blogger.comBlogger94125tag:blogger.com,1999:blog-998418780028044861.post-37723650182192650962012-08-07T11:42:00.000+02:002012-08-07T11:42:16.675+02:00Modular git with "git subtree"<p>One thing I always disliked about the way we organized our Horde repository was the fact that we have all library modules and applications lumped together in a single git repository. Of course there are some good reasons for that type of monolithic repo. But for someone just interested in our (really powerful) IMAP library this is a drawback: The library is hidden somewhere between the other libraries and if you want to work on it you will nevertheless have to clone the whole repository. And there are other situations in which small, module specific repositories would make sense. So far I wasn't aware of a solution that would allow for a reasonable compromise. <br />
</p><p>Originally I only knew that <tt>git submodule</tt> would allow including additional git repositories into a master repository. This approach has some drawbacks though. We could construct the current horde repository out of a bunch of submodules. But the work flow within this master repository would be significantly more cumbersome as <tt>git submodule</tt> interferes with the default way of working with git.<br />
</p><h2><b>git subtree</b> to the rescue!<br />
</h2><p><tt>git subtree</tt> however seems to allow for the perfect solution: Separate subrepositories can co-exist with the monolithic master repository. And any commits to either of them can be exchanged between them. The stream of commits to the monolithic master can even be transmitted automatically to the splitted repositories. None of these steps seem to introduce any additional overhead to any of these repositories.<br />
</p><h2>Installing "git subtree"</h2><p>The <tt>subtree</tt> command has been added to the <tt>git-1.7.11</tt> release. But as many distributions do not yet offer this variant you can install the tool in a more hackish way if desired:<br />
</p><pre>cd /usr/lib/git-core/
wget https://raw.github.com/apenwarr/git-subtree/master/git-subtree.sh
mv git-subtree.sh git-subtree
chmod 755 git-subtree
</pre><h2>Replacing a "submodule" with a "subtree"</h2><p>A while ago I pulled the Jenkins installation procedures into our <tt>horde-support</tt> repository using <tt>git submodule</tt>. In order to give <tt>git subtree</tt> a first test run I replaced the Jenkins submodule by the subtree approach. The first step had to be the removal of the old submodule:<br />
</p><pre>git rm .gitmodules ci/jenkins
git commit -m "Remove the jenkins installation procedures as a submodule. Prepares for replacement by 'git subtree'"
</pre><p>Now I imported the repository previously registered via <tt>git submodule</tt> using <tt>git subtree</tt>:<br />
</p><pre>git subtree add --prefix=ci/jenkins --squash https://github.com/wrobel/jenkins-install.git master
</pre><p>This pulled the external repository into the current <tt>horde-support</tt> repository at <tt>prefix</tt> "ci/jenkins" and squashed all commits of the imported repository into a single commit. The imported code is now an equivalent citizen to the rest of the code in the repository - none of the standard git work flows are affected in any way.<br />
</p><p>Of course the interesting question is whether updates to this imported code can be merged back into the original repository. I commited a small change within the imported code:<br />
</p><pre>git commit -m "Update to jenkins-1.475" ci/jenkins/jenkins.mk
</pre><p>This change can indeed now be splitted into the subtree again and exported to the original archive:<br />
</p><pre>git subtree split --prefix=ci/jenkins --annotate="(horde-support) " d73edc4878c8.. --branch ci-jenkins
</pre><p>What happens here is that <tt>git subtree</tt> splits the path specified with the <tt>prefix</tt> option into a separate branch named <tt>ci-jenkins</tt>. It will prefix any commit transported into this branch with <tt>(horde-support)</tt> to indicate the origin of the commit. Usually the branch range given here (<tt>d73edc4878c8..</tt>) is unnecessary for the operation. But the code within <tt>ci/jenkins</tt> had been included as submodule before commit <tt>d73edc4878c8</tt>. This part of the history should not be imported into the splitted branch.<br />
</p><p>After the splitting operation created the new <tt>ci-jenkins</tt> branch in my repository it should be equivalent to our original, imported repository. Thus I was able to push back to it:<br />
</p><pre>git push git@github.com:wrobel/jenkins-install.git ci-jenkins:master
</pre><h2>Using "git subtree" for the horde repository</h2><p>Can the subtree approach be used to have both a monolithic horde repository as well as the small modular repositories at the same time? This would be the best of both worlds: While we develop in the monolithic horde repositories we allow other developers to also watch and patch single modules. If the commits from the monolithic repo can be transferred to the modular repos on a regular basis while we can also import patches the other way around without blowing up any of the associated git repos: I'd be really happy.<br />
</p><p>I admit that I didn't test the subtree approach large scale yet - but everything I have tested so far indicates that the situation detailed above can indeed be achieved and automated. <br />
</p><p>In order to automate the splitting of the monolithic repository into different modules I would use an intermediate git repository that handles the splitting within a post-receive hook. Any pushing to a branch of this repository would then update the same branches in the various splitted repositories. The core of the splitting procedure in the post-receive hook I established looks like this so far:<br />
</p><pre>git config --bool core.bare false
git checkout $short_refname
git reset --hard HEAD
if [ -z "`git branch | grep subtrees/$short_refname`" ]; then
git branch subtrees/$short_refname
fi
git checkout subtrees/$short_refname
git merge $short_refname
git subtree split --prefix=$subtree --annotate="(horde) " --branch subtrees/$module/$short_refname --rejoin
git push git@github.com:horde/$module.git subtrees/$module/$short_refname:$short_refname
git config --bool core.bare true
</pre><p>The procedure runs in a loop that walks through the different modules of the horde repository. <tt>$module</tt> refers to the current module, <tt>$subtree</tt> to the path corresponding to this module, and <tt>$short_refname</tt> indicates the branch that was pushed to.<br />
</p><p>I'll walk you through the different steps...<br />
</p><p>The remote repository needs to be in a state where you can push updates to a branch to it: it needs to be "bare". Updates to the branch currently checked out in the remote repository would otherwise be impossible. <tt>git subtree</tt> however requires us to work on a real checkout. So before initiating the splitting process the repository is marked as non-"bare":<br />
</p><pre>git config --bool core.bare false
</pre><p>And subsequently the branch that was pushed to is being checked out and resetted to HEAD - this is the basis for <tt>git subtree</tt> to work its magic.<br />
</p><pre>git checkout $short_refname
git reset --hard HEAD
</pre><p>The splitting procedure benefits from using a separate branch that remembers the previous splitting operations using merge commits. It would also work without such a branch but the subtree operation would always have to walk through each and every commit of the repository again - a waste of time. Just in case this subtree specific branch does not exist it will be created with the next step.<br />
</p><pre>if [ -z "`git branch | grep subtrees/$short_refname`" ]; then
git branch subtrees/$short_refname
fi
</pre><p>All updates that were just pushed into the remote repository are now being merged into the branch specifically created for the subtree operation. Here the original line of development and the subtree marker merges (which do not affect the code itself) live together in one branch.<br />
</p><pre>git checkout subtrees/$short_refname
git merge $short_refname
</pre><p>This prepared the stage for the splitting operation which can now analyze the incoming commits for any changes to the module currently handled.<br />
</p><pre>git subtree split --prefix=$subtree --annotate="(horde) " --branch subtrees/$module/$short_refname --rejoin
</pre><p>If there were changes that affected the current module they will be pushed to the corresponding git repository on github using the following command:<br />
</p><pre>git push git@github.com:horde/$module.git subtrees/$module/$short_refname:$short_refname
</pre><p>And finally the repository will be marked as bare again to prepare it for the next commit:<br />
</p><pre>git config --bool core.bare true
</pre><p>Of course of all this is still somewhat untested. It still has to be shown to work large scale - with about one hundred different Horde modules at the same time. But at least it looks very promising.<br />
</p></p>Anonymoushttp://www.blogger.com/profile/05751776969338923853noreply@blogger.com7tag:blogger.com,1999:blog-998418780028044861.post-81109524133609626482012-05-25T22:50:00.000+02:002012-05-25T22:50:11.763+02:00Creating your own Horde module - Step 1<p>We always had a <a href="http://wiki.horde.org/CreatingYourFirstModule">wiki page</a> on how to create your own Horde module. The steps outlined there allow you to get started quickly.<br />
</p><p>There was no specific reason though why those steps could not be automated. Based on the core script presented on the aforementioned wiki page I created a complete script that allows you to initiate a new Horde module with a single command.<br />
</p><pre>framework/devtools/horde-generate-module.php fancy "Gunnar Wrobel <wrobel@horde.org>"
</pre><p>The line above will create the stub for the new Horde application <tt>Fancy</tt>.<br />
</p><p>Once you created the new module this way you still need to register the new application in the Horde registry. This can be done within the directory <tt>config/registry.d</tt>. For details see the <tt>README</tt> file in that directory.<br />
</p><p>In addition you can set an icon that should represent your module. Refer to the instructions on the <a href="http://wiki.horde.org/CreatingYourFirstModule">wiki page</a> for that.<br />
</p>Anonymoushttp://www.blogger.com/profile/05751776969338923853noreply@blogger.com1tag:blogger.com,1999:blog-998418780028044861.post-45528881206386601942012-05-25T13:15:00.001+02:002012-05-25T13:15:24.381+02:00Horde at LinuxTag 2012<a href="http://farm8.staticflickr.com/7236/7266862462_d283daa5a8_m.jpg" onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}"><img alt="" border="0" height="155" src="http://farm8.staticflickr.com/7236/7266862462_d283daa5a8_m.jpg" style="float: right; margin: 0pt 0pt 10px 10px;" width="240" /></a> <br />
<p>We have a great time here at LinuxTag. Somehow the <a href="http://www.kde.org">KDE</a> project managed to get us placed right beside them again. Apparently the constant stream of gummy bears we fed them during CeBIT has turned into a serious additiction. By now they kind of depend on us. And of course we can't let them off the hook now.<br />
</p><p>Beside that we had a good amount of chats with people interested in the Horde groupware. And we received a lot of positive feedback about the new design which is great. Looks like everyone is looking forward to it.<br />
</p><p>And since the mornings are usually somewhat quieter Jan even has some time to work on the redesign and commit code. Can't wait for Horde 5.0 anymore...<br />
</p>Anonymoushttp://www.blogger.com/profile/05751776969338923853noreply@blogger.com0tag:blogger.com,1999:blog-998418780028044861.post-70430451895613344292012-05-23T23:04:00.001+02:002012-05-24T06:42:01.301+02:00Generating default preferences for your Horde installation<p>While updating our demo server at demo.horde.org I also wanted to get some kind of automatic reset for the demo user preferences. The aim was to allow each user testing our demo system to play around with the system in every way desired - while at the same time resetting to the defaults once the next user logs in. This way the second user wouldn't get an unpleasant surprise and turn away from Horde if the first user selected our beloved "Barbie" theme. By the way: this one will be dropped with the redesign - I hope nobody flames our mailing lists and starts a revolution to get this pink atrocity back :) <br />
</p><p>In order to achieve that I needed to move from SQL based preferences to Session based preferences. Using the latter would purge any preference changes at the end of the session. In order to provide the users with some sane and useful defaults though I also Wanted to copy the old values from the SQL database into the <tt>prefs.local.php</tt> files. This way I can set the preferences so that users are immediately greeted with a portal screen that demonstrates the twitter and facebook integration.<br />
</p><p>Converting SQL based preferences to <tt>prefs.local.php</tt> default settings is something you might be interested in for your installation as well. It allows you to set appropriate defaults for one initial test user and convert those into site-wide defaults for your installation.<br />
</p><p>How to do that without much hassle? <tt>horde-prefs</tt> to the rescue!<br />
</p><p><tt>horde-prefs</tt> is a small tool that allows printing, exporting, and importing of preference values stored in a backend.<br />
</p><p>In order to use the tool you need to define a configuration file for the specific backend you want to access. For the demo server this has been a SQL database:<br />
</p><pre><?php>
$conf['driver'] = 'Sql';
$conf['params']['db'] = new Horde_Db_Adapter_Mysql(
array(
'persistent' => false,
'username' => 'root',
'password' => 'PASSWORD',
'socket' => '/var/run/mysqld/mysqld.sock',
'protocol' => 'unix',
'database' => 'horde',
'charset' => 'utf-8',
'ssl' => true,
'splitread' => false,
'phptype' => 'mysql',
)
);
</pre><p>Now you can access the stored preference values by calling the <tt>horde-prefs</tt> tool. As <tt>horde-prefs</tt> is not stored under <tt>/usr/bin</tt> on the home server but uses a non-standard location I prefix the command with an explicit PHP include path:<br />
</p><pre>php -d include_path="/var/www/pear/php" /var/www/pear/horde-prefs config.php guest print horde
</pre><p>The previous command prints all preferences values stored for the <tt>guest</tt> user for the application <tt>horde</tt>:<br />
</p><pre>...
$_prefs['twitter']['value'] = "a:2:{s:3:"key";s:50:"183748047-vr6RLMOiYhfbfTH3qI8Lc8E32jF4UGGFbIxdkZyt";s:6:"secret";s:42:"i2DGXInJBW4kk3r2bvBdrxzUKMxL6AYS4u97WAJDyQ";}"
$_prefs['upgrade_tasks']['value'] = "a:7:{s:5:"horde";s:6:"4.0.13";s:9:"kronolith";s:5:"3.0.5";s:8:"imp_auth";s:5:"5.0.8";s:3:"imp";s:5:"5.0.8";s:5:"turba";s:5:"3.0.7";s:5:"whups";s:10:"2.0-ALPHA1";s:6:"gollem";s:10:"2.0-ALPHA2";}"
...
</pre><p>To convert this into a format suitable for the <tt>prefs.local.php</tt> files I simply used <tt>sed</tt>.<br />
</p><pre>php -d include_path="/var/www/pear/php" /var/www/pear/horde-prefs config.php guest print horde | sed -e "s/^\([^:]*\): \(.*\)/\$_prefs\['\1'\]\['value'\] = '\2';/" >> /var/www/config/prefs.local.php
</pre>Anonymoushttp://www.blogger.com/profile/05751776969338923853noreply@blogger.com0tag:blogger.com,1999:blog-998418780028044861.post-11204023098541754072012-05-22T22:30:00.000+02:002012-05-22T22:30:20.163+02:00Getting CalDAV and CardDAV server capabilities within Horde<p>Would you like to sponsor CalDAV and CardDAV server capabilities within Horde? This has been a Horde feature wish for quite some time now. We would have liked to have it for Horde 4.0 already and it looked more feasible to get it with Horde 5.0. But ultimately we had to delay it again as <a href="http://log.pardus.de/2012/05/sneak-peek-of-new-horde-5-user.html">the redesign</a> had the priority.<br />
</p><p>We decided however that we would like to tackle the DAV topic with top priority right after Horde 5.0 has been released. And we already have some pledges of support to get the feature completed - but we would need a few more to get complete CalDAV and CardDAV support within Horde as the whole thing is a larger story.<br />
</p><p>If you have some interest in the functionality it would be great if you could contact us via <a href="mailto:info@horde.org">info@horde.org</a>. Or you could use the "Donate" button on our <a href="http://www.horde.org">homepage</a>.<br />
</p><p>Thanks for you support!<br />
</p>Anonymoushttp://www.blogger.com/profile/05751776969338923853noreply@blogger.com6tag:blogger.com,1999:blog-998418780028044861.post-63379626731084664362012-05-11T14:38:00.000+02:002012-05-11T15:03:53.105+02:00A sneak peek of the new Horde 5 user interface<a href="http://www.horde.org/images/screenshots/webmail/horde5_webmail_draft.png" onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}"><img alt="" border="0" src="http://www.horde.org/images/screenshots/webmail/horde5_webmail_draft-thumb.jpg" style="float: right; margin: 0pt 0pt 10px 10px;" /></a> <br />
<br />
<p>To get an idea on how Horde 5 will look like: click <a href="http://www.horde.org/images/screenshots/webmail/horde5_webmail_draft.png">the link</a> or the image of this post.<br />
</p><p>Why does Horde 5 get a face lift? Simply because the current UI was mentioned often enough as an issue by many Horde users. And since the Horde 4 release had a very technical focus the switch from Horde 3 to Horde 4 last year did not help - it even degraded consistency between the applications. At the same time the competition does not sleep and there are more and more large installations that offer their user base two different webmails - one of them being Horde for the power users that feel they need a lot of features but that care less about the UI. Time to get our act together.<br />
</p><p>So what is the primary target of the redesign? First and foremost we want to unify the main user interfaces. At the moment we have the static application views, the dynamic webmailer, and the dynamic calender as the core parts. All looked somewhat different. These are the elements that we wish to give a consistent look. The special views such as the minimal webmailer or the smartphone UI will remain untouched.<br />
</p><p>We also hope the new design looks somewhat fresher than what we had before but please keep in mind that we are oriented towards people that use the interface for their daily work. We do not aim for a UI that looks like the last hype. It should be functional instead.<br />
</p><p>The Horde LLC has been the driving factor behind the redesign. At least financially. A subset of the Horde core developers started the LLC a while back as a contact point for people that want to pay for Horde support or feature development. A part of the money that such contracts pay goes to the developers dealing with the particular customer request. But another part of the money remains within the LLC. The idea is to use the latter to drive features that we consider to be important for Horde and its community. The redesign is the first project that has been financed this way. The Horde team tried finding designers interested in contributing to an Open Source project several times before. This was unsuccessful however and paying a designer for the work remained the only reasonable alternative.<br />
</p><p>We contracted <a href="http://www.no-agency.de">No agency</a> for the design. After several rounds of communication between them and all Horde developers we managed to end up with the draft displayed above. This has been converted to HTML and CSS this week and will be hammered into code during the next week by <a href="http://www.janschneider.de">Jan Schneider</a>. We do hope to present you with an alpha of Horde 5 - including the draft of the new design - on the 22nd May of 2012.<br />
</p><p>Feedback and comments - as usual - are welcome!<br />
</p>Anonymoushttp://www.blogger.com/profile/05751776969338923853noreply@blogger.com11tag:blogger.com,1999:blog-998418780028044861.post-40028251609264398202012-03-08T08:45:00.000+01:002012-03-08T08:45:42.514+01:00Horde becomes biggest KDE sponsor!<a href="http://www.flickr.com/photos/wrobel/6963667739" onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}"><img alt="" border="0" height="240" src="http://farm8.staticflickr.com/7056/6963667739_7fea9e08ba_m.jpg" style="float: right; margin: 0pt 0pt 10px 10px;" width="135" /></a> <br />
<br />
<p>Here at <a href="http://www.cebit.de">CeBIT</a> we support our friendly neighbor project with a constant and vital support of gummy bears. As anyone knows these sweet animals can make the difference between one line of brilliant code and a dreadful spaghetti mess. Thus it is probably hard to deny that this fruitful collaboration turned Horde into one of the biggest KDE sponsors.<br />
</p><br />
<p>That being said: KDE, I'm already here, breakfast is ready ;)<br />
</p><br />
<p>Beside having fun in the open source area the second day was already packed with people here at CeBIT. We had plenty of Horde users which provided kind feedback. Some of them we could surprise with features they didn't know about. Others were happy to hear that "GPL" really means that they can use the software and modify it without being harassed with a lawsuit afterwards.<br />
</p><br />
<p>We had Horde newbies as well as free software newbies. Explaining how free software can result in a revenue was the easy part. Explaining why we have no strong interest in a product for obfuscating our code so that there is a decent protection against people trying to find security holes was ... sigh ... harder.<br />
</p><br />
<p>The most fascinating thing was a company that installed Horde and wants to run it from -25°C to 65°C - like putting Horde to the extreme. There were other extremes involved and I omit the details but it is always fascinating what people do with free software.<br />
</p>Anonymoushttp://www.blogger.com/profile/05751776969338923853noreply@blogger.com0tag:blogger.com,1999:blog-998418780028044861.post-52925721996248157962012-03-07T09:06:00.000+01:002012-03-07T09:06:47.312+01:00First day on CeBIT<a href="http://www.flickr.com/photos/wrobel/6815009056" onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}"><img alt="" border="0" height="75" src="http://farm8.staticflickr.com/7186/6815009056_250ccaa6a6_m.jpg" style="float: right; margin: 0pt 0pt 10px 10px;" width="100" /></a> <br />
<br />
<p>Wonderful start at CeBIT yesterday. Meeting people with an interest in Horde face-to-face is a refreshing alternative to the work behind the screen at home or at work. The positive feedback helps tremendously in building up energy.<br />
</p><br />
<p>The most important part yesterday was talking to well known contacts, chatting about the progress of some projects which will hopefully result in a few interesting feature additions to Horde during this year.<br />
</p><br />
<p>I also visited <a href="http://www.owncloud.com">ownCloud</a> at their Univention booth to chat a bit about integrating the tool with a webmailer.<br />
</p><br />
<p>And last but not least it is always fun seeing Jan in person. Nothing against having a "distributed" type of project with people scattered in Germany and US. But it would be so much fun having all of you guys here. Can't really wait for the next hackathon ;)<br />
</p>Anonymoushttp://www.blogger.com/profile/05751776969338923853noreply@blogger.com1tag:blogger.com,1999:blog-998418780028044861.post-86775089583482961022012-02-22T23:21:00.001+01:002012-02-22T23:43:29.635+01:00Recap of the verification that there was no backdoor in the Horde 4 packages<a href="http://www.facebook.com/photo.php?pid=31648348&l=6d4e4091e8&id=1434467167" onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}"><img alt="" border="0" height="75" src="http://farm5.static.flickr.com/4125/5178360024_ac2cd0201c_m.jpg" style="float: right; margin: 0pt 0pt 10px 10px;" width="100" /></a> <br />
<br />
<p>When we discovered the successful attack on ftp.horde.org two weeks ago we were of course frantic to determine which packages had been affected in addition to the one Horde 3 archive Jan identified as modified initially.<br />
</p><p>For the Horde 4 packages we had no hashes to verify the file integrity though. While PEAR supports signing of packages via GPG that seems to be a feature which is virtually unused. For one thing not that many PHP based projects use PEAR packaging and in addition there is no way to automatically verify package integrity on the user side when installing via PEAR. So we also didn't consider signing our packages when switching to installing Horde via PEAR.<br />
</p><p>Obviously you gain a different perspective on that issue once a hacker implanted a backdoor in some of your packages. Of course we invested a lot of time into securing our infrastructure now to ensure that such an event never happens again. On our side the file integrity is constantly monitored now. But we will also have to investigate how we can improve the PEAR based installation procedure so that it also allows for the required amount of security on the user side.<br />
</p><p>But if we had no hashes how did we ensure the Horde 4 packages were indeed unmodified? Git to the rescue! As we tag all our releases it was a matter of creating a short script to automatically compare the current state of the packages on our PEAR server against the state we had within git.<br />
</p><p>Without further ado - here is the <a href="http://files.pardus.de/horde4-pear-forensic/pear-recovery.sh">script</a> I used:<br />
</p><p><pre>#!/bin/bash
git reset --hard HEAD
git clean -f -d
STAMP=`date +%y%m%d-%H%M`
mkdir ../diffs-$STAMP
mkdir -p ../validate-$STAMP/pear.horde.org
mkdir -p ../validate-$STAMP/rebuild
for package in `cat ../pear-recovery-packages.txt | grep -v ".tar$"`
do
TAG=${package/.tgz/}
TAG=${TAG,,}
PPATH=${package/-*/}
if [ "x${PPATH/Horde_*/}" == "x" ]; then
PPATH=framework/${PPATH/Horde_};
fi
if [ "x${PPATH/groupware*/}" == "x" ]; then
PPATH=bundles/$PPATH;
fi
if [ "x${PPATH/webmail*/}" == "x" ]; then
PPATH=bundles/$PPATH;
fi
PRESENT=`git tag -l $TAG`
if [ "x$PRESENT" == "x" ]; then
echo
echo "======================================================================"
echo "Tag $TAG for package $package is missing!"
echo "======================================================================"
echo
echo "$package: TAG MISSING" >> ../status-$STAMP
else
rm *.tgz
rm -rf ../validate-$STAMP/pear.horde.org/*
rm -rf ../validate-$STAMP/rebuild/*
GIT=`git checkout $TAG`
horde-components -z $PPATH --keep-version
if [ -e $package ]; then
cp *.tgz ../validate-$STAMP/pear.horde.org/
cp ../pear.horde.org/get/$package ../validate-$STAMP/rebuild/
tar -C ../validate-$STAMP/pear.horde.org/ -x -z -f ../validate-$STAMP/pear.horde.org/*.tgz
tar -C ../validate-$STAMP/rebuild/ -x -z -f ../validate-$STAMP/rebuild/*.tgz
DIFF=`diff -Naur ../validate-$STAMP/pear.horde.org/${package/.tgz/} ../validate-$STAMP/rebuild/${package/.tgz/}`
if [ "x$DIFF" != "x" ]; then
echo
echo "======================================================================"
echo "Diff for package $package detected!"
diff -Naur ../validate-$STAMP/pear.horde.org/${package/.tgz/} ../validate-$STAMP/rebuild/${package/.tgz/} > ..$
echo "======================================================================"
echo
echo "$package: DIFF" >> ../status-$STAMP
else
echo
echo "======================================================================"
echo "$package CLEAN!!!"
echo "======================================================================"
echo
echo "$package: CLEAN" >> ../status-$STAMP
fi
else
echo
echo "======================================================================"
echo "Failed rebuilding package $package!"
echo "======================================================================"
echo
echo "$package: FAILED REBUILDING" >> ../status-$STAMP
fi
fi
done
</pre></p><br />
<p>The script walks through the list of packages we had on the PEAR<br />
server, moves back in time within our git repository to the<br />
appropriate tag, rebundles the package, extracts both the old and the<br />
new package and compares them using diff.<br />
</p><br />
<p>The <a href="http://files.pardus.de/horde4-pear-forensic/status-120208-1514">resulting status list</a> looked quite okay. There were a few release glitches where the tag was not on the exact commit that was used for building the package. In that case usually <a href="http://files.pardus.de/horde4-pear-forensic/diffs-120208-1514/">a small diff</a> resulted. The list was rechecked manually for any malicious traces - Horde_Imap_Client-1.4.3 was the only one were the diff was large but that turned out to be a result from a mishap while releasing that package version. A few times the diff led to a problem with rebuilding the package (indicated by "FAILED REBUILDING"). The package.xml was broken in those cases and needed a fix that was only committed after the tagging. Here I compared the git state directly against the extracted package. There was only a single case ("Horde_ActiveSync-1.1.11") where the tag was missing which required manually identifying the corresponding commit to verify that state against the old package contents.<br />
</p><br />
<p>Once the script was established the analysis ran for about two hours after which we were at least somewhat relieved. Having a backdoor in some Horde 3 packages was already bad enough - having that in Horde 4 would have been even more disastrous.<br />
</p>Anonymoushttp://www.blogger.com/profile/05751776969338923853noreply@blogger.com2tag:blogger.com,1999:blog-998418780028044861.post-35034780833862520522012-02-02T23:31:00.000+01:002012-02-02T23:35:56.501+01:00Horde on CeBIT 2012<a href="http://www.facebook.com/photo.php?pid=31648348&l=6d4e4091e8&id=1434467167" onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}"><img alt="" border="0" height="75" src="http://farm5.static.flickr.com/4125/5178360024_ac2cd0201c_m.jpg" style="float: right; margin: 0pt 0pt 10px 10px;" width="100" /></a> <p>It is <a href="http://www.linux-magazin.de/Themengebiete/Special/Cebit-2012/Projekte/Cebit-Open-Source-2012-Projektpraesentation-Horde?special=Cebit%202012&category=66122">official</a> now: Horde gets a booth on <a href="http://www.cebit.de">CeBIT 2012</a>! Sponsored by <a href="http://www.linuxnewmedia.de/">Linux New Media</a> the company behind the <a href="http://www.linux-magazin.de/">Linux Magazine</a>. Thanks! Simply awesome!<br />
</p><p>Hope to see you there!<br />
</p>Anonymoushttp://www.blogger.com/profile/05751776969338923853noreply@blogger.com0tag:blogger.com,1999:blog-998418780028044861.post-29668543078757712262012-02-01T01:11:00.001+01:002012-02-01T01:11:33.292+01:00Wiki time<p>Tonight it has been wiki time. The <a href="http://en.wikipedia.org/wiki/Internet_Messaging_Program">article on IMP</a> from the English wikipedia was a mere stub and I expanded it with a little bit of history. I will hopefully find the time to continue this later. I recently did the same for the <a href="http://en.wikipedia.org/wiki/Horde_(software)">Horde article</a>.<br />
</p><p>In addition I updated the <a href="http://http://wiki.horde.org/Deployments">list of Horde deployments</a> in our <a href="http://wiki.horde.org">Horde wiki</a>. I also added a <a href="http://wiki.horde.org/Providers">list of Horde hosting providers</a> and a <a href="http://wiki.horde.org/Distributions">list of alternate installation methods for Horde</a>. I couldn't refrain from adding a short abstract on why we only offer PEAR with Horde 4 to the latter article. <br />
</p><p>Of course corrections and additions are very welcome!<br />
</p>Anonymoushttp://www.blogger.com/profile/05751776969338923853noreply@blogger.com0tag:blogger.com,1999:blog-998418780028044861.post-50767364720770515812012-01-30T23:59:00.001+01:002012-01-31T00:07:48.872+01:00Horde is calling you<p>Yes, it's late... and I should either sleep or try to work for a final hour before collapsing onto a pillow. But instead I feel like writing a few lines. And calling strangers in the US...<br />
</p><p>I don't know the people I call there. But it is fun, I train my english, and on top they are usually happy about my call. <br />
</p><p>What we talk about? I explain them what a provider is. And why I'm actually unable to help them. Which maybe helpful information in its own right.<br />
</p><p>Of course these people called the <a href="http://www.horde.org">Horde LLC</a> before. The Horde webmail is running on so many servers around the world that we get a constant stream of requests for help from users that mistake Horde for their service provider. Usually we get a handful of e-mails every day asking whether we could reset the password, restart the server, or in general just "HELP!!!". Once in a week the line <blockquote>I can't log in. My username is ABC, my password is XYZ.</blockquote>reminds us how easy phishing is.<br />
</p><p>E-Mails are simple: sending out a friendly response with a link to <a href="http://wiki.horde.org/ICantGetMyMail">one of our most successful wiki pages</a> is an easy thing to do.<br />
</p><p>Phone calls are a different matter though. The phone number for calling us in the US has been set up by <a href="http://mojolingo.com/">MojoLingo</a> based on <a href="http://adhearsion.com/">Adhearsion</a> a while back. And of course we get a certain number of "I can't get my mail!" calls as well. We usually didn't answer these until <a href="http://mojolingo.com/">MojoLingo</a> helped us with setting up Voip access so that calls into the US generate negligible costs.<br />
</p><p>So I can sit here, run SipDroid on my little droid and call into US when I need a break or have a minute to spare. A few more pleas for help that do not get lost unanswered.<br />
</p><p>In case you ever get called by a stranger with a weird German accent after your Horde webmail broke you probably didn't read this blog post. <br />
</p>Anonymoushttp://www.blogger.com/profile/05751776969338923853noreply@blogger.com2tag:blogger.com,1999:blog-998418780028044861.post-46235499745860491472011-12-13T23:19:00.000+01:002012-01-24T10:14:01.529+01:00The Horde release train<p><a href="http://www.horde.org">The Horde project</a> did push out 906 <a href="http://pear.horde.org">releases</a> in the last 8 months since the initial Horde 4 release. <a href="http://pear.php.net">PEAR</a> packages, better release management tools and <a href="http://ci.horde.org">continuous integration</a> seem to pay off. The stream of bug fixes and improvements available to the users has switched to high speed.<br />
</p><p>At the same the time the use of PEAR packages and automatic DB migrations has lowered the effort of updates on the administrator side to an absolute minimum.<br />
</p><p><pre>pear upgrade -c horde</pre></p><p>And few clicks later you are up-to-date.<br />
</p><p>The code quality of the basic PEAR tools is an entirely different matter... BUT it is just soo damn cool anyway ;)<br />
</p>Anonymoushttp://www.blogger.com/profile/05751776969338923853noreply@blogger.com0tag:blogger.com,1999:blog-998418780028044861.post-5517068410396525922011-11-13T22:21:00.001+01:002011-11-13T22:26:15.184+01:00Accessing PEAR server information with PHP<p>The number of PEAR servers has increased dramatically in recent times. Maybe due to the availability of <a href="http://pirum.sensiolabs.org/">Pirum</a> - a simple PEAR server. Whatever the cause: there are plenty of package lists and package datasets available on the net these days. Accessing this data is not always easy though. </p><p>PEAR servers provide a well defined REST API that is usually queried using the PEAR toolset available via the basic <a href="http://pear.php.net/package/PEAR">PEAR</a> package. That package however does not provide anything that I would consider to be a decent developers API. </p><p>So let me provide you with an alternative that is available via the <a href="http://www.horde.org/libraries">Horde framework libraries</a>: The <a href="http://www.horde.org/libraries/Horde_Pear">Horde_Pear</a> library. </p><p>The package comes with the <tt>Horde_Pear_Remote</tt> class wich provides you with high-level access to the REST interface of a PEAR server. </p><p>Creating an instance of this class without providing any arguments to the constructor will allow you to access the PEAR server at <a href="http://pear.horde.org">pear.horde.org</a>. </p><p><pre>$pear = new Horde_Pear_Remote();
print(join("\n", $pear->listPackages()));
Horde_ActiveSync
Horde_Alarm
Horde_Argv
Horde_Auth
Horde_Autoloader
...</pre></p><p>Alternative servers can be specified in the first argument to the constructor:</p><p><pre class="brush: php; toolbar: false;">$pear = new Horde_Pear_Remote('pear.phpunit.de');
print(join("\n", $pear->listPackages()));
DbUnit
File_Iterator
Object_Freezer
PHPUnit
...</pre></p></p><p>The full set of the functionality provided by <tt>Horde_Pear_Remote</tt> is detailed on <a href="http://www.horde.org//libraries/Horde_Pear/docs/REMOTE_PEAR_SERVER">our website</a>. </p>Anonymoushttp://www.blogger.com/profile/05751776969338923853noreply@blogger.com0tag:blogger.com,1999:blog-998418780028044861.post-29224881498394735582011-11-04T12:56:00.001+01:002011-11-04T12:58:26.070+01:00The library section of the Horde website supports component documentation now.<p>Our new <a href="http://www.horde.org/libraries">libraries</a> section has been started a while ago to further push the PHP components we offer. Now this section supports publishing the component documentation as well. You can take a look at the <a href="http://www.horde.org/libraries/Horde_Cli_Modular/docs">documentation of the Cli_Modular</a> package for example. In fact: it is pretty much the only package that fully uses the new system already. So there is still work ahead. </p><p><b> If you use any of the components: We are grateful if you start writing a bit of documenation or describe some examples on how to successfully use the package. It will help us and others tremendously. Thanks! </b> </p><p>Let me try to explain how the system is currently intended to work - feedback and critical comments of course welcome: </p><p><ol><li> When creating new documentation for one of the Horde components you start a new section on <a href="http://wiki.horde.org/Doc/Dev">the developer documentation of our wiki</a>. See the <a href="http://wiki.horde.org/Doc/Dev/HordeCliModular">Horde_Cli_Modular</a> link below <b>Library components</b> for example. Please note: It is not mandatory to write the documentation in the wiki. You can have static documentation files within a component as well. This is just meant as an option for those situations where it makes sense to work on documentation files together with people that do not have direct git commit access. Hopefully there will be many component consumers interested in updating and fixing the documentation. </li>
<li> How you structure that new section is up to you. For a simple package you might wish to keep all documentation in a single file but for the more complex one it might make sense to have several files. In that case the new section page should probably just be a link list to the various wiki pages that document the component. In case of <a href="http://wiki.horde.org/Doc/Dev/HordeCliModular">Horde_Cli_Modular</a> I used a single page. </li>
<li> Once the documentation has been written in the wiki format it is time to download it into the <tt>doc</tt> directory of the component. The <a href="http://wiki.horde.org/Doc/Dev/Component/Components">horde-components</a> helper is what you should use for that operation. The tool expects to find a <a href="https://github.com/horde/horde/blob/master/framework/Cli_Modular/doc/Horde/Cli/Modular/DOCS_ORIGIN">DOCS_ORIGIN</a> file within the <tt>doc</tt> or <tt>docs</tt> (for the applications) directory of the component. This file must conform to the <a href="http://docutils.sourceforge.net/rst.html">reStructuredText</a> format and map remote URLs to local file paths relative to the component root. The <a href="https://github.com/horde/horde/blob/master/framework/Cli_Modular/doc/Horde/Cli/Modular/DOCS_ORIGIN">DOCS_ORIGIN from Cli_Modular</a> links <a href="http://wiki.horde.org/Doc/Dev/HordeCliModular">the wiki page</a> exported as <a href="http://docutils.sourceforge.net/rst.html">reStructuredText</a> to the path <a href="https://github.com/horde/horde/blob/master/framework/Cli_Modular/doc/Horde/Cli/Modular/README">doc/Horde/Cli/Modular/README</a>. The links can of course point anywhere so you are in principle free to use any remote source. But you should ensure the page provides readable <a href="http://docutils.sourceforge.net/rst.html">reStructuredText</a>. </li>
<li> Now you can run <tt>horde-components fetchdocs</tt> and it should fetch the URLs into the respective local files. You can then run <tt>horde-components update</tt> to refresh the file list in the <tt>package.xml</tt> file if the documentation files were not included before. </li>
<li> Once a component was released we can now run something like this: <tt>horde-components Horde_Cli_Modular webdocs -D ~/git/horde-web/ --html-generator=git/horde-support/maintainer-tools/docutils/html.py --allow-remote</tt> in order to update <a href="http://www.horde.org/libraries/Horde_Cli_Modular/docs">the documentation on the website</a>. Right now this does not happen automatically during a release but I plan to add this soon. </li>
</ol></p><p>There are of course still a few problems with this approach: </p><p><ul><li> The <a href="http://docutils.sourceforge.net/rst.html">reStructuredText</a> exporter of our <a href="http://www.horde.org/apps/wicked">wiki engine "Wicked"</a> is not yet complete. You may experience problems with the export if you use constructs it does not know yet. Please ping me if you experience problems with the export. </li>
<li> The wiki syntax and the syntax required by <a href="http://docutils.sourceforge.net/rst.html">reStructuredText</a> is not 100% aligned. You can write a wiki page that looks just fine but leads to errors or formatting problems in the <a href="http://docutils.sourceforge.net/rst.html">reStructuredText</a> export. After creating / updating a new wiki documentation page it make sense to run the <tt>horde-components fetchdocs</tt>/<tt>horde-components webdocs</tt> sequence to check for problems. </li>
<li> We still have some components with documentation files not in <a href="http://docutils.sourceforge.net/rst.html">reStructuredText</a> format. These will result in non-existing links on our website for now but I plan to fix the few faulty packages soon. </li>
</ul></p>Anonymoushttp://www.blogger.com/profile/05751776969338923853noreply@blogger.com0tag:blogger.com,1999:blog-998418780028044861.post-24524087122336338962011-11-04T10:17:00.001+01:002011-11-04T10:20:07.579+01:00Jenkins on ci.horde.org got a major update<p><a href="http://ci.horde.org">The Horde continuous integration setup</a> saw a major upgrade and a number of fixes during the last week. </p><p>The take home message for everyone that does not wish to read the full log: <a href="ci.horde.org">ci.horde.org</a> will now run <a href="https://github.com/horde/horde/blob/master/framework/bin/test_framework">horde/framework/bin/test_framework</a> after a new commit has been pushed. This reduces the load on the system significantly and you will get results faster. If you develop Horde code you should run <a href="https://github.com/horde/horde/blob/master/framework/bin/test_framework">horde/framework/bin/test_framework</a> as well in order to check if your recent commits could cause failures on our continuous integration server. </p><p>The full recap of the recent changes: </p><p><ul><li> Update to the newest version of the Horde components tool. This allows to use the improved templating system. </li>
<li> New configuration and build templates that allow to reduce the job build time by avoiding to rebuild the set of package dependencies if this is not necessary. This has an important implication: A code change in one package (e.g. Horde_Imap_Client) will be tested against unchanged dependencies (e.g. Horde_Mime). <b>Even</b> if the commit also touched one or several of the dependencies (e.g. Horde_Mime). The packages should be backward compatible - so this should result in no error. The dependencies of one package will only get updated in case the dependency list in the package.xml of that package changes. </li>
<li> Running <a href="https://github.com/horde/horde/blob/master/framework/bin/test_framework">horde/framework/bin/test_framework</a> has been integrated into the <a href="http://ci.horde.org/job/horde-git/">horde-git</a> job. With the new "rebuild dependencies only when necessary" policy (see above) the component jobs do not fully test the integrity of the latest commit anymore. When the dependencies do not get updated the focus of shifts a bit more towards checking for backward compatibility. In order to not loose the integrity check for the "bleeding edge" our <tt>test_framework</tt> script is now executed right after updating the code from git. </li>
<li> Running <tt>test_framework</tt> also allows to run component jobs only if the code of the component has actually been touched. This significantly reduces the load on ci.horde.org as it removes the need for rebuilding all components for each and every commit. </li>
<li> An update to the newer <a href="http://pear.php.net/package/PHP_CodeSniffer">CodeSniffer</a> which included an update to the checklist for the Horde style. The <a href="https://github.com/horde/horde-support/blob/master/ci/templates/pre-build/phpcs.xml">new ruleset</a> is available from our <a href="https://github.com/horde/horde-support">horde-support</a> repository. </li>
<li> A customizable <a href="https://github.com/horde/horde-support/blob/master/ci/templates/pre-build/phpmd.xml">ruleset</a> for the <a href="http://phpmd.org">PHP mess detector</a> has been added as well. We still need to tweak the exact configuration of PMD to match it with what we consider reasonable defaults for Horde code. </li>
<li> <a href="http://www.docblox-project.org/">DocBlox</a> has been added to allow generating experimental API documentation. DocBlox is significantly faster and less memory hungry than PHPDocumentor. It has already been adopted by large frameworks such as Zend. So I figured it is worth taking a look at it. Feedback welcome! </li>
<li> The Jenkins configuration has been updated with the latest fixes and improvements from <a href="http://jenkins-php.org/">the Jenkins-PHP project</a>. </li>
<li>The setup procedure has been fixed so that it should be possible to generate a local setup - comparable to <a href="http://ci.horde.org/">ci.horde.org</a> - again. </li>
</ul></p>Anonymoushttp://www.blogger.com/profile/05751776969338923853noreply@blogger.com0tag:blogger.com,1999:blog-998418780028044861.post-62129497100828178112011-10-18T08:33:00.001+02:002011-10-18T08:36:32.781+02:00demo.horde.org got updated<p>You might have noticed that <a href="http://www.horde.org">the Horde homepage</a> received a new <b>Demo</b> button within the main navigation recently. We did indeed manage to get a demo machine up and running at <a href="http://demo.horde.org">demo.horde.org</a>. The installation got updated to the latest releases just yesterday and <a href="http://www.horde.org/apps/ansel">Ansel</a> - the Horde photo management application - got added into the mix. Feel free to test drive this new application as well as the other components on <a href="http://demo.horde.org">demo.horde.org</a>. </p><p>If you experience any problems please let us know. </p><p>In principle you shouldn't be able to break anything on the installation. It is a <a href="aws.amazon.com">virtual EC2 machine</a> and we will update the image every once in a while when there are new releases available. Any data you store on the machine will be resetted at that point. </p><p>You should also be able to mail to the two demo users <a href="mailto:demo@demo.horde.org">demo@demo.horde.org</a> and <a href="mailto:guest@demo.horde.org">guest@demo.horde.org</a>. Mailing to the outside is not possible - for obvious reasons. </p>Anonymoushttp://www.blogger.com/profile/05751776969338923853noreply@blogger.com1tag:blogger.com,1999:blog-998418780028044861.post-71923952467589895062011-10-14T09:42:00.001+02:002011-10-14T09:44:31.873+02:00Horde_Push now supports sending to Blogger.com<p>Horde_Push supports sending to blogger.com now.<br />
</p><p>The Horde_Push library is a tool to send content elements to various external services. Such as Twitter, Facebook, Blogger and others. Right now it only all<br />
ows sending Tweets, publishing entries on blogger.com and sending e-mails.<br />
</p><p>The idea is to allow publishing content you curate on a Horde installation to the social networks out there. At the moment the code is not yet there as the <br />
integration into the base <b>horde</b> package is still lacking. Right now there is only a small command line helper that I use for testing.<br />
</p>Anonymoushttp://www.blogger.com/profile/05751776969338923853noreply@blogger.com0tag:blogger.com,1999:blog-998418780028044861.post-39026202074921504802011-06-21T00:35:00.000+02:002011-06-21T00:35:24.559+02:00Anatomy of a Horde test suite - III<p>
Ready for the next item on the test suite agenda? This time the topic is <b>Autoloading</b>. We use a rather simple autoloading setup for most component test suites. It requires no additional setup and works out of the box if you run <tt>php AllTests.php</tt>.
</p>
<p>
That is already nice and allows running the complete test suite without further ado. But I must admit that I want more. My default work mode looks like this:
</p>
<ul>
<li>open test case (and modify it)</li>
<li>hit <tt><f3> <f8></tt> to run <tt>phpunit</tt> on this single test case</li>
<li>hit <tt><f4> <f4></tt> to to jump to the code line that produced the first error</li>
</ul>
<p>
This allows me to add a new test definition and immediately run it so that I can check for problems. And I don't need to execute the full <tt>AllTests.php</tt> to get feedback on the new test. So I'm annoyed every time I hit a test case that does not allow me to do that. A working autoloading setup is the key for that.
</p>
<p>
Luckily not only my own preferences make using an <tt>Autoload.php</tt> file in a test suite attractive. There are a number of reasons why such a file can be useful. The <a href="http://wiki.horde.org/Doc/Dev/Component/Horde_Test">Wiki page for the Horde_Test component</a> details them and this is a copy of the relevant section:
</p>
<hr/>
<p>
The <tt>Autoload.php</tt> file is not required in a test suite but it is strongly recommended that you use it. It's purpose is to setup PHP autoloading so that all tests in the test suite automatically have access to all the classes required for executing the tests. The reason why it is not mandatory is that <tt>Horde_Test_AllTests</tt> already loads a basic autoloading definition that works for most framework components.
</p>
<p>
This means that running <tt>php AllTests.php</tt> usually does not hit any autoloading problems. Running a single test case (e.g. <tt>phpunit Horde/Xyz/Unit/UnitTest.php</tt>) is a different matter though.
</p>
<p>
The <tt>*Test.php</tt> files do not extend <tt>Horde_Test_AllTests</tt> and thus there is nothing that would magically setup autoloading if you try to run such a test suite in isolation. And running single test cases can be quite convenient if the whole test suite would take a long time to execute. Using an <tt>Autoload.php</tt> file alongside the <tt>AllTests.php</tt> file is the recommended way to provide a single test case with autoloading and thus enable commands such as <tt>phpunit Horde/Xyz/Unit/UnitTest.php</tt>. In addition the file is helpful for any case where you need slightly more complex loading patterns or want to pull in special files manually.
</p>
<p>
Once you created an <tt>Autoload.php</tt> file for your test suite it will also be heeded by <tt>Horde/Test/AllTests.php</tt>. The latter will avoid the basic autoloading setup if it detects the presence of an <tt>Autoload.php</tt> file for the current test suite. That one will be loaded and is assumed to contain the required autoloading setup.
</p>
<h4>The content of <tt>Autoload.php</tt></h4>
<p>
You should at least require the <tt>Autoload.php</tt> from <tt>Horde_Test</tt> in this file. This is also what <tt>Horde_Test_AllTests</tt> would do when choosing the simple autoloading setup.
</p>
<pre>
require_once 'Horde/Test/Autoload.php';
</pre>
<p>
It also makes sense to adapt the error reporting level to the same standards as required in the <tt>AllTests.php</tt> wrapper:
</p>
<pre>
error_reporting(E_ALL | E_STRICT);
</pre>
<p>
If you derive your test cases from a central test case definition you should load this one in <tt>Autoload.php</tt> as well:
</p>
<pre>
/** Load the basic test definition */
require_once dirname(__FILE__) . '/TestCase.php';
</pre>
<p>
Sometimes it makes sense to pull in the definition of test helpers that may be used throughout the test suite. They are usually not available via autoloading and need to be pulled in explicitely:
</p>
<pre>
/** Load stub definitions */
require_once dirname(__FILE__) . '/Stub/ListQuery.php';
require_once dirname(__FILE__) . '/Stub/DataQuery.php';
</pre>
<p>
Real world examples for <tt>Autoload.php</tt> helpers can be found in the <a href="http://git.horde.org/co.php/framework/Date/test/Horde/Date/Autoload.php?rt=horde-git&ws=1">Horde_Date</a> and the <a href="http://git.horde.org/co.php/framework/Kolab_Storage/test/Horde/Kolab/Storage/Autoload.php?rt=horde-git&ws=1">Kolab_Storage</a> components.
</p>
<p>
Within the test cases you only need to load the <tt>Autoload.php</tt> file which usually looks like this (and obviously depends on the position of the test case in the directory hierarchy of the test suite):
</p>
<pre>
require_once dirname(__FILE__) . '/../Autoload.php';
</pre>
<hr/>
<p>
You'll find additional background information on autoloading within test suite runs on the <a href="http://wiki.horde.org/Doc/Dev/Component/Horde_Test">Wiki page for the Horde_Test component</a>.
</p>Anonymoushttp://www.blogger.com/profile/05751776969338923853noreply@blogger.com0tag:blogger.com,1999:blog-998418780028044861.post-52292021580994413722011-06-20T23:54:00.000+02:002011-06-20T23:54:06.530+02:00Horde4 debian packages - Second round<p>
Nearly half a year passed since I started my first attempt at Debian packages for Horde4. Back <a href="http://log.pardus.de/2011/01/horde4-package-mill-for-debian.html">then</a> it was just about snapshots which I generated via a continuous integration setup. But Horde4 has been released now, there are packages and it is time for the real thing.
</p>
<p>
I did get <a href="http://files.pardus.de/horde4-debian-20110617/">a first set of packages</a> installable today but this is just the initial draft. The main point about it is that the majority of it is automated.
</p>
<p>
If you want more details and the installation steps then I would suggest to follow my discussion with Mathieu Parent on <a href="http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-php-pear">pkg-php-pear@lists.alioth.debian.org</a>. These were the mails exchanged so far (with the last two detailing my current status):
</p>
<ol>
<li><a href="http://lists.alioth.debian.org/pipermail/pkg-php-pear/2011-May/000106.html">On PEAR packaging (25.5., Mathieu)</a></li>
<li><a href="http://lists.alioth.debian.org/pipermail/pkg-php-pear/2011-June/000115.html">On PEAR packaging (7.6., Gunnar)</a></li>
<li><a href="http://lists.alioth.debian.org/pipermail/pkg-php-pear/2011-June/000117.html">On PEAR packaging (7.7., Mathieu)</a></li>
<li><a href="http://lists.alioth.debian.org/pipermail/pkg-php-pear/2011-June/000122.html">On PEAR packaging (17.7., Gunnar)</a></li>
<li><a href="http://lists.alioth.debian.org/pipermail/pkg-php-pear/2011-June/000125.html">On PEAR packaging (20.7., Gunnar)</a></li>
</ol>
<p>
There is not much more to say right now. Any testing and feedback is welcome and there is obviously still a lot of work ahead until this pops up in your default package channels. But it is at least on the horizon and the variant that I ship on files.pardus.de should become useable very soon.
</p>Anonymoushttp://www.blogger.com/profile/05751776969338923853noreply@blogger.com4tag:blogger.com,1999:blog-998418780028044861.post-33554803653114168732011-06-14T09:29:00.000+02:002011-06-14T09:29:11.338+02:00Anatomy of a Horde test suite - II<p>
This morning I completed the next step on the journey through Horde's test suites and added the description of the <tt>AllTests.php</tt> file to the <a href="http://wiki.horde.org/Doc/Dev/Component/Horde_Test">wiki page</a>. I am not going to copy the complete text here but instead focus on the use cases for this file as I still have a few question to the audience below.
</p>
<hr/>
<p><tt>AllTests.php</tt> is the only mandatory requirement for a Horde test suite. Everything else is optional but there has to be an <tt>AllTests.php</tt> file which serves as an entry point into the test suite.</p>
<p>This is the functionality expected from the file:</p>
<ol>
<li>It must collect all tests of the test suite.</li>
<li>It must allow to retrieve all tests of the suite via <tt>Horde_Xyz_AllTests::suite()</tt>.</li>
<li>It must allow running the test suite via <tt>phpunit AllTests.php</tt>.</li>
<li>It must allow running the test suite via <tt>php AllTests.php</tt>.</li>
</ol>
<p>The <tt>Horde_Test</tt> package already delivers a boilerplate <tt>AllTests.php</tt> class in <tt>framework/Test/lib/Horde/Test/AllTests.php</tt> and deriving an <tt>AllTests.php</tt> for a standard test suite becomes rather simple. The full code for this is presented <a href="http://wiki.horde.org/Doc/Dev/Component/Horde_Test">on the wiki page</a> and you can also look <a href="http://git.horde.org/co.php/framework/Date/test/Horde/Date/AllTests.php?rt=horde-git&ws=1"">at an example from our repository</a>.</p>
<hr>
<p>
Now I wonder if the items listed above are in fact all the requirements we have for this file.
</p>
<p>
Requirements (1) and (2) are obvious as this is functionality needed for our <tt>horde/framework/bin/test_framework</tt> helper that runs <b>all</b> framework tests. Though I assume nobody uses this one on a regular basis at the moment.
</p>
<p>
But I noticed that (3) does not work out of the box with the current PHPUnit. This led to a <a href="https://github.com/sebastianbergmann/phpunit/pull/271">pull request</a> as it definitely <b>should</b> (and can) work.
</p>
<p>
I usually run the tests with a rather long command line that ultimately boils down to <tt>phpunit Horde_Xyz_AllTests AllTests.php</tt> which is tied to a shortcut in Emacs. As the Lisp code I use for that extracts the class name automatically I never noticed that a plain <tt>phpunit AllTests.php</tt> does not work.
</p>
<p>
So are most people using <tt>php AllTests.php</tt>? How do <b>you</b> run the test suites or would like to run them? Can I get some feedback on this (either here, on IRC or via tweet)?
</p>
<p>
Anything additional I missed about the requirements for the <tt>AllTests.php</tt> file?
</p>
<p>
Next in the series will be on autoloading which should allow me to also look at the problems we still have with that in the application components.
</p>Anonymoushttp://www.blogger.com/profile/05751776969338923853noreply@blogger.com2tag:blogger.com,1999:blog-998418780028044861.post-81811669536312924922011-06-09T18:25:00.000+02:002011-06-09T18:25:43.269+02:00The Horde newsletter again<p>
Time flies and it is soon time for the next issue of the Horde newsletter. Which reminds me to post the <a href="http://eepurl.com/d0AoX">link to the last one</a> in case you missed it. You can subscribe to the monthly newsletter <a href="http://bit.ly/hdHJ08">here</a>.
</p>
<p>
As <a href="http://www.janschneider.de/">Jan</a> already <a href="http://www.janschneider.de/horde/jonah/stories/view.php?channel_id=5&story_id=336">mentioned in his blog</a> he gave <a href="http://www.radiotux.de">Radio Tux</a> an interview on Horde 4. I just checked by roughly translating half of it to English that I should be able to include a transcript of it in the next newsletter. It delivers a very good overview on all things Horde 4.
</p>Anonymoushttp://www.blogger.com/profile/05751776969338923853noreply@blogger.com0tag:blogger.com,1999:blog-998418780028044861.post-16809516703351040262011-06-09T15:24:00.000+02:002011-06-09T15:24:30.114+02:00Anatomy of a Horde test suite - I<p>
Just got issue <b>07/2011</b> of the <a href="http://www.linux-magazin.de/">the German Linux Magazine</a> in the mailbox and on the final page there is this little abstract about <b>08/2011</b> saying...
</p>
<p>
"<b>PHP Unit and Jenkins</b> - There are two things guarding against programming errors: unit tests covering your code and continuous integration systems that automate the testing. The next issue will demonstrate this based on a real example from a PHP web project." [translated from German].
</p>
<p>
The "PHP web project" is actually named "Horde" and hm... I guess this means <a href="http://log.pardus.de/2011/05/is-horde-php.html">I have to write this thing - ;)</a> . When agreeing to the article I immediately knew I wanted to combine it with <a href="http://wiki.horde.org/Doc/Dev/Component/Horde_Test">an overview on how the Horde test suites are arranged</a>. So far we have been lacking a summary in that area and it should help newcomers to the Horde project to get into testing mode as well.
</p>
<p>
My mind is currently fully tuned to unit testing and code quality and it is amazing how easy it is to write about this. The initial draft for the article already exceeded all limits when it comes to size. Though I got pretty positive feedback on it I will have to leave some stuff out. Those sections should make it to this blog instead so that I can link to it in the article.
</p>
<p>
Basically I will make this into a short series of blog entries on unit testing in Horde. I will include parts of the <tt>Horde_Test</tt> overview, personal musings, and stuff related to the article. Let's hope it is useful to some people out there.
</p>
<p>
Here we go with the introduction to the <tt>Horde_Test</tt> overview...
</p>
<hr/>
<h2>Introduction</h2>
<p>
The Horde Project has always had high standards when it comes to code quality. Of course these standards evolved with time and also with the progress the PHP community made. The code from <a href="ftp://ftp.horde.org/pub/imp/imp-1.0.0.tar.gz">IMP-1.0.0 (1998)</a> didn't come with unit tests. And somehow it lacked classes. And there is an awful lot of code mixed with HTML. Somehow this looks horribly like PHP3.
</p>
<p>
Oh, it <b>was</b> PHP3.
</p>
<p>
Of course PHP development changed over time and so did the Horde project. Nowadays each and every commit into <a href="http://git.horde.org">our repository</a> leads to the automatic execution of thousands of unit tests written by the Horde developers and they check our code for failures. Night and day <a href="http://ci.horde.org">our continuous integration server</a> broadcasts the current test status to us in particular but also to anyone else interested.
</p>
<p>
With the release of Horde 4 the test suites of the Horde components available via <a href="http://pear.horde.org">our PEAR server</a> all show some common patterns. There are certain Do's and Don'ts and a lot of playground in between. Often the <tt>Horde_Test</tt> component is involved. So it makes sense to associate the overview on the anatomy of Horde test suites with this particular module.
<hr/>
<p>
I must admit that I really like the way the Horde project approaches unit testing. There is no way we could be unit test purists which would be too extreme given the fact that the project already exists for more than a decade. But at the same time there was also no one complaining when testing entered the equation. It just felt like continuing to adhere to the quality standards that seem so familiar when it comes to Horde code.
</p>
<p>
So much for now. More technical stuff to follow soon...
</p>Anonymoushttp://www.blogger.com/profile/05751776969338923853noreply@blogger.com1tag:blogger.com,1999:blog-998418780028044861.post-91402596423620491442011-06-01T15:14:00.003+02:002011-06-01T15:18:00.640+02:00Mixing stable Horde components with snapshots<p>
One thing I completely love about Horde 4 being a component framework is the ability to have stable installations into which I can inject experimental snapshots of packages further developed, patched or hacked in some other way. Just an overview of one such installation:
</p>
<pre>
INSTALLED PACKAGES, CHANNEL PEAR.HORDE.ORG:
===========================================
PACKAGE VERSION STATE
Horde_ActiveSync 1.0.0 stable
Horde_Alarm 1.0.1 stable
Horde_Argv 1.0.1 stable
Horde_Auth 1.0.3 stable
Horde_Autoloader 1.0.0 stable
Horde_Browser 1.0.0 stable
Horde_Cache 1.0.3 stable
Horde_Cli 1.0.0 stable
Horde_Compress 1.0.1 stable
Horde_Constraint 1.0.0 stable
Horde_Controller 1.0.0 stable
Horde_Core 1.1.2dev201105300702 stable
Horde_Crypt 1.0.2 stable
Horde_Data 1.0.0 stable
Horde_DataTree 1.0.0 stable
Horde_Date 1.0.1 stable
Horde_Date_Parser 1.0.0 stable
Horde_Db 1.0.1 stable
Horde_Editor 1.0.1 stable
Horde_Exception 1.0.1 stable
Horde_Feed 1.0.1dev201106011224 stable
Horde_Form 1.0.1 stable
Horde_Group 1.0.0 stable
...
</pre>
<p>
How do you get these snapshots? Ensure you have the <a href="http://wiki.horde.org/Doc/Dev/Component/Components">horde-component</a> helper set up. Then enter the package you wish to snapshot (e.g. <tt>Horde_Core</tt>) and run:
</p>
<pre>
horde-components snapshot
</pre>
<p>
The snapshot package will be assembled in this directory, can be uploaded to your webserver and installed using:
</p>
<pre>
pear upgrade --offline --force Horde_Core-1.1.2dev201105300702.tgz
</pre>
<p>
What happens if you patched and deployed your package, sent the patch upstream, it gets accepted and a new package released?
</p>
<pre>
pear upgrade horde/Horde_Core
</pre>
<p>
Magic!
</p>Anonymoushttp://www.blogger.com/profile/05751776969338923853noreply@blogger.com0tag:blogger.com,1999:blog-998418780028044861.post-55129768978316843792011-06-01T12:55:00.002+02:002011-06-01T15:17:41.770+02:00Blogging to blogger.com via Horde_Feed<p>
With Horde 4 being available as PEAR based components a lot of functionality that was hidden behind the Horde groupware applications have suddenly become available as building blocks for your own PHP based software. There is still a lot documentation that should get written. As one piece of the puzzle I will use my blog to post short tutorials on interesting things you might wish to do with the Horde PEAR packages.
</p>
<p>
Let's start with blog posts today...
</p>
<p>
Creating Atom blog posts becomes a rather simple task with the help of the <b>Horde_Feed</b> package. Install it via PEAR first:
</p>
<pre>
pear channel-discover pear.horde.org
pear install horde/Horde_Feed
</pre>
<p>
Start with creating an instance of <tt>Horde_Feed_Entry_Atom</tt> and populate it with content similar to the example below:
<pre>
$entry = new Horde_Feed_Entry_Atom();
$entry->{'atom:title'} = 'Entry 1';
$entry->{'atom:title'}['type'] = 'text';
$entry->{'atom:content'} = '1.1';
$entry->{'atom:content'}['type'] = 'text';
</pre>
<p>
The <tt>type</tt> could also be <tt>html</tt> or <tt>xhtml</tt> if the text in the corresponding field is not plain text. See the <a href="http://en.wikipedia.org/wiki/Atom_%28standard%29">overview on the Atom format</a> for that.
</p>
<p>
Now it is sufficient to post the entry with:
</p>
<pre>
try {
$entry->save($blogUri);
} catch (Horde_Feed_Exception $e) {
die('An error occurred posting: ' . $e->getMessage() . "\n");
}
</pre>
<p>
In most situations this example will be somewhat too simplistic. Most sites will require you to authenticate before being able to add Atom entries. How such authentication information can be transmitted when posting the entry is detailed in the <a href="https://github.com/horde/horde/blob/HEAD/framework/Feed/examples/Horde/Feed/blogger.php">blogger.com example</a> that comes with the <tt>Horde_Feed</tt> package. At the time I'm writing this the authentication is not yet available in the released package though - you will have to wait for the next release to hit <a href="http://pear.horde.org">pear.horde.org</a>.
</p>Anonymoushttp://www.blogger.com/profile/05751776969338923853noreply@blogger.com0