Python egg fun

Now that I'm back working on some of my Python packages it was time to look at Python package management again. So far I just used the embedded distutils package but apparently "setuptools" is the thing that starts to be used widespread. So I looked at this in more detail and will summarize some notes here.

"setuptools" does actually not deliver too much fancy new functionality. The main benefit lies in the area of dependencies and the creation of distributable packages.

I basically used something like this as setup.py when I only used distutils:

from distutils.core import setup

PACAKGES = ['libpardus',        
            'libpardus.configs',
            'libpardus.utils',
            'libpardus.web']

import sys
sys.path.insert(0, './')             
from libpardus.version import VERSION

setup(name          = 'libpardus',
      version       = VERSION,                
      description   = 'p@rdus Python Library',
      author        = 'Gunnar Wrobel',
      author_email  = 'p@rdus.de',                       
      url           = 'http://libpardus.sourceforge.net',
      packages      = PACKAGES,
      license       = 'GPL',
      **extra
      )

Now, with setuptools things do not become much more complicated:

try:                                           
    from setuptools import setup, find_packages
except ImportError, e:              
    from distutils.core import setup
    extra = {}              
    PACAKGES = ['libpardus',        
                'libpardus.configs',
                'libpardus.utils',
                'libpardus.web']
else:            
    extra = dict(           
        install_requires = [
            'setuptools',
            'web.py',        
            'zope.interface',
            ],            
        extras_require = {
            'WEB': ['web.py'],
            }
        )                     
    PACKAGES = find_packages()

import sys
sys.path.insert(0, './')
from libpardus.version import VERSION
                                  
setup(name          = 'libpardus',
      version       = VERSION,                
      description   = 'p@rdus Python Library',
      author        = 'Gunnar Wrobel',
      author_email  = 'p@rdus.de',                       
      url           = 'http://libpardus.sourceforge.net',
      packages      = PACKAGES,
      license       = 'GPL',
      **extra
      )

I'm mainly using find_packages() to generate the package list and add some packages as basic requirements. In order to make this also work on "non-setuptools" system the whole is embedded in a try: ... except: ... statement.

The whole thing is not yet complete since I don't know every detail of "setuptools" yet, but the main point here is that things are not too different if you decide to upgrade from "distutils".

I also created a setup.cfg-file to make the creation of snapshot and release packages easier:

[egg_info]
tag_build = .dev
tag_svn_revision = 1                           

[aliases]
release  = egg_info -RDb ''
relpatch = egg_info -db ''  

Running python setup.py bdist_egg will now create an egg that I can use for testing purposes. It gets the current subversion revision attached to the file name (e.g. libpardus-0.9.2.dev_r38-py2.4.egg. Because of the alias definition

release  = egg_info -RDb ''

the command python setup.py release sdist will create a release source package (e.g. libpardus-0.9.2.tar.gz). The relpatch command is for releasing a patch level package after the main release.

But during development it is really nice to produce these testing eggs without the need to build/install the tools every time.

"setuptools" also makes it easy to handle foreign eggs. This way you can build whole test applications without modifying your site-wide python library. This can be done using the easy_install tool:

easy_install -zmaxd lib/ web.py

This way you would place web.py as a packaged egg within the lib directory.

Within a python library you can the either directly include the full egg filename in the sys.patch:

sys.path.insert(0, 'lib/web.py-0.23-py2.4.egg')

Using setuptools you can also make this:

from pkg_resources import require
require("web.py")

The details can be found on the setuptools homepage.

I'll probably continue this once I learn how to upload python packages to PyPI, the python package index.