Python Packaging with Setuptools

Using and creating Python packages

Ian Bicking


  • distutils in the standard library.
  • Standard way of packaging and installing packages.

Installing with Distutils

$ sudo python install

Global installation; for local installation (or if you don't have root):

$ python --install-lib=./app-packages

Make sure you update $PYTHONPATH! Other --install-* options control location of scripts, headers, etc.

Part 1:

  • Part of setuptools
  • Installs any distutil-based package
  • Can find packages on PyPI (now the "Cheese Shop")
  • But many (most?) packages in PyPI don't include the necessary information

Installation: Normal

Normal installation

$ Package

Installation: Development

Development installation

$ --editable --build <DIR> Package
$ # or just download and unpack the package
$ cd <DIR>/Package
$ sudo python develop

Installation: Isolated

Isolated (version-specific) installation

$ -m Package==X.Y
$ python
>>> import pkg_resources
>>> require('Package==X.Y')


$ kid
Searching for kid
Best match: kid 0.6.3
Running kid-0.6.3/ -q bdist_egg --dist-dir /tmp/easy_install-gsePfU/kid-0.6.3/egg-dist-tmp-WojETA
zip_safe flag not set; analyzing archive contents...
kid.importer: module references __file__
kid.test.__init__: module references __file__
Adding kid 0.6.3 to easy-install.pth file
Installing kid script to /usr/bin
Installing kidc script to /usr/bin

Installed /usr/lib/python2.4/site-packages/kid-0.6.3-py2.4.egg
Processing dependencies for kid


$ --editable \
  --build-directory ~/co \
  --find-links= \
Searching for Paste
Best match: Paste [unknown version]
Doing subversion checkout from to /tmp/easy_install-d75rz8/trunk
Processing trunk

Extracted editable version of Paste to /home/ianb/co/paste

If it uses setuptools in its setup script, you can activate it in
"development" mode by going to that directory and running::

    /usr/bin/python2.4 --develop

See the setuptools documentation for the "develop" command for more info.

Development notes

Things to note:

  • --find-links points to a page where you list distributions
  • A packages index is just a list of links
  • --build-directory and --editable keep the files around, and don't do anything with them...

Development installation

$ cd ~/co/paste
$ sudo python develop
running develop
running egg_info
writing requirements to ./Paste.egg-info/requires.txt
writing ./Paste.egg-info/PKG-INFO
writing top-level names to ./Paste.egg-info/top_level.txt
running build_ext
Creating /usr/lib/python2.3/site-packages/Paste.egg-link (link to .)
Adding Paste 0.0 to easy-install.pth file
Installing paster script to /usr/bin

Installed /home/ianb/co/paste

Development notes

More things to note:

  • develop installs a package without moving it into site-packages/
  • Paste.egg-link is the poor man's symlink to ~/co/paste
  • easy-install.pth also points to ~/co/paste
  • Python finds .pth files in site-packages and adds their contents to sys.path

Installing in Isolation

  • Libraries aren't always backward- or forward-compatible
  • When dependencies are automatically installed, there's greater chance of conflict

Installing in Isolation

Use "multi-version":

$ sudo python easy_install --multi-version
  • Does not add the package to sys.path
  • You must require the specific version; more on that later
  • Doesn't effect anyone else on the machine (as long as you are using good version numbers)

Isolated Checkouts

From a repository:

$ svn co flup
$ cd flup
$ # fix to use setuptools
$ sudo python egg_info --tag-svn-revision \
  develop -m
Because this distribution was installed --multi-version or --install-dir,
before you can import modules from this package in an application, you
will need to 'import pkg_resources' and then use a 'require()' call
similar to one of these examples, in order to select the desired version:

    pkg_resources.require("flup")  # latest installed version
    pkg_resources.require("flup==0.5-r1802")  # this exact version
    pkg_resources.require("flup>=0.5-r1802")  # this version or higher

Part 2: creating packages

Distutils features:

  • Packages include standard installation script (
  • Installation script also builds archives for distribution
  • Script can be registered and uploaded to PyPI automatically
  • Included with Python; good and bad


Setuptools' extra features:

  • It's just like distutils
  • That develop command we saw
  • Creates eggs: better because "Eggs" are a better visual than a "distutil package"


Setuptools' extra features:

  • Everything does, it does by tricking a package into using setuptools instead of distutils
  • Dependencies!

Creating a Package

Nevermind features...

Lay your files out like this:


Package Layout

  • Your "distribution" has a name: MyPackage
  • Not to be confused with your "package": mypackage (of course, probably will be confused)
  • Packages (and modules) all lower-case by convention

Package Layout

  • Documentation and (usually) tests go outside the package
  • mypackage/ is all that really gets "installed"
  • describes the package

A typical

from ez_setup import use_setuptools
from setuptools import setup, find_packages

      description="My Package, now featuring 10% more packaging!",
This is a boxy package...
      author="Ian Bicking",

More arguments:

      package_data={'mypackage': ['data/*.xml']},
      install_requires=['Paper>=1.0', 'UPSCode'],
      ) ez_setup

This boilerplate installs setuptools when the user (who is running hasn't installed setuptools:

from ez_setup import use_setuptools
use_setuptools() comes with setuptools, you include it directly in your archive. an explanation

  • All the metadata goes in setup()
  • Some of this is used to install the package
  • Some is used to create an archive of the package
  • Some is used for dependencies
  • Some is used for PyPI the arguments

The name of your distribution. Don't put spaces in it. Becomes the name of your archive.
The version. Suffixes like a1 and pre5 are sorted as you'd expect.
description, long_description:
For use by PyPI. long_description is in restructured-text format.
author, author_email, url:
Also used by PyPI. the arguments

Important if you aren't uploading to PyPI; the location where you'll upload your package.

These values can be edited through PyPI, if you need to correct the information on a released version. more arguments


You list all the packages that should be installed, including subpackages, like ['mypackage', ...].

find_packages() does this for you. The exclude argument keeps it from auto-detecting things that look like packages.

For files you want included. This is a dictionary of package-name ("" for all packages) to globs (e.g., "*.txt"). requirements

This is a list of requirements for this package. Each is a package a string like you would give to

If you have optional requirements, you can use "features", which are not explained here.

Your New Package

What fun you and your new package will have!

$ python --help-commands
Standard commands:
  build            build everything needed to install
  build_py         "build" pure Python modules (copy to build directory)
  build_ext        build C/C++ extensions (compile/link to build directory)
  build_clib       build C/C++ libraries used by Python extensions
  build_scripts    "build" scripts (copy and fixup #! line)
  clean            clean up output of 'build' command

Commands: build*

The build* commands build C code. C is not Python. These are not the codes you are looking for.

Or really: I write Python, and building C code isn't broken, and install runs these commands for you, so I know nothing of this.

Your New Package

$ python --help-commands
Standard commands:
  install          install everything from build directory
  install_lib      install all Python modules (extensions and pure Python)
  install_headers  install C/C++ header files
  install_scripts  install scripts (Python or otherwise)
  install_data     install data files

Install the library; more on that later. The other install* commands are for installing just pieces of the package, which is used in intermediate steps you are unlikely to use independently.

Your New Package

$ python --help-commands
Standard commands:
  sdist            create a source distribution (tarball, zip file, etc.)
  register         register the distribution with the Python package index
  bdist            create a built (binary) distribution
  bdist_dumb       create a "dumb" built distribution
  bdist_rpm        create an RPM distribution
  bdist_wininst    create an executable installer for MS Windows

Commands: distributing

Creates a .tar.gz or .zip file that contains your package. This is what you give other people.
Takes your package information and uploads it to PyPI (aka Cheese Shop).
bdist, bdist_dumb, bdist_rpm, bdist_wininst:
I've never seen bdist or bdist_dumb packages. These might be nice for Windows or RPM users. But sdist packages can be turned into bdist* packages by the user (you need a compiler if there's C code; for this reason a Windows installer is nice, but it's not as important for other users).

Your New Package

$ python --help-commands
Extra commands:
  rotate           delete older distributions, keeping N newest files
  develop          install package in 'development mode'
  setopt           set an option in setup.cfg or another config file
  saveopts         save supplied options to setup.cfg or other config file
  egg_info         create a distribution's .egg-info directory
  upload           upload binary package to PyPI
  alias            define a shortcut to invoke one or more commands
  bdist_egg        create an "egg" distribution
  test             run unit tests after in-place build
  easy_install     Find/get/install Python packages

Commands: setuptools

All these commands come from setuptools.

Use this to keep a limited number of nightly snapshots around.
Talked about this earlier; installs a package without copying it into site-packages.

Commands: options

Command-line options to can also go in setup.cfg, distutils.cfg, and other locations.

Save options, without finding the file or translating the command-line options to the configuration options.
Same idea, slightly different interface.
Take a bunch of commands and options, and give it one name.

Commands: eggs

Eggs put metadata in MyPackage.egg-info directory. This metadata is used from the outside. You can modify the version string from outside with this (like adding a Subversion revision).
Creates a .egg file. This is a zip file you can put right in site-packages, amongst other things. Good for plugins, used for install, but mostly you can ignore these.

Commands: manage

PyPI Is CPAN: upload your package(s) directly to PyPI.
Two frontends to the Same Thing
Run unit tests. Limited to unittest currently.