Ian Bicking: the old part of his blog

workingenv.py update

I've been using workingenv more lately, and I thought I'd talk more about its goals.

If you haven't used it, or didn't read/remember the original post, the basic idea is that you do this:

$ wget http://svn.colorstudy.com/home/ianb/workingenv/workingenv.py
$ python workingenv.py NewEnvironment
$ cd NewEnvironment
$ source bin/activate

This creates an isolated environment in NewEnvironment/ that contains its own libraries isolated from the rest of the system. As such it is very similar to virtual-python.py but perhaps slightly simpler, in that it doesn't create a new python executable or symlink over the entire standard library. Honestly virtual-python works quite well too, though I suppose workingenv works on Windows (without symlinks); at least, I hear it does, I've never tried it.

This is great for tutorials and examples, since the result is a very controlled system. It's common for people to forget that they have all sorts of hidden requirements installed on their system, and since workinenv removes all of those libraries (including everything in site-packages unless you give the --site-packages option) you won't forget anything. It's very repeatable.

There's some other features lingering in workingenv that might be additionally useful. In particular --requirements, which lets you give a set of requirements to be installed by easy_install when the environment is set up (you can also track the requirements over time to confirm you are up-to-date). For instance:

$ python workingenv.py \
  --requirements=https://svn.openplans.org/svn/training/pylons/requirements.txt \
  NewPylonsEnvironment

This is the Pylons environment we used for the a mini-tutorial in the framework at TOPP. If you look at the file, it's just a text file with one requirement per line. You can also use -f someplace_to_find_links, which will in effect add that option to all subsequent easy_install invocations. Pylons has such a page at http://pylonshq.com/download/, TurboGears at http://www.turbogears.org/download/, etc. And -e Package to install an editable version of a package, which you can use for all your internal/in-development packages. Later on you can rerun workingenv.py to update the environment and requirements, or run workingenv.py --confirm NewPylonsEnvironment/ to check if anything has changed in the requirements file, or any requirements aren't in sync.

Some other things I'd like to add:

Notably another project with some similar goals is zc.buildout. zc.buildout is more declarative, and works off a configuration file instead of interactive commands. For example, you install something something in a buildout by changing the configuration and rebuilding. You install something in a working environment by using easy_install. zc.buildout is also more general -- it has recipes to do more than just install eggs, it can also build packages with configure; make; make install (cmmi), and you can add new recipes for more complicated setups. I've found it more difficult to get my head around, especially how to support both production and development deployments. But if you are interested in this sort of thing, you should certainly give it a look (I've noticed it already has considerably more documentation than when I last looked).

Created 11 Oct '06

Comments:

I have to say I've been using workingenv for a while now and it is really helpful. Everyday I need to work with more than one module from subversion and this is a nice way to go. I was having a little bit of trouble getting 2.4 and 2.5 environs going side by side (I think the local easy_install script was getting overwritten) but maybe it was fixed when I blew away my env and started over, can't recall. I'll let you know if it comes up again.

Is there a way to repair an existing workingenv? I know you have been tweaking issues for compatibility w/ setuptools here and there (invalid site.py error, etc). Obviously I haven't looked at the new --confirm option. Maybe that does it.

One thing I'm trying to find time for in my company is convert our hand-rolled continuous integration package to buildbot. I think workingenv would be a nice way to automate the buildbot environment for each test suite. Has anyone tried something like this yet?

# Kumar McMillan

Is there a way to repair an existing workingenv? I know you have been tweaking issues for compatibility w/ setuptools here and there (invalid site.py error, etc). Obviously I haven't looked at the new --confirm option. Maybe that does it.

The new version saves your configuration information in .workingenv/*, so you can simply re-run python workingenv.py existing-env. It'll ask you before overwriting any files.

One thing I'm trying to find time for in my company is convert our hand-rolled continuous integration package to buildbot. I think workingenv would be a nice way to automate the buildbot environment for each test suite. Has anyone tried something like this yet?

I haven't, but it definitely seems like a good way to do this.

# Ian Bicking

For reasons unknown the setuptools egg was not being expanded, apparently with my switch to Python 2.5 or related changes.

I forced it with a change to workingenv.py

517c517

< os.system('%s %s --install-dir=%s --script-dir=%s --always-copy '

> os.system('%s %s --install-dir=%s --script-dir=%s --always-copy --always-unzip '

# Rick Thomas

I'm getting this error only on one of my machines and having trouble tracking down the cause:

site.py is not a setuptools-generated site.py; please remove it.

well, obviously the site.py is not setuptools-generated but on other machines with the latest version of workingenv and setuptools I don't see this error. The only difference (I can tell) is that on the erroring machine python 2.5 is installed. I don't see how that would be related. I tried killing setuptools in the base (2.4 and 2.5) and the env, re-building the env; still no luck. Any other ideas?

# Kumar McMillan

I am having problems/am stumped on how to install database packages such as pysqlite.

using:
(foo)jlh@dmt600u:~/Projects/sa$ easy_install pysqlite

I get the error:

"No local packages or download links found for pysqlite error: Could not find suitable distribution for Requirement.parse('pysqlite')"
# Jeff H