Ian Bicking: the old part of his blog

Re: Packaging Python

Suppose a security bug is found in a popular library.

If I'm using a Debian-based system like Ubuntu, a notification will appear on my screen. I click the "install update" button. The system downloads installs a new version of the library package. I'm done.

On the other hand, suppose I'm on a system without proper packages, where each application installs the library into its own copy of the environment. Then I need to hunt through every application on my system, see if it uses the library, and install the security fix for each application that does. Or, every application author needs to update their application packages to include the security fix, and I need to install all of the updated packages.

This is just one of the reasons that libraries benefit from proper packaging even more than applications do.

Comment on Packaging Python
by Matt Brubeck

Comments:

Suppose a security bug is found in a popular library.

Then the security bug exists in every application using that library, and they should all be updated, and new versions of the system-level application packages will emerge for you to install. Problem solved.

# Ian Bicking

This is pretty fast problem solving. But it doesn't match reality.

Look for example at xpdf; its code is duplicated in gpdf, kpdf, some command-line utilities, poppler, I am probably missing some. Waiting for all of them to update to the newer xpdf code is unrealistic. It has been shown as not working and causing delays.

You write they should all be updated. While only one single package could. I prefer that second option.

# Frederic Peters

This is something that should be handled with proper tool support.

Look for example at xpdf; its code is duplicated in gpdf, kpdf, some command-line utilities, poppler, I am probably missing some. Waiting for all of them to update to the newer xpdf code is unrealistic.

Why is it unrealistic? Because updating the code causes problems in those packages? Updating the code via dynamic linking doesn't improve that any. Because the maintainer isn't sufficiently available to make the update in time, or in a coordinated fashion? Debian does non-maintainer updates to solve this sort of problem; that kind of update just needs to be done more widely.

# Ian Bicking

It is not easy because gpdf, kpdf, etc. don't have much knowledge about the xpdf code they ship; so it takes time.

And the developers who will bundle many Python packages to get an "application package" won't know details about all those Python packages, same problem.

Even worse it gets much harder for an entity that cares (say Debian) to apply the fixes; 1) they must be applied in many different places and 2) those different places have different versions.

As for the tools to handle all of this, they do not exist for the moment.


IIRC you had slides opposing developers and packagers ("deployers", maintainers, sysadmins, whatever the name), iirc. All of this, For development of applications, much more comfortable and flexible for the developer, isolated development environments concerns developers. Please don't forget the other side.

# Frederic Peters

I sort of agree.

There are fewer libraries in Python that can even have true security flaws. Those libraries are usually used in such a fashion that they _can_ be installed system-wide. In fact they should be included with the Python distribution.

All those helper libraries should be bundled separately with the application (as is usually done on Windows) and makes deployment easier.

But what about GUI development? For development, the sandbox approach doesn't work. I find wxWidgets is a pain sometimes because I need to tweak a widget to work well with my applications, since they often don't make subclassing easy. As a developer, sandboxing doesn't make sense because I want to be able to add this tweak to all my applications. But I do want to take advantage of new features, so something like working-env makes sense for all my legacy apps. Maybe that's a problem with the library though.

But what about web development? For hosting, the sandbox approach doesn't work so well for deployers (bundling CherryPy.) In this case it's nice to have a real server that can stay stable underneath.

However, I do agree that the majority of libraries don't belong in site-packages at all. Library dependencies should be resolved by the developer, not the deployer.

# Kevin Deenanauth

But what about GUI development? For development, the sandbox approach doesn't work. I find wxWidgets is a pain sometimes because I need to tweak a widget to work well with my applications, since they often don't make subclassing easy. As a developer, sandboxing doesn't make sense because I want to be able to add this tweak to all my applications.

You mean you edit the wx code directly? In that case you should probably make a mini fork. Monkeypatching is probably a more realistic option, though. It's a messy situation any way you do it -- I think it's cleaner with monkeypatching though. Monkeypatching someone else's (or everyone else's) application is a little more difficult with isolated installations. So I don't know.

But what about web development? For hosting, the sandbox approach doesn't work so well for deployers (bundling CherryPy.) In this case it's nice to have a real server that can stay stable underneath.

I think the opposite is true. The hoster really doesn't care about CherryPy, and generally doesn't want to maintain or update that for you. On both sides of that relationship people just want things to work and not to have to coordinate and communicate. Bundling CherryPy or whatever your web framework is is probably going to work better for everyone.

# Ian Bicking