Ian Bicking: the old part of his blog

WSGI

Things in WSGI-land have been going well lately, I think. Paste is at 1.0 and doing fine there. Pylons has been gaining in popularity, and it's been doing really well as a traditional and familiar framework on the outside, but with WSGI just under the skin. It's been a compelling introduction to WSGI for many people. Other WSGI-based frameworks have been appearing; I especially like niche frameworks like wsgicollection because they wouldn't be viable frameworks at all if they had to stand on their own. But with WSGI you really can just solve one problem, and not worry about all possible problems.

There's been several recent articles on WSGI. I'm never sure how WSGI should really be presented -- on one level it's an implementation detail of higher-level frameworks. Creating WSGI middleware and dispatchers and whatnot is not terribly advanced, and is genuinely useful even if you aren't inclined to write your own framework from scratch, but it's not a prerequesite to getting things done in WSGI-based environments. But WSGI's success will definitely rely on the community of people that feel comfortable with both the details and the architecture of these systems, so building wide familiarity is important. I've personally come to believe that talking about the architecture of WSGI is more important than the details (which are easy enough to pick up if you are motivated).

I started talking about more standards on Web-SIG. Python is ripe for new standards in lots of areas, and WSGI stacks are among them. The first proposal I introduced just because it seemed like low-hanging fruit, but people seemed to really like it. I guess it's not surprise that low-hanging fruit is usually the best stuff to pick. The proposal describes an environment key (wsgiorg.routing_args) that dispatchers can use to store captured information. In the process we discussed a fairly light process for introducing and accepting proposals, centered now on a wsgi.org page. I also added a bunch of ideas for specs to the bottom of that page; so if you see one you are thinking about yourself, now you know someone else is thinking about it too.

In my own coding, I've been working on WSGIRemote, which is a library to help with making subrequests, both external requests and what should be very fast internal requests (not that I've benchmarked anything). This should make apps using REST internally more manageable -- they should scale down a lot further, requiring less setup and less general overhead. Of course the performance characteristics will change a lot when you actually move services to other processes or servers, but I'm more concerned with manageability at this point. A lot of the tools in Paste (and Paste Deploy) are about managing lots of small pieces without letting that turn into external complexity. One aspect of this is to let applications self-provide the services they need, because most of the time simple internal implementations of an API will be sufficient. Turning a Python library into a service is kind of like adding another level of formality to it; and I think more formality is useful as we try to share pieces of code.

Another earlier-stage project is WSGIFilter, perhaps notably one of the only frameworks for building WSGI middleware. It's not a very big framework, to be sure (it's really just the one file) but it should help implement output filtering in WSGI, which can be really tricky.

WSGI development can seem kind of plodding at times, because we're building up slowly from a low-level foundation. But as the tools emerge they work together really well; I think we're starting to see real returns from that process. If there was a motto for using these WSGI tools together, it might be making large problems smaller -- it's not doing much for you, it's not fancy or magic, things like "convention over configuration" don't even mean anything on this level, it's not code or a framework, it's just an idea and an architecture. Describing what that really means remains a challenge.

Created 13 Nov '06

Comments:

I added a couple more specs tonight, for the heck of it:

# Ian Bicking