Python is often criticized -- as much from within as without -- for its proliferation of web frameworks. And yes, there's a lot -- and the list on that Wiki doesn't even cover them all, or account for the miniature frameworks built into applications. For instance, Leonardo and PyPI each have ad hoc frameworks, just to give two examples.
People suggest different ways of "solving" this problem of diversity, and say that this is the reason Python isn't as popular for web programming as PHP or Java or some other language. But I think it's time we just get over it -- the "problem" can't be solved. Java and PHP both have lots of frameworks, just like Python; this is simply the inevitable result of a diverse use base, and there's simply no way to stifle diversity and reimplementation in an open source community. Instead of reducing the options, we should work on handling the situation we have, of enabling a multitude of options instead of just complaining about it.
What does that mean?
We don't need some Killer Framework To Rule Them All -- we should work on an army of frameworks instead of a single Goliath.
The obvious way to do this is with WSGI. So far my experience with using other people's WSGI-enabled applications (or applying WSGI changes myself) has been quite successful -- when you follow the spec things Just Work.
But it shouldn't stop with simple WSGI backends to current frameworks (even if that's the right place to start). We should start pushing functionality deeper into the WSGI stack as middleware, and providing fine-grained WSGI applications that can be usefully integrated into other applications.
I think Python has a unique opportunity here:
Instead of wishing Python web programming was more like language X, we have the opportunity to do something new. We can turn this perceived flaw of diversity into a real benefit.
This isn't innovative like a whiz-bang new feature, and it's not new like proprietary-world new. This is the kind of innovation open source has been good at; integration, making things work together in new ways, realizing the benefits of distributed development. This is real innovation, it's just not innovation that the proprietary world tends to pay attention to. But working programmers know the value of this work.
This is what I've been working on in Paste; but an integration framework is only as good as the number of things that are integrated. I could use more help; simply using Paste is help. So please, the project needs more eyes and more diversity. And it matters.
While I agree somewhat that the sheer number of frameworks isn't necessarily a problem, I don't sense lock-in or incompatibility as being a huge stumbling block for many. I still think it's a business-like concern over support (i.e. documentation, books (eventually) and number of other users of the software) that is most people's concern.
"Am I going to build our department intranet in this framework and then it'll be dead by the time I'm done?" Sure people have the source, but in many cases not the time or skills to evolve and improve a framework like a community of users.
This is one thing that has drastically helped R*ils. As many similar projects do, they have a smaller core of developers who are dang good, and a much larger orbit of others -- designers, general webmonkeys, occasional programmers, beginners, etc. Many Python projects seem to get stuck at the small group of good devs - and never make it to the "attracting a crowd" part. Many possible reasons, some valid, some junk, but I still think this is the magic [but mundane] step to increasing visibility (https://ianbicking.org/why-web-programming-matters-most.html).
Eventually, a Python framework will show up in a nice package, with a non-PythonGuru friendly face and crowd (lesser mortals aren't scared away by the high priests on the project), and start attracting much more attention. Maybe Mssr Holovaty's package, maybe not (haven't seen it!). But I think once that (inevitably) happens, it will set the terms. Sorta like how the IMG tag came about maybe.
That said, if it happens on a shared spec like WSGI and your other efforts, that's even better. But I suspect that will be resolved by a large-visibility project choosing to implement it rather than the project needing to for its own success. i.e., top-down as opposed to bottom-up. Just missing the obvious top right now...
While I agree somewhat that the sheer number of frameworks isn't necessarily a problem, I don't sense lock-in or incompatibility as being a huge stumbling block for many. I still think it's a business-like concern over support (i.e. documentation, books (eventually) and number of other users of the software) that is most people's concern.If you can switch frameworks reasonably easily (and incrementally), then you can follow support and popularity. That's why I think lock in is a big problem. That said, in practice lock in isn't terrible right now, with one notable exception (that starts with a Z), but it's still more than it should be.
That said, if it happens on a shared spec like WSGI and your other efforts, that's even better. But I suspect that will be resolved by a large-visibility project choosing to implement it rather than the project needing to for its own success. i.e., top-down as opposed to bottom-up. Just missing the obvious top right now...I think (?) that Zope 3 probably already runs on WSGI. When the Twisted people asked Jim Fulton what it would take to get them to use Twisted instead of their Medusa fork, he asked them to do it via WSGI. Twisted web2 (which had a 0.1 release recently) included native WSGI support, and Zope 3 should at least come along soon. I doubt Zope 2 would be that hard either, though what I'd really like is a Zope 2 WSGI Server Product, so that you could run WSGI applications under Zope. Really I should just stop talking about it and write it, because it shouldn't be that hard.
# Ian Bicking
I should expand on the importance of being able to switch:
An environment where people can move from one framework to another will allow a real Darwinian process, and the emergence of a smaller set of "winners". Right now frameworks linger on forever, because there's no honest way for them to die. Either the developers abandon their users -- many of whom are often clients who trusted the developer's original choice of framework -- or the project keeps going and further development continues on it. Neither is good; either you are bad to your customers, or you create a partisan environment where each group of developers works in isolation to create the same set of features in their different frameworks.
The coupling of typical frameworks makes this worse. It's hard to be a polished framework when you have to implement a large number of features. As a result, frameworks typically implement a few features well, and some features poorly. As a result the selection process is haphazard, because you have to decide which features are important to you (and you don't really know), and you have to accept compromises in other features. With a decoupled stack, you can choose the best piece for each level of that stack. Sometimes there will still be vague choices -- different valid quality implementations, which appeal to different segments. But I think at each level there will arise a small set of choices, each with real reasons for existing even in the presence of other implementations.
# Ian Bicking
You are of course right on the huge value of being able to switch - or "evolve away" from a specific framework. As seen in the Java world with some people moving away from Struts to more recent setups like Spring, JSF, [today's new favorite], etc., but not needing to move it all at once.
I guess I was loosely speaking more so of the possible reasons for thriving/dying of contenders. Most likely both parts are needed: not being locked-in at a lower level will allow more dynamism in the upper levels, which should lead to a few "obvious choices" as you point out [and I tried to point out as needed to really expand the visibility of pyweb stuff].
Perhaps akin to having your living room furniture bolted down and/or integrated into your flooring - requiring tearing up the floor to change things [current scenario], and having all your furniture on wheels on a nice smooth floor - easy rearrangement [planned]. Alright I'll stop killing analogies now...
Whenever I hear the word "framework" my alarm bells ring. And I think that's the case with many people in IT.
But sometimes it does make sense to us a framework.
Choosing a (web) framework means taking risks. (Risk of "wrong" web framework selection, risk of steep learning curve, etc.) The smaller the choice the easier is the selection.
To be on the safe side, many people want to go mainstream. But there is no Python web framework mainstream. Sigh.
In order to be successful and survive, frameworks need a critical mass of users and developers. I think the diversity of Python web frameworks prevents one of those to get the critical mass. (I would say that Rails reached such a critical mass.)
As long as none of the Python web frameworks reaches such critical mass, they can't profit from self-energising (positive feedback) effects.
# Bishi
I know people are wary of frameworks, and I am too. But to clarify the discussion, "framework" can mean two things:
- A set of conventions.
- A library that tries to generalize logic.
The first is the natural result of dealing with a specific domain. This is why all web applications use a framework, even if it's a custom framework for the application. This set of conventions needs to be sufficient, as well -- the standard cgi module isn't sufficient, so if you use it you'll still end up creating a framework if your application is very large or general at all. And even that module is a mini-framework -- just not a very good mini-framework.
There's a real overhead to conventions, so if a framework is just (1) then it's still a problem. There's new words, new constructs, new interfaces, and those tend to infuse the code that uses a framework. If they don't, if those concepts are internal to the library, then it's not very framework-like. But it can't be helped sometime, you can only hope you use terminology that is clear, minimal (but sufficient), and maps well to the domain, and is conventional. Custom frameworks are most minimal, but least conventional, and tend to embody many nonconventional ideas of the application author.
The second -- generalizing logic -- is much more dangerous, though often just as necessary. For instance, if you didn't generalize logic a web application might start with:
if PATH_INFO.startswith('/contact'): import contact_form contact_form.run() elif PATH_INFO.startswith('/article') and PATH_INFO.endswith('/view'): import article_view article_view.run()And so on. Because a simple thing like URL parsing is very logic-heavy. And if you use something like SimpleHTTPServer, you'll end up writing exactly that code. Or you'll end up writing code that imports modules and does path matching generally, moving that "code" into "data" (a URL mapping or an algorithm for resolving URLs). Then you've created your own logic-generalizing framework.
In this case you hope the framework is minimal and transparent, and follows several important rules (which I have in my head and somewhere on disk, but I haven't written down on this blog).
Paste has a little of both kinds of framework, and leaves a bunch up to frameworks that run on top of it. This causes problems, but I can only say that I think hard about the issues and I'm designing Paste with these issues in mind. Decoupling makes a framework more minimal, because it is locally-understandable, and Paste is all about decoupling. A common infrastructure encourages convention. And a wide base demands that the domain be well covered.
# Ian Bicking
> Adopting a framework is less commitment, and moving to a different framework no longer requires anything like a full rewrite.
The idea of switching frameworks easily just doesn't make sense to me. For the sake of this comment, I think a framework has two parts:
- A framework lays out a set of interfaces that people can plug into. When people choose Aquarium, Aquarium lets them swap out Web servers, how they do their URLs, how they do their sessions, etc. All of this is tied to using simple API's that Aquarium provides, and then swapping in new pieces that implement those API's. Those API's are fixed so that what implements those API's can change. You can't have the backend and the API's changing at the same time, nor should you need to.
- A framework often lays down a methodology for solving a domain of problems. For instance, Aquarium lays down the way I think many Web applications should be developed. For instance, it distinguishes controllers from views. Changing from having controllers and views to something else, such as having just views and other stuff (like Zope) can never be easy. It requires a rewrite.
So if you want to change frameworks all the time, the API's have to be overly simple and it can't suggest a methodology. What's the point? ;) You're better off just providing a library for whatever real functionality you provide and not calling it a framework at all.
(That's just my $0.02. Keep up the good work :) )