Ian Bicking: the old part of his blog

Re: Full Stack vs. Glue

Summarizing a comment I made in reply to you on my blog:

The full-stack/glue distinction is very real, and can be revealed by a couple of practical tests; the best one is to ask where a bug report for a major component goes -- if it gets directed to a third-party project, then you're glue. If you handle it yourself, then you're full-stack. Again, it's the difference between supporting code that helps other things work together, and supporting a full set of components.

And Django doesn't claim to be decoupled, it claims that its components are loosely coupled. That may seem like semantic hair-splitting, but it is a significant difference -- while we obviously don't go out of our way to make it hard to swap components, having "plug and play" core components is not one of our design goals; instead we focus on the components we have, on making them better and making them work better together. That is, again, the difference between full-stack and glue in a nutshell: is your goal to primarily provide and support one set of components, or is your goal to primarily provide and support ways to hook arbitrary sets of components together? For Django it's the former and so we're full-stack. For Pylons it seems to be largely the latter, and so Pylons is glue.

The coupled/decoupled alternative also really feels like the same thing in different words: the primary difference is, again, whether you're aiming to support a single set of components together, or to support swapping and interchanging of many components and sets of components. I'll admit "glue" can be something of a loaded word in the Python community (I guess it reminds us too much of Perl -- which is a shame, because good glue code is some of the most insanely useful stuff out there), but "coupled" is, as you've said, going to suffer from the same problem. We need to come up with some better terminology, I think...

Comment on Full Stack vs. Glue
by James Bennett


Could the difference between a 'glue' and 'full stack' framework be summarised as ... If, as a framework representative I find a third party module that does something significantly better;

Or would it be OK to use the third party module as long as it becomes a component part of the "full stack" framework..

Or! If MochiKit, Kid, CherryPy and SQLObject all shared a single svn repository and all prefixed their project names with 'TG' would it then become a "Full Stack" framework?

If so then the definition is virtually meaningless..

# Tim Parkin

Tim, personally I think it comes down to development focus: does the framework primarily focus on a single set of components, and provide support and maintenance for those components and for the code it uses to hook them together? If so, it's full-stack ("coupled" in Ian's terminology), and you don't have to develop all your components in-house to do that -- you could, for example, use someone else's open-source components and distribute patched versions which work with the rest of your framework (and to be a good citizen you'd submit those patches back to the original project, of course). On the other hand, you can primarily focus on developing interfaces and adapters which aren't meant for any specific component or set of components, but rather to be generic enough to plug any component into any slot in the framework; that's what I mean when I say "glue" and what I think Ian means when he says "decoupled". And you don't have to use only third-party stuff as your default set -- you could develop an ORM, template system, etc. and offer them as the default set.

Either way it's a lot of work -- supporting a specific component stack is non-trivial, and neither is developing a full WSGI system with the necessary support and adapters to truly support "plug and play" components. I'm incredibly glad there are dedicated people on both sides who are willing to put in the time and effort to give the Python community the embarrassment of riches we have in the web-dev sector now, because I remember all too well the days when I desperately wanted to use Python on the web but couldn't because Zope was the only option and wouldn't fit in my head :)

# James Bennett

This is where things get confusing.

TurboGears does, as a simple matter of fact, pull together a single set of components, and encourage their use. We are trying to tell a unified story about how everything works together, and we've had help from CherryPy, Kid, and other components in bringing everything together in the way we want.

Let's look at the way we work with templating systems. We clearly aren't distributing a patched version of Kid, but the Kid maintainer is doing it because he uses it in TurboGears. When you use a kid template you just specify the name of the template in the @expose decorator, and everything "just works."

So far we are just like Django and the Django templating system. Simple defaults, and easy configuration make life good. But we do have a differnt philsophy. The maintainers of Kid understand that some people will want to use Kid in other ways -- outside of the framework -- and they work to support that.

At the same time we also understand that some people have hundreds of thousands of lines of template code written with Cheetah and we support them if they want to use TurboGears. So, we've allowed them to define the engine they want used as well as the template name in the @expose decorator.

And that's the difference between the TG/Pylons way and the Django way. It's not Glue VS Full Stack (TurboGears at least tries very hard to be Full Stack). It's about code reuse. We are open to pulling in code from outside and working with those developers to get what we need from their components, and we are open to the idea that others will want to use some of our components outside of our world.

Of course this is a trade-off, we're dependent on the good judgement of the people who work on Kid, unless of course we want to fork the project.

Now, I think TurboGears is doing the right thing here, but I also think there is a benefit to the centralized control that the Django guys have.

The documentation story in TurboGears is harder, because we have to pull together a clean story that includes each of the components we use. And because those projects all have a lot of internal innovation going on, it's a lot harder to keep up with all the cool new stuff that's coming in. We're definitely struggling with how to do that in the best way.

So, I recognize that we're traiding away some control of our own destiny, and inviting extra documentation and bug-tracking work, in order to get better code-reuse, cleaner decoupling, and more end-user freedom.

As Jacob once said, "it's trade-offs all the way down." I'm happy with the trade-offs we make, even though there is a cost.

I'm sure you're happy with the trade-offs you make. But you should admit that there is a real cost there too.

# Mark Ramm

you know what reminds me of Perl ? this:

>>> from django.template import Context
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
EnvironmentError: Environment variable DJANGO_SETTINGS_MODULE is undefined.

theres nothing "loose" about that, id characterize it more as "arc-welded to a hardcoded idea of runtime configuration".

# mike bayer

NOOO! not the P-word, NOOOOO!

at the same this this is a straw-man argument,

seriously do you think it would be difficult to load the config from another file? As of now very few people would want to use the Django templating system outside of Django. And since there are already lots of settings needed to run a webapp, why not keep them in the same python file as the rest. Is your problem that the setting are in a python file? That is a different issue altogether and many would prefer Python to some kind of configuration mini-language.

In the end this whole discussion comes down the argument of the whole being more valuable than the sum of parts. For any subpart of Django there is a python module/package that is "better". Yet no one has put together a whole package like Django did.

Here is another tidbit, the stuff works, I used cherrypy, paste whatever and you know what, over the long term eventually each had shown some very serious problems, that come from them being more about the concept rather than a program that you need to make a living off. It will take years to iron out those kinds of problems.

Django works, it will hold up in traffic, it will not blow up in your face. That quality it is far superior to any other. Interoperability, reuse, decoupling are fine in theory, in practice adhering to them too strictly just makes life more difficult.


# PixieDust

if i read you correctly, you agree with my sentiment that django's components are not very loosely coupled, your argument being essentially, "so what ? it works." that is completely fine, and im glad django is there, running very well, and attracting lots of new people to Python. its just that, there are definite advantages to componentized and environment-agnostic architectures, and im glad the community has now accepted that there will always be multiple web framework approaches (in constrast to the calls for "one true framework" a couple of years ago) so that those of us who want them can choose to use our own frameworks like Pylons which make this a priority.

# mike bayer

The full-stack/glue distinction is very real, and can be revealed by a couple of practical tests; the best one is to ask where a bug report for a major component goes -- if it gets directed to a third-party project, then you're glue. If you handle it yourself, then you're full-stack.

Like I said, I think Django's choices are easier to justify based on development methodologies and community building, for reasons like this. It's clear who responds to bugs. Django is never stuck when they really want to make a change in something, but the upstream maintainer doesn't agree or doesn't have the same priorities. Django can provide a very concise and clear set of documentation, and activity is all centralized around one basic set of metaphors, mailing lists, bug trackers, etc. And I actually like centralized development, so I'm quite sympathetic to these arguments. (Though as a counterexample: Mozilla is a single project, and it's hardly easy to navigate as a developer.)

As an engineering principle I don't think this distinction is meaningful. That is, from an engineering perspective you can get to the same solution but with slightly different names, and based on names one solution is "full-stack" while the other is "glue". And I don't think that's a reasonable distinction to make. But of course from a mathematical/symbolic point of view names don't mean much, but for humans names do mean a lot.

Again, it's the difference between supporting code that helps other things work together, and supporting a full set of components.

But see, there you start drifting. This is not the difference. You can achieve this regardless of how the packages are laid out.

# Ian Bicking

I agree that the package layout doesn't necessarily impact it (see my other reply in this thread -- it's certainly possible, for example, to be "coupled" while using third-party components, and "decoupled" while using in-house components as the default). And I still agree wholeheartedly that we don't have a good way to talk about this, because the terms that come to mind are all loaded with connotations from other things. I do think there's a clear difference in focus between Django and some of the other frameworks, though -- we are focused pretty much exclusively (partly for historical reasons and partly because of available developer time) on our single stack of components, while other frameworks focus much more on adaptability and interchangeability. And I still think that's a choice which will end up affecting every part of a framework, and that it's not possible for most projects to pull off an equal focus on both, because both routes require major investments of developer time and effort.

# James Bennett

Yea, the Django people have a lot easier time with documentation than we do. On many mornings I find myself envying them.

Which is not to say that I'd give up the code-reuse paradigm in order to make my life writing documentation easier.

# Mark Ramm