Ian Bicking: the old part of his blog

A More Perfect App Server: WSGI Transaction Manager

Chris McDonough is speculating on a "more perfect app server". Of course, I'd say that would be a WSGI stack (a-la Paste). But how that works might not be obvious. I thought I'd speculate on what a "transaction manager" might look like in this system. I'll just include part of the untested code:

class TransactionManagerMiddleware(object):

    def __init__(self, application):
        self.application = application

    def __call__(self, environ, start_response):
        environ['paste.transaction_manager'] = manager = Manager()
        # This makes sure nothing else traps unexpected exceptions:
        environ['paste.throw_errors'] = True
        return wsgilib.catch_errors(application, environ, start_response,
                                    error_callback=manager.error,
                                    ok_callback=manager.finish)

class Manager(object):

    def __init__(self):
        self.aborted = False
        self.transactions = []

    def abort(self):
        self.aborted = True

    def error(self, exc_info):
        self.aborted = True
        self.finish()

    def finish(self):
        for trans in self.transactions:
            if self.aborted:
                trans.rollback()
            else:
                trans.commit()

And you use it a little like this:

def my_app(environ, start_response):
    manager = environ['paste.transaction_manager']
    manager.transactions.append(MakeTransactionObject())
    ...

application = TransactionManagerMiddleware(my_app)

I created the function paste.wsgilib.catch_errors for this, but only because it's something that's come up before. catch_errors would be pretty simple if it wasn't for the iterable that applications can produce.

This is only one of several features Chris mentions, but my point is one of proof-of-concept -- most of the features he wants can be implemented separately and in a highly decoupled way using WSGI middleware (or sometimes simply mundane libraries). The advantages aren't just testability, but also simplicity (include what you care about, replace what's broken for you) and the potential for the kind of distributed development that open source relies on.

Created 11 Jul '05

Comments:

The transaction manager in zope is decoupled and already provides a much more robust and well tested interface.

http://svn.zope.org/ZODB/trunk/src/transaction/

Only real dependency is the zope.interface package.

I know its fashionable to rewrite things from scratch in python, and to bash on zope, but its just sad sometimes how much stuff people put out that just can't meet any real world applications beyond their initial design constraints or toy apps, because they were never written with flexibility in mind or the needed domain expertise.

the zope developers are solving complex real world problems today, might as well learn something from their expertise.

# anonymous

Sure; this was just an example, and I wasn't proposing it replace anything; that's why it just went in a scratch directory. The code you link to would replace Manager, and provides interfaces for the transactions themselves; except it calls Manager a Transaction, and the objects it contains DataManager. A lot of the additional code it provides seems to be for logging and debugging, and two-phase commit. All good code, I'm sure.

In generally WSGI/middleware applies to the integration of code like that into the request cycle; if the transaction handling code was linked to WSGI that would be a bad sign. It's best that the integration code be short, since the transaction code should also be usable outside of a web request, and if the integration code is long then that's a sign the code being integrated is obtuse or incomplete.

# Ian Bicking

wrap the Zope transaction manager into a WSGI application interface as above and the world can take advantage of its great maturity.

# mike bayer

I think for 90% of the use cases, Zope's implementation is far too complicated. In typical cases, you're using a DBAPI driver and you have a database connection (perhaps pooled, but this is an orthogonal issue) per request. In the end, if there is an exception, you rollback; otherwise you commit. It isn't that hard. I'd rather not include a huge chunk of code like that in my webserver unless I absolutely need it. Minimilism is a very good thing.

# Clark Evans

+100 on the pipelining of components ala WSGI/Paste to get some of this stuff done. I think wherever it's appropriate to factor something out into WSGI middleware (authentication is a great example, as are sessions, an error renderer, and maybe a txn manager), it feels really good to do so. I'm going to try to take a closer look at Paste as I continue thinking about what would make a great appserver stack.

# Chris McDonough