While we're at it: I noticed several times that the availability of your blog also was very bad (to me it was something like 25% of my tries this site wasn't available). Was that also due to your "attempt to avoid database" or were there other reasons. You - as someone who is quite exposed on the Python web development front - shouldn't have such a bad availability (as this can also fuel the anti-python fud). If this is the fault of your hosting company you should obviously change to another one.
It's mostly the software that's causing the problem. There's been a few times when the Subversion server (previously under the same Apache instance) caused some collatoral damage, but mostly the software itself. There's two scaling problems the software has: everything is kept in flat files, but the actual data model is fairly relational. And there's now quite a few posts and comments (about 8000, I guess), and some of the operations require scanning the contents of those files (not many, but a couple). To help that situation, I then foolishly thought I'd keep indexing information in a BDB database. That's fairly fast, but maintaining indexes in an ad hoc fashion is a horrible idea -- it's hard, and you accomplish nothing you couldn't do better with a more general system. And BDB does not operate well under load, especially with concurrent access. That ended up being the biggest problem, with the process just freezing up regularly (usually with no error, it simply stops responding and won't shut down properly).
Right now I don't think I use anything in this blog that I use regularly elsewhere.# Ian Bicking