Ian Bicking: the old part of his blog

Kernel, meet Python

I just started reading about LUFS-Python via Jkx@Home, and now in another entry by Simon Willison.

LUFS is Linux Usermode File System; a virtual file system implemented by a normal process (as opposed to being compiled into the kernel). It's a neat idea, and one which should have come about a long time ago. This kind of feature was one of the selling points of GNU Hurd (but that shouldn't scare you away from it ;).

Simon comes up with a couple ideas I hadn't thought of -- it wouldn't have occurred to me to use LUFS to underly other protocols, like FTP or SMB, or even DAV. But once you have files, these servers already exist.

I'd definitely like to experiment with this, but at the same time there were significant problems the last time I used LUFS. At the time I was looking for a WebDAV filesystem for Linux, and I didn't want to install Code and a bunch of other things that davfs2 required (though I may yet try that again). LUFS also seemed like the Right Way. Now the DAV option seems to have disappeared, or maybe I was imagining it and I was only using sshfs; either way, I did manage to get it working. Besides having bugs (which happens), the things was really slow. Not just network-filesystem slow, but I-can't-use-this slow. My perception was that things that seemed trivial to me (like tab completion) resulted in expensive calls. The performance characteristics of these filesystems just didn't map to my expectation, or the expectation of the tools I use. I doubt grep or find would be a very effective. If you take a different track, something like Tramp for Emacs (which I find absolutely indispensable) works at a higher (application) level, and has the opportunity to use that to its advantage [ouch, Savannah is down, but you can read some here for now]. This is also the advantage of similar systems like KDE's ioslaves, and Gnome VFS.

Some of this can probably be cleverly fixed, through judicious use of caching and prediction. This is why Python could make this a lot more interesting, because Python is such an accessible language that you can build all these features without going mad. For most of these filesystems, the bottlenecks (I predict) are at a level where Python is Fast Enough, and a well optimized Python program could beat a poorly optimized C program.

The other big issue I see is that filesystem semantics aren't correct for some of these virtual filesystems. We see this in Windows Web Folders, where applications require special modification (even if minor) to use these files correctly. Ioslaves and VFS try to do the same thing. But if you work in an entirely application-neutral manner, like LUFS, how do you deal with varying semantics? These are things like WebDAV locking or versioning, or RDBMS transactions, and it continues for almost all things that seem at first blush to be filesystem-like.

I have too much stuff to work on, but this still has me quite excited, I'll definitely have to play with it.

Created 10 Dec '03
Modified 14 Dec '04