Ian Bicking: the old part of his blog

FileSystemView vs. LocalFS

From a comment in my last post I looked at FileSystemView as an alternative to LocalFS. FSView certainly feels cleaner, but I'm not 100% sure... here's my initial thoughts:

My conclusion: unless I'm missing something, it's mostly a wash (at least given the development I've already done on LocalFS). FileSystemView may have more of a future; it's not even clear who is developing LocalFS at this point (though I assume the SF project is defunct). FSView avoids some of LocalFS's complications by being read-only; of course, filesystem permissions can make LocalFS thoroughly read-only too, avoiding some potential problems. FSView's policy of looking in Products annoys me -- it might make sense in the context of CMF or some other system or company, but it makes it more difficult to adapt to our local conventions. I dislike putting policy into code; internally to an organization it's probably good, but for public code it's a burden.

There was also a suggestion to use APE. APE is a much larger project, but also seems to be pretty actively developed. In the past it didn't seem that compelling to move stuff out of the ZODB and into a database or filesystem if the objects were still opaque. But apparently it has custom mappings for some types (probably all the types I'm interested in), which would presumably make the files pretty transparent.

Unfortunately APE requires Zope 2.7, and we're using Zope 2.6 here. There's more and more reasons we need to upgrade to 2.7... but anyway, both LocalFS and FileSystemView work fine with 2.6, and I don't want to delay this by adding another task to the mix -- I'd rather switch products later than delay moving code into the filesystem/repository. Ideally the product will allow the logical placement of code on the filesystem, so that moving to a different product won't be too complicated.

Then, just now, I get a suggestion for zfolder2product. This also requires Zope 2.7, and seems to dump ZODB objects to the filesystem, then allow you to put them back into the ZODB from those files -- the object that recreates those ZODB objects is the "product" you create. I suspect the synchronization is less transparent than LocalFS or FSView (which are basically live views of the filesystem), but I'm not able to try it yet.

Actually, the one thing I'm worried about is security, something that has always made me uncomfortable in Zope (part of the nature of its pervasive object publishing metaphor). FSView doesn't allow permissions to be set at all (below the root folder), and LocalFS does but it lies -- it forgets all settings. This might be something to I can hack onto LocalFS, but it makes me uncomfortable, and my initial experiments have been painful (I blame Acquisition once again... blast you Acquisition, you are my nemesis!)

I suspect APE supports security restrictions and all those other possible annotations well, and zfolder2product can probably support it with a little work because of its imperative nature (you could just add another task to the end of the ZODB-object-creation step that fixes up the permissions).

Created 22 Feb '05

Comments:

APE saves everything - permissions, properties, source, etc, since it is a ZODB storage layer and not some magic folder or object that reads and writes things it knows to the file system. So everything gets saved. When it serializes an object, it goes through a chain of objects that can store different pieces of it, and an event tracks what persistent attributes have been dealt with. So something with properties uses the properties serializer which writes the properties to a file (or RDBMS table, etc) in a usable format, and marks those attributes as having been dealt with. Then the next serializer might be the one responsible for storing permission settings (since those are also persistent attributes). The next might serialize the text source (Python Script, ZPT Page, etc) or binary source (file, image), etc. If after all of these serializers have been used and there are still persistent attributes remaining that haven't been stored or marked as used yet, those get written out in a pickled format. Changes made on the file system, or new files added in the file system, get picked up immediately. And because it's a ZODB storage, nothing changes in how you interact with the object in the ZMI.

My terminology and specifics about how it works may not be 100% correct, but that's my understanding of how it worked as of Ape 0.7. Since 0.7, the design has actually been simplified.

Ape 0.7 worked with Zope 2.6, and I think I still have a copy of it. Or maybe it was Ape 0.6. If you're interested, I've put those old copies online at toulouse.share.

If I recall correctly, things did get a little funny when CVS was installed (basically because all of the CVS directories and their contents show up in Zope). But it worked really really well for dumping a site to the file system, editing it on the file system, and then even copying it back into a regular FileStorage based area of the ZODB - it did it all transparently, and nothing was lost.

# Jeff Shell

Thanks -- I'm trying it out now. It seems like it creates an ini file for the properties; a boring example script called test gives me a test.py file and a .test.py.properties file that looks like:

[classification]
class_name=Products.PythonScripts.PythonScript.PythonScript

[security]
local-role role="Owner" username="ianb"

That seems pretty workable. You have to edit a configuration file to make new directories available, so it can't be done purely through the ZMI (and I suspect requires a server restart).

It looks like a pretty good basis for putting legacy ZODB code into a repository, without having to mess with the code (which would make us unwittingly turn legacy code into current code, which is what we're trying to avoid ;)

# Ian Bicking