I was looking at Phillip's
CLOS-style
generic functions
recently, and it got me thinking about metaprogramming in Python. If
you haven't looked at those generic functions, it's a very neat idea
and implementation, and can be expressed quite elegantly in Python
(at least after adding 2.4's @ decorator syntax). (This post
is another recent item that got me thinking about metaprogramming.)
Python doesn't have much of anything like macros (which CLOS generic methods are implemented with, I believe), but it doesn't seem like a big impediment. And there's a lot of metaprogramming that is done in Python, without a macro in sight. By metaprogramming I mean programming structures that operate on other programming structures. Every decorator is a case of metaprogramming; the decorator is a function that takes a function as an argument, and returns a function (or function-like) object.
There's a lot of ways you can do metaprogramming in Python. Decorators are one way, and metaclasses are another. But it can be easier than that; you can modify classes on the fly, you can you can do all sorts of things with higher-order functions (i.e., functions that take functions as arguments), and even do funny sorts of things with import hooks, the new module, etc.
What makes this possible is the fact Python is a scripting language.
I know some people hate that term; but I think it means something
very specific, not merely derogatory. A Python module is a
script; when Python loads a module, it reads the file,
evaluates the first line, the second line, etc., until it's done.
Functions aren't declared; rather def creates a
function on the spot; you can redefine the function later, or define
it inside another scope, or delete it, or whatever. This causes
problems (like the problem of changing code in long-running
processes, or circular references); but it also has some advantages.
When we use a module, we can forget that we aren't using the source
we see in its .py file. We're using the objects that are
created by that source. When loading a module, we're running
a bunch of statements; some statements are rather benign with few
side effects (mostly def); others are somewhere in
between (like class); and others are executed
immediately, and we only have access to their effects. Decorators
and metaclasses are two instances; but you can put any statements in
a module, and any of them can modify what's in the module. You can
even modify modules after they are loaded (a
monkey patch).
There are some problems. You can't look inside code programmitically (or at least it isn't easy), and lazy evaluation is fairly limited. The generics package is forced to use strings for its expressions, then compile and execute them in a closed environment. SQLObject uses SQLBuilder which uses Python's magic methods, but is limited; Numeric uses something similar. Still, there's a huge amount you can do, and all without any macros, all simply due to the highly imperative nature of Python source.