Ian Bicking: the old part of his blog

Re: The challenge of metaprogramming comment 000

I've made extensive use of Scheme macros on a project with many novice coders. The macros define a domain-specific language, and have been largely responsible for the success of the project to date (i.e., asking our coders to use a general purpose language for domain-specific problems would almost certainly have been a disaster).

We seriously evaluated using Python for this project, and reached two conclusions:

  1. Python was a far easier language to learn than Scheme.
  2. There was no good way to represent our problem domain in Python. Users would be forced to add gratuitous levels of class structure and indenting, and they're have to do it right every time. Unfortunately, some of the defining characteristics of novice programmers are that they don't reliably follow style-guides, they don't understand complex coding conventions, and they don't deal well with extra levels of indentation.

So we wrote ten pages of fairly complex macro code which massively reduced the complexity of hundreds of pages of domain code. Management knows how hard it will be to hire someone to maintain the macro code, and they still consider it an acceptable tradeoff--we gain so much elsewhere that a small amount of guru-level code is OK.

I've contributed to largish Python projects (including Zope) and these problems do occur elsewhere. The Zope security system, in particular, requires programmers to add security "declarations", which are actually fiddly little snippets of executable code, each of which needs to be in almost exactly the correct place, and few of which are meaningfully error-checked. If Python had a decent macro system--or even C#-style class and method adjectives--Zope's security system could be made completely declarative, which I think would be a win. Pythonistas may disagree, of course.

I'm not arguing that Python should include macros, not even simple and elegant ones (if such are possible). After all, language design is an art, and Guido needs to balance many competing factors to make a widely accessible language.

I am, however, arguing that on certain real-world projects, Python's lack of macros cripples it so badly that even as esoteric a language as Scheme can ultimately be more usable for novice programmers. Python is a general-purpose notation--perhaps the best I've ever seen--but a general-purpose notation can be significantly harder to apply than a good domain-specific notation.

Comment on The challenge of metaprogramming comment 000
by EK


I think "cripple" is a little extreme. But I do agree that we need good metaprogramming constructs, and I'm really very interested in that, I'm just not that psyched about some particular techniques.

And there's a lot more opportunities to do metaprogramming now than there were a few years ago -- in part because of changes in the language (mostly Python 2.2), and in part because we're learning better techniques to use those features. There are a lot of avenues for exploration, most of which are probably a bad idea ;) And we have what I'm guessing is the same as method adjectives in Python, I believe at the bequest of Zope people, though it was too late for them to use them -- functions can have arbitrary attributes with arbitrary values (since Python 2.0 or 2.1?) And of course classes have always had this ability. Or maybe you are thinking of something else?

# Ian Bicking