Ian Bicking: the old part of his blog

The challenge of metaprogramming comment 000

Hi, Andreas! I worked with you in the early years of the open-source Gwydion Dylan project, and I'm glad to see you're making so much progress. Dylan is still one of my favorite languages.

I stand by the argument that Dylan macros are a kludge. They work fine for relatively simple source-to-source transformations--largely thanks to a lot of "do what I mean" magic in the spec--but as soon as you try to do anything complicated, they're at least an order of magnitude harder than creating LALR(1) grammars. Look at how the pattern "var :: type = expr" actually binds; it defaults in helpful ways, but it's incredibly arbitrary and ad-hoc. These individual kludges add up, and by the time you're building a large, novel macro, they become pretty overwhelming.

(I'll have to take a look at the dexpr proposal. It sounds quite promising, but I imagine it still has a few kludgy issues with Dylan's low-level pattern grammar.)

I think the best macro system, to date, is the combination of Scheme R(n)RS high-level macros and the Chez Scheme low-level macro system. It has all the hygiene Common LISP lacks, and it's only slightly harder to write programmatic macros in Chez Scheme than in Common Lisp.

I think the answer for truly good infix macros requires an extensible grammar system. However, every extensible grammar system I've examined to date has been ugly. That's because they all allowed programmers to extend an underlying LALR(1)-style grammar (or something similar). LALR(1)-style grammars, unfortunately, are tricky and--most importantly--completely non-modular. You can't easily combine multiple macros from different modules, you can't produce good error messages, and you can't formally explain why certain things work and other things don't.

Basically, I think the key to good infix macros is to replace LALR(1) with a different, more-modular parsing model, and then adapt as much of the LISP/Scheme approach as possible. LALR(1), quite frankly, is a clever kludge, and you can almost certainly get by with something a lot simpler if you make a couple of assumptions. I'd be delighted to discuss those assumptions with you in e-mail.

Comment on Re: The challenge of metaprogramming comment 000
by Eric Kidd


Hi Eric!

Didn't hear of your for quite a while... If you have the time, please visit us at IRC channel #dylan, on irc.freenode.net. I'm sure we have much to discuss.

But yes, implementing infix macros by adding rules to the grammar is a thought that has crossed my mind. This would be much more straightforward to implement, and give better error reporting.

In case you haven't seen it yet, take a look at Peter Housel's Monday project (http://monday.sf.net). It contains a nice extensible parser framework.

# Andreas Bogk