Ian Bicking: the old part of his blog

Hash table accused of impersonating OO

From a post by Ted Leung came some links to a Python to Scheme translator that was presented at the 2003 Scheme Workshop.

I found this comments on Python interesting:

Here is my take on Python after working through this "exercise": Python pretends to be an OO-based language yet it is really a hashtable-based that creates an illusion of OO-oriented programming. For simple programs, this works all right but most Pythonistas seem to actually exploit the translucent nature of objects and classes. It is a hopeless task to reason about modules or program pieces in this language. (source)

It is an interesting comment, though certainly greatly biased because the author came at it from an implementation perspective, rather than the perspective of writing Python code. Though really I would say (to quote import this) Namespaces are one honking great idea -- let's do more of those!. Really all the hash tables (dictionaries) are namespaces, and all the namespaces look the same, and none of them are magical.

A surprising number of languages -- Scheme included (where they can't decide on a-lists or hash tables) -- make dictionaries way too difficult. Considering that I use dictionaries almost as much as lists, it's really other languages that should be answering for their deficiencies.

But anyway, I don't actually disagree about what he has to say. Well, you can reason about code, just not in an analytic, programmatic sort of way (which is what he wants to do, because he's writing a compiler). I guess I'm not as concerned, because I prefer programmer empowerment to compiler empowerment (and there are more than enough languages out there that are biased to the compiler, probably because the first programmer is usually a compiler writer ;)

Another comment:

It reminds me of Lisp in its extreme youth where the value and function cells were kept on the symbol property list and people felt free to manipulate them in undisciplined ways. Its very flexible and powerful, but the lack of discipline makes it impossible to reason about. Since those days we've learned that we can achieve the same results in a structured manner. Python is still young. (source)

I'm sympathetic to things that will make a program easier to understand, and I can see how manipulating the object model (which is so very easy in Python) can be an issue. There are greater sins (like Acquisition), but cleverness is a dangerous thing, and there is a great deal of room for cleverness in Python. (Though Lispers aren't one to talk, what with their All Powerful macros) I myself have been prone to cleverness from time to time. I sometimes think the best libraries are the ones that seem most simplistic, sometimes painfully explicit -- they are the easiest to understand, and the least likely to surprise. But then, I tend not to write libraries like that, so there's obviously some disconnect in my thinking.

Created 02 Feb '04
Modified 14 Dec '04

Comments:

I think his comments are correct. He state it's impossible to _reason_ about Python code. That's right. If you ever played with computer aided reasoning and software specifications, you know what he is talking about. There are actually very few languages where it's doable to reason about at least a subset of that language - most functional languages somehow are in that category. There are even fewer languages where you can reason about the full language with all constructs. I am not even sure that Haskell fully qualifies here, but that's one language where I wouldn't be too surprised if it would be possble.

But then, I think that reasoning about software is as much a dead tree as artificial intelligence - at least in the "pure" sense. There are many good ideas coming from the reasoning croud, though, so we shouldn't put them off to bad ;-)

Many new compiler optimizations actually come from reasoning about code. For example type inference is something that best works if your language is structured in a way that makes reasoning possible or at least partly possible - ML or Haskell rely heavily on it, many Lisp and Scheme compilers use it to produce much faster code by infering concrete types where possible and generating concrete code instead of generic code.

AI gave us much better search algorithms and data inference algorithms that run many optimized database engines and rule engines nowadays, btw. ;-)

So I wouldn't take his critic as too critical - it does say much more about the possibility for transparent and automatic optimizations than it says about the actual language itself. It _is_ important if you work on a Python compiler, though, as it says you a lot about the language. Efficient compilation of Python code might need a helping hand by the programmer (PyRex actually shows this quite nicely).
# Georg Bauer

I second Georg here. These comments probably aren't meant to criticize, they are just dry observations from a compiler writer. And they're true (well, at least the first one is). However:

"""Python pretends to be an OO-based language yet it is really a hashtable-based that creates an illusion of OO-oriented programming. For simple programs, this works all right but most Pythonistas seem to actually exploit the translucent nature of objects and classes. It is a hopeless task to reason about modules or program pieces in this language."""

Yes, that's really all what objects are about, they're dicts with some sugar and special rules. Some people think that "true" OO is the way Java, C++ or Eiffel do it... maybe this compiler writer is one of them. I disagree; at least Smalltalk's notion of OO was never like that.

I read a paper a few months ago, stating that OO doesn't fulfill its promises *because* Java, C++ and the like twisted it, turning something that was originally dynamic into a static and rigid mechanism.

It's understandable that a compiler writer would like to reason about "modules or program pieces"; most Python programmers don't feel the need for this, though. I guess they'd much rather just *use* the language. :-)
# Hans Nowak

Hmm. Smalltalks OO is quite different from most all other OO implementations. The only systems that are a bit like Smalltalk OO are those implemented in Scheme or other Lisps. So it's not like much other implementations :-)

But OO itself is a rather weird concept, allmost every class of languages has it's own idea what it is. Most Algol derived languages see OO as structures with code. Most scripting languages somehow have a objects/classes as hashes (or something similar) idea. That's easy to see why it happened: hashes are quite handy when modelling OO in those class of languages.

Scheme usually uses different approaches: Objects are much more encapsulated, often they are implemented as a bunch of defines (the methods) over some closure (the state). So someone coming from Scheme background will see much more the control flow aspect of objects than the data representation aspect! That's where this special criticism might come from: for someone coming from a defines-over-closures-over-instance-variables aspect, this hashes/dicts-as-objects must look really ugly.

I used to upset Common Lisp programmers by giving references from Common Lisp idioms to Perl idioms and showing where similar concepts existed. Most times they didn't get over the step to accept that Perls object _implementation_ are hashes, just because that was a handy datatype to use. Common Lisp people are _really_ weird with respect to OO, as their CLOS is much more than what people usually see in OO: they have structure implementations, that's standard. But methods in Common Lisp are not bound to one class, they are specified on all parameters. So you actually don't have the view on objects as code+data - you have objects as data and generic functions as code, where the compiler (and sometimes the runtime) does the magic to automatically find the most specific method for a generic function. This is very nice if you think in functional programming terms. Actually I would prefer that every day over Pythons rather crude object system. Only problem is that I would need all those nice extension modules and especially the nice and terse syntax of Python. I can't stand the bloated identifiers common lisp uses :-)

And let's don't even start to discuss Haskell Type Classes ...

There is a very good book on all this: "Theoretical aspects of Object-Oriented Programming", MIT Press, Gunter and Mitchell Editors. Highly recommended if you want to see what OO actually can be. Other stuff to study would be the AI stuff about frame languages like loops or flavors - those were predecessors to the Common Lisp CLOS system and heavily propagated the objects-as-data-chunks-with-slots with seperate function binding ideas.

It's rather bad that nowadays primitive object oriented systems like Java or C++ dominate the thinking of language designers, as that largely hinders language design. Ok, there is allways Ruby, that seems to be written by a Smalltalk+Scheme-Fan.
# Georg Bauer

Another way of thinking about OO is in terms of messages -- at least, that was Alan Kay's preferred metaphor (even though classes then became the norm). Python does very well at this, and if all you think about is messages then Python is more complete than most OO systems. Classes are just one kind of object, and types are one kind of object, but Python's object system is really defined by the set of rules that control getattr (or its syntactic form of ".").

Classic message-based programming paradigms like proxy objects are very simple in Python. In fact, I think they are simpler than in Smalltalk, though they are complicated by the presence of first-class functions/methods. But then, the hackability of methods might be a big plus (or maybe it's too clever and leads to things like Acquisition ;)

Anyway, I don't entirely disagree with the people I quote, but I think that they might not have noticed the namespaces because they were paying too much attention to the hash tables. Dynamic namespaces are the essential component of Python, and of course you'll implement those with hash tables. But you'll never see the namespaces from the implementation alone, they are merely implied.

Really, the justification of Python is also what they use as a criticism: "most Pythonistas seem to actually exploit the translucent nature of objects and classes". This translucent nature causes problems, and it would just suck if it wasn't also very useful. But it is useful, and the intelligent programmer can utilize Python's internal mechanisms to implement many things that would have to be done more explicitly (and with no greater efficiency) in another language.
# Ian Bicking

Oh, of course it is usefull. And PyDS uses several of those weird things :-)

The problem isn't that it's usefull or not. The problem is, that the quotes came from a very different angle. Lisp OO systems have their own weird way doing stuff that make both life easier and more complicated. If you ever happen to look at Metaclasses in Common Lisp, you will see discussions about metaclasses in Python rather silly ;-)
# Georg Bauer

I was just pointing out a contrast. For instance, in Tcl you can do something like:

set x 1+2
set y [expr 3*$x]

And y == 5, because [expr 3*$x] expands to [expr 3*1+2]. Crazy stuff. Useful? Not at all. A pain for compiler writers? Indeed. So sometimes you can get the worst of both worlds, a form of generality that no one wants, but somehow can't be eliminated.

(Of course, none of us are really disagreeing on anything here, just to clear that up)
# Ian Bicking

I, in fact, as a programmer, find it a lot easier to think of Python in terms of hashtables and namespaces rather than objects. That's perhaps because of how it's implemented. I came to Python from a OO mindset and found it difficult to learn in the beginning, but from the point I realised it's name-value pairs all the way, it started feeling like a toy.
# Manish Jethani