Implementing multimethods in pure Clojure

47 views
Skip to first unread message

Alexander Yakushev

unread,
Dec 11, 2010, 3:32:55 AM12/11/10
to Clojure
I am currently giving some lectures about Clojure to a group of
students. One of the Lisp features I promote to them is the ability to
write language in the language itself. So during the lecture when I
talked about multimethods one student asked if one could write own
multimethods implementation if Clojure didn't have them. I went all
like "Sure, you just..." and stuck. My first thought was to save
multimethod into an atom and the list of methods into metadata of that
atom, but as I found out atoms cannot hold metadata.
Desperately I looked into Clojure sources for implementation of
multimethods and saw that it is done using the MultiFn Java object. I
guess this is done for performance and there is still a way do this in
pure Clojure (I mean, in pure Lisp).
Thanks in advance for helping!

Ken Wesson

unread,
Dec 11, 2010, 10:37:54 AM12/11/10
to clo...@googlegroups.com

As a first stab, I'd use an atom holding a list of dispatch-fn,
map-of-argfn-return-values-to-methods, default-method

Something like:

(defmacro defmethod [name dfn]
`(def ~name (atom
[~dfn
{}
(fn [& args#]
(throw (IllegalArgumentException. (str "no dispatch for " args#))))]))

(defn add-method [name dfn-val meth]
(swap! name assoc-in [1 dfn-val] meth))

(defmacro addmethod [name dfn-val bindings & body]
`(add-method ~name ~dfn-val
(fn ~bindings ~@body)))

(defn set-method-default-meth [name meth]
(swap! name assoc 2 meth))

(defmacro setmethod-defaultmeth [name bindings & body]
`(set-method-default-meth ~name
(fn ~bindings ~@body)))

(defn call-method [name & args]
(let [a @name]
(apply (get-in a [1 (apply (a 0) args)] (a 2)) args)))

This doesn't bind the method name itself to a callable function; you
have to (call-method name args ...) rather than (name args ...). It's
also untested. But it should give you some idea of how something like
this can be implemented.

The default method is called if the dispatch function's return value
isn't found in the map. The default default method is the IAE throw in
the first macro. Methods can be replaced by doing a fresh addmethod
with the same dispatch value, but I didn't bother to include a
deleter. It should be simple enough to implement with dissoc. (It's a
shame there isn't a dissoc-in. Oh, wait, you can easily write your
own:

(defn dissoc-in
"Removes an entry in a nested associative structure.
(= (dissoc-in {:a {:b 1 :c 2} :d {:e 3 :f 4}} [:a :c])
{:a {:b 1} :d {:e 3 :f 4}})"
([m keys]
(if (= 1 (count keys))
(dissoc m (first keys))
(let [ks (butlast keys)
k (last keys)]
(assoc-in m ks (dissoc (get-in m ks) k))))))

(this one IS tested).)

Making the thing work with (name args ...) is not too too difficult.
You'd have to have defmethod output both a def of an atom like above,
but with a gensym for a name, and a defn with the specified name that
has the body of call-method, more or less, but with the name arg fixed
to the gensym. There'd also need to be a global names-to-gensyms table
somewhere to make addmethod and the like work.

This variation looks like (defmacro defmethod [name dispatch-fn] `(do
(def ...) (defn ~name ...))).

Another alternative is for defmethod to def the name to a function
that closes over the atom and, via special sentinel arguments,
implements all the functionality of addmethod etc. as well as
call-method. When called without sentinel arguments it does a normal
call-method; when called as (name :add dispatch-val meth) it adds a
method; etc.

This variation looks like (defmacro defmethod [name dispatch-fn] `(def
~name (let [a (atom ...)] (fn ...)))).

Meikel Brandmeyer

unread,
Dec 11, 2010, 11:56:31 AM12/11/10
to clo...@googlegroups.com
Hi,

I suppose you are Unlogic from IRC. I don't whether you saw it, but I posted some rough sketch: http://paste.pocoo.org/show/303462/

It just introduces the function binding, no other global objects are introduced. The methods are stored in a map in an atom in the metadata of the Var of the multimethod. I haven't tested things, though. Implementing isa? dispatch and other sugar is left as an excercise to the astute reader. ;)

Sincerely
Meikel

PS: atom may have metadata! For comparison, see also the metadata on the Var.

user=> (def x (atom 0 :meta {:meta :data}))
#'user/x
user=> (meta x)
{:meta :data}
user=> (meta #'x)
{:ns #<Namespace user>, :name x, :file "NO_SOURCE_PATH", :line 2}

Alexander Yakushev

unread,
Dec 11, 2010, 5:01:24 PM12/11/10
to Clojure


On Dec 11, 5:37 pm, Ken Wesson <kwess...@gmail.com> wrote:
> On Sat, Dec 11, 2010 at 3:32 AM, Alexander Yakushev
>
> Making the thing work with (name args ...) is not too too difficult.
> You'd have to have defmethod output both a def of an atom like above,
> but with a gensym for a name, and a defn with the specified name that
> has the body of call-method, more or less, but with the name arg fixed
> to the gensym. There'd also need to be a global names-to-gensyms table
> somewhere to make addmethod and the like work.
>
> This variation looks like (defmacro defmethod [name dispatch-fn] `(do
> (def ...) (defn ~name ...))).
>
> Another alternative is for defmethod to def the name to a function
> that closes over the atom and, via special sentinel arguments,
> implements all the functionality of addmethod etc. as well as
> call-method. When called without sentinel arguments it does a normal
> call-method; when called as (name :add dispatch-val meth) it adds a
> method; etc.
>
> This variation looks like (defmacro defmethod [name dispatch-fn] `(def
> ~name (let [a (atom ...)] (fn ...)))).

Thanks a lot! That was particularly what I was asking. I got the idea
of defining some map of dispatch-values - functions, but my thoughts
ended up either at defining multimethod macro and multimethod map atom
separately, or using syntax like ((call method-name) args...). The
last part of your response was a great help for me!
Message has been deleted

Meikel Brandmeyer

unread,
Dec 11, 2010, 5:24:52 PM12/11/10
to clo...@googlegroups.com
Hi,

Am 11.12.2010 um 23:10 schrieb Alexander Yakushev:

> Thanks for your response! Your example is very useful, though I wanted
> to implement the multimethods without that multi-call layer, so it
> will look just like an ordinary function. Thanks to Ken Wesson I
> already have an idea how to do this.

I'm a bit confused. It just looks like a normal function call.

(my-defmulti foo type)

(my-defmethod foo String [x] (str "A String: " x))

(foo "Hello, World!")

So it just looks like an ordinary function. Extracting the multi-call function saves code size, eases macro development and allows to change the underlying driver function for all multimethods while working on it. Very helpful, because you don't have to re-call the my-defmulti macro, but still have the changes take immediate effect.

> Oh, that's my fault, I tried with-meta function on the atom and it
> wouldn't work. Still, after I defined an atom with some metadata in
> it, how can I change it thereafter?

I believe, you can't. You have to create a new atom.

Sincerely
Meikel

Ken Wesson

unread,
Dec 11, 2010, 5:36:48 PM12/11/10
to clo...@googlegroups.com

You can "change" the metadata on the object held by the atom (if that
object supports metadata) via (swap! a with-meta ...).

One thing a bit annoying is if you want to alter the metadata in an
incremental way. To do that atomically requires a closure. Or defining
a swap-meta! function, like so:

(defn swap-meta! [a f & args]
(swap! a
(fn [x]
(with-meta x (apply f (meta x) args)))))

That abstracts the "do it with a closure" method into a single function.

user=> (def x (atom (with-meta [] {:foo 1})))
#'user/x
user=> (meta @x)
{:foo 1}
user=> (swap-meta! x assoc :bar 2)
[]
user=> (meta @x)
{:bar 2, :foo 1}

Alexander Yakushev

unread,
Dec 12, 2010, 5:14:35 AM12/12/10
to Clojure


On Dec 12, 12:24 am, Meikel Brandmeyer <m...@kotka.de> wrote:
>
> I'm a bit confused. It just looks like a normal function call.
>
> (my-defmulti foo type)
>
> (my-defmethod foo String [x] (str "A String: " x))
>
> (foo "Hello, World!")
>
> So it just looks like an ordinary function. Extracting the multi-call function saves code size, eases macro development and allows to change the underlying driver function for all multimethods while working on it. Very helpful, because you don't have to re-call the my-defmulti macro, but still have the changes take immediate effect.

Oh, I'm so inattentive! I had not understand at first how the last
macro my-defmethod worked, but saw the call-multi function and thought
it was about calling additional function to extract the method. Now I
got I fine and see that it is really what I need! I very much
appreciate your help.

Alexander Yakushev

unread,
Dec 12, 2010, 5:22:30 AM12/12/10
to Clojure


On Dec 12, 12:36 am, Ken Wesson <kwess...@gmail.com> wrote:
>
> You can "change" the metadata on the object held by the atom (if that
> object supports metadata) via (swap! a with-meta ...).
>
> One thing a bit annoying is if you want to alter the metadata in an
> incremental way. To do that atomically requires a closure. Or defining
> a swap-meta! function, like so:
>
> (defn swap-meta! [a f & args]
>   (swap! a
>     (fn [x]
>       (with-meta x (apply f (meta x) args)))))

I still very often mix up the state and identity in Clojure. What I
tried to do is to add metadata to the atom itself, not to the value it
holds. Now I see that it was kind of stupid:).
Your swap-meta! function is of great use, from now on I will use in
such cases. Thank you!
Your swap-meta!

Alan

unread,
Dec 13, 2010, 3:27:57 PM12/13/10
to Clojure
That function is already written for you.

user=> (def x (atom (with-meta [] {:foo 1})))
#'user/x
user=> (meta @x)
{:foo 1}
user=> (swap! x vary-meta assoc :bar 2)
[]
user=> (meta @x)
{:bar 2, :foo 1}

Alan

unread,
Dec 13, 2010, 3:31:22 PM12/13/10
to Clojure
See my reply to Ken. I recommend against writing swap-meta! in your
own code, except maybe as a shorthand for (swap! foo vary-meta);
certainly don't implement it from the ground up when the language
already gives you the function you want.

On Dec 12, 2:22 am, Alexander Yakushev <yakushev.a...@gmail.com>
wrote:

Ken Wesson

unread,
Dec 13, 2010, 3:42:12 PM12/13/10
to clo...@googlegroups.com
On Mon, Dec 13, 2010 at 3:27 PM, Alan <al...@malloys.org> wrote:
> That function is already written for you.
>
> user=> (def x (atom (with-meta [] {:foo 1})))
> #'user/x
> user=> (meta @x)
> {:foo 1}
> user=> (swap! x vary-meta assoc :bar 2)
> []
> user=> (meta @x)
> {:bar 2, :foo 1}

Not exactly. My swap-meta! is a bit more concise to use. (Where did
you find vary-meta? There seems to be a lot of stuff that's there, but
hardly anyone knows about.)

Meikel Brandmeyer

unread,
Dec 13, 2010, 5:38:44 PM12/13/10
to clo...@googlegroups.com
Hi,

Am 13.12.2010 um 21:42 schrieb Ken Wesson:

> (Where did
> you find vary-meta? There seems to be a lot of stuff that's there, but
> hardly anyone knows about.)

http://clojure.github.com/clojure/

Hope that helps.

Sincerely
Meikel

Ken Wesson

unread,
Dec 13, 2010, 5:52:53 PM12/13/10
to clo...@googlegroups.com

That's not what I meant. I figure all of us have tabs permanently open
to there (I have two actually). What we don't have is the whole thing
memorized, or the time to read it all rather than use it for reference
and, sometimes, control-F it for things. In particular, my due
diligence with atoms and metadata consisted of searching for functions
that work directly with atoms. Of course vary-meta isn't one of them.
My question of Alan was really more a matter of how he found it --
what called his attention to it.

I don't think there's actually a serious problem with the
documentation here; people can't memorize the documentation as there's
too much of it, nor read it all in advance for the same reason, and
search on particular topics won't always find something useful that's
not quite directly related to that topic, but there's little that can
be done about it. The documentation can't be made much shorter without
making it incomplete, nor can every search scenario be anticipated in
advance. So, sometimes people will not know about various functions,
especially more obscure ones that are uncommonly used.

The only problem, in other words, and it is a minor one, is when
people post (possibly only implied) criticisms of other people for
happening to not know about one of those things. That will lead to a
fair amount of time being wasted both on posting such criticisms and
on defending against same by pointing out what due diligence one
performed and why it didn't find whatever particular thing.

Note: that last is not directed at you, in particular, Meikel. It was
Alan (notably the only partly-anonymous user in this thread thus far)
that was a bit abrupt and seemed to be suggesting that I had done
something wrong, or at least questionable, and then explicitly
recommended other people ignore my code (despite the fact that it
works perfectly), and most likely he simply either got up on the wrong
side of the bed or didn't quite mean it that way but misstated
something. At the same time, it was also Alan who called our attention
to the existence of the vary-meta function. Make of that what you
will.

Meikel Brandmeyer

unread,
Dec 14, 2010, 4:22:26 AM12/14/10
to clo...@googlegroups.com
Hi,

Am 13.12.2010 um 23:52 schrieb Ken Wesson:

> That's not what I meant. I figure all of us have tabs permanently open
> to there (I have two actually). What we don't have is the whole thing

> memorized, or the time to read it all rather than use it for reference […]

My solution to this problem is actually quite simple: I took the time to read it. Not all at one day, but slowly function after function. If I saw a function in other people's code, which I didn't know, I looked it up. Or I read through the overview list and thought „WTF does alter-var-root do?“

Then it was either something „Dang! I need this twice a day. Why don't I know about it?“. Then I would certainly not forget it anymore. Otherwise I would at least remember „There was something.“ and know where to search in the documentation to find the details again if needed.

Of course there are also functions, which I constantly forget. for is such a candidate. When it was new I *always* forgot about it. I built complicated mapcat-reduce-map combinations, which where a simple, short for. Even nowadays I have to remind myself about for every now and then. Such things trickle in slowly.

However, this process is called „learning“ and there is no short-cut to it. This process takes time. There is no „I know Kung-Fu.“ in the real world. cf. http://norvig.com/21-days.html

Sincerely
Meikel


James Reeves

unread,
Dec 14, 2010, 4:52:33 AM12/14/10
to clo...@googlegroups.com
On 14 December 2010 09:22, Meikel Brandmeyer <m...@kotka.de> wrote:
>
> Am 13.12.2010 um 23:52 schrieb Ken Wesson:
>
>> That's not what I meant. I figure all of us have tabs permanently open
>> to there (I have two actually). What we don't have is the whole thing
>> memorized, or the time to read it all rather than use it for reference […]
>
> My solution to this problem is actually quite simple: I took the time to read it.

One can also apply a measure of common sense. If you're writing a
function that has a general use, such as changing the metadata, you
may very well consider the possibility it has already been written. If
you then open the API page and search for a function that contains
"meta", you'd soon run across vary-meta.

- James

Ken Wesson

unread,
Dec 14, 2010, 7:42:32 AM12/14/10
to clo...@googlegroups.com
On Tue, Dec 14, 2010 at 4:52 AM, James Reeves <jre...@weavejester.com> wrote:
> On 14 December 2010 09:22, Meikel Brandmeyer <m...@kotka.de> wrote:
>>
>> Am 13.12.2010 um 23:52 schrieb Ken Wesson:
>>
>>> That's not what I meant. I figure all of us have tabs permanently open
>>> to there (I have two actually). What we don't have is the whole thing
>>> memorized, or the time to read it all rather than use it for reference […]
>>
>> My solution to this problem is actually quite simple: I took the time to read it.
>
> One can also apply a measure of common sense.

I resent the implication that I didn't.

> If you're writing a function that has a general use, such as changing
> the metadata, you may very well consider the possibility it has already
> been written. If you then open the API page and search for a function
> that contains "meta", you'd soon run across vary-meta.

Yes, but if you look for functions that work on atoms you won't, and
someone looking for functionality like swap-meta! is at least as
likely to think "atoms" as "metadata" when formulating a search.

I don't think there's any point in endlessly rehashing this and trying
to point a finger of blame. Nobody has actually done anything wrong
here, as near as I can tell, OTHER than trying to turn this into a
blame game by getting critical.

javajosh

unread,
Dec 14, 2010, 2:23:28 PM12/14/10
to Clojure
I wouldn't worry too much about your reputation. Your posts are top
notch, and you obviously know the language better than 90% of most
clojure users. Have confidence and laugh if you think someone is
disparaging: actions speak far louder than words.



On Dec 14, 4:42 am, Ken Wesson <kwess...@gmail.com> wrote:

Ken Wesson

unread,
Dec 14, 2010, 2:45:40 PM12/14/10
to clo...@googlegroups.com
On Tue, Dec 14, 2010 at 2:23 PM, javajosh <java...@gmail.com> wrote:
> I wouldn't worry too much about your reputation. Your posts are top
> notch, and you obviously know the language better than 90% of most
> clojure users.

Thank you.

> Have confidence and laugh if you think someone is
> disparaging: actions speak far louder than words.

Yeah, perhaps I should. I just worry that the same attitude aimed at
another user might provoke escalation, even a flamewar.

Reply all
Reply to author
Forward
0 new messages