There is a serious problem when teaching macros to beginners: the real
power of macros is only seen when solving difficult problems, but you
cannot use those problems as teaching examples. As a consequence,
virtually all beginner's introductions to macros are dumbed down:
usually they just show a few trivial examples about how to
modify the Scheme syntax to resemble some other language. I did the same
too. This way of teaching macros has two negative effects:
- beginners are naturally inclined to pervert
the language instead of learning it;
- beginners can easily dismiss macros as mere
syntactic sugar.
The first effect is the most dangerous: the fact that you can implement a C-like
for loop in Scheme does not mean that you should use it! I
strongly believe that learning a language means learning its idioms:
learning a new language means that you must change the way you think
when writing code. In particular, in Scheme, you must get used to
recursion and accumulators, not to imperative loops, there is no other
way around.
Actually, there are cases
where perverting the language may have business sense. For instance, suppose
you are translating a library from another language with a
for loop to Scheme. If you want to spend a minimal effort in the
translation and if for any reason you want to stay close to the
original implementation (for instance, for simplifying maintenance),
then it makes sense to leverage on the macro
facility and to add the for loop to the language syntax.
The problem is that it is very easy to abuse the mechanism. Generally
speaking, the adaptibility of the Scheme language is a double-edged
sword. There is no doubts that it increases the programmer
expressivity, but it can also make programs more difficult to
read. The language allow you to invent your own idioms that
nobody else uses, but perhaps this is not such a good idea if you care
about other people reading your code. For this reason macros in the
Python community have always been viewed with suspicion: I am also
pretty confident that they will never enter in the language.
The second effect (dismissing macros) is less serious: lots of
people understimate macros as mere syntactic sugar, by forgetting that
all Turing complete languages differ solely on syntactic sugar.
Moreover, thinking too much about the syntactic sugar aspect make them
blind to others and more important aspects of macros: in particular,
the fact that macros are actually compilers.
That means that you can implement
with macros both compile time checks (as I have stressed in episode #10,
when talking about guarded patterns) and compile time computations
(I have not discussed this point yet) with substantial
benefits for what concerns both the reliability and the performance of
your programs. In episode #11 I have already shown how
you can use macros to avoid expensive function calls in benchmarks
and the example generalizes to any other situations.
In general, since macros allows you to customize the evaluation mechanism
of the language, you can do with macros things which are impossible
without them: such an example is the test macro discussed
in episode #11. I strongly suggest you to read the third comment
to that episode, whereas it is argued that it is impossible to
implement an equivalent functionality in Python.
So, you should not underestimate the power of macros; on the other
hand, you should also not underestimate the complexity of macros.
Recently I have started a thread on comp.lang.scheme with 180+ messages
about the issues I have encountered when porting my sweet-macros
library between different Scheme implementations, and the thread ended
up discussing a lot of hairy points about macros (expand-time vs run-time,
multiple instantiation of modules, separate compilation, and all that).
I am not an advocate of macros for enterprise programming. Actually,
even ignoring the issue with macros, I cannot advocate
Scheme for enterprise programming because of the lack of a standard
library worth of its name. This was more of an issue with R5RS Scheme,
but it is still a problem since Scheme has an extremely small standard
library and no concept of batteries included à la Python. As a
consequence, everybody has to invent its own collections of utilities,
each collection a little bit different from the other.
For instance,
when I started learning Scheme I wrote a lot of utilities;
later on, I discovered that I could find the same utilites, under
different names and slightly different signatures, in various Scheme
frameworks. This never happened to me in Python to the same extent,
since the standard library is already coding in an uniform way most of
the useful idioms, so that everybody uses the library and there is
less need to reinvent the wheel.
On the other hand, I am not a macro aficionado like Paul Graham, who says:
When there are patterns in source code, the response should not be
to enshrine them in a list of "best practices," or to find an IDE
that can generate them. Patterns in your code mean you are doing
something wrong. You should write the macro that will generate them
and call that instead.
I think Graham is right in the first part of its analysis, but not in
the conclusion. I agree that patterns are a code smell and I think
that they denote a lack in the language or in its standard library. On
the other hand, the real solution for the enterprise programmer is not
to write her own macro which nobody knows, but to have the feature
included in the language by an authoritative source (for instance
Guido van Rossum in the case of Python) so that all users of the
language get the benefit in an uniform way.
This happened recently in Python, with the ternary operator, with the
try .. except .. finally statement, with
the with statement, with extended generators and in many other
cases. The Scheme way in which everybody writes his own language
makes sense for the academic researcher, for the solitary hacker,
or for very small team of programmers, but not for the enterprise.
Notice that I am not talking about specialized newly invented
constructs: I am talking about patterns and by definition, according
to the GoF, a pattern cannot be new, it must be a tried and tested solution
to a common problem. If something is so common and well known
to be a pattern, it
also deserves to be in the standard library of the language, or
in a standard framework. This
works well for scripting languages, which have a fast evolution,
and less well in languages designed by committee, where you can
wait years and years for any modernization of the language/library
(we all know Paul Graham is coming from Common Lisp, so his position
is understandable).
In my opinion - and your are free to disagree of course - the
enterprise programmer is much better served by a language without
macros but with a very complete library where all useful constructs
have been codified already.
After all, 99.9% of the times the enterprise
programmer has to do with already solved problems: it is not by
chance that frameworks are so used in the enterprise world. Notice
that by "enterprise programmer" I mean the framework user, not
the framework writer.
Take my case for instance: at work I am doing some Web programming,
and I just use one of the major Python web frameworks (there already
too many of them!); I do quite of lot of interaction with databases,
and I just use the standard or de facto standard drivers/libraries
provided for the task at hand; I also do some scripting task:
then I use the standard library a lot. For all the task I
routinely perform at my day job macros would not help me a bit:
after all Python has already many solutions to avoid boilerplate
(decorators, metaclasses, etc.) and the need for macros is not
felt. I admit that some times I wished for new constructs in Python:
but usually it was just a matter of time and patience to get them in
the language and while waiting I could always solve my problems
anyway, maybe in a less elegant way.
There are good use cases for macros, but there also plenty of workarounds
for the average application programmer.
For instance, a case where one could argue for macros, is when there
are performance issues, since macros are able to lift computations from
runtime to compile time, and they can be used for code
optimization. However, even without macros, there is plenty of room
for optimization in the scripting language world, which typically
involve interfacing with C/C++.
There are also various standard techniques for code generation in C,
whereas C++ has the infamous templates: while those are solutions
very much inferior to Scheme macros, they also have the enormous
advantage of working with an enterprise-familiar technology, and you
certainly cannot say that for Scheme.
The other good use for macros is to implement compile time checks:
compile time checks are a good thing, but in practice people have
learned to live without them by relying on a good unit test coverage, which
is needed anyway.
On the other hand, one should not underestimate the downsides of
macros. Evaluation of code defined inside of the macro body at
compile time or suspension of evaluation therein leads often to bugs
that are hard to track. The behaviour of the code is generally not
easy to understand and debugging macros is no fun.
That should explain why the current situation about Scheme in the
enterprise world is as it is. It is also true that
the enterprise programmer's job is sometimes quite
boring, and you can risk brain atrophy, whereas for sure
you will not incur in this risk if you keep reading my Adventures ;)
You may look at this series as a cure against senility!
In this appendix I will give the solution to the exercise suggested
at the end of episode #10, i.e. implementing a Python-like for
loop.
First of all, let me notice that Scheme already has the functionality
of Python for loop (at least for lists) via the for-each
construct:
> (for-each (lambda (x y) (display x) (display y)) '(a x 1) '(b y 2))
abxy12
The problem is that the syntax looks quite different from Python:
>>> for (x, y) in (("a", "b"), ("x", "y"), (1, 2)):
... sys.stdout.write(x); sys.stdout.write(y)
One problem is that the order of the list is completely different, but
this is easy to fix with a transpose function:
(define (transpose llist) ; llist is a list of lists
(if (and (list? llist) (for-all list? llist))
(apply map list llist)
(error 'transpose "Not a list of lists" llist)))
(if you have read carefully episode #8 you will notice the
similarity between transpose and zip). transpose
works as follows:
> (transpose '((a b) (x y) (1 2)))
((a x 1) (b y 2))))
Then there is the issue of hiding the lambda form, but this is an
easy job for a macro:
(def-syntax for
(syntax-match (in)
(sub (for el in lst do something ...)
#'(for-each (lambda (el) do something ...) lst)
(identifier? #'el))
(sub (for (el ...) in lst do something ...)
#'(apply for-each (lambda (el ...) do something ...) (transpose lst))
(for-all identifier? #'(el ...))
(syntax-violation 'for "Non identifier" #'(el ...)
(remp identifier? #'(el ...))))
))
The important thing to notice in this implementation is the usage of a guard
with an else clause: that allows to
introduce two different behaviours for the macro at the same time.
If the pattern variable el is an identifier, then for is
converted into a simple for-each:
> (for x in '(1 2 3) (display x))
123
On the other hand, if the pattern variable el is a list of
identifiers and lst is a list of lists, then the macro
also reorganizes the arguments of the underlying for-each
expression, so that for works as Python's for:
> (for (x y) in '((a b) (x y) (1 2)) (display x) (display y))
abxy12
Previous Topic
