Summary
I received an email from a compatriot lamenting the planned demise of reduce() and lambda in Python 3000. After a few exchanges I think even he agreed that they can go. Here's a summary, including my reasons for dropping lambda, map() and filter(). I expect tons of disagreement in the feedback, all from ex-Lisp-or-Scheme folks. :-)
Advertisement
About 12 years ago, Python aquired lambda, reduce(), filter() and map(), courtesy of (I believe) a Lisp hacker who missed them and submitted working patches. But, despite of the PR value, I think these features should be cut from Python 3000.
Update: lambda, filter and map will stay (the latter two with small changes, returning iterators instead of lists). Only reduce will be removed from the 3.0 standard library. You can import it from functools.
I think dropping filter() and map() is pretty uncontroversial; filter(P, S) is almost always written clearer as [x for x in S if P(x)], and this has the huge advantage that the most common usages involve predicates that are comparisons, e.g. x==42, and defining a lambda for that just requires much more effort for the reader (plus the lambda is slower than the list comprehension). Even more so for map(F, S) which becomes [F(x) for x in S]. Of course, in many cases you'd be able to use generator expressions instead.
Why drop lambda? Most Python users are unfamiliar with Lisp or Scheme, so the name is confusing; also, there is a widespread misunderstanding that lambda can do things that a nested function can't -- I still recall Laura Creighton's Aha!-erlebnis after I showed her there was no difference! Even with a better name, I think having the two choices side-by-side just requires programmers to think about making a choice that's irrelevant for their program; not having the choice streamlines the thought process. Also, once map(), filter() and reduce() are gone, there aren't a whole lot of places where you really need to write very short local functions; Tkinter callbacks come to mind, but I find that more often than not the callbacks should be methods of some state-carrying object anyway (the exception being toy programs).
So now reduce(). This is actually the one I've always hated most, because, apart from a few examples involving + or *, almost every time I see a reduce() call with a non-trivial function argument, I need to grab pen and paper to diagram what's actually being fed into that function before I understand what the reduce() is supposed to do. So in my mind, the applicability of reduce() is pretty much limited to associative operators, and in all other cases it's better to write out the accumulation loop explicitly.
There aren't a whole lot of associative operators. (Those are operators X for which (a X b) X c equals a X (b X c).) I think it's just about limited to +, *, &, |, ^, and shortcut and/or. We already have sum(); I'd happily trade reduce() for product(), so that takes care of the two most common uses. The bitwise operators are rather specialized and we can put fast versions in a library module if there's demand; for shortcut booleans I have the following proposal.
Let's add any() and all() to the standard builtins, defined as follows (but implemented more efficiently):
def any(S):
for x in S:
if x:
return True
return False
def all(S):
for x in S:
if not x:
return False
return True
Combine these with generator expressions, and you can write things like these::
any(x > 42 for x in S) # True if any elements of S are > 42
all(x != 0 for x in S) # True if all elements if S are nonzero
This will mostly give us back the quantifiers from ABC: ABC's IF EACH x IN s HAS p(x): becomes if all(p(x) for x in s):, its IF SOME x IN s HAS p(x): becomes if any(p(x) for x in s):, and IF NO x IN s HAS p(x): becomes if not any(p(x) for x in s):. (Except that in ABC, the variables would be bound and usable inside the IF block; if you need that in Python you'll have to write an explicit for loop and use a break statement.)
I expect that even if you disagree with dropping reduce(), you will agree that adding any() and all() is a good idea -- maybe even for Python 2.5!
+1 for any() and all() in Python! It was only last week that I wrote almost that exact code in a library of 'helper functions' for an internal software product with a Python API.
With list comprehensions and generator expressions, Python's functional programming builtins seem to grind a little, I don't think their removal in Python 3000 would be a sore loss
I won't miss reduce() at all and I'd certainly welcome any() and all(). I've always thought I'd miss map() and filter() quite a bit and lambda slightly less. After further thought I can write my own map/filter/lambda for my typical use cases.
Where I still use map/filter over listcomps/genexps is in the degenerative case of making one call to a predefined function on each item in the sequence. It just seems clearer to me to read:
dirs = filter(os.path.isdir, os.listdir('.'))
instead of:
dirs = [pathname for pathname in os.listdir('.') if os.path.isdir(pathname)]
If I want something as complex as an expression then I switch to listcomps or genexps. filter and map can be implemented very easily in just a few lines so they'll be easy to replace if I really do miss them.
As for lambda... I do a lot of GUI programming and to reduce dependencies between different parts of my application I often pass callables around (with any parameters curried to them) instead of values. The callable can be stored in order to retrieve the value at any time. lambda reduces the penalty on the caller for this level of indirection when the full power isn't needed. For example to set the label of a button, a function is passed and the button stores that function and periodically calls it to determine if it's label has changed. For the simple case of a static label, a call looks like this:
button.setlabel(lambda: 'Click Me!')
If the function is ever more complex than returning a value/variable then I use a nested function. So I can easily have:
def wrapper(value): def f(): return value return f
What about ensuring the proposed changes to itertools (and/or the consuming iterators module) go ahead, which would include any() and all(), for 2.5? They could then easily move to builtins later.
I have to admit that even I was not totally sure about closures being able to be **exactly** equivalent to a lambda until a second ago (didn't think that default argument for closures were re-evaluated each time they were returned; learn something new every day).
While being able to fit stuff on a single line is fun for toy examples, it obviously is not usually the cleanest solution (I offer my one-liner for removing duplicates in an iterable that I recently posted to python-dev as an example; futzing with short-circuiting of 'or' and 'and' is never pretty). So doing away with lambda is, in my mind, a definite at this point.
And as for adding 'any', 'all', and 'product', I say go for it. And as for the loss of 'map', 'filter', and 'reduce', good riddance!
I just came across a slightly expanded lambda use in my GUI code. It's the same kind of thing as the button label above, but this time the caller wants to bind it to a member variable and have the button automatically pick up changes to that member variable. Today, leveraging nested scopes:
Close, but again it doesn't read as cleanly (assuming nested scopes are second nature).
Is there any interest in a further confined lambda like contruct that takes no parameters, but just works as a way to defer/repeat evaluation of an expression in a certain scope?
Ouch. I won't miss reduce at all, and I can live without map and filter (although they're faster than a listcomp if the function is a builtin), but lambda I will definitely miss. It's just too convenient for working with delayed calculation idioms.
I think that maybe you're missing the fact that sometimes you need to pass a filtering or transformation function *into* something that might then be using it in a comprehension. Such functions can go beyond a simple itemgetter or attrgetter, but still fall well short of deserving a 'def' suite on a separate line.
So, if you get rid of lambda in Py3K, can we at least repurpose backquotes to allow quoting expressions inline? Or maybe allow an inline 'def' instead of lambda?
> I just came across a slightly expanded lambda > use in my GUI code. It's the same kind of thing as the > button label above, but this time the caller wants to bind > it to a member variable and have the button automatically > pick up changes to that member variable. Today, leveraging > nested scopes: > > button.setlabel(lambda: self.nextaction) > > Without lambda, still leveraging nested > scopes: > > def nextaction(): return self.nextaction > button.setlabel(nextaction) > > Not too bad, but not quite as convenient as it is now and > having a name is redundant. Here's an attempt at a > reusable wrapper: >
> +1 for all() and any(). > > filter(os.path.isdir, os.listdir('.')) > > would become > > any(os.path.isdir(i) for i in > os.listdir('.')) > > very nice.
I don't think that's how to use "any". Probably you mean this
> I expect that even if you disagree with dropping > reduce(),
Yes. Even in the case of a simlpe addition of two objects that are not numbers applying sum() is currently no replacement:
class X: def __add__(self,other): return "sum of two elements"
>>> a,b = X(),X() >>> a+b sum of two elements
>>> reduce(lambda x,y:x+y, (a,b)) sum of two elements
>>> sum((a,b)) Traceback (most recent call last): File "<interactive input>", line 1, in ? TypeError: unsupported operand type(s) for +: 'int' and 'instance'
So reduce() will obviously return into my codebase even it is not __builtin__ by default anymore, lambda will be trashed ( and replaced by operator.<method> ) and i have to implement it by myself which is no big deal:
def reduce(func,args): res = func(args[0],args[1]) for item in args[2:]: res = func(res,item) return res
>you will agree that adding any() and all() is a > good idea -- maybe even for Python 2.5!
I ran into the above-mentioned problem with sum() over non-numbers just recently. I think sum() should just add the elements in the sequence instead trying to add 0 all the time when i'm not specifying a starter value. In other words, these should work: