The secrets of unbound super objects

Let me begin by clarifying a misconception about bound super objects and unbound super objects. From the names, you may think that if super(C, c).meth returns a bound method then super(C).meth returns an unbound method: however, this is a wrong expectation. Consider for instance the following example:

>>> class B1(object):
...     def f(self):
...         return 1
...     def __repr__(self):
...         return '<instance of %s>' % self.__class__.__name__
...
>>> class C1(B1): pass
...

The unbound super object super(C1) does not dispatch to the method of the superclass:

>>> super(C1).f
Traceback (most recent call last):
  ...
AttributeError: 'super' object has no attribute 'f'

i.e. super(C1) is not a shortcut for the bound super object super(C1, C1) which dispatches properly:

>>> super(C1, C1).f
<unbound method C1.f>

Things are more tricky if you consider methods defined in super (remember that super is class which defines a few methods, such as __new__, __init__, __repr__, __getattribute__ and __get__) or special attributes inherited from object. In our example super(C1).__repr__ does not give an error,

>>> print super(C1).__repr__() # same as repr(super(C1))
<super: <class 'C1'>, NULL>

but it is not dispatching to the __repr__ method in the base class B1: instead, it is retrieving the __repr__ method defined in super, i.e. it is giving something completely different.

Very tricky. You cannot use unbound super object to dispatch to the the upper methods in the hierarchy. If you want to do that, you must use the two-argument syntax super(cls, cls), at least in recent versions of Python. We said before that Python 2.2 is buggy in this respect, i.e. super(cls, cls) returns a bound method instead of an unbound method:

>> print super(C1, C1).__repr__ # buggy behavior in Python 2.2
<bound method C1.__repr__ of <class '__main__.C1'>>

Unbound super objects must be turned into bound objects in order to make them to dispatch properly. That can be done via the descriptor protocol. For instance, I can convert super(C1) in a super object bound to c1 in this way:

>>> c1 = C1()
>>> boundsuper = super(C1).__get__(c1, C1) # this is the same as super(C1, c1)

Now I can access the bound method c1.f in this way:

>>> print boundsuper.f
<bound method C1.f of <instance of C1>>

The unbound syntax is a mess

Having established that the unbound syntax does not return unbound methods one might ask what its purpose is. The answer is that super(C) is intended to be used as an attribute in other classes. Then the descriptor magic will automatically convert the unbound syntax in the bound syntax. For instance:

>>> class B(object):
...     a = 1
>>> class C(B):
...     pass
>>> class D(C):
...     sup = super(C)
>>> d = D()
>>> d.sup.a
1

This works since d.sup.a calls super(C).__get__(d,D).a which is turned into super(C, d).a and retrieves B.a.

There is a single use case for the single argument syntax of super that I am aware of, but I think it gives more troubles than advantages. The use case is the implementation of autosuper made by Guido on his essay about new-style classes.

The idea there is to use the unbound super objects as private attributes. For instance, in our example, we could define the private attribute __sup in the class C as the unbound super object super(C):

>>> C._C__sup = super(C)

With this definition inside the methods the syntax self.__sup.meth can be used as an alternative to super(C, self).meth. The advantage is that you avoid to repeat the name of the class in the calling syntax, since that name is hidden in the mangling mechanism of private names. The creation of the __sup attributes can be hidden in a metaclass and made automatic. So, all this seems to work: but actually this not the case.

Things may wrong in various cases, for instance for classmethods, as in this example:

def test__super():
  "These tests work for Python 2.2+"

  class B(object):
      def __repr__(self):
          return '<instance of %s>' % self.__class__.__name__
      def meth(cls):
          print "B.meth(%s)" % cls
      meth = classmethod(meth) # I want this example to work in older Python

  class C(B):
      def meth(cls):
          print "C.meth(%s)" % cls
          cls.__super.meth()
      meth = classmethod(meth)

  C._C__super = super(C)

  class D(C):
      pass

  D._D__super = super(D)

  d = D()

  try:
      d.meth()
  except AttributeError, e:
      print e
  else:
      raise RuntimeError('I was expecting an AttributeError!')

The test will print a message 'super' object has no attribute 'meth'. The issue here is that self.__sup.meth works but cls.__sup.meth does not, unless the __sup descriptor is defined at the metaclass level.

So, using a __super unbound super object is not a robust solution (notice that everything would work by substituting self.__super.meth() with super(C,self).meth() instead). In Python 3.0 all this has been resolved in a much better way.

If it was me, I would just remove the single argument syntax of super, making it illegal. But this would probably break someone code, so I don't think it will ever happen in Python 2.X. I did ask on the Python 3000 mailing list about removing unbound super object (the title of the thread was let's get rid of unbound super) and this was Guido's reply:

Thanks for proposing this -- I've been scratching my head wondering what the use of unbound super() would be. :-) I'm fine with killing it -- perhaps someone can do a bit of research to try and find out if there are any real-life uses (apart from various auto-super clones)? --- Guido van Rossum

Unfortunaly as of now unbound super objects are still around in Python 3.0, but you should consider them morally deprecated.

Bugs of unbound super objects in earlier versions of Python

The unbound form of super is pretty buggy in Python 2.2 and Python 2.3. For instance, it does not play well with pydoc. Here is what happens with Python 2.3.4 (see also bug report 729103):

>>> class B(object): pass
...
>>> class C(B):
...     s=super(B)
...
>>> help(C)
Traceback (most recent call last):
  ...
  ... lots of stuff here
  ...
File "/usr/lib/python2.3/pydoc.py", line 1198, in docother
   chop = maxlen - len(line)
TypeError: unsupported operand type(s) for -: 'type' and 'int'

In Python 2.2 you get an AttributeError instead, but still help does not work.

Moreover, an incompatibility between the unbound form of super and doctest in Python 2.2 and Python 2.3 was reported by Christian Tanzer (902628). If you run the following

class C(object):
    pass

 C.s = super(C)

if __name__ == '__main__':
    import doctest, __main__; doctest.testmod(__main__)

you will get a

TypeError: Tester.run__test__: values in dict must be strings, functions or
classes; <super: <class 'C'>, NULL>

Both issues are not directly related to super: they are bugs with the inspect and doctest modules not recognizing descriptors properly. Nevertheless, as usual, they are exposed by super which acts as a magnet for subtle bugs. Of course, there may be other bugs I am not aware of; if you know of other issues, just add a comment here.

Appendix

In this appendix I give some test code for people wanting to understand the current implementation of super. Starting from Python 2.3+, super defines the following attributes:

>> vars(super).keys()
['__thisclass__',
'__new__',
'__self_class__',
'__self__',
'__getattribute__',
'__repr__',
'__doc__',
'__init__',
'__get__']

In particular super objects have attributes __thisclass__ (the first argument passed to super) __self__ (the second argument passed to super or None) and __self_class__ (the class of __self__, __self__ or None). You may check that the following assertions hold true:

def test_super():
    "These tests work for Python 2.3+"

    class B(object):
         pass

    class C(B):
        pass

    class D(C):
       pass

    d = D()

    # instance-bound syntax
    bsup = super(C, d)
    assert bsup.__thisclass__ is C
    assert bsup.__self__ is d
    assert bsup.__self_class__ is D

    # class-bound syntax
    Bsup = super(C, D)
    assert Bsup.__thisclass__ is C
    assert Bsup.__self__ is D
    assert Bsup.__self_class__ is D

    # unbound syntax
    usup = super(C)
    assert usup.__thisclass__ is C
    assert usup.__self__ is None
    assert usup.__self_class__ is None

The tricky point is the __self_class__ attribute, which is the class of __self__ only if __self__ is an instance of __thisclass__, otherwise __self_class__ coincides with __self__. Python 2.2 was buggy because it failed to make that distinction, so it could not distinguish bound and unbound methods correctly.