Summary
The Closure proposal from Neal Gafter et al. introduces something that is like an object, a closure, but not quite. To enable interoperation with normal objects a form of autoboxing is used, this autoboxing has two forms, normal closure conversion and restricted closure conversion. This blog examines an alternative.

---

Advertisement

Disclosure and Gripe

I should begin by disclosing that I have presented an alternative to the closures proposal, C3S that is based on objects. So obviously I don't whole heartedly agree with the closures proposal as it stands. However I would like a shorter syntax for creating one off instances of interfaces and classes.

Now the gripe: I have tried to have a discussion about alternatives to the BGGA proposal on Neal's blog. Neal chooses to moderate his blog and what typically happens is that I make a comment, which he allows, then he replies with a rebuttal, I disagree with the rebuttal and post a second comment. This second comment is then moderated. I am not sure if Neal is doing this deliberately or not, but either way the result appears that a question was raised and addressed and the issue has gone away. From my point of view this is far from the truth. The issue of this blog, closure conversions, is one such case and hence my own blog entry.

I have invited Neal to have a right of reply and I sincerely hope he does state how he sees the moderating of comments on his blog.

As an aside: an un-moderated blog has the advantage of providing a record of discussions. With a moderated blog you can never be certain that you are seeing the 'whole truth'.

Closure Conversion

Closures are structurally typed not name typed, for example suppose you had a military application:

interface Missile { boolean fire( long timeInMS ); }
interface Ship { boolean depart( long timeInMS ); }

You have two distinct methods with different names Missile.fire and Ship.depart, even though they have the same signature boolean xxx( long ). With a closure there are no names so a closure with the right signature can gets converted via a closure conversion to either of these:

Missile m = { long timeInMS => ... }; // closure get boxed into a Missile
Ship s = { long timeInMS => ... }; // closure gets boxed into a Ship

Note a closure only gets converted once, i.e. you can't say s = m and if you say { long => void } c = { long timeInMS => ... } then a closure conversion to a synthetic interface goes on behind the scenes and hence you can't say m = c. This automatic boxing is necessary to allow interoperation of closures with interfaces. Closure conversion only works with interfaces that have exactly one method. If there are two methods, or if there is a class then there are no closure conversions. Therefore not all existing APIs can be use directly with closures via closure conversions, e.g. AWT and Swing: AbstractAction, MouseListener, KeyListener, etc. There are some suitable interfaces in AWT like ActionListener, but it is annoying that it is not universally possible to use a closure to define abstract methods.

Restricted Closure Conversion

So what's the problem other than incompatible existing APIs? Well if the closure is stored somewhere and then executed at a later time there are a number of features that don't work. Closures allow non-local returns, for example. If an each method were added to java.util.Collections then:

static <T> boolean isKeyPresent( List<T> list, T key ) {
  Collections.each( list, {T x => if ( x.equals( key ) ) return true; } );
  return false;
}

There is an alternative syntax for things like each methods, but to keep this blog simple I didn't use that syntax in the above example.

The return statement inside the closure not only returns from the closure but also from the enclosing method, isKeyPresent.

Now if a closure is to be executed after it is made then an exception can result when the code is run, e.g. if a closure is passed to another thread, stored in a data structure for execution later, or made by another method:

static { => void } makeMethod() {
  return { =>
    ...
    return { => ...; }; // Error not caught by the compiler! Returns from makeMethod and the closure
  };
}

static void main( String[] notUsed ) { makeMethod().invoke(); }

This is an error since the return inside the closure inside makeMethod tries to return from makeMethod (and the closure) but the closure is executed inside main, not inside makeMethod, and hence the code throws an exception when run. The above example is an error — the return keyword should be omitted inside the closure (but with an inner class you would need the return and therefore this is likely to be a common error since people will have to mix closures with inner classes).

To try and prevent these exceptions occurring a marker interface, RestrictedFunction, can be used to mark an interface but not the closure. Therefore if the above example is rewritten to introduce an interface the error can be prevented:

interface VoidMethod extends RestrictedFunction { void invoke(); }

static VoidMethod makeMethod() {
  return { =>
    ...
    return { => ...; }; // Error now caught by the compiler
  };
}

But to prevent the error you need to modify interfaces to extend RestrictedFunction and this may not be possible if you don't control all the source, i.e. you will be relying on all the Java libraries, including third party, to be modified to be closure compatible.

An Alternative

An alternative to RestrictedFunction is given in C3S and is to make a closure a normal object that behaves like an instance of an inner class and for the special features, like non-local returns, to require named qualification, e.g. the examples above:

static { => void } makeMethod() {
  return { =>
    ...
    return { => ...; }; // Error! The closure returns a void
  };
}

or

static { => void } makeMethod() {
  return { =>
    ...
    makeMethod.return { => ...; };
  }; // Error non-local return used inside an un-executed closure
}

and

static <T> boolean isKeyPresent( List<T> list, T key ) {
  Collections.each( list, {
    T x => if ( x.equals( key ) ) isKeyPresent.return true; // Qualified return
  } );
  return false;
}

I.E. explicitly say what is to be returned from if it isn't the immediately enclosing scope.

For name scoping retain the current inner class rules, i.e. if a name is inherited and also exists in an outer scope then the inherited name takes precedence and the outer scope name needs to be qualified. The only inherited names in a closure are method names from Object and the method name invoke, therefore name clashes will be rare. If from within a closure you wanted to refer to the enclosing classes toString method, for example, then just like an inner class you would write [class name].toString().

Tennent's Correspondence Principle

Neal Gafter, one of the proposers of closures for Java, has suggested that Tennent's Correspondence Principle is applied to design closures. Tennent mostly demonstrated his principle by enclosing some code inside an inner procedure (void method in Java speak). The principle is that if you enclose an expression inside an inner procedure and execute the procedure then the meaning of the expression shouldn't change. This principle precludes the return statement and is therefore problematic to apply to Java closure proposals since Java already has return. In Java you can't have inner methods but if you could then:

int outer() {
  return 1; // Expression to be enclosed
}

Then enclose the statement in an inner method and execute the method (assuming Java had inner methods):

int outer() {
  void inner() { return 1; } // Error! inner is a void method
  inner();
} // Error outer doesn't return a value

The solution from Tennent is to ban return statements and have the name of the method as the return value, e.g.:

int outer() {
  outer = 1; // Expression to be enclosed (equivalent of return)
}

Becomes:

int outer() {
  void inner() { outer = 1; } // OK
  inner();
} // OK outer is 1

With name qualified returns it is better than standard Java unqualified return, though far from perfect:

int outer() {
  void inner() { outer.return 1; } // OK
  inner();
  return 0; // Dummy return to keep compiler happy
}

Therefore named returns help, but are not perfect. Surprisingly, considering that Tennent's Principle is often quoted when discussing closures, the present closure proposal has the same problem as named returns:

int outer() {
  { => return 1; }.invoke();
  return 0; // Dummy return to keep compiler happy
}

The compiler cannot in general follow control flow inside the closure and therefore cannot prove that outer always returns a value (at least for complex closures — unlike the example which the compiler may be able to do).

Summary

Closure conversions, both normal and restricted, have a number of catches, a bit like autoboxing of primitives does. The problem is that a closure, like a primitive, is a bit like an object but not quite and this difference is patched over by boxing the closure within an object. I think we should be striving for a more consistency, i.e. everything as an object, not introducing more none-object types.

I have presented an alternative that retains the power of the closure but without the downside, in particular I have presented a closure that is an object and therefore doesn't require any boxing. What do others think?

Talk Back!

Have an opinion? Readers have already posted 30 comments about this weblog entry. Why not add yours?

RSS Feed

If you'd like to be notified whenever Howard Lovatt adds a new entry to his weblog, subscribe to his RSS feed.

Digg | del.icio.us | Reddit

About the Blogger

Dr. Howard Lovatt is a senior scientist with CSIRO, an Australian government owned research organization, and is the creator of the Pattern Enforcing Compiler (PEC) for Java. PEC is an extended Java compiler that allows Software Design Patterns to be declared and hence checked by the compiler. PEC forms the basis of Howard's 2nd PhD, his first concerned the design of Switched Reluctance Motors.