Back Story

Some months ago, I ran into a small problem with the Scala REPL (Read-Evaluate-Print-Loop) under MS Windows.

This was using JLine, which hooks a native library into the command window to find the window size and to support key bindings such as tab completion. JLine is a great solution, except when you need to run the REPL in a nested process with redirected I/O, or connect to it remotely via a protocol such as Telnet. In such cases you have to sacrifice functionality.

Seeing the situation, I felt that Scala could benefit from a swing-based REPL. This would not need JLine to support the full range of functionality in 2.8, would be embeddable, and would allow for connecting to a remote JVM. It also looked like a good project to stretch my growing experience of coding in Scala and gave me a chance to experiment with a bit of swing in the language.

For various reasons, this project has been postponed (demand from the community wasn't strong at the time, JEditorPane and JTextPane still need to be ported over to the scala swing library, and it looked unlikely that I would have it done in time for the then-predicted 2.8 release), but it did expose me to lots of code that looked a bit like this:

class MyClass(peer: OtherClass) {
  def method1 = peer.method1
  def method2(n:Int) = peer.method2(n)
  def method3(s: String) = peer.method3(s)
  def prop1 = peer.prop1
  def prop1_=(n:Int) = peer.prop1_=(n)
  def prop2 = peer.prop2
  ...
}

Pulling in a bunch of methods and properties from elsewhere is why traits were invented, it's the entire reason that Scala has "with" as a keyword. But, as discussed in part I, traits/mixins are of no use in this situation because the OtherClass instance, peer, already exists.

Still, the code contains a lot of boilerplate - the sort of thing that IDEs can readily generate for Java projects. This just has to be a code smell. Surely, there's something the compiler can do to help?

Dynamic Mixins

What I first wanted is a way to change an expression like this:

val x = new Foo with Bar

and instead write something like this:

val f = //create or fetch an instance of Foo
val x = f with Bar

Thus decorating a pre-existing instance of Foo with extra functionality. This capability is often referred to as "dynamic mixins" on the Scala mailing lists.

Although such syntax may be possible [1] , it would require a change to the parsing rules of Scala and is really beyond the scope of a compiler plugin.

Instead, I settled on the @mixin (formerly @proxy) annotation to extend the language in a more plugin-friendly fashion:

class Bar(@mixin foo: Foo) {
  ...
}
val f = ...some method returning an instance of Foo...
val x = new Bar(f)

I then sought feedback about this approach via the Scala mailing lists and the responses were unexpected! Instead of just seeing my proposal as a dynamic alternative to mixins, there was a lot of curiosity about other ways in which this annotation might be used. Since that time I've expanded the scope of the plugin and I'm now testing it in a number of scenarios that are increasingly removed from the original goal. The remainder of this article covers some of the uses...

Customising accessors

Non-private member variables in Scala are generated as a hidden field, plus a getter and (in the case of vars) setter method. Sometimes it's desirable to override these generated methods, perhaps for logging, security or to notify change listerners.

One way to achieve this is to create a private member and hand-craft the accessors:

class Foo {
  private[this] val x0 : Int = 0
  def x = { x0 }
  def x_=(value : Int) = {
    // custom behaviour goes here
    x0 = value
  }
}

This needs a naming convention to be chosen for the private fields. It also needs the developer to write a getter method identical to the one the compiler would have generated, which feels a bit redundant.

It's easy enough to avoid the proliferation of private names by holding them all in a singleton object:

class Foo {
  private[this] object props {
    val x : Int = 0
  }
  def x = { props.x }
  def x_=(value : Int) = {
    // custom behaviour goes here
    props.x = value
  }
}

This approach then lends itself to using autoproxy for re-use of the compiler-generated methods:

class Foo {
  private[this] @mixin object props {
    var a : Int = _
    var b : Double = _
    var c : String = _
  }

  //The following defs are mixed in: a, a_=, b, b_=, c
  //a delegate for c_= is NOT created as it already exists below

  def c_=(str : String) = println("c set to " + str); props.c = str
}

Thus allowing accessors to be selectively overridden, with minimal boilerplate.

Mixing-in Parameterized Traits

Due to type erasure, the types Foo[String] and Foo[Bar] are identical at runtime, with the type safety only being enforced within the compiler. Scala 2.8 offers the @specialize annotation, manifests and view bounds to help out with many of the problems that this can cause, but there is one problem that can't be handled - mixing in differently paramaterized versions of the same trait:

trait Foo[T] {
  def foomethod(arg: T) = println(arg)
}

//can't do this!
class Bar extends Foo[String] with Foo[Int] {
  def barmethod = println("bar method")
}

Once again, autoproxy can help here:

trait Foo[T] {
  def foomethod(arg: T) = println(arg)
}

//this works though!
class Bar {
  @mixin val foo1 = new Foo[Int]
  @mixin val foo2 = new Foo[String]
  def barmethod = println("bar method")
}

This approach is fine for methods such as foomethod above that have different type signatures in each of the mixed-in traits, they will just end up as overloaded versions in the Bar class.

However, for any method that exists in both Foo[Int] and Foo[String] with the same signature, the generated delegate will use the version from Foo[Int] as it was the first mixin listed in Bar. Depending on your needs, this can be seen as either a problem to work around or a useful design feature.

The Future

The plugin is still in desperate need of testing in as many scenarios as possible. But I'm hopeful of having it up to production quality in time for the 2.8 Scala release.

One problem I would like to tackle though is javabean properties. Currently, if a field is annotated as @BeanProperty then it has getXXX and setXXX methods generated in addition to the standard scala getters and setters, when such an object is mixed-in then separate delegates will be created for the Scala and the JavaBean accessors. Ideally, it should be possible to mixin an object, override the Scala accessors, and have the JavaBean accessors delegate to the overridden implementation instead of the original.

It would also be a very useful extra if I could take an existing object using Javabean style properties, and proxy it using Scala style properties.

And The Next Article?

I'll cover how the plugin is implemented. This involved some fun interactions with the typer phase of the compiler, which fails if delegates have not yet been generated, but also needs to be run to provide information used in generating the delegates.