Foreword

Once upon a time, there was a language, it was called Scala.

People came to look at the language, and they announced that it was “pretty impressive”, but they thought that the feature set was changing too fast, and that nobody was using it yet outside of hobby projects. It sure did look good, but nobody wanted to risk their career on it, it was too young, it might not last.

Then something happened; Scala grew up. Twitter announced that they used it to replace some of their Ruby back-end, and SAP were using it, and EDF. The news spread and new developers also came to see that Scala was “pretty impressive”, whilst the early pilgrims began to look at it with renewed interest and a glint in their eye.

What they saw was a language that is now mature and ready to be used in anger. With the approach of version 2.8, Scala is coming of age, and it is indeed “pretty impressive”!

Programming is Life

Languages evolve, they spawn children, and they give rise to distinct species. Much like the formation of life on a young earth, programming languages came to existence in a bubbling primordial soup of CPU instruction sets and mathematical ideas. Unlike the formation of life, there was no slime involved [1].

But this isn’t the vicious tooth and claw battle of natural selection, although it sometimes feels that way with the ongoing wars about tabs vs. spaces and exactly where you should stick your brackets. Languages are picked with as much care as a breeder of thoroughbreds will choose horses. Just as in biology, this artificial selection is not without its problems, and inbreeding is an ever-present risk. Fortunately, there is a flip-side to this coin, good husbandry and horticulture also takes advantage of hybrid vigour. This is where two parents are specifically chosen for their differences to create offspring that is stronger than either, the idea being that both sets of weaknesses will be cancelled out. The same thinking has been applied in the world of programming languages too [2], with many now fusing concepts from Object-Oriented and Functional styles to offer the programmer incredible power and expressiveness.

Scala is one of them.

Get on with it!

Most folk reading this will be Java programmers, so I’d like to explain some of the ideas behind functional programming before covering the sordid details of how functions and objects interact.

The internet is already full of better explanations that I can ever dream of writing, so I’ll keep it short.

What is a function?

In mathematics, a function is a thing that takes one value (the input) then uses it somehow to produce another value (the output). This definition covered almost every eventuality for a long time, even when mathematicians started seriously stretching the definition of what a number is; complex numbers, matrices, vectors, co-ordinates (both Polar and Cartesian), quaternions... Lots of things can be treated as a number if you squint in just the right way.

This was fine for a long time, then programmers were invented, and then computers were invented [3]. Once computers settled into the collective consciousness and matured a bit, the programmers began to think differently about them, one such thought was that looking at oversized print-outs with endless lists of three letter opcodes was a pretty good way to give oneself a headache.

If only they could group together some of these sequences and give them a name then we’d have a neat way to reuse stuff and all that time spent repeating things could be freed up to, hopefully, be spent down the pub (or looking at quaternions, or writing pong). Because so many of these programmers were also mathematicians, and because so many of their programs were working on mathematical problems, it was decided that the idea of a function fitted very nicely with the idea of a pre-packaged unit of behaviour, and so 2nd generation languages were born.

Trouble in Paradise...

The fit wasn’t perfect though. There was the question of functions that needed to work with multiple inputs or (more frustratingly) multiple outputs. Fortunately some mathematicians already allowed for functions with multiple inputs, so this idea was adopted very early on and ways were figured out to return multiple values (usually by subverting the input parameters and writing back out to them). Other mathematicians (like Haskell) didn’t like multiple inputs and came up with the idea of higher-order functions instead, functions that could return other functions, or take functions as arguments, but these were harder to implement for the programmers and so got ignored at first.

Another discrepancy with the functions of programming was in side effects. A function might do a different thing every time it was used with the same input (such as when reading a file), or it could do something besides just returning a value (such as printing a line to the console). Even worse, it was possible that a value might have been changed after being used as an argument to a function! As a program needs to have side effects in order to do more than warm the CPU this wasn’t worried about very much, except by the mathematicians that didn’t especially like beer but still had free time on their hands.

So the functions of programming were not the same as the functions of mathematics. They had a new (and less precise) definition: A procedure, or bunch of instructions, with a name, that can optionally have one or more inputs and one or more outputs, some of which can be both at the same time, and might also do other stuff on the side.

A reprive ahead of its time

Of course, a lot of mathematicians weren’t happy about the way functions were being undermined, so a new breed of programming language was created to give back some of what was lost and once again put this stuff on a solid theoretical framework.

Functions became first class entities and not just a name for a bunch of code. This allowed for Haskell's higher-level functions to be used in designing software. Language features also evolved to encourage immutable values, so that functions couldn’t muck about with their inputs. Currying was implemented and Tuples were used, so that functions could give a respectable facade of only having one input and one output. Interesting stuff was also being done [4] to restrict those pesky side-effects, if they couldn’t be totally eliminated, then they could certainly be put in their place! This family was known as “functional” programming languages, due to the importance placed on returning functions back to their mathematical roots. The family includes such languages as Lisp, Scheme, Caml, Erlang, F#, Clojure, etc.

Being a good example of sound engineering, functional languages are elegant in design, comprehensible [5], efficient and internally consistent. Also, as so often happens with Good Ideas™, they spent a long time being largely ignored by the mainstream. The programmers [6] were still very conscious of why they liked functions in the first place; the need to split their systems up into distinct, named units of behaviour that could be isolated and reused in different ways. This need was still itching and so the Object-Oriented paradigm was born as a way to scratch it [7]. For the time being, functional programming was delegated to being a hobby, and also a handy way to find cute new material for the occasional dissertation or thesis. It was believed that functional languages were slower than their imperative cousins, but this was mostly taken on faith and nobody really went out of their way to prove it. It was also believed that, although FP looked very neat for small programs and for demos, it wouldn’t be able to scale up to programs that were hundreds of thousands of lines long, it wouldn’t be maintainable.

Renaissance

But this stuff wasn’t just a hobby. In common with all the best revolutions it was just bubbling underground for a few years until the world was ready for it. Mainstream programmers were increasingly finding that there were things they could easily think of, but which couldn’t be easily expressed in their (object oriented) code. Even simple ideas like “take this list of strings, and turn it into a list of uppercase strings” would run into subtle problems. Sometimes the list of uppercase strings would be missing the first entry because they’d started counting from 1 instead of 0, sometimes another user would find that their list had all been turned to uppercase for them even though they didn’t want it to be, there were also a lot of null pointer exceptions.

Slowly, people began clamouring for closures, and continuations, and for other tricks to make their programs more robust and easier to maintain. But these weren’t things that could be built from objects, so enhanced for-loops were invented, and anonymous classes, and the visitor pattern, and the command pattern; none of them matched perfectly what the programmers were thinking, but they still helped and made a lot of problems more manageable [8] (even if they did create a small swamp of boilerplate code). The time was ripe for a paradigm shift, and functional programming was awaiting its grand entrance.

Feature Envy

Via Erlang, Ericsson demonstrated that the FP paradigm could scale to massive systems. It could be made fast, maintainable, testable, and remarkably free of errors. This was FP outside of the labs and it was looking better than the object-oriented systems. Ericsson programmers were getting more beer time than ever before. The gauntlet had been thrown down!

On the other side of the fence, programmers were looking at FP with thoughtful expressions and a hint of jealousy. Java was getting bloated, and every new feature seemed to come with its own requirement for boilerplate code. Even small programs are now littered with annotations and template parameters and duplicate type declarations, big programs are even worse. To add insult to injury, the debate about how to add closures doesn’t look like reaching an early conclusion, and we still have the endless lists of get/set methods in java beans. Something had to give.

Despite this, Java is not without a LOT of strengths. The Virtual Machine is one of the most mature and optimised on the market and can be found everywhere, from washing machines and mobile phones through to countless web servers and desktop computers. The ecosystem with its range of open source libraries and frameworks for the language is just astounding, the paid-for stuff is pretty hot too. Under the Java umbrella, the world has been flooded with projects that flourished and matured following corporate investment and millennia of collective development time.

That’s an awful lot to give up just for a few new language features

We can have our cake... and eat it too!

What was needed was a way to keep all that juicy java goodness, but to have it a la carte with a refreshing order of FP at the same time. Scala stepped in to meet this need, although it wasn’t the only contender. Pizza came first, but it’s seen as a prototype for Scala nowadays.

Other languages learned to run on the JVM too, such as JavaFX, JRuby, Jython, Groovy etc. Most of them also had closures and other elements of FP, but in the nation of Java these newcomers weren’t quite native citizens, behaving more like immigrants with a shiny new work visa and a foreign accent.

The fashion for dynamic languages didn’t help either; by pretending that types don’t exist it becomes difficult to generate compiled code that “speaks” idiomatic Java. This can be big problem if you’re producing objects to be handed to a 3rd party library. In some cases the language barrier was so challenging that it needed an interpreter, such as the JSR233 Scripting API or the Bean Scripting Framework.

Scala came to the party with a new trick, it integrates so tightly with Java that it can speak types like a native. Instead of acting like a migrant worker this is a naturalised, passport carrying, citizen. From the outside world Java and Scala compile to classes that look identical [9], this is hardly surprising as it was an explicit goal in the design of the language. It’s even more impressive when considering that Scala is a fully-fledged FP language, embracing and unifying the twin worlds of objects and functions in a beautifully consistent manner.

Because of this tight integration, Scala can be used as a drop-in replacement for java and nothing forces you to write a single line of code in a functional style. Features like type inference, traits, clean property handling [10], and the fusion of member variables with constructor arguments all allow it to be used as a kind of java-with-a-cleaner-syntax. Even foregoing the benefits of FP, it can be argued that Scala is still a worthy successor by being much more object-oriented than Java:

• Everything is an object, including number literals and functions. In Java, methods aren’t objects, nor are primitives. 2.toString is a valid statement in Scala.
• It does away with static class members, these are mostly an anachronism dating back to Java’s heritage as a C++ successor. C++ was a hybrid language itself, with the goal of supporting the procedural style of C whilst also adding support for objects. Static members aren’t very object-oriented as they can’t implement an interface, or take advantage of polymorphism and overloading in the same manner as instance members. It’s also not possible to pass a class as an argument to a function that will then use its static members [11]. Instead, Scala offers singleton objects which have none of these failings. The new concept of companions allows a singleton to access the restricted members on instances of a same named class, thus duplicating the privileges of static members.
• All properties on an object are implemented behind-the-scenes as a hidden field and a pair of methods to get and set the field [12]. No more public access to your object’s internal data that can be directly modified by anyone.

In the future, virtual classes may well be added to this list, making Scala’s support for objects even more impressive.

The world of functional programming is supported just as well:

• tail-call optimisation of recursive functions
• pattern matching
• first class and higher order functions
• partial functions (not defined for all possible input values)
• currying and partial application of functions
• closures
• concise language syntax to declare immutable values, and a good library to work with immutable collections.
• continuations (scala 2.8 plus)

So what does this all mean?

FP has proven itself, and a growing number of developers are now coming to recognise the problems that are best suited to this approach. Scala demonstrates how functional design patterns can be added to the programmers’ arsenal without needing to sacrifice the benefits of object-orientation. It also shows how the two styles can complement each other to produce a robust well-rounded language, without any hint of multiple personality disorder.

Once you get beyond basic syntax and concepts of closures, first class properties, higher order functions, traits, immutable refs, etc., the deeper potential of Scala reveals itself in the interplay between features. Some of the design choices made over the lifetime of the language have resulted in powerful synergies [13]; and learning this new generation of object-functional design patterns perhaps offers the key to being truly effective in Scala.

With future articles, I hope to further explain this exciting language by covering some of the ideas that are helping to spread its growing popularity.

What happened to the next AutoProxy article?

The plugin is work in progress

I'm still fighting the compiler over correct generation of field setter delegates, when I've resolved this problem it'll be part of the write-up and I'll post my experience.

In the meantime, this is an article I wrote a couple of months ago, but never found the time to proof-read and format.