This article is sponsored by the Java Community Process.

A conversation with Patrick Niemeyer

---

The BeanShell scripting language has gained a large following since Patrick Niemeyer first started to experiment with scriptable Java[1]. Today, BeanShell is a thriving open-source project, and BeanShell scripting has been incorporated into dozens of commercial and open-source projects, such as BEA's WebLogic, the NetBeans IDE, Sun's Java Studio, OpenOffice, JEdit, and many others. The Java Community Process recently accepted BeanShell as a JSR, the first step for BeanShell to become an official Java standard.

Artima's Frank Sommers interviewed Patrick soon after the JCP vote. Patrick became involved with Oak (Java's predecessor) while working at Southwestern Bell, and gained fame in the Java community both as author of the best-selling O'Reilly book Learning Java, and as creator of BeanShell.

Dynamic scripting with BeanShell

Frank Sommers: What is BeanShell, and what does it offer to a developer?

Pat Niemeyer: BeanShell is a scripting language for Java. At a basic level, BeanShell is a Java source interpreter. The BeanShell syntax is based on, and is a superset of, the Java syntax. You can feed it standard Java code, the kind that you'd give to the Java compiler, and BeanShell will dynamically interpret that code for you at runtime.

More important, however, is that BeanShell extends the Java language syntax into the realm of scripting languages in a natural way, allowing you to "loosen" or "scale" Java from a static to a dynamic programming style. That, in turn, allows you to use Java in all sorts of new ways. BeanShell scripts can work with "live" Java objects and APIs just as any other Java code would, as part of your application, or with a stand-alone interpreter.

By a dynamic programming style, I mean that BeanShell offers features of a traditional scripting language: You can use "loosely" typed variables, without declaring them first. You can write simple, unstructured scripts consisting of just arbitrary statements and expressions, or toss in a few methods if you like. You can supply scripted or compiled "commands" which act just like Java methods, but are loaded on demand when called by the script. You can dynamically change the classpath and reload classes in BeanShell as well as create new classes on the fly. These are examples of using Java language constructs in "scripty" ways.

Beyond that, BeanShell offers additional, highly dynamic language features, such as the ability to implement a "meta-method" to handle method invocations, variables, methods, and namespaces that can be inspected programmatically, and the ability to manipulate the call stack and perform evaluations in powerful ways.

Those latter capabilities are not there so much for day-to-day use, as they are to enable powerful BeanShell commands. BeanShell commands are methods that are a loaded in dynamically. You can mix and match all of this loose syntax with the standard, fully typed Java syntax and classes. That makes cutting and pasting or migrating code between scripts and Java very easy.

Rapid prototyping and dynamic program configuration

Frank Sommers: What are some useful ways BeanShell can help developers in their daily work?

Pat Niemeyer: BeanShell started as a playground for experimenting with Java APIs and "live" objects. There is something immensely satisfying about typing new JButton(), and seeing the thing pop up right on the screen, then having a way to tweak its methods and watching what it does. I still use BeanShell that way for rapid prototyping.

When I try to remember how the Java regular expression API works, for instance, I start typing in BeanShell. When I'm curious what fonts are available in my current environment, I type a line in BeanShell. There is that prototyping aspect to it, and also there is a role for BeanShell in education. I include BeanShell with my book, Learning Java (O'Reilly & Associates), because BeanShell is an excellent companion for learning or teaching the Java language[2].

In addition, BeanShell as a scripting and extension language allows users of your Java application to customize application behavior with their own scripts. That's similar to how Emacs is extended via Lisp, for instance, with the difference that BeanShell and Java share a common syntax. Often, it's not reasonable to require a regular user of an application to understand Java and have a full blown development environment just to supply a simple macro or "if-then-else"-type customization for that application.

James Gosling famously noted once that all configuration files eventually become programming languages. BeanShell's simplified syntax is perfect for user scripts, expressions, rules engines, and the like. BeanShell scripts can be used as a general replacement for Java property files, allowing you to not only supply simple values, but to perform more complex logic, if needed. For instance, you can invoke or supply methods, construct objects, and perform procedural setup, all with the familiar Java syntax, but using that syntax in a scripting way.

BeanShell can also do fantastic things for dynamic deployment and debugging. With a couple of lines of code you can create a server from your application, access that server with a web browser, and get a shell inside your running application. You can poke at your app's internals live, even fix things in some cases that way. There are an endless number of things you can do with a dynamic language married to Java.

Frank Sommers: Can you give us a taste of a few dynamic BeanShell features?

Pat Niemeyer: Let me mention just three examples here, but there are many others. First, BeanShell has a handy "invoke" meta-method:

invoke( name, args ) { ... } 

// Or, equivalently, with all the type information... 
void invoke( String name, Object [] args ) { ... } 

Implementing a method with those signatures in your script allows you to handle method invocations for non-existent methods:

invoke( name, args ) { print( name + " invoked!" ); } 

justmadeup( 1, 2, 3 ); // justmadeup invoked! 

That's very useful if you don't feel like implementing all methods of a class or interface, or if you wish to implement your own command loading scheme for your scripts. You can even use this pattern to implement mappings or variable argument list schemes for BeanShell methods, by looking up other methods and dispatching invocations to them.

Next, there is the magic "this.caller" reference that lets you refer to the namespace - the scope - of the caller of your method, to do whatever you'd like on their behalf.

callMe() { 
    this.caller.foo = 42; 
} 

callMe(); 
print( foo ); // 42 

That may not seem all that magical until you look at what's going on. The script that you invoked has access to the environment of the line of code that called it, enabling you to do all sorts of things on the caller's behalf. For instance, you could set or get variables, and define or look up methods. That capability is used in many standard BeanShell commands to do things such as taking into account the classes imported in the caller's namespace or, more generally, to evaluate code and leave the side-effects in the caller's scope. The general eval() method works in this way.

Another example of a dynamic BeanShell command takes advantage of the above feature. The importObject() command imports the methods and variables of a Java object into your scope. That's a lot like the Java 5 static import feature, but it works relative to a "live" object. For example:

map = new HashMap(); 
map.put("foo", "bar"); 

importObject( map ); 

print( get("foo") ); // bar 

This code snippet "mixes in" the methods and variables of the map instance into your current scope, and you can then use those methods and variables as if you had scripted them yourself, or if they were built-in commands. That feature can be very convenient when you want to script part of your application, as it allows your script to "step into" the scope of one of your objects.

These are just a few interesting things to give your BeanShell scripts more leverage and help you keep them both simple and Java-like. I also think it's interesting that the features I just highlighted sort of fell naturally out of the design, and didn't require any special effort to add to the language.

Java scripting standards

Frank Sommers: With JSR 274 now accepted as a JSR, the JCP seems to have three different scripting-related JSRs: "Scripting for the Java Platform" (JSR 223), "The Groovy Programming Language" (JSR 241), and now BeanShell (JSR 274). Why have three separate scripting-related JSRs, and what is the relationship between those efforts?

Pat Niemeyer: "Scripting for the Java Platform" defines the new javax.script API for working with scripting languages from Java[3]. That API will ship with Mustang (Java 6.0). I was a member of that expert group as well. Developers who have used the IBM/Apache Bean Scripting Framework (BSF)[4] are already familiar with what this API offers. The javax.script API insulates you from the mechanics of looking up a scripting engine, figuring out how to bind variable values to a script, and execute a script.

The javax.script API was crafted to work with a broad range of scripting languages, and has some powerful features, such as a truly pluggable environment. If I want to script an Ant task via a new scripting language I just invented, for instance, all I have to do is drop in the JAR file for my language, and I'm done. Ant doesn't care what language I use for the script, assuming the Ant task understands the javax.script API.

The scripts themselves are in different languages, and are not interchangeable. However, javax.script goes a long way towards allowing an application to be scriptable and extensible via whatever scripting language a user wants to use. For instance, you can expose variables of your application to any script via a simple Map interface, and any script will see those variables just as if those variables were declared inside the script.

Next came Groovy[5] and BeanShell. I'm ashamed to say that I don't know very much about Groovy. However, I like what I've seen, and I hope that they are very successful. That said, when I saw the Groovy presentation at JavaOne last year, I had some mixed feelings. Let me explain.

I always thought that someday we'd need a standard for a "native" Java scripting language, with a Java-like syntax. I imagined that might grow out of the "Scripting for the Java Platform" (JSR 223) effort when that JSR was done, and that BeanShell could be a model for what that kind of a language could look like.

Some aspects of Groovy struck me as very similar to BeanShell: For instance, the Java-like syntax and the optional typing. But then Groovy goes off in very novel and interesting directions. I think that Groovy really wants to be an alternate programming language for the Java VM, and that working with Java is perhaps of secondary importance to them. By contrast, BeanShell is foremost a bridge between Java and the sorts of scripting problems javax.script deals with. The extent that BeanShell has become, or will become, a general-purpose programming language on its own, is yet to be seen.

The BeanShell JSR was prompted by the fact that the javax.script API will ship with Mustang and that the Java platform needs [scripting language] implementations for that API. The keepers of Java seem to think it's appropriate to ship a few, light-weight scripting languages with the platform. Since the core BeanShell is very small, only about a 150K JAR file, it fits that criteria.

Both Groovy and BeanShell will be available via the javax.script API. I do not yet know if BeanShell or Groovy will ever be bundled with the J2SE platform. For BeanShell, that is certainly the hope, but that will ultimately depend on what comes out of this JSR, and on the JCP. If and when that happens, I think will be a very good thing for Java.

Language design in the JCP

Frank Sommers: BeanShell has been around for several years now, and seems a fairly mature tool. How does the JCP benefit the BeanShell project at this stage? How will standardization of BeanShell benefit existing BeanShell users? How will it help the broader Java developer community?

Pat Niemeyer: BeanShell will benefit the Java platform by enabling all the capabilities I was talking about earlier, right in the core. There is a difference between being able to rely on a feature in the core APIs and having to supply it, small though that feature physically may be. That Java will have a standard scripting language, and that developers will implicitly know and understand that language because it uses Java syntax, will be a boon for all of the reasons I cited before.

As well, there is the impact of standardization on the BeanShell project and community. I think the JSR work, and the scrutiny that comes with it, is going to be a very good thing for BeanShell. The process of writing a language spec, and the input of the very smart people on the expert group and beyond, will take this little language that has grown up in fits and spurts over the years and really shake it down and work out the kinks. The existing community will continue as it always has, driving the language forward, and now supplying the reference implementation and compatibility test kit for the JSR.

Frank Sommers: Design-by-committee has often handicapped programming languages in the past. Larry Wall, Guido van Rossum, James Gosling, and other successful language designers advocate the "benevolent dictator" approach, where users may suggest features, but ultimately the project's "benevolent dictator" decides what to include and what to reject. How does running a scripting language project through the JCP change those dynamics? Wouldn't expert group members feel dejected if you refused to include features they suggested just because you didn't think those features fit your vision of BeanShell?

Pat Niemeyer: Over the years I have served as a hopefully "benevolent dictator," as well as a bottleneck, on progress in the BeanShell language. Many people have contributed to the project, but I have maintained strong views on keeping the language simple, minimal, and free of clutter.

This was a trade-off, but I think that has paid off in that there are very few things in the language that I would now like to remove. We are in a perfect position to be Java 5-compatible, and can take advantage of those new language features without having stepped all over them by prematurely implementing our own syntax. Some of that was by design, some of that was because I did not have enough time—that's why I was a "bottleneck"—to think about all the ramifications of a change, and therefore wasn't comfortable deciding many issues.

I believe that I will be able to maintain this focus with the expert group, and I hope that it will not be as painful as you suggest. I am looking forward to the influx of opinion, and I only hope that I can keep up with the project so that I don't slow it down too much. If everyone in the community or expert group seems to disagree with me, I'll probably go with the consensus. But then again, I might not (smile).

As for features, I do have some exciting ideas on how we can incorporate new specialized syntax into the language in the future in a pluggable way, without adding baggage to the core. I hope to present some of those at the JavaOne BeanShell BOF this

Resources

[1] JSR 274. The BeanShell Scripting Language
http://www.jcp.org/en/jsr/detail?id=274

[2] Patrick Niemeyer, Jonathan Knudsen. Learning Java Second Edition. O'Reilly and Associates, 2002.

[3] JSR 223: Scripting for the Java Platform
http://www.jcp.org/en/jsr/detail?id=223

[4] Apache Bean Scripting Framework
http://jakarta.apache.org/bsf

[5] JSR 241: The Groovy Programming Language
http://www.jcp.org/en/jsr/detail?id=241

See also:

BeanShell.org
http://beanshell.org

BeanShell Wiki
http://beanshell.ikayzo.org/docs/display/BeanShell/Home

Dwight Shih, Hibernate Prototyping with the BeanShell
http://blog.ideoplex.com/software/2003/08/18.html

Talk back!

Have an opinion? Readers have already posted 1 comment about this article. Why not add yours?

About the author

Frank Sommers is a Senior Editor with Artima Developer. He also serves as chief editor of the Web zine ClusterComputing.org, the IEEE Technical Committee on Scalable Computing's newsletter, and is an elected member of the Jini Community's Technical Advisory Committee. Prior to joining Artima, Frank wrote the Jiniology and Web services columns for JavaWorld.