Summary
You go to JavaOne in hopes of hearing about realy cool things. Every once in a while, you get something extremely cool--elegant, even--something like Project Jackpot. If you need to clean up your source files en masse, do it elegantly, and do it quickly, then Jackpot is the tool you've been waiting for.
Advertisement
Early in my career, I saw my mentor (a certain David Smith), become outraged when
keypunch operators were spending hour after hour making changes to files that
could just as easily have been programmed. "Sure," he said to the project
manager, "You don't mind doing the job with manual labor, because
you're not the one who's doing the work." His reaction reflected a
deep-seated appreciation for the value of human effort that turned out
to be contagious.
It has been many years since I first heard that, but it's a lesson that always
stuck with me: Never do large amounts of mindless, repetitive work if you can
possibly get the computer to do it for you.
When you think about it, it only stands to reason:
The computer will work all day and all night, never taking a break, without
complaining.
The computer will do what you told it to do, with no mistakes, at high
speed.
Once you've taught the computer how do to the task, it doesn't forgets. You
can ask it to do the same thing next year, and every year thereafter.
When it comes to making a large number of changes to a massive collection of
source files--or even just examining them to look for patterns--Jackpot was made
to order. Better yet, Jackpot puts an expert at your elbow, so you can get their
advice on changes you should make, but may never have thought of on your own.
Jackpot is a tool for source code examination and transformation that lets you
automate the process of inspecting, reporting on, and modifying source
files. It was developed by James Gosling and Tom Ball, with the help of Neal
Gafter, who made the javac compiler reentrant so Jackpot could use it.
To define the changes you want to make, you create a pattern-based series of
production rules and save them in a file. (I think of that file as a filter.)
The rules have the form: source-pattern=>result-pattern, where => is
the production operator. When Jackpot runs, code that matches the source
pattern on the left is transformed into the result pattern on the right.
Note: You can also write your own transformation class, instead of a
series of rules. It requires knowledge of the Abstract Syntax Tree
produced by javac, and an understanding of the Visitor pattern--the
most useful pattern there is for operating on tree structures.
Jackpot is based on javac, so anything the compiler can process, Jackpot can
process. And in addition to using javac to read the source file coming in,
Jackpot uses javac to verify that resulting source code, after all changes
have been applied, will still compile. That helps rules-authors to avoid
mistakes.
There is tremendous power in the concept of a rule set. In essence, it
captures the expertise of the author. When you run that rule set,
you have the expert at your elbow, suggesting changes. The transformations
might produce cleaner code, better performing code, more localizable code, or
adhere to any other principles that might guide an expert when modifying
a class.
In effect, Jackpot puts a variety of experts at your beck and call, so you
can ask them:
"If you wanted to make this code (cleaner, faster, localizable, take your
pick), how would you modify it?"
In addition to making changes, Jackpot can be useful for examining code,
as well. In all, Tom Ball noted several general categories of use for Jackpot:
Source auditing: Examining code for adherence to conventions (like lint).
Source archeology: Finding patterns in existing code and figuring out how it works.
Refactoring: Making the code better.
Reengineering: Converting to new APIs or new technologies.
Pretty-printing: Making code more readable and maintainable.
Some of the rule sets that Tom Ball demonstrated were designed to:
Clean up code (40-some rules that eliminate extraneous code)
Make code that compiles faster
Convert code that uses StreamTokenizer to use the new Scanner class
Simplify loops so they run faster
Some other possible applications for transformation rules include:
Making code localizable
Optimizing code for performance
Removing debug logic before compiling the production version
There is a lot to know in many of those areas. It's difficult to acquire the
knowledge necessary to make code that is maintainable and localizable and
high-performance and well-documented and makes use of the latest APIs--and
oh, by the way, it has to run properly, too. Jackpot, and the library of
transformation rules that will surely grow up around it, will make it possible
to get the expert advice you need to achieve all of those goals.
When you write a production rules, you include meta-variables that define
place holders in the current syntax tree. At the moment, meta-variables
are defined as syymbols that have a trailing underscore (although that
may change if a better suggestion comes in).
For example, in the expression, if(v==true), v is a literal that only matches
the symbol "v" in the source code. But in the expression, if(v_==true), v_ is a
meta-variable that matches any expression in that position in the syntax tree.
Having defined v_, you can now use it the right-hand side of the
production. For example: if(v_==true)=>if(v_)
That rule says to change an expression like if(succeeded==true) to if(succeeded) --an obvious simplification.
You can also qualify an expression with a "where" clause like you used to see in
mathematics texts. In this case, you introduce the where clause with a
double-colon (::) For example, you could rewrite the previous expression like
this: if(v_==true)=>if(v_)::v_instanceofBoolean
(You can also check to see if v_ extends some class, or is a superclass of it,
among other things.)
Under the Hood, Jackpot stores, examines, and/or operates on three trees provided
by javac:
Abstract Syntax Tree (AST) (syntax hierarchy--code nesting)
Type System (class hierarchy--inheritance)
Symbol Table (instantiation hierarchy--ownership)
Using those three hierarchies allows precise operation. That operation occurs at
high speed, for a variety of reasons:
First, the javac compiler isn't generating byte codes and it's not doing
optimizations. It only parses the class files and produces an AST (abstract
syntax tree). So that part of the process takes seconds, even for a large
number of files.
Second, Jackpot is optimized for the AST. Patterns are found as early in
the tree in possible, before the exponential explosion drains performance.
Third, although Jackpot creates a new AST for the result, it shares
copies of existing nodes. So a minimum number of changes made, and most
of those involve swapping a few pointers around.
Note: Node-sharing posible, because no node knows what its parent
is.
When Jackpot modifies a node, that node and its parents are copied to
to the new tree. When copying parents, Jackpot follows a linear chain up
to the root, without the exponential explosion that occurs when you follow
the branches down towards the leaves, so it operates in essentially linear
time.
Undo is accomplished by removing the node, along with its parents--again,
a linear operation that is both fast and safe, because the original nodes
remain untouched.
In short, Jackpot runs very quickly, in essentialy linear time (Order N, where
N is the product of the number of rules to process and the amount of code
you're operating on).
You can use a rule set to reformat an entire file, although in many cases
that's undesirable. You can also use them to format the results of
source code modifications.
Reformatting an entire file is useful when you're the sole author, especially
when you've inherited a project. But for a large project with multiple authors,
where the files are under source control, massive reformatting generally
isn't a good idea--unless everyone knows it's coming and all files are checked
in first. Otherwise, format changes will swamp the real differences, making
them virtually impossible to locate.
When making a small change to a source file, therefore, it's generally not a good
idea for Jackpot to reformat the entire file. On the other hand, it's impractical
for Jackpot to deduce the style that happens to exist in that file. So when Jackpot
makes changes, it uses your specified style settings (of which there are many) to
format the new code.
As a result, the new code may or may not adhere to the style exhibited in that
file. But it will definitely conform to your coding standard--and you can always
run a Jackpot pretty-print filter on the whole file, if you have that luxury.
Jackpot doesn't parse code comments the way that javadoc does, for example, so
it won't help very much in that area. For that, you still need to use a tool
like DocCheck and make changes manually. (DocCheck is available at
http://java.sun.com/j2se/javadoc/doccheck/)
Note: As the author of DocCheck, I can tell you that it does a
thorough job. Another tool I wrote that may be published by Sun's
java tools group in the near future is CommentMerge--a heavily-tested
program that merges API comments into existing source code. At some
point, I would dearly love to see the javadoc syntax tree grafted onto the
AST, and use Jackpot rules to implement a merge of DocCheck and CommentMerge
for comment transformations!
The only other thing that Jackpot doesn't do, besides slicing your bread,
is checking out files from the source control system. If you're making
changes to a large number of files, that's obviously an issue. The solutions
are to:
Use Jackpot as part of an IDE that will check out the files
automatically.
Use Jackpot to generate a list of files that need to be changed,
and funnel that list to a script that checks them out. (That
would be an excellent job for Groovy, in fact. See JavaOne, Day 1:
It's a Groovy Day!, at http://www.artima.com/weblogs/viewpost.jsp?thread=116723)
Although the Jackpot engine is working, it isn't currently available because
the NetBeans integration isn't totally working, at the moment. (The IDE will provide
a GUI, so you can examine suggested changes and choose whether or not to apply them.)
To find out when Jackpot is released as a NetBeans module, register at http://www.netbeans.org/.
On the other hand, Tom Ball also left open the possiblity of creating a command-line
version that could be built into an ANT task and eventually integrated into other
IDEs. The best way to check progress on that front is most likely to monitor his
blog at http://weblogs.java.net/blog/tball/.
"Currently only Smalltalk applications have the benefit of the tool. ... The search and replace functions of the Brant Roberts Rewrite Rules use a traditional technique of algorithms based upon generating a compiler parse tree. This technique could be applied back to other languages to produce a Rewrite Compiler whose output is rewritten source code, not machine level code. The key point is that a parse tree is essential to achieving the correct transformation of complex expressions, and a compiler seems the appropriate place for meta knowledge about the target language and system."
"The Smalltalk Refactoring Browser was originally developed to study refactorings. After building specific transformations, we added a general rewrite engine to make it easier to add future transformations. However, the syntax for this was too complex for normal users. Therefore, we simplified the syntax and added a userinterface so that programmers would have access to the rewrite engine. Since then, several people have used the rewrite engine to transform their programs."
Previous Topic
