Summary
Java is in need of a sort of "Grand Unified Theory" for distributed applications. EJB, Jini, JXTA, and JMS have more in common than just the letter J, though you wouldn't know it from reading the specs.

So what do you mean Grand Unified Theory?

Last week, for the first time since 1998, I made it out to San Francisco for JavaONE. The intervening years have seen the crest of the tech wave that had only been building back in 1997 when I attended my first J1. The changed atmosphere of this edition produced a vague feeling that there was some thread running through the whole thing that I just wasn't picking out. Then, while listening intently to a conference session concerning a JSR with which I was totally unfamiliar, it dawned on me that my unfamiliarity with the topic at hand was itself the key factor.

Back in '98, I felt that I understood Java as a technology. There was no session that year that I could attend where I didn't have at least a passing familiarity with the topic and some exposure to the API involved - and I didn't consider myself at all special in that regard. JavaDoc and O'Reilly books on Java were just the stuff you read in your spare time if you were a geek. But things have changed since then.

There are now so many pieces of the Java specification that it is entirely possible to work heavily with one API and still be completely incapable of explaining how to relate that API to another one which has some overlapping features. Don't believe me? Try this - in one nice, neat list describe the decision tree you'd use to determine whether to house an object in a distributed application in a) an EJB container, b) a JXTA peer group, c) a Jini service or d) broadcast it in a JMS topic. I've certainly tried it and the result somehow left me feeling stupid.

My thesis for the next few posts to this blog will be that Java is in need of a Grand Unified Theory for distributed applications that would enable me to write just such a list.

Let me assure you that I don't have the elements of the list predetermined and that I'd greatly appreciate feedback from the community on what I'm missing. (I should note that I derived a great deal of guilty satisfaction recently from hearing some of the people who wrote the APIs mentioned above say that they really weren't able to make such a list either ).

Sounds pretty vague to me

This particular musing is not a hazy abstraction for me. It is directly related to problems I face in a current project and that I've simply got to solve in the coming months.

I'm building out a compute farm for a coarse-grain, numerically intensive problem that has got to dramatically scale up. ( I need at least a 50X increase in throughput over the current multi-threaded app). A distributed technology that allows me to quickly add new hardware at near-linear cost per compute cycle is the only way I see that we can make that happen. So I hope to be mixing blades, 8-way symmetric servers and maybe some desktops into the computational mix, scattering those resources across several different locations and trying to do it all on the cheap.

I've already made certain technology choices, so I can at least begin to fill out my decision tree. Configuration, control and logging of the entire beast is going to be vested in objects living in a J2EE container, so that I can take advantage of JMX. Moving the computational tasks around is going to be a Jini/JavaSpaces responsibility. (Marrying those two technologies in a reasonable way is the subject of much hacking at present.) JXTA offers some interesting notions that I intend to explore for sharing spaces across sites. And finally, JMS is of interest because there could be real-time data which every computation will need to be aware of. How all of this eventually will shake out is still somewhat mysterious to me, though.

Okay, so its one interesting facet of a tiny grain of sand on the computing beach

I've read Jim Waldo's posts to this site with interest, because I agree with him that a) this sort of scalable, distributed application is going to a Very Important Trend in computing and b) this will require some form of mobile code. I've certainly mistaken the problem I happen to be working on for the Gulf Stream of computing currents before, so I could be way off here - but if I am, this is still the most fun I've had writing code in a while.

Hopefully I can get some more details and thoughts posted in the next week or two. In the meantime, I look forward to hearing the thoughts of others on unifying Java distributed technologies into a broader framework.

Talk Back!

Have an opinion? Readers have already posted 13 comments about this weblog entry. Why not add yours?

RSS Feed

If you'd like to be notified whenever Van Simmons adds a new entry to his weblog, subscribe to his RSS feed.

Digg | del.icio.us | Reddit

About the Blogger

Most who've heard of Van Simmons before will remember him in his former role as President of VNP Software. Since 1990, Van has usually made a living architecting, designing, writing and generally being frustrated with large systems for Wall Street firms. Originally he hacked his way about these in Objective-C on the old NeXT platform, but starting in 1996, he realized that he was flogging a dead horse and transferred his allegiance to the (then) immature Java platform. These days, Van is an employee of a firm with a lot of numbers to crunch, so he's spending his days trying to figure out how to get an hours worth of computation done in a minute of elapsed time. Venting, musing and guffawing on this topic is what he plans to write about.