I took a break for the 3 pm session - up in the vendor area, there were customers that wanted to talk to me. So now, I'm back downstairs, and attending a talk on one of the things I've warned against - technology migrations. Karen hope, late of USF&G (which itself was bought out) is giving the talk. Karen started with Smalltalk in 1992, and has worked in VW, VSE, and VA. She worked in Smalltalk continuously until 2004 - with 1994 - 2004 being with the insurance company. She's now an independent consultant.

The application she spent time on was an insurance policy writing system (commercial, small business liability). It won a number of internal and external awards and kudos, including a Smithsonian Innovation award in 1997. The application also won early adopter awards from an insurance industry consortium between 2001 and 2003 (ACORD).

There aren't any code examples here, or consultant bashing. The decision to translate the app was made by corporate, not by the developers or application architects. The actual migration dates from 2004 - and the recurrent theme here is simple: Smalltalk good, Java bad. [ed: I'd modify that by again stating my general theory that technology migrations are almost always a very bad idea, and always spend huge wads of cash in order to stand still).

Karen's opinion after this experience: She would not want to work in Java, and would always recommend Smalltalk.

Heh - Karen adds an acknowledgement to Jeff Sutherland for a 1995 talk called "Smalltalk, C++, and OO Cobol: The Good, the Bad, and the Ugly". That appeared in Object Magazine in May of 1995.

Back to the application. With final deployment in 2004, the application was multi-product, multi-language, and multi-interface: Smalltalk, Java, HTML, WebSphere, HTTP, RMI, ACORD XML, app specific XML, JSPs and Javascript. The initial application was all VSE, and ran on a notebook. It evolved steadily over time. It started with a single business product, and ended up as a full end to end insurance policy writing system.

There were over 2000 domain classes (describing the insurance domain). It used an O/R framework (TopLink), homebrew UI/MVC frameworks, and included a workflow management system. They were one of the early adopters of the ACORD XML system, which allowed them to do comparative pricing quotes in 3rd party systems. The application architecture:

• 4 Smalltalk multi-processor servers (hardware). 6 edits/rating Smalltalk image clones, 4 domain Smalltalk image clones, and 2 XML translator Smalltalk image clones.
• 2 additional servers - downstream/extractor Smalltalk clones
• Bootstrap Java code NT service managed groups of clones on Smalltalk servers. Each ran as an independent Windows process. Each clone had independent caching strategies, but had access to a shared cache.
• Round Robin dispatching initially, went to a more sophisticated load balancer over time.

The system ended up having over 99% uptime statistics. The average real-time edit/rate took 8 seconds. The average number of rates was 200/hour, and they had 75 developers on average. Between 1996 and 2006, they had over 40 production code base releases. So in 1996, what was at stake? There was a total redirection of IS resources (they actually started in 1993). At start, they had 2 Smalltalkers. They went on a massive training/hiring cycle. They created an unprecedented partnership between the business and IS.

In 1993, they chose Digitalk (VSE). At the time, Windows was not predominating, so they wanted to keep their options open - recall that Digitalk did create a Mac port early on. What about 1997 (ParcPlace-Digitalk merger era, VSE was taken down by PPD management). They decided to stay with Smalltalk, even though they were on a moribund product. They were able to push 4 production releases a year - they did not believe that they could do this with a different technology. They started moving parts of the application to VAST.

Then in 1998, USF&G was purchased. This changed things. The new management team also had a policy writing system, and the new CIO had been a part of that. They ended up with years of political infighting over which system would prevail. This was also when the depths of "Smalltalk is dying" and "Java is the future" was out there. Simultaneously, the fat client application was getting fatter, so they did need to make some kind of change to address that. Moving parts of the application to a server looked like a win in maintenance and deployment terms (no need to deal with variant clients, etc).

In 1999-2001, they started moving to a SOA based architecture as they converted code from VSE to VAST. They needed to respond to competitive pressures to allow for browser based rate quotes. They also made the database less application specific, allowing for more general reporting. It also simplified their O/R mappings, and made asyncronous processing easier. They also tightened the UI/domain separation, which had creeped away from MVC over time. At this point, they were adding Java components as part of the server-side WebSphere deployment. In 2002, the VSE client was retired, and all Smalltalk code was now server-side VAST.

Then management decided that they should transfer the whole thing to Java [ed: this is the point where I'd call what they did a mistake. VSE was moribund - a migration was necessary. VA --> Java? Lots of money to stand still].

They started with a "throw the code over the wall" service - a North Carolina based company. The project was cancelled after they had a partly working system. The test cases they had delivered to the vendor allowed them to do a VA Java based system that ran and reproduced their test cases (and found errors in a few test cases and in Smalltalk code). There were some Java patterns in use that they didn't like, and they worked with the vendor to change them.

So here they were with a cancelled, partly working system. Management again decided that they wanted to go to Java anyway. In 2003, they employed a NY based company to do the translation. They worked hands on with the project team, and the company had to devote resources to the job full time. As with the prior effort, this was late, but not unreasonably so. The new code replaced the Smalltalk server with a WebSphere server. An additional task was to change from using DOM (XML) objects to actual domain objects on the back end. So that had to move into the new server as well. This worked, and was deployed (partially) in 2004. It looked like it worked, was robust, and stable.

There was still the downstream processing to translate - at this point, yet another insurance company bought them out. So between 2004 and 2006, the entire application was announced to be dead, with the new company's .NET/VB system being the incumbent.

Now onto the migration efforts from Karen's perspective. She has nothing bad to say about them. The code delivered for translation was very large, and beyond what they initially expected. Interestingly, BFS is still running at this point - the company in question hopes to sunset "soon".

What were the bad things? Moving from dynamic typing to static. It made the developers vastly less productive. There were things that were hard to translate:

• Blocks (not iterators) - areas where they passed blocks around. These were very hard to migrate, and had been natural to Smalltalkers. The resultant Java code did not look at all like what they had started with.
• Java Dates suck. Rates are date driven, and this gave them a lot of trouble. They had to spend a lot of time to correct discrepancies between processing differences between Smalltalk dates and Java dates. They had to re-engineer the whole area.
• Heavy use of Class behavior and Class Instance variables. You just cannot get the same kind of behavior between the two languages - you solve problems differently
• In Smalltalk, methods would have multiple return types - systems could return domain objects, or error objects. This required major re-engineering efforts. Again, you solve these problems differently in the two languages.
• Primitive types or Java Objects for magnitude type things. The Smalltalk code makes assumptions that Java cannot satisfy easily.

The Smalltalk code was "too Smalltalky" for the translation (again, design is language specific IMHO). They had DNU handlers, descendants from nil, #perform, etc. Then there's the whole utility class issue - final classes in Java (String, Date) that would simply be subclassed in Smalltalk. Joshua Bloch, call your office :/

There were problems dealing with marshaling errors on the Java side - these were easy to debug in Smalltalk, but a near impossibility to deal with on the Smalltalk side (something Brian Foote spoke of this morning, actually - the Java side has thrown the information away). Then there are casting errors. RMI was slow, especially with ST to Java (and vice versa) marshaling. maintaining parallel Smalltalk and Java domain objects made forward progress on code hard to do. The new system was not an improvement for users - the values they were getting from the existing system was different than the ones from the new ones (the problem was in the test cases, but the users didn't care).

Specious reasons to translate:

• "Smalltalk is dead" - not correct
• "No one knows Smalltalk" - teaching someone Smalltalk is trivial. The learning curve is not steep, and the big problem is at the domain level, not the language level. The Smalltalk code was easier to understand and maintain
• "Write once, run anywhere" - So is Smalltalk
• "Java is free" - The amount you spend on the IDE/tools is trivial compared to the productivity costs
• "Java is State of the Art" - When they translated, it was simply Smalltalk in Java. It didn't use any "bleeding edge" stuff
• "Get it translated and you're done" - the basic translation is hardly the end, as outlined here. You ten need to maintain the new Java code, which is more Smalltalk-like than stock Java developers will like

You'll need more RAM to develop in Java and simultaneously looking at the Smalltalk. Lots of the normal Smalltalk code will end up being unreadable Java. The bad Smalltalk code will be awful on the Java side. If only a few people understand the Smalltalk, no one will understand the Java code.

What would Karen do differently? They should have just re-written the entire ratings module, and possibly taken advantage of (now available) off the shelf components. Possibly would look at Web Services. Would examine other things than just Java. The reality is that these decisions are never made by the developers, they are political decisions made by management. A question came up about how they would have improved performance in Smalltalk - bearing in mind that she would have had engineers who were busy with migration to attack that - and architectural changes (to a server based scheme) would have helped regardless. [ed: I'd also refer to Travis' talk, which is my experience as well - an algorithmic change is quite frequently the right answer).

What about the good? The new system was faster (rate processing went from 8 seconds to 2) - but it's not an apples to apples comparsion. The Smalltalk system was a fat client, and the Java system was a server based system. They also ditched an EJB component that they had been using with the Smalltalk system. They ended up with higher up time and better processing of large (complex) policies. And, of course, the executives were happy. The business partners were tolerant at best... (lots of money was spent).

Here's my take - They started translation efforts in 2001, and kept at it - one way or another - until late 2004. So that's over three years of churn that could have been spent on actually solving problems that had value to the business. It also involved large sums of cash flow on two levels: Consulting fees to two different firms, and the soft cost of retraining a large number of developers who were competent with the Smalltalk tools, and had to ramp up on new stuff. Useful quote from Karen in this regard: The users didn't care that they were migrating - they wanted the same behavior.