The Artima Developer Community
Sponsored Link

Java Community News
The Big Rewrite: Why So Hard?

30 replies on 3 pages. Most recent reply: Jan 3, 2007 11:29 AM by Carsten Saager

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 30 replies on 3 pages [ 1 2 3 | » ]
Bill Venners

Posts: 2284
Nickname: bv
Registered: Jan, 2002

The Big Rewrite: Why So Hard? Posted: Dec 27, 2006 6:11 PM
Reply to this message Reply
Summary
In his weblog, Chad Fowler suggests that "The Big Rewrite," a project in which you reimplement an existing product likely with a new technology, is a "longer, harder, more failure-prone path than you expect." What's your experience?
Advertisement

In The Big Rewrite, Chad Fowler launches a series of blog posts about the perils of doing complete rewrites of existing applications. He writes:

Throughout my career in software development, I’ve been involved in Big Rewrite after Big Rewrite. I suspect it’s because I have an interest in learning eclectic computer languages, operating systems, and development environments. Not being just-a-Java-guy or just-a-Windows-guy has led to me becoming a serial rewriter. I’ve been on projects to replace C, COBOL, PHP, Visual Basic, Perl, PLSQL, VBX (don’t ask!) and all manner of architectural atrocities with the latest and greatest technology of the day.

In many cases, these Big Rewrite projects have resulted in unhappy customers, political battles, missed deadlines, and sometimes complete failure to deliver. In all cases, the projects were considerably harder than the projects’ initiators ever thought they would be.

What is your experience with Big Rewrites, and what would you advise someone who is considering doing a Big Rewrite? When a Big Rewrite does make sense, what's the best way to approach it?


Nemanja Trifunovic

Posts: 172
Nickname: ntrif
Registered: Jun, 2004

Re: The Big Rewrite: Why So Hard? Posted: Dec 27, 2006 8:15 PM
Reply to this message Reply
>what would
> you advise someone who is considering doing a Big Rewrite?

My advise: Don't.

Joel has a nice article on the topic: http://www.joelonsoftware.com/articles/fog0000000069.html

Peter Booth

Posts: 62
Nickname: alohashirt
Registered: Aug, 2004

Re: The Big Rewrite: Why So Hard? Posted: Dec 27, 2006 8:43 PM
Reply to this message Reply
Sometimes a Small Rewrite can produce surprising results.

A few jobs ago I was given the task of rewriting an existing C++/X11 app in Java/Swing. It wasn't s large application - it took a handful of inputs, made some DB queries in response and then drew a few graphs. To this day I don't know why the project occurred - the existing app appeared fine. But one thing I realized quickly was that there's no better spec than an existing app.

I estimated it would take eight weeks to do this project. The first functional version was there in maybe 12 days and the finial product turned out to be smaller, faster, and cleaner than the original , all of which were surprises. I imagine that rewriting the same app in Ruby/Rails today would again provide an opportunity for further efficiencies.

Sometimes a change in technology gives you an excuse to throw out stuff that people feel attached to undert the guise of "not supported with the new library."

Yagiz Erkan

Posts: 1
Nickname: yagiz
Registered: Jan, 2004

Re: The Big Rewrite: Why So Hard? Posted: Dec 28, 2006 3:30 AM
Reply to this message Reply
> My advise: Don't.

So, should we just leave the job to another company/implementor?

Vikas Hazrati

Posts: 3
Nickname: vhazrati
Registered: Oct, 2006

Re: The Big Rewrite: Why So Hard? Posted: Dec 28, 2006 3:34 AM
Reply to this message Reply
Probably not, but may be smaller steps and smaller increments can help and a big rewrie may not be necessary at all. Refer to this http://blog.xebia.com/2006/11/29/agile-software-architecture-the-smallest-change-with-the-highest-business-value/

robert young

Posts: 361
Nickname: funbunny
Registered: Sep, 2003

Re: The Big Rewrite: Why So Hard? Posted: Dec 28, 2006 5:37 AM
Reply to this message Reply
Remember the olde sayinge, "Your product will be replaced by a smarter, better one. Either by you or a competitor. Take your pick."

What fools people is the assumption that the M$ experience is general rather than rare. It is rare. Now, given that the U$ government has been in the hands of Right Wing folk for a while might lead one to extrapolate that monopoly practices will no longer be punished (e.g. the baby bell saga). Only if that continues to be true can one get away with foisting crappy systems. They will be replaced by better.

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: The Big Rewrite: Why So Hard? Posted: Dec 28, 2006 6:59 AM
Reply to this message Reply
> Probably not, but may be smaller steps and smaller
> increments can help and a big rewrite may not be necessary
> at all.

I was thinking the same thing. Luckily for me, the system we use is a large conglomeration of loosely coupled processes. You can leave things alone until they have to be rewritten. If you are trying to rewrite a monolithic application, however, this becomes much more tricky to pull off.

Merriodoc Brandybuck

Posts: 225
Nickname: brandybuck
Registered: Mar, 2003

Re: The Big Rewrite: Why So Hard? Posted: Dec 28, 2006 7:00 AM
Reply to this message Reply
> Throughout my career in software development, I’ve been
> involved in Big Rewrite after Big Rewrite. I suspect it’s
> because I have an interest in learning eclectic computer
> languages, operating systems, and development
> environments. Not being just-a-Java-guy or
> just-a-Windows-guy has led to me becoming a serial
> rewriter. I’ve been on projects to replace C, COBOL, PHP,
> Visual Basic, Perl, PLSQL, VBX (don’t ask!) and all manner
> of architectural atrocities with the latest and greatest
> technology of the day.

Sounds nifty. You said not to ask, but I can't help it. VBX? What was that system for?

I think Big Rewrites for the the sake of the technology are doomed to have problems. The original write in most cases I'm aware of weren't to get some new technology out there. They were to solve an actual problem somebody had. The motivation was completely different. Originally the software was likely written because somebody needed problem X solved, not because somebody wanted to see what they could do with .NET 3.0.

> In many cases, these Big Rewrite projects have resulted in
> unhappy customers, political battles, missed deadlines,
> and sometimes complete failure to deliver. In all cases,
> the projects were considerably harder than the projects’
> initiators ever thought they would be.

In any of those cases were any of the original people involved still around? How long had the product been maturing before the Big Rewrite, on average?

I think the problem is the same as with weight loss. People sit on their arses for years accumulating extra baggage around the middle and then expect to take it off in a matter of weeks. If it took you many years to put the baggage on, what makes you think it will come off in a matter of weeks? The rewrites I've been a part of have been for products that have been around for years if not a decade or more. It's not like these systems remain stagnant. They constantly evolve during this time. So you take a system that has been evolving for years and expect to totally replace it in 6 - 18 months? That's wacky. And I would guess most people here would say 18 months would be generous. 6 - 12 seems to be a more typical expectation of these initiators in my experience. I don't know what these initiators base the expectation on in a lot of cases, but that seems to be the general belief. "We can have that done in about a year!". Yeah. Right.

> What is your experience with Big Rewrites, and what would
> you advise someone who is considering doing a Big Rewrite?
> When a Big Rewrite does make sense, what's the best way to
> approach it?

They've always been painful.

I think the best way to do a Big Rewrite is to approach it as a series of small rewrites whenever possible. Like Peter pointed out, a working app is a great spec and small rewrites can be very satisfying. However, the rewrite process doesn't seem to scale very well. I've been best served by breaking the existing app as much as possible and rewriting parts of it. As long as you don't have to totally scrap the interface this works.

If you really have to do a Big Rewrite and it's monolithic in nature my only advice would be the same advice that most people gave me when we were considering having work done on the house. Double your time and cost estimates. That might put you somewhere in the ballpark. I think people are generally optimistic and/or naive. Not that this is bad, but it seems to be a benefit only to endeavors without a fixed timetable.

James Watson

Posts: 2024
Nickname: watson
Registered: Sep, 2005

Re: The Big Rewrite: Why So Hard? Posted: Dec 28, 2006 7:36 AM
Reply to this message Reply
> I think the problem is the same as with weight loss.
> People sit on their arses for years accumulating extra
> baggage around the middle and then expect to take it off
> in a matter of weeks. If it took you many years to put the
> baggage on, what makes you think it will come off in a
> matter of weeks?

I think this is really the key to why rewrites are so hard. People think that because it took 6 weeks to write the first version, that it should take on the order of 6 weeks to rewrite. As you point out, there's always a lot more there than what was done in that initial implementation, not to mention all the time spent fixing bugs.

The other thing that comes along with this is that I've never seen a piece of software that has good documentation for the initial work and all the changes. Even if these are documented, they often don't match up with what the code does. Having documentation that actually tells you the current state of the system would actually be the most useful but I have doubts I'll ever see something like this unless I do it myself. I always get blank stares when I suggest it, anyway.

sajeev gopinathan

Posts: 1
Nickname: jeevs
Registered: Dec, 2006

Re: The Big Rewrite: Why So Hard? Posted: Dec 28, 2006 8:43 AM
Reply to this message Reply
Most often technology support limitations are the the cause of big rewrite. Why its so hard it because , with change of technology, sometimes its hard to match the user experience of a old system with the new system. And mostly to match the performace and features of the old system, lot of investment in hardware and extra effort is required.

Eirik Maus

Posts: 15
Nickname: eirikma
Registered: Oct, 2005

Re: The Big Rewrite: Why So Hard? Posted: Dec 28, 2006 2:20 PM
Reply to this message Reply
I joined my current employer may 2004 to work on a project re-creating a medium sized cobol (mainly) batch application with java, spring, hibernate, jms and all that. At that time we were about 5 or 6 months from release. We are still 5 or 6 months from release....

Eirik Maus

Posts: 15
Nickname: eirikma
Registered: Oct, 2005

Re: The Big Rewrite: Why So Hard? Posted: Dec 28, 2006 2:23 PM
Reply to this message Reply
okay, typo, it was in 2005, but the point is still valid.

Bill Venners

Posts: 2284
Nickname: bv
Registered: Jan, 2002

Re: The Big Rewrite: Why So Hard? Posted: Dec 28, 2006 4:09 PM
Reply to this message Reply
It didn't occur to me when I posted this item yesterday, but I once lost a lot of money because of a Big Rewrite that failed so badly the company went out of business. I was working as a consultant and unfortunately I was owed I think it was over $10k at the time. They had a product that was written using some tool that sat on top of DOS, and they wanted to rewrite the thing in C++ for Windows 3.1. It blew up into a massive project, in big part because it wasn't managed well (they didn't have a software manager for a while.) But they embarked on this project with no experience in the company using C++ or Windows. They made a vague contract with a customer for a fixed amount of money to do this replacing, and after that ran out, they really had trouble paying for the continued development. So they started borrowing money from a customer, maybe the same one waiting for the very late Big Rewrite software, and eventually that customer just decided to cut their losses. Everyone lost their jobs. Some of the founders, who had been required to put in more money themselves lost even more.

Chee Seng Chua

Posts: 62
Nickname: quai83
Registered: Nov, 2006

Re: The Big Rewrite: Why So Hard? Posted: Dec 28, 2006 6:10 PM
Reply to this message Reply
Hi guys,

I am in the position where I am going to be responsible for a Big Rewrite. The rewrite aims to change an existing 2-tiers Fat Client application written in Delphi to N-tiers web application written with Java backend and Rich Client frontend. The Rich Client technologies is still to be decided, but probably not going to be Ajax. It is based on the fact that although Ajax has improved greatly UI experience for webpage-based application, it is still some way to go to close up to traditional desktop-based application, both in terms of UI experience support and IDE support. I plan to pick richer web client platform such as Flex 2 for frontend, hopefully it can cut down the project risk.

Any advices for a Big Rewrite is appreciated, thanks Bill. ;)

dave wave

Posts: 5
Nickname: wave
Registered: Dec, 2006

Re: The Big Rewrite: Why So Hard? Posted: Dec 29, 2006 3:27 AM
Reply to this message Reply
> I plan to pick richer web client platform
> such as Flex 2 for frontend, hopefully it can cut down the
> project risk.
>

In my experience it's all about managing risk, how big are the risks, the 'hopefully' in your sentence is a red flag that should be examined, don't proceed until all the hopefully's and the I thinks are gone or at least a strategy for managing them has been decided.

Write down all the risks and review it with the team, some obvious ones are
1) new technology (flex) - big
2) requirements - have they changed, are they well understood - don't know
3) development staff availability - for flex probably not available - so big, training required
4) hardware availability - don't know
5) etc.

The other thing you can do is suggest strategies as part of the risk management if the worse case occurs, if there is no alternative then you've identified the highest priority risks that must be evaluated and eliminated.

Management must be made aware of these risks, a risk / benefit analysis should be made, what will flex, as an example, give them, how much will it cost, what support infrastructure will be necessary, etc. It doesn't have to be huge, a couple of pages to start with, the most important thing to outline are unknowns and assumptions.

If these are presented up front an informed decision can be made, and everyone can act on the risks. The assumptions can be questioned (and if necessary pointed to when the blame is going around). Too often it's the other way around, someone says it has to be done in x months, here's a technology that looks good, let's do it, this ends in tears, because developers are always optimistic about how long things take and how good a new technology is, there's always problems, things never work the way you think they would, eliminate all the known gotchas, and wait for the unknown gotchas to get you.

In my experience the best way to mitigate the risks is focus on the unknowns and high risk items, and create small pilot projects to test if the assumptions are valid. For instance if setting up a flex environment and getting samples going takes 2 months instead of the 2 days you thought it would, well there's a problem, however if you've got it working that afternoon well things are good.

In many ways new tech is a separate project from the actual business solution, and all the requirements for the technology should be evaluated before the actual project. Of course the real world is never this neat, and deadlines are always here before you thought they would be.

Good luck.

Flat View: This topic has 30 replies on 3 pages [ 1  2  3 | » ]
Topic: Rebooting Java Media Previous Topic   Next Topic Topic: Groovy 1.0 Released

Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2019 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use