Summary
This may come as no surprise, but we deliver crap. Why this is so has been bothering me for a decade. Maybe this is why.
Advertisement
It is a premise of mine that nobody who develops largish systems should ever be anything but embarrassed. Our (that is, the industry's) track record in delivering systems is pretty abysmal. Late, over budget, buggy systems is the norm. To be clear I'm not talking about little bitty 100k LOC guys, I'm talking about 1M+ LOC systems that are expected to have very long lifetimes. You can disagree with this premise all you like but I think the evidence is on my side.
I constantly think about this. Constantly. I have been, for about 10 years or so, trying to understand why it's so hard to build quality systems. I've tried lots of things; tools, processes, languages to make things better. Or at least appear better to me. Unquestionably some things have been enormously helpful. Java was, in my mind, an enormous leap in popularizing useful things like GC and OOP (well sort of. OOP today is crap, but that's a different story). Language aware IDEs like IntelliJ and Eclipse have, at least for me, dramatically changed how I talk to my code. For the better I think. Nevertheless I routinely feel unsatisfied with how things are. Incomplete or something. I dunno. Just not right.
This morning while watching the sun rise and contemplating some recent circumstances I've been dealing with I realized why the state of developing systems is the way it is. The name for my pain is...Managers.
Before you tune out, or nod your head in vigorous agreement, this is not a "Managers Are All Idiots" diatribe. On the contrary, I've found most of the professional managers I've worked with to be well intentioned, reasonably intelligent people. To be sure some I've worked with and for some that have been horrifically stupid (<= carefully chosen word) but then I can say the same thing about some developers. No, it's not that I've decided that managers are stupid. They're not. They're children. This is what I realized with my coffee. Thank you Peets.
Most of the managers I've worked with made a conscious decision to track on the management career path roughly because they were interested in it. (I have no idea why anyone wants to herd cats for a living, but evidently some do and that's ok I guess.) I don't know of many development managers who weren't code jockeys for a little while. Sometime in their early 30s though they started on the management track. This is the crux of the problem I think.
In my early 30s I was, like a lot of developers I know and have and do work with, annoyingly cocky. I was confident I knew how to write software correctly and, I'm now very embarrassed to say, there was not a lot that someone could tell me about writing software. This is exactly the time when many managers decide to become managers. And now their software world view ossifies with this sort of cockiness, this know it all, it's not that hard, perspective built in.
So when I now go to a manager and attempt to explain that the coupling in the system is going to crush us in a year or so unless we take steps to address it now, their world view doesn't include a perspective that let's them say "Oh right. I remember making this mistake.". They can only recall that in their short developer lives simple code hygiene was never really an issue. (As a side note I giggle, absolutely fucking giggle, when I see job postings for senior(?!?) software engineers with 7+ years experience. 7 years. Senior. What's a Junior engineer? 30 days and read a book? Busts me up.) This means that getting this sort of work into the list of tasks is never really with these manager types full buy in. When push comes to shove this kind work is always dropped off of the schedule (sprint list, whatever) because the folks who decide what's on the task list don't have any basis for valuing it. They can see the value of features and functions, but decoupling the system is not something you really value until you've felt the pain of not actively doing it.
I think there is some element of truth in this new perspective of mine. Managers should not be managers until they've written code for something like 20 years or so. They shouldn't be deciding things that they have no real experience in. Of course nothing is going to change. Not yet. Frankly I think we'll continue to develop systems the same way, and deliver the same crappy results, until some Titanic sort of event occurs. I don't know what it will be but, eventually, certainly, we're going to kill a lot of people, or put a large number of people into the poor house overnight or some equally human outcome. I'm not talking about the annoyance of some Outlook worm that brings your network to it's knees. That's really an efficiency issue. Eventually our hubris, our lackadaisical attitudes will catch up and people are going to be seriously and righteously pissed off. That's when the manager inversion will be corrected. It won't be features, it will be correctness that drives and to do that managers will have to know correct from a hole in the ground. Too bad we'll wait so long.
I would say that either A) you read my mind or B) you sit in the cubicle next to me. I know B is not the case, so it must be A :-)
I've been dealing with this directly for a year now and indirectly for much longer. And I must be some mutant because I just hit 30, I realize I can still learn a lot, have no desire to be a manager and can't for the life of me see how people with such short sightedness ever got to the positions they did. Some days I want to hurl my monitor through a window. Those are the good days.
My last manager missed coding and developing so he left for a position where he could get back to that. My current manager isn't a developer by any stretch of the imagination. Most of the time this doesn't cause any issues, because said manager is smart and is, unlike a lot of people I have worked with over the years, is willing to listen. But every now and then you come across something and cannot do a single thing to impress the importance of the issue on this sort of person's mind because they have no experience with the problems that are there and people don't often take you on faith. Or you spend more time documenting what you are going to do than actually getting the work done.
So I document a lot of crap and in X time frame, I'll at least have my observations and comments to back me up when somebody comes down from on high and asks why something is such a piece of crap. It's most maddening because people seem to care more about the status of the work than the work itself. Is it me, or is that crazy? I should just have to worry about getting the software developed in the most efficient, reliable way possible but instead I have to worry more about covering my a$$ than getting quality work done. And alas this isn't new or specific to this job. And it wasn't an issue until relatively recently.
Anyway, I've really enjoyed your writing up to this point. Partially because I agree with some of it and partially because your writing always seems to engender a decent discussion, which is another lament I have about my current working environment. Very few people seem to care enough about what we do to talk about it and figure out how to make it better. <sigh>.
Back to playing LOTR Return of the King. It is Saturday, after all, and the kids are sleeping :-)
It's not just that people became managers after only four to five years coding it is just that computing has changed a great deal since they were programmers and they no longer have the hands on insight that playing with the code gives you. They have to rely on 'management skills' (writing reports, writing schedules, writing gantt charts, writing anything ... but code).
The reason that they don't appreciate the problems of scalability of client server systems is that client server systems didn't exist when they were coders and so they cannot assess the issues that are presented to them.
I have seen project managers trying to do the old mainframe DB commit / rollback model within a web application. It worked for them when they programmed mainframes so they are going to do it again. And when you finally get them to understand the concept called 'state' they just keep trying more and more convoluted code to get the commit / rollback to work.
This also applies to companies who start small and then grow big. One company I joined started as five people and a handfull of quick basic programs. It ended up as multi- million pound company whose products were priced like telephone numbers.
The company now had in excess of 80 programmers and this is where the problems started.
The Director of the company was ok, his job / skill were much the same for the task - the clients were bigger but the job had not really changed. The same for the Sales Director.
But the technical people had real problems, they were all 'senior this' and 'chief that' but they were not really particulary good programmers, they were just in the right place at the right time. The people they were hiring and trying to manage were much more skilled and experienced (for the founders of this company this was their first real job).
Not only did their programming skills suck, so did their management skills. But by god they had 'senior' and 'chief' in their title and so they weren't going to take any lip from a programmer with 20 years experience.
The good news was that the director was no fool and sidelined the founders into 'special projects' whilst he brought in some new management.
A possible scenario of what exactly would happen after the hitech Titanic disaster: large groups of software engineers (say, 10, 30 or whatever) led by managers will be replaced by smaller groups (2, 3, 4) of programmers with no manager at all.
They will spend nearly the same time for the same projects as the large group, because the need to coordinate interfaces would be minimal and because each of these 2, 3, 4 programmers is a real talented and experienced programmer, as opposed to someone who's just read a book on Java or C++.
They will produce less bugs, because they will wright software in a language/IDE they like, not something that the boss considers 'right' just because it's STANDARD or because everyone uses it. They will choose tools, libraries, components they feel comfortable with. They will think about reusability to save their time.
And finally, they will cut costs dramatically because, obviously, the group consists of 3 instead of 30 people.
I too have felt this pain, but I can't help thinking that there is another corollary to this, one which makes a lot of developers uncomfortable.
Sure. As developers, we all agree that to really understand the issues affecting software development, you need to spend your time "in the trenches". Most of us would also agree that candidates for management and other decision-making positions should be pulled from the pool of people who really understand these issues.
But, if you ask professional testers, they often have a similar idea. If you are faced every day with flaky implementations of misunderstood specifications you begin to wish that all software developers start as testers, and only progress to developing software once they really understand the issues involving specifications, testing, integration and delivery.
At first blush this sounds reasonable, doesn't it? But as a developer, I'm pretty sure that you don't want to be forced to spend five years or more clicking the same buttons over and over again and raising fault reports which will just be swept under the carpet. It's not what you got into the business for!
Now look at it from the point of view of managers. If you were interested in managing - planning, organizing, getting people motivated, etc. - would you want to be forced to spend years mucking around with software just to be given a chance to do what you want to do with your career? And what if that meant five years of testing and then as many again in development.
It's a nice idea, but I can't see that it would ever work.
The immaturity and inexperience of both developers and managers certainly contributes to lots of bad software. I've been programming (and sometimes managing) for 25 years, and every week I see both managers and programmers making mistakes that were understood and written about 20 or 30 years ago.
Projects frequently fail because participants focus too much on technical issues. Anyone who has worked on a serious project has experienced this: During testing the team suddenly realizes that an important business requirement was not understood. Probably lots of them. During a demo or training an actual user asks some unexpected questions, revealing the development team's shallow understanding of the business problem. At this point no one wants to go back to requirements or design so the problem gets glossed over, the user reassured with promises of version 2.
Programmers (and former programmers acting as managers) often shy away from criticizing other programmers, even from establishing who is senior and who is junior. Programmers are expected to pretend that someone with one year of experience is just as qualified as someone with ten. This was most evident during the dot-com boom hiring frenzy but still happens enough to make the profession seem amateurish and undisciplined.
I think programmers frequently fall in love with technology and don't spend enough time learning their craft. Mastering Java is hard and takes time, but learning how to understand a business need and translate that into working, useful software is harder and takes a lot more time.
I don't think that the managers missing development experience is the real problem.
Our domain is so complex, that we have to develop specialties. We need good developers that have several years of experience in development and we need managers with several years of experience in management. A long career in development doesn't make a good manager.
The real problem is, that we need to work together. And working together always involves both sides. Management needs to include development into their decisions and need to listen to the developers concerns. But developers also need to listen to management and understand their concerns.
We developers too often focus our attention primarily on technology. Our own views are predominant. We easily ignore the management view.
But lets face it: We depend just as much on management as management depends on us!
> So let's work together and listen to each other!
That would imply that there is a give and take, beyond the current management gives, development staff takes model, which seems to be where I'm stuck. And a lot of other people I know are stuck, too.
To comment on Frank's point, where I'm at the development staff does a LOT of testing and some of us go onsite to help out with early adopters (pre-beta) of our software, so I think the development staff at my office gets their testing time in. They also get to see real life use of the software up close and personal. Every onsite visit I've had has gone well and the face time with the customer has been very helpful for me.
I'm also not sure how being a tester would help you with understanding flaky specifications. It sounds like you make the grave error that the specification is good and merely misunderstood by the development staff which has no grasp of the business problem. I think that's a bad assumption. Granted the flaky implementations would go down (a monumental achievement in its own right), but that doesn't solve the misunderstood spec problem because, in a lot of cases, there is no spec beyond "Can you make it do 'y'"? At which point you can only get information about the facets of 'y' that you either already understand or know to ask questions about. So you're left shooting in the dark most of the time. If you're lucky you get one of those piece of crap key chain penlights to help you slog through the darkness.
And maybe I'm generalizing here, but, in my experience, management's concerns entail getting the maximum amount of buzzwords into the next release with the minimum amount of effort and cost. It sells products and is cost efficient. At least in the short run.
> Managers should not be managers until > they've written code for something like 20 years or so. > They shouldn't be deciding things that they have no real > experience in.
Software sucks because by-and-large people suck. I don't think requiring experience is an answer that works across the board. I'm sure there are plenty of examples of 20 yr experienced managers who suck as managers as well.
Programmers suck too, because, oops, they're people as well. Bruce Eckel waxed eloquent on this topic as well [ http://mindview.net/WebLog/log-0038 ]:
"On the other hand, the WBC summit was really about psychology: why do programmers write poor code and don't seem to care about it, and what can we do to convince them to write better code and to care? At best, we succeeded in enumerating the problems that we had seen, so compared to the other summits we reached no conclusions. But perhaps the struggle was the important thing, and like Weinberg's PSL company simulation, we all needed to have these ideas inserted so we could struggle with them over the ensuing years."
You're now in the realm of psychology and religion and the soul. Why is it people do the things they do?
Isn't it, just ever so slightly, suspicious when the reason "why software sucks" is some non-programmer is too callow to make a decision the programmer would approve?
Shouldn't the programmer take some responsibility for what happens? Obviously they've failed to influence the decision, they've failed to persuade the manager that a project risk should be addressed. Maybe they need to learn how to influence and persuade?
So when I now go to a manager and attempt to explain that the coupling in the system is going to crush us in a year or so unless we take steps to address it now, their world view doesn't include a perspective that let's them say "Oh right. I remember making this mistake". They can only recall that in their short developer lives simple code hygiene was never really an issue.
Or maybe they have different information - if we don't get the current tasks done by X the whole project is going to be cancelled.
> Isn't it, just ever so slightly, suspicious when the > reason "why software sucks" is some non-programmer is too > callow to make a decision the programmer would approve? > > Shouldn't the programmer take some responsibility for what > happens? Obviously they've failed to influence the > decision, they've failed to persuade the manager that a > project risk should be addressed. Maybe they need to learn > how to influence and persuade? > > > So when I now go to a manager and attempt to explain > that the coupling in the system is going to crush us in a > year or so unless we take steps to address it now, their > world view doesn't include a perspective that let's them > say "Oh right. I remember making this mistake". They can > only recall that in their short developer lives simple > code hygiene was never really an issue. > > Or maybe they have different information - if we don't get > the current tasks done by X the whole project is going to > be cancelled.
So the manager has made promises to people based on what information? If tasks done by X is the reason, and the developer said 'yeah, I can get that done' and then they don't, they shouldn't take some responsibility, they should take all of it. And tasks done by X vs project cancellation is an important piece of information to share. At least it takes out some of the 'what they hell are they smoking' factor of the exchange. At least you know where the person making the request is coming from and you can adjust accordingly. Even that information isn't shared sometimes so all you know is somebody made an unreal request without having a clue of the work involved.
So if one day, some manager comes in and says 'You need to have X done by this date' and that borders on the impossible or jumps right over the line without looking back, what the hell is said programmer supposed to do? They've had no influence on the decision. It's been made already. Any attempt to 'influence and persuade' is seen as 'developers not wanting to do any work'. I take that last quote directly from one of my manager's managers. I about hit the roof when I heard it, because it is the biggest load of crap I've ever heard.
So instead, you get to cut corners and do a less than good job just to appease the powers that be. Because of its inherent flexibility, not to mention the load of 'learn to be a pro-programmer in 6 hours' type books, software development gets a lot of unreal expectations put on it. I don't think you would be hovering over a brain surgeon, requesting that he be done by 6 pm so the patient can get back to his research on non-linear combinatorial gobble-de-gook when said surgery can't possibly be done until midnight or later because, hey, the doctor want's the guy's brain to be operational when he's done.
Not that most programming projects even remotely approach brain surgery but getting some input from the actual guys doing the work, and then actually giving some weight to that input, would make people's lives a hell of a lot easier.
I think what is most suspicious is that many, many programmers have the same issues and nothing seems to be done about it by anybody. There are articles, books, entire schools of thought surrounding the 'why software sucks' issue. And just about all of them center around the process. This is because process is where you have the most control and you can affect the widest number of people. And a lot of experience bears out that the places where most of the problems occur is in the transfer of knowledge, requirements, etc. from party A to party B. How many managers do you know of that come from a non-software background that have read 'The mythical man-month?' Of those, how many act like the wisdom in that book had any impact whatsoever on their thinking? There are quite a few other good treatises on the topic and I'm sure you can substitute the title in the above questions and get the same dismal results.
I've never had a problem implementing something I thought of because I know what I want. There are a few people I've worked for that I've not had problems implementing anything because they knew what they wanted and they could communicate it in a clear manner. I've also worked for people that I have had problems implementing solutions for because either they didn't know what they wanted or they couldn't communicate it. Short of reading their mind, you can't get what they want. I'm no mind reader. So all I have to go on in these cases is the information I've been given and the information I can glean from either reading between the lines or asking any concerned parties what they actually want, assuming they are even available to ask. And I don't know about you, but my crystal ball tends to be a little fuzzy, so my best guesses are just guesses. Sometimes they're right and other times, guess what, they're wrong! Imagine that!
With some people you can go through this back and forth of playing guessing games and they don't mind, because they realize that you don't have all the information you need and you are trying to deliver the best possible product to them. Others end up thinking you are lazy, an idiot or both. You can't even discuss the problem with these people to try and 'influence and persuade' because their expectations are based on some fantasy where programming is easy work that any monkey can do. To these people I simply say 'If it's that easy, you do it and let me know how it goes'. Somehow, that seems to get people's attention most of the time and they'll at least listen. Other times it degenerates into a good fight and hey, I like a good fight as much as the next guy.
Sadly, there are far too many pointy haired managers running around.
'yeah, I can get that done' and then they don't, they shouldn't take some responsibility, they should take all of it. They should take responsibility for letting everyone know they are not going to get it done asap.
And tasks done by X vs project cancellation is an important piece of information to share And it may even be information that cannot legally be shared.
'You need to have X done by this date' and that borders on the impossible ... So instead, you get to cut corners and do a less than good job just to appease the powers that be
Let's see, the task borders on the impossible, yet it can be done if we cut corners?
-snip-
How many managers do you know of that come from a non-software background that have read 'The mythical man-month?'... How many programmers do you know who haven't even heard of... (the blame game is easy to play, costs nothing - we don't need to change, someone else does - and the blame game changes nothing)
I've never had a problem implementing something I thought of because I know what I want And you don't blame yourself for coming up with later additions and changes ;-)
I've also worked for people that I have had problems implementing solutions for because either they didn't know what they wanted or they couldn't communicate it. And it seems you didn't have the skills to tease-out the information you needed for the next step? And did you negotiate an agreement to do a quick prototype to find out what was required?
Others end up thinking you are lazy, an idiot or both. You can't even discuss the problem with these people to try and 'influence and persuade' because their expectations are based on some fantasy where programming is easy work that any monkey can do. Isn't part of your role setting clear expectations?
'If it's that easy, you do it and let me know how it goes' How fortunate they don't interpret that as a resignation.
Sadly, there are far too many pointy haired managers running around And a fair number of pointy headed programmers.
> 'yeah, I can get that done' and then they don't, they > shouldn't take some responsibility, they should take all > of it. > They should take responsibility for letting everyone know > they are not going to get it done asap.
You must know more understanding people than I, because this never does anything but get a lot of people angry. If you say you are going to get something done, get it done. If it can't be done, sure you let them know, but in most cases that hasn't changed the due date. So the date stays but the functionality or the robustness or the security or some other facet that's less important to the people determining whether you are employed or not gets dropped and isn't a concern. That is until you are getting tracked on bug count and the like. Suddenly, it's a huge deal. But still not one we are going to take much project time to address.
> And tasks done by X vs project cancellation is an > important piece of information to share > And it may even be information that cannot legally be > shared.
Legality or lack thereof doesn't change the stupidity of this state.
> 'You need to have X done by this date' and that borders > on the impossible > ... > So instead, you get to cut corners and do a less than good > job just to appease the powers that be > > Let's see, the task borders on the impossible, yet it can > be done if we cut corners? >
Yes. The impossibility usually lies in delivering the requested functionality in the requested time frame. With enough time, just about anything is possible. If you cut the right corners, time is less of a factor. So is robustness and stability, but people only seem to care about that after it is too late.
> > How many managers do you know of that come from a > non-software background that have read 'The mythical > man-month?'... > How many programmers do you know who haven't even heard > of... (the blame game is easy to play, costs nothing - we > don't need to change, someone else does - and the blame > game changes nothing)
True. But hey, what's good for the goose is good for the gander. It is hard for people without a background in software to manage software projects from what I've seen.
> I've never had a problem implementing something I > thought of because I know what I want > And you don't blame yourself for coming up with later > additions and changes ;-)
Of course. I dock myself two weeks pay and write my mother a very stern letter :-). In all seriousness, I don't change the requirements 3 days before the release and then expect the release date to stay the same. Been there, done that and it stinks.
> I've also worked for people that I have had problems > implementing solutions for because either they didn't know > what they wanted or they couldn't communicate it. > And it seems you didn't have the skills to tease-out the > information you needed for the next step? And did you > negotiate an agreement to do a quick prototype to find out > what was required?
In some cases I do, in others I don't. I'm wondering why it is that you are taking the stance that it is always the programmers fault. That somebody who is a developer must somehow always be able to compensate for the faults/shortcomings/weaknesses/etc. of others. In the cases I've dealt with where getting real requirements have been a problem (despite my bitter tone, there has been only one in the past ten years or so...), questions are seen as pestering and in some cases the person isn't available most of the time. This is a HUGE timesink. Like Artima.com, but at least I get something out of Artima.com besides having my will to live sapped.
Do something for me. Look at the phone at your desk. Ask it for the next great addition to the current project you are on. Tease the requirements out of it. Seriously, depending on who you are dealing with, the inanimate object is better. At least it can't give you wrong information and then ask 'why did you do that?' In those cases it's fun to come back with 'uh, you told me to. Look here.' That always goes over well.
> Others end up thinking you are lazy, an idiot or both. > You can't even discuss the problem with these people to > try and 'influence and persuade' because their > expectations are based on some fantasy where programming > is easy work that any monkey can do. > Isn't part of your role setting clear expectations?
It's hard to set clear expectations in cases where there is no exchange of information.
> 'If it's that easy, you do it and let me know how it > goes' > How fortunate they don't interpret that as a resignation.
Or unfortunate, depending on your viewpoint.
> Sadly, there are far too many pointy haired managers > running around > And a fair number of pointy headed programmers.
> > They should take responsibility for letting everyone > > know they are not going to get it done asap. You must know more understanding people than I, because this never does anything but get a lot of people angry Not as angry as when they the due date arrives, someone has to ask if it's completed and they say no; not as angry as when they pretend it has been completed and it isn't until someone tries to use it...
If you cut the right corners, time is less of a factor. So is robustness and stability, but people only seem to care about that after it is too late. Seems like you have a technical challenge - what do you need to do differently to produce better quality code, faster?
It is hard for people without a background in software to manage software projects from what I've seen Could it just be hard to manage software projects, period?
I'm wondering why it is that you are taking the stance that it is always the programmers fault. I'm just turning around what you write and sending it back. It isn't "the programmers" fault or "the managers" fault - they are both at fault when they play "the blame game" and they are both contributing to the poor level of communication by erecting caricatures about the others.
That somebody who is a developer must somehow always be able to compensate for the faults / shortcomings / weaknesses / etc. of others. And yet, you expect a manager to always be able to compensate for... ? You don't have responsibility for the "faults" of the managers (or other developers) - you do have responsibility for doing whatever you can, to make a success out of the situation. Complaining is at least a sign that you're unhappy with the current situation - but nothing will change until after you change what you are contributing to the problem.
questions are seen as pestering and in some cases the person isn't available most of the time. This is a HUGE timesink Seems like from their perspective, dealing with you is a HUGE timesink. Why do you think your time is somehow more valuable? What can you do to make answering your questions less time consuming?
Flat View: This topic has 34 replies
on 3 pages
[
123
|
»
]
Previous Topic
