In my experience with software development at bigco, it's the lead architect-type person, who has the direct ear of upper management and has already made a decision well in advance of the developers/designers even hearing about the project. This decision will not be communicated in advance, rather people must generate several good ideas that will then be pared down to something close to what the architect originally had in mind, though they will all be moderately unsatisfactory due to the obvious limitations in this process. The result is a product that no one is completely happy with but at the same time no one can completely fault because of the extensive research and consensus that went into its development. This lack of enthusiasm will trickle down through the rest of the project until everyone is just looking for a way to get onto some other, newer project that is still untainted yet will inevitably undertake the same cycle in due course.
If a truly excellent idea should arise from the independent investigation, it will be shelved for a 2.0 product release due to the "perceived risk" or "out of scope requirements".
From the outside, however, this process looks exactly like the diagram in the article.
Pardon my cynicism, I suppose this is why I got fed up with commercial software development...I hear it's not like this everywhere;)
I love discussing implementation details of different designs, but it has always been to my disadvantage to acknowledge what's good about a competing vision and lay out in detail the pros and cons of my own idea, because that just gets turned around to support the high-level idea of the senior employee who's been with the company longer (and has already earned the trust of the IT director).
The devil is always in the details, so I love discussing all "gotchas" before code is written. But I guess early in the project, people like to believe that the new design will solve all problems and will only take two months to implement, and a frank discussion can only dampen that enthusiasm. So it tends to get suppressed rather than encouraged.
How could we avoid this? The article says it should be the one who is better at self criticism. But how do we unbiasedely pick this person? You imply non-commercial development is different. How so?
Let me give an example of LSMB's newer architecture which is still being refined.
One member says "Hey, we should use stored procedures and here's why." I say "No, we shouldn't and here's why." We argue back and forth. We end up using stored procedures in a way that avoids my objection. We both win.
One member says "Hey, we should ditch LaTeX for Docbook XML." I say "No we shouldn't because then I have to spend time I could be spending programming learning a new toolchain and documentation master format." He says "No worries. I am sure someone else can maintain the docs." I say "Great." But nobody does so we stick with LaTeX.
The goal outright has to be to ensure that everyone's legitimate objections are met to the extent possible....
A better approach would be for the architect to make the first proposal. "Here's what I am thinking. Any objections? Any criticism?" And then fire off a discussion as to what the shortcomings of the approach are. These can then be the subject of a second iteration or where the architect really is wrong he can go back to the drawing board. Ideally this is done before going to upper level management.
The only problem here is that opening up your ideas to such criticism takes a sense of security and confidence that is lacking in most management types in corporate America today. People's whose primary skills include playing politics don't tend to be confident standing on their own...
Now, here's an example. In LSMB we added the ability to store files as attachments to transactions. I sent off a proposal and a few people liked it and one person really didn't. He talked a bit about how to detect and reduce file duplication in the database. There were some low-level I/O problems with his proposals, but he had other substantive objections which were correct. So we went back and forth on this a while and the end result was that I came back a week later with a new proposal taking a lot of that into account.
Picking great leaders? That's a tough one. An easier question is to ask "What prompts a Lead 'Hat' stick with an idea though it doesn't seem to be the 'best' idea?"
Developers/designers using OP's two-way criticism style are really removing their own egos and emotions from the discussion and letting ideas honestly fight for themselves. I suspect that the Lead "Hat" is a conduit between developers/designers and executives and it's naturally difficult to convey, let alone objectively discuss corporate objectives in a development/design meeting.
The takeaway for Lead Hat lurkers on HN would be to (unfortunately) expose developers/designers to a wider scope of the project when the ideas are fighting, so they can understand where you're coming from.
Productive self-criticism is a hard thing to select for. It shouldn't be that hard to determine if someone has really thought through the ramifications of their own product design, but it can be much harder to determine if they've really thought through someone else's. Organizing people across teams any larger than a few people is a also hard problem, one for which I unfortunately don't have a ready solution. In small teams, this kind of thing is much easier to spot and, hopefully, correct.
One way would be to do some prototype and demo your idea.
At the same time it is no guarantee that this would work all the time. It depends on factors like how cohesive the team is, whether there is openness to listen to conflicting thoughts, ideas etc. Otherwise your prototype might get as well shot down with some specious argument.
In my experience with software development at bigco, it's the lead architect-type person, who has the direct ear of upper management and has already made a decision well in advance of the developers/designers even hearing about the project. This decision will not be communicated in advance, rather people must generate several good ideas that will then be pared down to something close to what the architect originally had in mind, though they will all be moderately unsatisfactory due to the obvious limitations in this process. The result is a product that no one is completely happy with but at the same time no one can completely fault because of the extensive research and consensus that went into its development. This lack of enthusiasm will trickle down through the rest of the project until everyone is just looking for a way to get onto some other, newer project that is still untainted yet will inevitably undertake the same cycle in due course.
If a truly excellent idea should arise from the independent investigation, it will be shelved for a 2.0 product release due to the "perceived risk" or "out of scope requirements".
From the outside, however, this process looks exactly like the diagram in the article.
Pardon my cynicism, I suppose this is why I got fed up with commercial software development...I hear it's not like this everywhere;)