Monday, February 16, 2009

Agile in a Ossified Culture

A friend of mine (call him Fred) is trying to get a project approved in his company.  The point of the project is to replace a complicated, multi-step manual process with an automated tool.  In addition to being manual, the process relies heavily on tacit knowledge (i.e., it exists only in the users' heads).  The approach being suggested was agile: several short iterations geared toward delivering immediate business value while beginning to draw the tacit knowledge out of the users' heads.

To give you an idea of how well it is going, Fred is currently in month five of preparing/revising/defending the project plan.  He is hoping to go forward for actual approval in another six weeks.  Assuming all goes well, after budget approvals, etc. are done, the first line of code may be written in another eight weeks (OK, I lied; Fred got so frustrated that he's been doing some coding during this protracted farce).  Honestly, if the waterfall group got together with the PMP certification folks and the most conservative accountants and CMMI/ISO9002 group you could imagine, the resulting process would be cavalier in comparison.  There are a minimum of 16 meetings to review the final plan and budgets before it can be approved.

What is hard to understand is that everyone in Fred's company seems to be terrified by the prospect that version 1.0 won't contain every possible bit of detail, kind of like Athena appearing fully formed from the head of Zeus (although Zeus got whacked in the head with an axe to let Athena out, a feeling Fred is becoming quite familiar with).  What's more, they seem completely focused on having the perfect plan.  Remember this is a manual process that revolves around tacit knowledge.  One of the company's technology leaders asked how the data integrity of the results could be guaranteed.  Currently the "data integrity of the results" is based upon rules that only exist in people's heads, and on manual entry of the results into a back-end system.

Fred's company has a plan-driven culture.  The company essentially believes that planning predicts the outcome.  Requiring detailed plans is their way of preventing wild, poorly thought out wastes of the company's money.  Fair enough.  What I find puzzling it that they can't see that they are losing all of the business value that could be had now if they would just get out of their own way  Given that they seem to agree that the project should and will be done, opportunity cost associated with the delay seems like a foreign idea.

Agile development is often described as a strategy of mitigating risk of poorly understood and/or volatile requirements.  It is well-known that requirements churn have blown many a budget, and eventually killed many projects.  One thing that seems less emphasized is Agile's focus on early delivery of the highest priority requirements.  This can allow the business to realize some value from the software early, even while the plan-driven folks are still in the requirements huddle.

Fundamentally, I'm left wondering whether it is possible for Fred to succeed with his approach.  One possibility is obviously faking a detailed plan, and then doing he intended to do all along.  As long as the results are good, this can be done.

But if Fred wants to play it straight, can he?  How does one sell that to management that doesn't want to hear it?  Is it possible to use Agile be effectively used in a plan-driven culture such as exists in Fred's company?