By Gary Stocks, executive at BSG

We often hear horror stories about business projects ending heavily over budget, with the flexible nature of agile being used to justify the lack of governance. These stories often deter decision-makers from opting for an agile approach, as they are typically accountable for spending and project outcomes, so an approach that appears to lack the necessary assurance is likely to be quickly vetoed.

But, ignoring agile can have equally disastrous implications, especially when we consider that, on average, large IT projects run 45% over budget and 7% over time, while delivering 56% less value than expected. As more and more projects are run in an agile manner, businesses are seeing the value it can bring, such as delivering change faster and more flexibly than traditional linear delivery methods, such as waterfall-based approaches. And according to the 2011 CHAOS Manifesto, agile projects are three times more successful than waterfall projects.

Because solutions are designed and built in stages, agile approaches are able to deliver value incrementally and iteratively, based on priorities defined by the business. Waterfall-based approaches, on the other hand, define a detailed set of requirements upfront and build the solution according to those. With agile, the business is able to continually adjust and refine its requirements, whereas system requirements in waterfall-based approaches are fixed from the beginning. This means that, when following an agile approach, developers are able to adjust the solution based on be business’ evolving needs.

Although agile approaches have become more widespread, the majority of agile projects have been smaller and more focused, being applied to relatively simple solutions with fewer integration points and co-located teams. The challenge now is to successfully apply agile thinking and practices to enterprise-wide solutions with large budgets and more stringent investment approval and governance requirements.

What is enterprise governance?

Large, complex enterprise-scale programmes and projects, like the development of core systems, are usually governed by stringent controls because they typically involve significant financial, time and resource investment. These controls include upfront investment approval, which is typically gained by drawing up a business case with quantified costs and benefits, which are estimated based on the scope of work and defined outputs.

Additional controls, such as oversight of project execution, also form part of the governance requirements. This oversight is included in order to ensure achievement of business outcomes and the definition of decision-making responsibilities. During project execution, the steering committee regularly holds the project sponsor accountable by comparing actual progress with planned outputs. Significant risks and issues are reviewed and mitigation decisions are made.

There are various levels of governance, delegated from the investment committee to the project sponsor, who in turn delegates to the project manager, and so on. The important thing to note is that funding is tied to a defined final output.

Does agile contradict governance fundamentals?

Not necessarily, but a shift is needed in order for agile programmes and projects to be effectively governed. This shift takes place at three levels:

1. Investment governance: agile approaches are designed to release working software more frequently. As a result, an accurate upfront estimation of scope is not always possible. Project sponsors should insist on a high-level effort and cost estimate, based on a subset of requirements that can be extrapolated to the full set of requirements. This can then be used to secure an initial budget. It is also critical that an experienced agile practitioner educates the investment committee on the finer details of agile principles and successes.

2. Architecture governance: agile approaches don’t naturally lend themselves to architecture governance because the architecture of an agile solution emerges, rather than being specified upfront. By facilitating engagement between the architecture function and the agile solution team, it is possible to gain, upfront, agreement and alignment regarding the level of detail required for the architecture, as well as how it will be defined during the build process. The success of this also depends on continued communication and alignment between the agile solution team and other enterprise support functions, such as integration and marketing.

3. Project governance: governance structures and tools are often tightly integrated with linear, waterfall approaches, which can mean they aren’t always effective in agile approaches. Businesses should use a high-level definition of the solution as the basis for the agile team’s mandate and insist on a detailed definition of each iteration, before commencement. This will help to break the project into smaller sections, thus making it easier to govern. Frequent reviews, burn down charts and visible storyboards also help to build confidence with stakeholders.

Despite the apparent governance challenges, enterprise-wide agile programmes or projects can be successfully managed within the frameworks of traditional governance, provided that the necessary changes are understood and communicated. As with any change exercise, strong leadership is critical to ensure its success.