agile projects fund build measure learnOne of the key concepts from Eric Ries’ seminal book, the Lean Startup, is “Build Measure Learn”, or BML. This means that rather than deliver projects in a sequence, organisations should build something small, measure behaviour and learn from those measurements, then loop back and build again (i.e. extend or change the product).

While many agile veterans will say “oh sure, we’ve been this for years, this is just iterations”, they are wrong. Agile doesn’t say anything about learning and looping back, it just says to keep building in short sprints. There is nothing in the agile canon about measuring and learning from your product. The team is just meant to keep pulling things off the backlog and building them. This is where Lean Startup has the advantage over Agile: the approach is not just iterative, but circular.

The main problem with doing Build Measure Learn in many organisations it that traditional project funding leaves no money for further iterations! A project has funding for a given time period to deliver some amount of scope (in Agile as well as Waterfall). Once the scope has been delivered (via Agile or Waterfall methodologies), and the funding has run dry, then the project is “finished” and we all pack up and move on to another “project”. We need a different funding model, whether you are waterfall or agile.

How BML actually works

Build Measure Learn is not a path or a flow, it is a loop. You go round and around. You build something, you take measurements from it, and you learn from those measurements, then you build on the same product again based on the learning you just got from those measurements.

Build Measure Learn also includes Hypotheses

You actually start with hypotheses, to be exact. “We hypothesise that customers will want to use an option to authenticate on our site via a Facebook login rather than creating a new account with us”. Then you build that, then you measure its usage. “We learnt that customers didn’t want to use their Facebook account to log in to our site. We have a new hypothesis, that customers will want to use their Google account to authenticate on our site”. Then you build that, and so on.

Learning canvases

Some projects begin with a “Learning Canvas”, where you set out all of the things you are interested in learning, and importantly, why. It is crucial to map these learnings back to outcomes or impacts for your business, otherwise the learnings are pointless. The Learning Canvas will also state the initial set of hypotheses for the project (these will be updated regularly as you build, measure, learn, and produce new sets of hypotheses).

Learning and analytics

Measurements and learnings are crucial for anyone following an agile / lean startup model. Your application needs analytics and they need to be as open, accessible and close to real-time as possible. Getting a report mailed to three senior managers once a month is not good enough. There are plenty of off-the-shelf analytics tools out there, pick one of them and plug it in. Or build your own (not recommended). Also, beware of vanity metrics (a blog post for another day).

Where waterfall (and most Agile!) funding fails

Projects only have money for one iteration

Most projects (both Waterfall and Agile) have funding for one big “iteration”, even if that is made up of many sprints. “Here’s a million bucks, spend six months building this thing, in short sprints, then great, we’re done”. You’re not done: you need to keep learning and iterating. Or if the product sucks, kill it. Don’t ever be afraid to kill badly performing features or applications. Steve Jobs cancelled nearly every project and product when he returned to Apple, and just focused on a few (outrageously successful) ones.

Scope and benefits are defined upfront

This is a traditional pitfall of Waterfall projects, but sometimes organisations starting to move to Agile suffer from this as well: you define all the scope upfront. In an agile context, this means you write out all your epics and features as soon as you secure your funding, and don’t change them, because your funding was based on benefits, and those benefits are often tied to those epics and features. The fact is, your benefits are unknown and have extreme uncertainty. And they shouldn’t be specifically tied to scope items anyway – the business wants benefits, and shouldn’t really care what scope they get them from. You will discover what benefits, if any, your project will deliver as you start delivering it. If it isn’t delivering any, kill it and start a different project.

How we can improve it

This problem isn’t unsolvable. It requires changes, not so much in how we do software development, but in how we fund software development. This is not a technology problem, it’s an accounting problem. There are two solutions, an easy one that is not very good, and a harder one that is much better.

First option: include post-project funding

The first option is to secure funding for a number of sprints after the main release or launch of the product. This will enable you to learn, pivot if necessary and iterate based on your learnings. Make sure there are no benefits or scope whatsoever tied to this funding: the scope will be defined “just in time” based on learnings from the analytics you built into the product. This is a slight improvement but it doesn’t give you many chances to innovate.

Second option: move to a iterative funding model

The better model involves a major restructuring of how your projects are funded. You need to move away from a “project” mindset to a “product” mindset. Where a product is funded in perpetuity, with no particular scope attached, until it no longer delivers sufficient benefits. At which point it is killed (and either decommissioned, or rolled into BAU / maintenance mode). Until then, it is funded sprint by sprint, with no scope attached to that funding. This is a truly agile funding / accounting model and will enable a team to continue moving around the Build Measure Learn loop, until it no longer becomes useful or valuable to do so.

Question: have you encountered difficulties in implementing Build Measure Learn? How did you overcome them?

  1. I’m keen to explore the idea of having this perpetual funding model for a given product or channel. A product or channel will have an ongoing BAU budget allocated, which would be re-defined yearly due to changes in operating costs. If we fund product enhancements the same way, then we would need to determine a minimum amount for the investment funding to be effective. That would probably be most effective by starting at looking at what the costs are for a single scrum team to operate over a period of time (ideally for the full 12 months of a year and ongoing). Now this might then work for said team to deliver some outcomes by continuously looping through refined hypotheses that support an idea. Where the challenges would come up is what if there are multiple ideas for enhancements to a product? Maybe that decision would still fall to the product owner, if the ideas are contained to the single product. There will still be undoubtedly situations where there are cross-product ideas, especially with a growing focus on creating ecosystems for customers to engage in. Subsequently, there would still be situations where, in an organisation, there would need to be a prioritisation process to determine what to invest in. Perhaps the funding would be guaranteed (e.g. $X for product A to have 2 scrum teams working for the financial year, $Y for product B, etc) and then the product owners would need to collaborate in a prioritisation process to determine what ideas should be evaluated for their riskiest assumptions. I’d wonder how that collaboration and discussion would go down in real life, where undoubtedly each product owner would be driven to achieve the company vision and/or their personal KPIs. What about investments in new product areas are thrown into the mix? There would always be a dilemma of opportunity costs for each investment. I definitely think you’re onto something and it’s worth exploring further to see how it a different model would really work in an organisation.

    Anyways, great blogs. Keep it up!

    • I think this is a good model, but the approach should be more iterative, with a much smaller timescale. Maybe every few months, or even shorter, you would review the funding for all your products / channels. And then decide if you want to keep doing BML loops and adding enhancements and features and fixing bugs, or put it into maintenance support (just fix critical bugs), or kill / decommission it. If you have multiple ideas for enhancements, then yes, that would be a product priority decision. Thanks for the comment and feedback!

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes:

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>