Data-driven estimation the power of the Monte Carlo Simulation

Nearly thirty years ago I worked in the Construction industry and one of my roles was to create estimates for how long a future novel piece of work would take in time/effort; resources and then attribute a risk profile and lastly the margin of return known as gross profit. At that point we have something like ten different teams. I say teams. We called them gangs. Whilst some of the gangs were relatively stable others were not given the fluid nature of the jobs in the pipeline team members needed to move for short periods of time. I would often seek to bring gangs together in response to the size, complexity and milestone (payment dates) as back then much of the work we did was earned value given key milestones.

How did we grow a successful business? There were many key factors that made the family business distinctive: branding; attitude to safety, strong leadership; performance-related pay and bonus schemes; highest standards of training; professional accreditation and many more. But for getting our estimates right? Monte Carlo was key. We were a deeply data-driven and data intelligent business.

Stated simply, the Monte Carlo simulation furnishes the decision-maker with a range of possible outcomes and the probabilities they will occur for any choice of action.. It shows the extreme possibilities—the outcomes of going for broke and for the most conservative decision—along with all possible consequences for middle-of-the-road decisions.

Stated briefly, Monte Carlo simulation provides a number of advantages over deterministic, or “single-point estimate” analysis including:

  • Probabilistic Results. Results show not only what could happen, but how likely each outcome is.
  • Graphical Results. Because of the data a Monte Carlo simulation generates, it’s easy to create graphs of different outcomes and their chances of occurrence.  This is important for communicating findings to other stakeholders.
  • Sensitivity Analysis. With just a few cases, deterministic analysis makes it difficult to see which variables impact the outcome the most.  In Monte Carlo simulation, it’s easy to see which inputs had the biggest effect on bottom-line results.
  • Scenario Analysis: see exactly which inputs had which values together when certain outcomes occurred.  This is invaluable for pursuing further analysis.
  • Correlation of Input model interdependent relationships between input variables.  It’s important for accuracy to represent how, in reality, when some factors goes up, others go up or down accordingly.

Of course, you do need a decent understanding of statistics. To this day I am still very grateful for my practical training back then, as well as completing advanced statistics such as multiple regression, analysis of variance and mixed analysis of variance when I completed my MSc in Organisational Psychology at Cardiff University.

One of the most exciting prospects of the current online Kanban tool is that we are starting a journey of moving from guess-estimates (data neutral based on faulty human reasoning and bias) to a data-driven science of estimation. And delight of delights? This tool has an in-build Monte Carlo simulation! Yes…dreams sometimes can come true…

Have a good weekend,

Jason

 

 

 

 

Appreciating the value of SAFe

SAFe4.0_transparent_background 8.5x11

Over the last few days I have been taking the time to carefully reflect on the reasons why I really appreciate the SAFe framework. I’ve put the link in here for you SAfe 4 and there are a number of case-studies detailed in here SAFe case studies

I will neatly ‘side-step’ the positivist hierarchy of evidence question for the time-being as I think that might muddy the proverbial waters in terms of my appreciating what it offers, for me, and perhaps for you. Stated simply, case-studies offer three things for the interested professional:

  1. Credibility. Many Senior Executives find it helpful.
  2. Insights and Learning: The Case-Studies and CoP help foster respectful collaboration
  3. Evidence. Many large public and private organisations want underpinning evidence for the ‘case for change’ or an associated business case for validation.

But this does not really capture what I have in mind and this is the consultancy cycle approach to incremental change. It is fair to say that I’ve been using this model for over 16-years now. Stated simply, it starts with a problem that needs to be solved. It also sets aside any notion of a prescribed methodology, or indeed methodologies, and instead actively seeks out the established ‘evidence-base’ for what has effectively worked in similar situations/contexts or what we might call case-studies?

To make my point a little more ‘real’ let me provide three hypothetical scenarios and the ways by which the SAFe framework would, perhaps, offer something of value, insight and help.

Remember, of course, that one of the foundations of the agile movement is all around incremental change. That is to say, that we are looking to make small, testable improvements from the current state to the desired future state. We collect data/evidence as we test our hypothesis to this end.

Also, remember that SAFe is a framework and therefore you can select the parts that you wish to test as hypothesis to help you gain more agility.

Scenario One:

The organisation wants to empower its teams to use the most appropriate methodology and associated tools so that they can take seriously the ideas of the self-empowered or organising team.

One of the strengths of SAFe is the operational ease by which each team can adopt, test and refine its own lean-based methods such as Scrum, Kanban, ScrumBan or any refinement that the team makes as part of its own individual agility maturity. We don’t need, anticipate or expect that innovation is quashed by ‘corporate policy’ or the illusion that if every team used the same tools then life would be simpler! SAFe is ace in this regard!

Scenario Two:

There is significant technical debt because projects are being stopped and started. The dependencies are out of synchronisation, and even completed projects are left on the shelf completed without any genuine business value being realised.

Thankfully SAFe has lots to offer in the ‘strategic portfolio operational’ space. At the Enterprise there are key strategic themes. In turn at the Portfolio there is a ‘work-in-progress’ limit to the number of projects that are in the Portfolio strategic pipeline. Thus, the value stream per theme is clear; with enabler projects and Epics being clearly worked up and approved in a ‘light: tight’ governance role. This simply means that the business value of working software is known prior to it being started. SAFe also has a very realistic portfolio budgeting method that lends itself to ‘light: tight’ financial planning. This model is very similar to that advocated the National Audit Office for financial budgets that have a range of variables and costs with the assumptions (and sensitivity analysis) explicit.

Notice though, that if any project has emergent problems and has to stop whilst those problems are solved, that the WiP ensures that there is a worked-up (i.e. ready to go) project for that team. Thus, there are no idle, redundant or sunk costs due to poor sequencing or Portfolio synchronisation. SAFe is first-class in this area!

Scenario Three:

In a word the next problem is all around system improvements. Consider a context with SOA architecture and three projects needing to ‘call’ various SOA services before the transition to a fully production/live services.

In this regard SAFe has lots to offer! Consider the cadence or rhythm of the software (fully tested and system Demo to all stakeholders including the business Users). The neat release train ensures that all the teams know when to have their Epics completed to ‘hit the next train’. This makes System Assurance testing co-ordination that much simpler too. In effect the Business Users have shippable working software more frequently and better tested across the Enterprise.

SAFe also has a very sensible 10 or 12-weeks planning session for all the teams, or silos, within IT or ‘brand IT’. In this way it ensures that the front-line staff across the whole of IT all have co-created a plan that they are all equally aligned with and committed to. (I’ve blogged previously about systemic alignment).

For me, this is very powerful. It shifts the thinking from silo or ‘part’ to the ‘us’ or the ‘whole’ IT family or system. I love this for the collaborative hope that it offers. And given the significant number of businesses across a range of Industries that have, and are, successfully using SAFe this is encouraging to me.

Summary

I hope that I’ve demonstrated the rationale for why I can appreciate the SAFe framework when we are seeking to improve our agile maturity? I hope that whilst you may prefer a different scaled framework, or none at all, given your specific/particular circumstances or contextual factors, that for others SAFe is both a fab place to start that journey, or indeed help the maturity?

Take care, Jason

 

Jason is a Certified Scrum Professional; as well as a Business Psychologist and Agile Project Manager. 

Scrum-Professional-Seal-sm whatisbp

 

Association_for_Project_Management_(logo)