Data-driven estimation the power of the Monte Carlo Simulation

Nearly thirty years ago I worked in the Construction industry and one of my roles was to create estimates for how long a future novel piece of work would take in time/effort; resources and then attribute a risk profile and lastly the margin of return known as gross profit. At that point we have something like ten different teams. I say teams. We called them gangs. Whilst some of the gangs were relatively stable others were not given the fluid nature of the jobs in the pipeline team members needed to move for short periods of time. I would often seek to bring gangs together in response to the size, complexity and milestone (payment dates) as back then much of the work we did was earned value given key milestones.

How did we grow a successful business? There were many key factors that made the family business distinctive: branding; attitude to safety, strong leadership; performance-related pay and bonus schemes; highest standards of training; professional accreditation and many more. But for getting our estimates right? Monte Carlo was key. We were a deeply data-driven and data intelligent business.

Stated simply, the Monte Carlo simulation furnishes the decision-maker with a range of possible outcomes and the probabilities they will occur for any choice of action.. It shows the extreme possibilities—the outcomes of going for broke and for the most conservative decision—along with all possible consequences for middle-of-the-road decisions.

Stated briefly, Monte Carlo simulation provides a number of advantages over deterministic, or “single-point estimate” analysis including:

  • Probabilistic Results. Results show not only what could happen, but how likely each outcome is.
  • Graphical Results. Because of the data a Monte Carlo simulation generates, it’s easy to create graphs of different outcomes and their chances of occurrence.  This is important for communicating findings to other stakeholders.
  • Sensitivity Analysis. With just a few cases, deterministic analysis makes it difficult to see which variables impact the outcome the most.  In Monte Carlo simulation, it’s easy to see which inputs had the biggest effect on bottom-line results.
  • Scenario Analysis: see exactly which inputs had which values together when certain outcomes occurred.  This is invaluable for pursuing further analysis.
  • Correlation of Input model interdependent relationships between input variables.  It’s important for accuracy to represent how, in reality, when some factors goes up, others go up or down accordingly.

Of course, you do need a decent understanding of statistics. To this day I am still very grateful for my practical training back then, as well as completing advanced statistics such as multiple regression, analysis of variance and mixed analysis of variance when I completed my MSc in Organisational Psychology at Cardiff University.

One of the most exciting prospects of the current online Kanban tool is that we are starting a journey of moving from guess-estimates (data neutral based on faulty human reasoning and bias) to a data-driven science of estimation. And delight of delights? This tool has an in-build Monte Carlo simulation! Yes…dreams sometimes can come true…

Have a good weekend,

Jason

 

 

 

 

‘Lessons to be Learned’ is this anything more than a rhetorical device?

network
You might recognise the picture above as a human neural network. This is the miraculous or amazing ways by which the human brain disseminates information that we all use to make sense of our external (environmental) and internal (psychological) worlds. I believe that this is a useful metaphor for learning at a number of levels, including individually, team and organisationally.

This blog is all about learning. But the motive behind writing it is all about relieving pain, upset, misunderstanding and disappointment. Consider the following scenario:

A loved one is treated poorly by a provider of care/ treatment. This results in serious harm or death. Given the seriousness of the error an external review is completed taking several months. Consequently a comprehensive 157 page report is produced. The same day this becomes available a press statement including the phrase that we all recognise instantly: “The organisation accepts the reviews findings and acknowledges that lessons must be learned” is given to the national, regional and local media.

Even though several months have passed for you the associated emotions remain raw or visceral and there is this sense of an injustice. You are energised to do three things. Firstly, you read carefully all similar reviews of the same type of error over the last 10-years. Secondly, you ask the organisation to demonstrate the ways by which they have implemented the ‘lessons learned’ from these previous reviews that you have found. Lastly, you informally ask friends that work in that organisation in what ways the ‘lessons learned’ from these reviews has shaped their professional practice over the last decade.

It is an obvious point but worth exploring none-the-less, what difference will it make to you if you discover that very little genuine learning has taken place? And, of course, if you discover that learning from previous reviews has indeed re-shaped professional practice and informed the ethical culture of the organisation what difference would this make to your sense of justice/injustice?

Around 7-years ago I was asked to review (yes me too!) and then design and implement a learning architecture that would address the points that I have outlined above. This is roughly what we co-created.

As a matter of interest this was within healthcare, but as I said before I believe the design features, or principles, could be applied in most organisational contexts such as banking/finance, IT, social services, foster care, schools/education, and the military, and the voluntary sectors such as charities and churches.

We mapped the following key learning building blocks:-

  • Insights from International, National Reviews
  • Learning from other external Reviews
  • Learning from Organisational ‘near miss’ events (logged)
  • Insights from Programme and Project ‘Lessons Learnt’ reviews/ evaluations
  • Learning from Conferences and other CPD events
  • Team Based Learning
  • Individual Learning (PDR)
  • Communities of Practice (Professional)

With bi-directional information and learning ‘flows’ we anticipate the following direct outcomes and wider systemic benefits.

Outcomes:

  1. The rate, or pace, of learning across the network ‘nodes’ should increase over time- given that we have specifically “designed-in” the learning connectedness
  2. There should be some correspondance of learning from each ‘output’ to the relevant learning ‘input’. In this way, we can see that any relevant learning from say a specific project review/evaluation should expected to be found, in say, the ‘community of practice’ for programme/project managers
  3. The same of course, could be said for any professional such as nursing, teachers, investment bankers and so forth
  4. Consequently, the knowledge management skill par excellence- is extracting the right degree of learning granularity from each knowledge input to each learning output

Benefits 

  • Having professional ‘communities of practice’ connected to learning knowledge management will enable skills and knowledge transfer in the most effective ways
  • The added-value from international/national reviews has genuine legitimacy to individual learning- with explicitly mapped transfer points or nodes across the network
  • The learning network is a key enabler for cultural and team climate improvements to this end
  • Individual learning evidently ‘scales-up’ to organisational learning. For example, an individual attending a CPD event would share a brief of that learning that is disseminated to each and every node

whatisbp
 Jason is a Business Psychologist.