Data-driven estimation the power of the Monte Carlo Simulation

Nearly thirty years ago I worked in the Construction industry and one of my roles was to create estimates for how long a future novel piece of work would take in time/effort; resources and then attribute a risk profile and lastly the margin of return known as gross profit. At that point we have something like ten different teams. I say teams. We called them gangs. Whilst some of the gangs were relatively stable others were not given the fluid nature of the jobs in the pipeline team members needed to move for short periods of time. I would often seek to bring gangs together in response to the size, complexity and milestone (payment dates) as back then much of the work we did was earned value given key milestones.

How did we grow a successful business? There were many key factors that made the family business distinctive: branding; attitude to safety, strong leadership; performance-related pay and bonus schemes; highest standards of training; professional accreditation and many more. But for getting our estimates right? Monte Carlo was key. We were a deeply data-driven and data intelligent business.

Stated simply, the Monte Carlo simulation furnishes the decision-maker with a range of possible outcomes and the probabilities they will occur for any choice of action.. It shows the extreme possibilities—the outcomes of going for broke and for the most conservative decision—along with all possible consequences for middle-of-the-road decisions.

Stated briefly, Monte Carlo simulation provides a number of advantages over deterministic, or “single-point estimate” analysis including:

  • Probabilistic Results. Results show not only what could happen, but how likely each outcome is.
  • Graphical Results. Because of the data a Monte Carlo simulation generates, it’s easy to create graphs of different outcomes and their chances of occurrence.  This is important for communicating findings to other stakeholders.
  • Sensitivity Analysis. With just a few cases, deterministic analysis makes it difficult to see which variables impact the outcome the most.  In Monte Carlo simulation, it’s easy to see which inputs had the biggest effect on bottom-line results.
  • Scenario Analysis: see exactly which inputs had which values together when certain outcomes occurred.  This is invaluable for pursuing further analysis.
  • Correlation of Input model interdependent relationships between input variables.  It’s important for accuracy to represent how, in reality, when some factors goes up, others go up or down accordingly.

Of course, you do need a decent understanding of statistics. To this day I am still very grateful for my practical training back then, as well as completing advanced statistics such as multiple regression, analysis of variance and mixed analysis of variance when I completed my MSc in Organisational Psychology at Cardiff University.

One of the most exciting prospects of the current online Kanban tool is that we are starting a journey of moving from guess-estimates (data neutral based on faulty human reasoning and bias) to a data-driven science of estimation. And delight of delights? This tool has an in-build Monte Carlo simulation! Yes…dreams sometimes can come true…

Have a good weekend,

Jason

 

 

 

 

Appreciating the value of SAFe

SAFe4.0_transparent_background 8.5x11

Over the last few days I have been taking the time to carefully reflect on the reasons why I really appreciate the SAFe framework. I’ve put the link in here for you SAfe 4 and there are a number of case-studies detailed in here SAFe case studies

I will neatly ‘side-step’ the positivist hierarchy of evidence question for the time-being as I think that might muddy the proverbial waters in terms of my appreciating what it offers, for me, and perhaps for you. Stated simply, case-studies offer three things for the interested professional:

  1. Credibility. Many Senior Executives find it helpful.
  2. Insights and Learning: The Case-Studies and CoP help foster respectful collaboration
  3. Evidence. Many large public and private organisations want underpinning evidence for the ‘case for change’ or an associated business case for validation.

But this does not really capture what I have in mind and this is the consultancy cycle approach to incremental change. It is fair to say that I’ve been using this model for over 16-years now. Stated simply, it starts with a problem that needs to be solved. It also sets aside any notion of a prescribed methodology, or indeed methodologies, and instead actively seeks out the established ‘evidence-base’ for what has effectively worked in similar situations/contexts or what we might call case-studies?

To make my point a little more ‘real’ let me provide three hypothetical scenarios and the ways by which the SAFe framework would, perhaps, offer something of value, insight and help.

Remember, of course, that one of the foundations of the agile movement is all around incremental change. That is to say, that we are looking to make small, testable improvements from the current state to the desired future state. We collect data/evidence as we test our hypothesis to this end.

Also, remember that SAFe is a framework and therefore you can select the parts that you wish to test as hypothesis to help you gain more agility.

Scenario One:

The organisation wants to empower its teams to use the most appropriate methodology and associated tools so that they can take seriously the ideas of the self-empowered or organising team.

One of the strengths of SAFe is the operational ease by which each team can adopt, test and refine its own lean-based methods such as Scrum, Kanban, ScrumBan or any refinement that the team makes as part of its own individual agility maturity. We don’t need, anticipate or expect that innovation is quashed by ‘corporate policy’ or the illusion that if every team used the same tools then life would be simpler! SAFe is ace in this regard!

Scenario Two:

There is significant technical debt because projects are being stopped and started. The dependencies are out of synchronisation, and even completed projects are left on the shelf completed without any genuine business value being realised.

Thankfully SAFe has lots to offer in the ‘strategic portfolio operational’ space. At the Enterprise there are key strategic themes. In turn at the Portfolio there is a ‘work-in-progress’ limit to the number of projects that are in the Portfolio strategic pipeline. Thus, the value stream per theme is clear; with enabler projects and Epics being clearly worked up and approved in a ‘light: tight’ governance role. This simply means that the business value of working software is known prior to it being started. SAFe also has a very realistic portfolio budgeting method that lends itself to ‘light: tight’ financial planning. This model is very similar to that advocated the National Audit Office for financial budgets that have a range of variables and costs with the assumptions (and sensitivity analysis) explicit.

Notice though, that if any project has emergent problems and has to stop whilst those problems are solved, that the WiP ensures that there is a worked-up (i.e. ready to go) project for that team. Thus, there are no idle, redundant or sunk costs due to poor sequencing or Portfolio synchronisation. SAFe is first-class in this area!

Scenario Three:

In a word the next problem is all around system improvements. Consider a context with SOA architecture and three projects needing to ‘call’ various SOA services before the transition to a fully production/live services.

In this regard SAFe has lots to offer! Consider the cadence or rhythm of the software (fully tested and system Demo to all stakeholders including the business Users). The neat release train ensures that all the teams know when to have their Epics completed to ‘hit the next train’. This makes System Assurance testing co-ordination that much simpler too. In effect the Business Users have shippable working software more frequently and better tested across the Enterprise.

SAFe also has a very sensible 10 or 12-weeks planning session for all the teams, or silos, within IT or ‘brand IT’. In this way it ensures that the front-line staff across the whole of IT all have co-created a plan that they are all equally aligned with and committed to. (I’ve blogged previously about systemic alignment).

For me, this is very powerful. It shifts the thinking from silo or ‘part’ to the ‘us’ or the ‘whole’ IT family or system. I love this for the collaborative hope that it offers. And given the significant number of businesses across a range of Industries that have, and are, successfully using SAFe this is encouraging to me.

Summary

I hope that I’ve demonstrated the rationale for why I can appreciate the SAFe framework when we are seeking to improve our agile maturity? I hope that whilst you may prefer a different scaled framework, or none at all, given your specific/particular circumstances or contextual factors, that for others SAFe is both a fab place to start that journey, or indeed help the maturity?

Take care, Jason

 

Jason is a Certified Scrum Professional; as well as a Business Psychologist and Agile Project Manager. 

Scrum-Professional-Seal-sm whatisbp

 

Association_for_Project_Management_(logo)

 

 

 

Dynamic Leadership: Thinking and Acting Dialectically.

power

Dynamic Leadership: Thinking and Acting Dialectically.

If you read any serious review of strategy or public sector reforms in one guise, or another, you will find the leadership theme writ large. “More leadership is needed” or words to that effect. In terms of my own professional practice I moved away from more ‘fixed’ conceptions of leadership based, for example, on traits or a personality type many years ago.

That is not to say that they don’t have a part to play. Just not in the style of coaching and development that I specialise in. My niche tends to be for those in current leadership positions or roles and are leaders de facto. Therefore, my needs are different. Quite distinct. Unique? Well… not quite that far! But each of them is without a doubt a unique individual.

One model I have found that adds value is one that I developed around 7-years ago whilst completing some first-person inquiry work. I thought I might share it and you can see if it catches your imagination, interest or even curiosity?

Rather than seeing key concepts as ‘fixed’ it creates a context that is much more dynamic or fluid. It is grounded by leader-member exchange theory (LMX) and then fused or integrated with dialectical theory or dynamics.

For those of you that have not heard of this before consider a form of magnetic power like the one in the picture.

mag1

You might remember at school a simple experiment that used magnetic ‘power’ to drive a small object such as a toy car, for example? If not, then imagine one now.

Dialectical power is stating things as plainly as possible… the real energy from the two opposing forces: like the two poles of a magnet, North and South or positive and negative. You’ll soon notice that lots of practical things have two opposites: good and bad; eternity and mortality; the sacred and the profane; inside and out; back and front; etc.

In terms of developing leadership awareness or ‘talent’, skills and responsiveness to a given context my clients have found the following model adds value. For our model imagine two sets of poles or opposites:-

  • Vision/Far Away versus Present/ The-Here-and-Now
  • Individual/Team Needs versus Task/Delivery/Execution

Most of the leadership empirical evidence tends to support the view that leaders will have strengths or preferences for one of the two poles. For example, a Visionary leader may have a compelling strategy. And they might also prefer to meet the individual and team’s emotional, psychological and training needs. However, as you can see their ‘blind spot’ is that they are not strong on focussing on the here and now and the absolute need to deliver a product and/or service to their customers or service users. This analytical framework can be found in many organisations, as well as many a discussion in the staff canteen when front-line staff are ‘getting it in the neck’ from disgruntled customers due to delivery delays!

My clients report that they have found that the practical added value of this model lies in the ability to empower them in these five key realities:-

  1. More flexibly in harnessing organisational energy (a great part of the model is making energy explicit. I’ll blog soon on various organisational energy flows especially around innovation).
  2. Responding to emergent business/customer intelligence in more responsive and effective ways
  3. Team members connecting their work-load to the broader Vision and thereby enhancing meaning; job satisfaction and retention rates
  4. Improved delivery of key products/services to customers
  5. Improved cognitive and emotional capacity through reflective professional practice

I hope this model might help you to see yourself in a dynamic, fluid and changing leadership context and that you can respond in more grounded and reflective ways.

Take care Jason

whatisbp        Association_for_Project_Management_(logo)

Jason is a Business Psychologist as well as a qualified Project Manager professional.

‘Agile is King and PRINCE remains the heir’? Not always…Project Design Choices.

At first glance the following sentence, or statement, will seem self-evident: “The project design will have a significant  bearing on the project’s success”. Fair enough I hear you cry! But what does the empirical evidence-base say about when, or what variables or factors determine when you should ‘design in’ adopting an Agile approach.. as opposed to say a more traditional approach like the classic PRINCE2?

AgileD

In this blog I’d like to focus on the team and project complexity factors. So I will ignore the organisational and the customer factors. I’ll cover those off in a later blog.

What do we know?

Firstly, that there are discernible team factors that should be analysed prior to adopting an Agile approach. These are as follows:-

  • Project teams committment (affective and psychological)
  • Internal project communication style/methods
  • Project teams expertise in the technical knowledge and actual delivery experience
  • Team dynamics and composition/maturity

What we know is that when the team is less mature, relatively inexperienced, and with low levels of technical skills and delivery experience and where the team dynamics are less collaborative and more ‘silo’ focussed then the evidence is that…traditional PRINCE2 project management is the best project design and this will predict project success. This holds true even when the project complexity is low and the project length and scope are short and narrow. A fascinating reality.

However, the opposite also holds true for Agile. What we know is that we should ‘design in’ an Agile approach in contexts where we have well established technical experts that have actual grounded experience of the given project’s demands. We also know that they all need to have worked ‘outward focussed’ and in typically more multi-vendor project designs in the past. In situations such as described and when these skills and experiences are found… but where the team is new then a professional PM can add value by developing the new team successfully in an Agile method as the Scrum Master.

This latter set of variables also holds true in project contexts where there is a need for pace, or urgency as well as innovation in the technical ‘fusion’ or integration of multi-suppliers. In this ‘innovation space’ Agile is ‘King’ and Prince remains in his younger, less mature but none-the-less ‘royal’ household!

I would add to this latter point that having a PM with actual experience of collaborative methods as well as some grounded, actual delivery, of complex projects (multi-million pound)would also be a critical success factor to this end. She, or he, can literally add value to the project outcomes in the ways outlined.

Happy project design!

As you can see your choice will have evidence-based consequences. Choose wisely!

All best Jason

whatisbp

Social Marketing. Health and Social Care: Ignore this Tool at your Peril!

One way of health promotion is to develop a campaign that seeks to address the population as a whole or total. Change for Life would fit under this generalist approach.

Another effective approach or option is social marketing which is a more sensitive and accurate methodology that uses the latest thinking in marketing segmentation for targeted health engagement.

 

BlueGirls

An example might be helpful. Jedele & Ismail (2010) report the following social marketing campaign evaluation. They were interested in two things:-

  1. Raising awareness of and
  2. Screening for oral cancer.

And given this was a social marketing campaign in specific segments of the total population, namely African-Americans in this case as this segmentation were a high-risk group in the Detroit/Wayne County Michigan area.

The programme benefits were:-

  • Citizen engagement at this segment level
  • Reduce the death rate
  • Improved detection rate earlier in the pathway
  • Increased screening rates at years 1, 2 and then 3.

The social marketing campaign costs were calculated per individual. What is important to understand is that the ways by which the campaign was designed, developed and implemented was participatory. Stated simply they spoke for themselves.

During the campaign:-

  • 1,327 radio spots were aired
  • 42 billboards displayed
  • 2 excellent Newspaper adverts were placed in papers read by that segmentation
  • 242 educational sessions were completed
  • The hotline (manned by African-Americans) received a stunning 1,783 calls
  • Interestingly 67% of the hotline callers stated that their call was prompted by the radio advertisement

Outcomes

The clinic screened 1020 adults with a total cost of $ 795, 898 which is some $ 783 per patient (directly screened) as there would be wider systemic benefits for raising awareness too. The outcomes were reported as:

  • 3 cancers detected (early stage detection)
  • 2 pre-cancers
  • 12 tumours detected

This is one example of an effective social marketing campaign. There are others from a community-based stroke preparedness intervention (see Boden-Albala et al., 2014).

And, in terms of emergency care flows a neat study from McGuigan & Watson (2010) looked at the beliefs, concerns and factors influencing patients decisions to attend their local emergency department (ED). They unpacked all the segmentation data and rightly concluded that:

A targeted social marketing campaign is needed to address the misconceptions of people who present at EDs” (p. 34).

This latter inquiry reminds me of a campaign at Barnsley that a colleague completed. This insight data-driven inquiry looked at attendance rates spikes and trends at the local ED. She led a team that developed an innovative campaign that addressed specifically young girls aged between 16-24 and then, very specifically, ‘binge-drinking on a Friday night’. The insights from the intelligence led to a successful campaign for this particular segment or group and thus reduced both attendance as well as admission rates for this group.

To meet rising demands and associated costs on health and social care I’d suggest social marketing is an essential tool.

References

Boden-Albala et al (2014). Methodology for a Community-Based Stroke Preparedness Intervention. Stroke.

Jedele & Ismail (2010). Evaluation of a multifaceted social marketing campaign to increase awareness of and screening for oral cancer in African-Americans. Community Dental Oral Epidemiology, 38: 371-382

McGuigan & Watson (2010). Non-urgent Attendance at Emergency Departments. Emergency Nurse, Vol 18 (6), 34-38.