In my current role, I work with companies on advanced analytics projects. We build predictive models, which sometimes predict events accurately and sometimes not. In cases where models are inaccurate, it is often due to missing parameters. For example, if one is modelling a process or a machine failure, one may consider all the operational parameters, such as rotor speed, fuel injection rate, etc., but miss some quality or environmental parameters, like chemical content in the fuel, operator experience, etc. This can lead to unintended, and sometimes fatal, conclusions.
One such conclusion was drawn (initially) by the investigative team of Flight 1549, which landed on Hudson River on January 15, 2009 after a bird hit damaged the plane’s engines.
What’s the story?
Chesley B. Sullenberger, the pilot of Flight 1549, became an overnight hero when he executed a miraculous landing of the disabled plane on the Hudson River, saving the lives of all 155 people on board. However, even as the entire nation celebrated his miraculous achievement, the insurance company and investigation committee were busy creating a case against Sully by creating multiple simulations and models that would prove his decision of landing on water wrong.
What was the initial conclusion of the investigative team?
The investigative team built various simulation models with starting data points similar to what Sullenberger faced before the bird hit. However, contrary to the actual event, the pilots operating the simulations landed the plane safely at the nearest airport within the available time. Hence, the investigative team concluded that there was no need to land the plane on the river and risk the lives of the crew and the passengers.
However, Sullenberger had a different viewpoint on the simulations. The flawed conclusion of the investigative team is presented in recent biopic, “Sully.”
How did the simulated model differ from the actual event?
The variables used in the simulated model were quite different from those that Sully faced during the actual event. The simulations had a major advantage in terms of knowledge from hindsight, often called hindsight bias. In other words, the simulation pilots knew a priori the exact course of action immediately after the plane hit the birds. As a result, the simulator was able to making a successful landing at the nearest airport every single time, which led the investigative team to conclude that Sullenberger’s decision to land the plane on the water was wrong. Little did the investigative team know that their models were missing a key variable/data point which would send all their arguments to a trash bin.
The missing key variable
Sullenberger pointed out that the investigative team was missing a key variable in their simulation models: the human factor. The human factor refers to the reaction time it takes for a human to come to terms with the gravity of a situation, to process and try various options, and to make a decision.
When the investigative team added a 35-second reaction time in the simulated model to account for the time lost in decision-making, all their simulations resulted in a fatal crash.
What does this tell us?
Today, organizations are extremely serious about data analytics as they look to increase significant competitive differentiation and positive business impact. Critical business decisions (sales forecasting, price negotiations, customer retention strategy, etc.) are being made based on the results of data analyts’ models. Adding to the complexity of creating these accurate models is the information deluge that can put even a seasoned data scientist off track.
These complexities are similar to what Sully faced when his plane struck the birds. Imagine the pressure he faced, with only 208 seconds to execute an emergency landing. What he pulled off in that short amount of time is nothing less than a miracle.
As data analytics becomes ever more important in the organizational context, analysts will face pressure, much as Sully did. To arrive at the best, most accurate outcomes, it is imperative that they take stock of all key information points while ignoring the “noise.” Perhaps that will give us models that work as well as Sully’s mind.
For more insight on tapping the power of analytics at your organization, see How To Set Up Successful Teams For Advanced Analytics.