Stephen Walmsley and Andrew Gilbey recently published a research article on perception and decision making by pilots flying VFR under marginal weather conditions (doi: 10.1002/acp.3225). The authors noted that VFR pilots have a propensity to be biased in their perception of minima MET and to make risky decisions that may put them into IMC inadvertently.
“Pretty much all the pilots we tested fell prey to these biases,” Dr Gilbey would later tell The Independent. “It’s just a very human thing to do.” (http://tinyurl.com/hu2zomz)
This is serious stuff, of practical relevance to all pilots flying VFR … unless, of course, what was reported was seriously flawed.
And here is the conundrum of the situation: as Moynihan put it, “Everyone is entitled to his own opinion, but not to his own facts,” and Walmsley and Gilbey seem to have engineered their own facts when some results didn’t go in the direction they expected. This article is a call to pilots to remind academicians of some basic facts regarding VFR flying in New Zealand.
Pilots—reported the authors—“place too much importance on reports that may no longer be valid and then fail to adjust their perceptions in the face of evidence to the contrary”.
That is, a group of pilots were given a good forecast beforehand and then were presented with images showing weather conditions with a cloud base at 2500ft and horizontal visibility of 16km. These pilots reported, on average, a cloud base at 2468ft and visibility of 15.50km. How did they fail to adjust their perceptions to the evidence (i.e. the presented images)? I wonder.
A second group of pilots were given a bad forecast beforehand—1300ft/8km or 1200ft/7km—then presented with the same images. These pilots reported a cloud base at 1756ft and visibility of 11.48km. Here it is obvious that these pilots did not perceive reality as well as the previous group, but didn’t they actually adjust to some degree?
Both groups also reported that conditions were somewhat safe to continue their VFR flight to destination (the first group being slightly more positive about it), an appropriate response given that both perceived conditions to be above minima Met.
What where the conclusions? The authors implied that the pilots in the first group were risk takers—“pilots exposed to the good forecast were more likely to feel that it was safe to continue the flight”—and that the pilots in the second group were wimps—“pilots exposed to the poor forecast prior to their flight may tend to divert unnecessarily”—even when the authors never asked about intention to divert.
In a later study, Walmsley and Gilbey presented pilots with several scenarios, also in marginal weather conditions. One group of pilots were told that another pilot had flown through previously and landed safely; a second group were told that another pilot had flown through, got into IMC and crashed; and a third group, acting as control, were told nothing about flight outcome.
All groups were asked their opinions regarding the behaviour of the third-party pilot. “How do you rate the quality of his/her decision to flight under those conditions?”
“Don’t know,” responded the first and control groups; “Somewhat poor,” responded the second group.
“How risky do you think the pilot’s behaviour was?” “Don’t know,” responded the first and control groups; “Somewhat risky,” responded the second group.
According to the authors, however, that a group of pilots responded to have no opinion didn’t matter: “Pilots interpreted the decision of pilots who flew into deteriorating weather conditions more favourably when the outcome was positive than when it was negative.”
Or, as The Independent put it, “Pilots are likely to assess their flying decision as the correct one if they are told the flight went ‘well’ afterwards, and their decision as dangerous if told that it ended up ‘crashing’.”
The authors also asked the pilots whether they would be able to safely conduct the same flight themselves. All groups responded with the equivalent of a “no way, José”, although the first group were closer to a “somewhat yes”.
What is more interesting is that all groups gave a somewhat optimistic assessment irrespective of condition; that is, they responded with the equivalent of being capable of flying somewhat better than a pilot who landed successfully, somewhat better than a pilot who crashed, and somewhat better than any other pilot irrespective of outcome. In a nutshell, their assessment was dependent on risk perception, not on outcome.
For the authors, however, pilots “exposed to a positive outcome were more likely to conduct the same flight themselves, when compared to pilots who had been exposed to a negative outcome … [which] might increase the risks faced by the subsequent flight compared to those faced by the earlier flight, but with no guarantee that the outcome will be good”.
Where did the authors go wrong? One of the authors is a flight instructor, so it could not be due to not knowing minima Met for New Zealand. The other author is an academic. The research was based on a PhD thesis, again in academia.
And here is where the problem lies: in the Ivory Tower and on its preference for playing games of numbers rather than getting its head out of the window and scanning reality. And although the research methods and results are largely correct, the interpretation of those results and the conclusions achieved by the authors are factually wrong.
The consequences are not just restricted to an academic paper that few may read but have already spilled onto the wider community, demonising pilots at international levels on unwarranted claims: “Sorry to say, but your pilot’s decisions are likely just as irrational as yours and mine”—wrote the British Psychological Society; “Pilots ‘very likely’ to misjudge flying conditions due to irrational decisions”—reported The Independent; “Why not trust pilots blindly”—reported Slate; and “Travelling by plane just got more threatening”—reported RT.
The current state of the situation is that the authors are resisting a correction of the original article. I have called for its retraction but it may all be in vain.
I believe Massey University should issue a public statement distancing itself from the conclusions reached. More importantly, I see that here is where the aviation community should also join to reject the above false conclusions and the unwarranted portrayal of pilots as irrational and risk takers.
I have prepared a Manifesto to voice the main facts that the Ivory Tower has no right to alter for its own purposes.
Feel free to copy and tweet to Massey University (@MasseyUni) or the editor (@WBPsychology):
Manifesto of facts from pilots to academicians regarding misinterpretations in doi: 10.1002/acp.3225
When presented with scenarios whereby the cloud base is at 2500ft above ground level and horizontal visibility is 16km, VFR-flying pilots may report the cloud base to be at 2468ft and horizontal visibility to be 15.50km. It cannot be reasonably said or implied that pilots show an inaccurate perception of weather conditions.
Were those pilots to assess such conditions as safe enough to continue the flights, it cannot be said that their assessments are irrational.
When presented with scenarios whereby the cloud base is at 2500ft above ground level and horizontal visibility is 16km, VFR-flying pilots may report, for whatever reason, the cloud base to be at 1756ft and horizontal visibility to be 11.48km. It thus may be said that such pilots show an inaccurate perception of weather conditions.
Were those pilots to assess such conditions as safe enough to continue the flights, however, it cannot be said that their assessments are irrational as the perceived conditions are still above the minima MET for legal VFR flight.
Furthermore, were those pilots to assess the perceived conditions as relatively safe to continue flight, it cannot be said that they also have a higher chance of diverting unnecessarily if they have not provided an answer supportive of such propensity.
When presented with scenarios of third-party pilots flying in ‘marginal’ weather conditions who, nonetheless, successfully completed the flights, VFR-flying pilots may report neither favourable nor unfavourable opinions regarding those third-party pilots’ decision making or risk taking propensity. When VFR-flying pilots report neither favourable nor unfavourable opinions, it cannot be said or implied that they show a favourable assessment.
When asked about their own decision to fly in conditions similar to those flown by third-party pilots, VFR-flying pilots may report (some degree of) positive confidence of being capable of carrying out similar flights in a safer manner than the third-party pilots did. If the reported confidence is relatively constant irrespective of risk perception (e.g., a VFR pilot reports to be capable of flying somewhat better than a pilot who landed successfully, somewhat better than a pilot who crashed, and somewhat better than any other pilot irrespective of outcome), then it cannot be said or implied that third-party outcomes have a direct effect on his or her self-assessment.
- Report by Jose Perezgonzalez and photography by John King.