MIAMI — Nate Silver and his FiveThirtyEight.com site took on the idea of “fastest flights” recently and it got me thinking: What am I actually looking for when it comes to getting somewhere by air? Is the fastest total travel time most important? Or likelihood of delay? Or magnitude of delay once it sets in? Or something else entirely??
Silver chose gate-to-gate time as the metric, suggesting this:
Airline A says it will fly you from Seattle to Portland, Oregon, in 45 minutes, but actually takes 60 minutes. Airline B says it will fly the same route in 75 minutes, and actually takes 70 minutes. Which flight would you rather take?
And the background says that the data is accounting for that. I’m not so convinced it is the correct question. I agree with him that simply looking at the DoT delay numbers is a very, very bad way to judge airlines for many reasons, but I’m still not sure he’s asking the correct question. Mostly I consider this:
Does it really matter that a flight lands early if I cannot schedule my other activities based on that expectation?
Put another way, variance from reality rather than scheduled time is fun to consider, but I cannot plan activities or an onward connection around anything other than the scheduled arrival time. So in the end that’s what really matters to me as a customer. And, believe it or not, the worst of the big 10 carriers reporting data was only ~11 minutes late on average. The best was just under a minute late on average. No airline has managed to build a schedule which, on average, it can beat.
Sure, but who is delayed most often? The DoT says a flight is delayed only if it arrives 15 minutes or more late at the destination gate. But the data is available to look at all flights. Hawaiian Airlines has a surprisingly high number of delayed flights when you consider the data this way, but scores very well in the DoT stats; I’ve looked at the Hawaiian DoT stats previously and shown where much of that bias comes from. At the other end of the spectrum sit Frontier and Southwest who each have nearly half of all flights arriving delayed. Considering only DoT numbers those two are still not great.
And, once the flight is delayed, what is the impact? Just how badly is it going to mess up my plans?? The numbers in the first chart suggest that I’m quite likely to arrive close enough to on time anyways, but when things are “truly” delayed by DoT standards what is the real impact there? Hawaiian and Delta fare the worst by that metric while Southwest and Frontier – the two most likely to see you delayed – actually fare the best.
Does this mean that you should purposefully choose Southwest or Frontier when given the option because even if late it won’t be as bad as the others? Probably not, mostly because with stats covering this many samples trying to apply it to any single experience is unlikely to net a pleasing result. But it is a good reminder that the same data can show widely varying “results” and that choosing which question to ask and what to measure is usually more important than just publishing pretty charts.