Jason,@jasonc_nc
“When a measure becomes a target, it ceases to be a good measure.”
Let’s talk about how Goodhart’s Law shows up in American fire department’s single-minded focus on response time and how it becoming the only target degrades the thing it was supposed to be a proxy for.
In most things public safety focuses on two things - cause, and outcome. If you can reduce the cause of injuries or deaths, while improving outcomes for those that still occur, public safety is improved. Fire departments measure neither of these in any real way. They measure call volume and how fast they respond. The result has been a perfect example of Goodhart’s Law in practice, where speed itself is treated as proof of success.
For some background we’ve seen no improvement (slightly negative, in fact) in deaths per fire since 1980, despite speed of response being the core thing optimized for across those 4+ decades. Meanwhile downstream harms from focusing on this one metric, especially when combined with another flawed proxy - the size of apparatus being used as proof of capability - have been overwhelmingly bad. The proxies ate the original goal of fewer overall injuries and deaths.
When a metric becomes the target, an institution will naturally protect the metric’s performance above all else. Think of standardized testing scores becoming a substitute for learning, or an emphasis on a quarterly earnings crowding out focus on long term business health. The goal (learning, a stable company) gets replaced with the metric (test scores, gamed earnings before a deadline). And this is exactly what happened for fire departments.
Rather than asking what helped or harmed overall public safety, fire departments optimized for wider turning radius, faster streets, and anything else that served the metric. The problem is these same wide, fast streets encourage higher vehicle speeds, dramatically increasing ~preventable~ injuries and deaths that are far higher in number than fires.
In response public health researchers and community groups push for traffic calming efforts, street diets, and so on. Fire departments (rationally, considering the metric as a proxy for safety) then use their political leverage and regulatory authority to fight against such efforts.
The relationship between street design and traffic deaths is some of the most robust public safety research we have. But since fire deaths and traffic deaths are measured and attributed separately, a fire department’s “balance sheet” of public safety only counts the first as a concern. And so a speed bump or a slightly slower street is treated as an impossible safety issue, while a design actively injuring and killing people is given no weight.
The end point is this: We are overdue on revisiting the metrics fire departments are designed around, especially in light of its evolution from “fire” to being predominantly EMS with a quite small % of fires. This begins with a balanced, honest accounting of how the design of streets impacts public safety, and a rational discussion of the tradeoffs between fire department preferences and where remaining life safety improvements lie.
