Here’s a dynamic I’ve seen at least a dozen times:Alice: Man that article has a very inaccurate/misleading/horrifying headline.Bob: Did you know, *actually* article writers don’t write their own headlines?…But what I care about is the misleading headline, not your org chartAnother example I’ve encountered recently is (anonymizing) when a friend complained about a prosaic safety problem at a major AI company that went unfixed for multiple months. Someone else with background information “usefully” chimed in with a long explanation of organizational restrictions and why the team responsible for fixing the problem had limitations on resources like senior employees and compute, and actually not fixing the problem was the correct priority for them etc etc etc. But what I (and my friend) cared about was the prosaic safety problem not being fixed! And what this says about the company’s ability to proactively respond to and fix future problems. We’re complaining about your company overall. Your internal team management was never a serious concern for us to begin with!Kelsey Piper wrote about the (horrifying) recent case where Hantavirus carriers in the recent outbreak on a cruise ship were released and sent back to their home countries on (often) public airplanes. No systematic quarantine seemed in place, and only some of the exposed people were even instructed to self-quarantine. Now in light of new information we think it’s very unlikely that this’d end up being a pandemic (the virus isn’t contagious enough at human-to-human transmission). But sure seems like pure luck rather than careful risk-benefit analysis; we only learned about the low contagiousness from negative tests after the cruise ship passengers were sent home.Seems pretty incompetent for humanity to manage a potential future pandemic this way! Tweeters disagreed. They argued that everything’s fine because in fact the WHO as an advisory body can’t enforce legal quarantines on sovereign states. Huh? Why is that relevant here? If this hantavirus outbreak was in fact as contagious as COVID (while maintaining the ~30% fatality rate common for past infections), Nature’s not going to be like “oops my bad. I was planning to kill 2 billion of you but I misunderstood your world’s by-laws for which entities are responsible for enforcing quarantines. I’ll just let y’all have a pass on this otherwise fatal pandemic and take my business elsewhere until you sort it out.”In each of these examples, people’s reactions were something like explanation-as-exoneration: treating the descriptive fact of why something happened as if it answered the normative question of whether it should have.This is a cognitive mistake or logical fallacy that is so wrong I’m not even sure how to address it. Like in the examples above, people weren’t even originally blaming the group that someone else rushed to defend! But even granting that they were, how does shifting the blame address the underlying problem?The reasoning has to be something like “these people are (implicitly) blaming some group G for an alleged problem P. If I can demonstrate that these people are wrong to blame group G, then I’ve demonstrated that they’re Wrong. As Wrongness is a transitive property, therefore we can be sure that problem P isn’t real (???) and we no longer need to be worried about P”Maybe I’m strawmanning, but I really don’t understand the logic here!In some of those cases, like the second example about prosaic AI safety, clearly there’s a specific party feeling accused and defensive. So self-serving bias is at play. But most of the times I’ve encountered this fallacy in the wild it’s from seemingly disinterested third parties! So I really don’t know what makes people react in this way.Another good adjacent reason stems from “ought implies can.” If it turns out a problem somebody complains about is impossible to solve (or practically infeasible, or too expensive, etc), it (sometimes) helps to inform them of this so they can set realistic expectations and/or complain about more tractable problems. This is both true for physical impossibilities and answers of the form “if everybody would just.” But saying that one person or institution that you might think is at fault is not at fault isn’t exactly a proof that solving a problem is impossible! I don’t really see how it’s even evidence, most of the time.Overall I’m pretty confused by this pattern of thinking. On the other hand we might have discovered a novel fallacy, so that’s fun! ^Though in my experience if you email the writer about a bad headline usually they can get it resolved anyway. Discuss Read More
Bad Problems Don’t Stop Being Bad Because Somebody’s Wrong About Fault Analysis
Here’s a dynamic I’ve seen at least a dozen times:Alice: Man that article has a very inaccurate/misleading/horrifying headline.Bob: Did you know, *actually* article writers don’t write their own headlines?…But what I care about is the misleading headline, not your org chartAnother example I’ve encountered recently is (anonymizing) when a friend complained about a prosaic safety problem at a major AI company that went unfixed for multiple months. Someone else with background information “usefully” chimed in with a long explanation of organizational restrictions and why the team responsible for fixing the problem had limitations on resources like senior employees and compute, and actually not fixing the problem was the correct priority for them etc etc etc. But what I (and my friend) cared about was the prosaic safety problem not being fixed! And what this says about the company’s ability to proactively respond to and fix future problems. We’re complaining about your company overall. Your internal team management was never a serious concern for us to begin with!Kelsey Piper wrote about the (horrifying) recent case where Hantavirus carriers in the recent outbreak on a cruise ship were released and sent back to their home countries on (often) public airplanes. No systematic quarantine seemed in place, and only some of the exposed people were even instructed to self-quarantine. Now in light of new information we think it’s very unlikely that this’d end up being a pandemic (the virus isn’t contagious enough at human-to-human transmission). But sure seems like pure luck rather than careful risk-benefit analysis; we only learned about the low contagiousness from negative tests after the cruise ship passengers were sent home.Seems pretty incompetent for humanity to manage a potential future pandemic this way! Tweeters disagreed. They argued that everything’s fine because in fact the WHO as an advisory body can’t enforce legal quarantines on sovereign states. Huh? Why is that relevant here? If this hantavirus outbreak was in fact as contagious as COVID (while maintaining the ~30% fatality rate common for past infections), Nature’s not going to be like “oops my bad. I was planning to kill 2 billion of you but I misunderstood your world’s by-laws for which entities are responsible for enforcing quarantines. I’ll just let y’all have a pass on this otherwise fatal pandemic and take my business elsewhere until you sort it out.”In each of these examples, people’s reactions were something like explanation-as-exoneration: treating the descriptive fact of why something happened as if it answered the normative question of whether it should have.This is a cognitive mistake or logical fallacy that is so wrong I’m not even sure how to address it. Like in the examples above, people weren’t even originally blaming the group that someone else rushed to defend! But even granting that they were, how does shifting the blame address the underlying problem?The reasoning has to be something like “these people are (implicitly) blaming some group G for an alleged problem P. If I can demonstrate that these people are wrong to blame group G, then I’ve demonstrated that they’re Wrong. As Wrongness is a transitive property, therefore we can be sure that problem P isn’t real (???) and we no longer need to be worried about P”Maybe I’m strawmanning, but I really don’t understand the logic here!In some of those cases, like the second example about prosaic AI safety, clearly there’s a specific party feeling accused and defensive. So self-serving bias is at play. But most of the times I’ve encountered this fallacy in the wild it’s from seemingly disinterested third parties! So I really don’t know what makes people react in this way.Another good adjacent reason stems from “ought implies can.” If it turns out a problem somebody complains about is impossible to solve (or practically infeasible, or too expensive, etc), it (sometimes) helps to inform them of this so they can set realistic expectations and/or complain about more tractable problems. This is both true for physical impossibilities and answers of the form “if everybody would just.” But saying that one person or institution that you might think is at fault is not at fault isn’t exactly a proof that solving a problem is impossible! I don’t really see how it’s even evidence, most of the time.Overall I’m pretty confused by this pattern of thinking. On the other hand we might have discovered a novel fallacy, so that’s fun! ^Though in my experience if you email the writer about a bad headline usually they can get it resolved anyway. Discuss Read More