Yesterday I took a day off after my second vaccine shot and promptly made the mistake of getting myself into a Twitter fight about the effectiveness of mask mandates (sadly the other party deleted their tweets leaving only my side – despite making good points that had me thinking). I compounded this mistake by citing a paper that I felt (and still feel) provides data in support of such mandates but had attracted the ire of a bunch of epidemiologists who had called for its retraction.

The good news is that all of this got me to look more seriously at the data, as well as at other research. On the basis of this I am confirming my assessment that mask mandates were justified and that more states should have implemented them early on.

But first a step back. What go this whole thing going? Well I noticed a bunch of people on Twitter claiming that mask mandates never worked (and I suppose by implication should have never been issued). What do they base this claim on? Well in recent months it appears that states which have lifted their mask mandates have fared similar to ones that still have them as can be seen in the following chart:

Now I don’t find this convincing because we are deep into this pandemic and at this point people’s mask wearing behavior is much more likely to be influenced by what they know about them than by a mandate. This is not dissimilar from seat belts. If you removed the seatbelt mandate in some states tomorrow but not in others, I highly doubt you would suddenly get a massive shift in seat belt wearing behavior.

Why? Because people aren’t stupid. That is people recognize when a technology provides a real safety advantage. True for seat belts and true for masks for an airborne virus. This is different from saying that there aren’t some stupid people, because, well, there are.

So if you want to look for evidence of the effectiveness of mask mandates you have to look early in the pandemic, not late. That’s what the paper in question did, which was published in June of 2020. It accumulates a bunch of evidence from different sources, including the following chart:

Here is why I think this chart provides evidence for the effectiveness of a mask mandate. New York City, which was hit by one of the hardest early outbreaks had rapidly spiking cases. Social distancing measures helped bend that curve down, but there is another distinct change in trajectory following the mask mandate. So simply based on a comparison with itself (not with the rest of the US, which I will get to in a second) this is evidence supporting the effectiveness of a mask mandate.

Now let me comment briefly on the retraction letter that was signed by a whole bunch of professors. At this point in time we already knew that the virus was airborne, as they thankfully note in the letter. So when it comes to public health measures our prior should be that masks work. When you see a chart that shows a mask mandate bending down the curve that is further evidence supporting this position. In a pandemic with exponential growth potential every day matters. So asking for better statistical evidence, as the letter does, is an egregious reversal of the burden of proof. There would have to be strong evidence that mask mandates do NOT work.

Now admittedly the paper makes some silly mistakes such as saying that only NY had a mask mandate in April. It is true that a few other states did also, but if anything that makes the comparison to the rest of the US stronger, not weaker. I should also note that NY was the only state with a comprehensive mask mandate in April. In any case though I wondered what the data would look like if one compared the four states with mask mandates as of April 15, 2020 (New York, New Jersey, Pennsylvania and Maryland) with the rest of the country. The data is easily available and it took me about 30 minutes of Excel wrangling (well Apple Pages, but same difference) to produce the following graph (blue are new cases in states with early mask mandates, green are the rest of the US both scaled to per 1000 person):

So similarly strong evidence that states with mask mandates (this now covers 47 million people or 14% of the US population) achieved a much bigger turnaround in new infections per thousand than did those without. Note also that this turnaround was much harder to accomplish given the more explosive initial growth in those states.

But might this not be a case of Simpson’s paradox? States peaking later and later could aggregate up to form such a pattern also. But looking at the state level data rules that out. The first wave crested pretty much all across the US by the end of April (even in South Dakota, which is about as far from either coast as you can get).

What about the fact that maybe some states had not yet ramped up their testing and were undercounting cases? Absolutely. It is possible that there would have been a hump for the states that had no mask mandates and hence a downward slope after that even without a mask mandate. But even with testing ramped further as time goes on (into June) new cases for early mask mandate states go significantly below that for the rest of the country.

Of course, it turns out not to be hard to find other studies coming to similar conclusions about the effectiveness of mask mandates. Here is one at the state level (also from June 2020), a later state level study using hospitalization data, and another one at the county level (in Kansas).

So bottomline: there is strong evidence from the early phases of the pandemic that mask mandates work and that they would have saved lots of lives had they been broadly adopted.

And notes to self: first, don’t engage in a Twitter fight on a day off and second, pick a less controversial paper to make your point.