Andrew McKay 07/15/20
Several days ago, a Twitter user put out the allegation that the FDOH test numbers are wrong because many of the labs in the daily data sheet are reporting 100% positive test results. I addressed these concerns on-air and hoped that people would handle the story responsibly. MyFox35 in Orlando was the first to publish anything as far as I’m aware, and now we are seeing this story circulate massively. Unfortunately, the accounts I’ve read are not handling the facts with as much good judgment as I would have hoped for. And as a result, many people are pointing to this as the smoking gun evidence that all this COVID-19 stuff is just fake news or exaggerated. So, here are some observations about this story for people who want to understand it better.
1. There are roughly 1,175 labs listed on the daily statewide report. By my count 465 of them are listed with a 100% positive result status. Obviously at first glance, this is very troubling. But first glances aren’t the whole story. See the report here (lab numbers start around page 26): http://ww11.doh.state.fl.us/comm/_partners/covid19_report_archive/state_reports_latest.pdf
2. One part of the story not being reported is that there are also a significant number of labs reporting 0% positive. By my count, there are 151 labs reporting 0% positive. I have yet to see a news account of this story that includes that fact. Of course 0% is also surprising, but it doesn’t fit the “all the numbers are inflated” narrative. I’m not saying MyFox35 suppressed the fact. My guess is they just didn’t notice, since 0% shows up as a blank line on the report. (See picture, green lines)
3. If you look at a lot of the numbers, it’s immediately evident that the percent positive in any given lab could be almost any number from 0 to 100. Given the probabilities involved in testing, this isn’t really all that surprising. (See photo, a sample page from the whole report)
4. The issue of probabilities becomes even more pertinent when you see that the vast majority of labs reporting 100% (and 0%) are only reporting very small total numbers. Of the 465 100% labs, 456 of them have reported 100 or fewer results. The largest (Lab24 Inc) has 464 test results, which is still very small. By contrast the seven largest labs like Quest and Labcorp have reported over 100,000. As you might expect, the closer you get to the top of the chart, the more homogeneous (similar) the percentages become. But even so, they vary among the top 7 between 3% and 13% positive. If labs resulting 100,000 plus tests can vary by 10%, it’s easy to imagine that labs resulting 100 or less can certainly have unusual numbers. Making this even more poignant is that 380 of the 456 labs actually report only 10 or fewer results…roughly 250 of them are 3 results or fewer.
5. There is also a screening question. Perhaps some of these labs are only handling high probability cases to begin with (or low probability ones). The source of the samples can always bias numbers one way or another.
6. Nevertheless, as MyFox35 reports, the test positive rates still don’t seem right, as Orlando Health and Orlando Veterans Medical Center say the positivity rates don’t match their data. Moreover, if it were purely a probability issue, you would expect the 100% and the 0% to be the same or close and they aren’t. So is there any other part of this?
7. As MyFox35 reports, many labs aren’t doing what FDOH has told them to do: report all results including negatives. For whatever reason, this appears to be a compliance issue. From the article: “FOX 35 News went on to speak with the Florida Department of Health on Tuesday. They confirmed that although private and public laboratories are required to report positive and negative results to the state immediately, some have not. Specifically, they said that some smaller, private labs were not reporting negative test result data to the state. ‘The Department immediately began working with those labs to ensure that all results were being reported in order to provide comprehensive and transparent data,’ a spokesperson for the Florida Department of Health said. ‘As the state continues to receive results from various labs, the Department will continue educating these labs on proper protocol for reporting COVID-19 test results.’”
8. So if a lab is at 100% and they are small to begin with, it almost certainly can be explained as A) statistical probability, especially for the very small labs B) screening criteria or C) a failure to report negative tests.
IMPLICATIONS
9. There is no evidence to suggest that the total number of positive results is wrong. That’s a key non-implication. The skeptical take on this story is that somehow case numbers are exaggerated, but there is no evidence for this.
10. The more likely significance is that negative test numbers haven’t been reported to FDOH, which means there is a chance that our positivity rates are slightly too high. I say slightly too high because, again, this appears to affect a very small number of our total test volume in Florida. Keep in mind that even if negative results weren’t reported, the only way they change the positivity rate is if they are results from people who haven’t otherwise had a result recorded. A person can test 10 times with 8 negatives and two positives. FDOH records this as a single result. In other words for the negatives to even count in the data against positivity, they would have to be negatives on people who had no other test results already recorded.
11. As a consequence, the most likely significance of this data issue is very, very small and not the huge scandal the original Tweet claimed nor the smoking gun that all of our numbers are inflated in Florida.
12. More information could be forthcoming, and if it is relevant, I am obviously eager to revise any of my analysis. In this business, I only have the information to date to work on.
