Faulty Covid-19 Tests, But Results Delivered in 15 Minutes

Mish

A study indicates Abbott ID Now failed to detect 48% of the positive cases found by another machine.

Fast results are one thing but accuracy is another. 

The Abbott Lab test takes only 15 minutes. But what about accuracy?

Please consider Abbott’s Coronavirus Test Falls Short of Rival Device.

The study involved 101 emergency-room patients who were referred for testing at the hospital during a three-day period. For each patient, two swabs were taken. One swab was tested using a device the NYU lab had already validated, Cepheid’s GeneXpert, which takes about 45 minutes to complete a test. The other swab was taken using Abbott’s recommended method and tested in the ID Now. 

The GeneXpert detected 31 positive cases among those samples. By contrast, the Abbott device detected only 16 out of the 31. “The remaining 15 samples were falsely negative,” the authors reported in the study. 

In another portion of the study, the authors examined 15 positive specimens that had been stored in a liquid on both devices, finding that the ID Now missed one-third of those cases. Last month, Abbott changed the ID Now’s instructions to warn against using the preservative, which it said had contributed to earlier concerns about test accuracy.

In April, The Wall Street Journal reported that several lab directors said the Abbott ID Now was producing false negative results using samples stored in the preservative as much as 25% of the time.

25% to 50% False Negatives

But consider the advantages.

  1. ID Now is a very catchy name. GeneXpert? Come on.
  2. Faster Results - 15 minutes vs 45 minutes
  3. Fewer Reported Cases 

That sounds like triple bragging rights.

Mish

Comments (76)
No. 1-24
Sechel
Sechel

They've been talking about this for days. Seems to come up with false negatives, but not false positives. It's apparently not sensitive enough. The tests are dependent on getting an adequate sample from the subject.

Low accuracy from the tests could diminish their usefulness as a catch-all screening tool for public-health workers and medical personnel treating patients. Maybe it has some value in a white house setting where people now get retested daily...bu not a winner

Trump touted the Abbott machine with great fanfare in the White House Rose Garden after the Food and Drug Administration approved it for emergency use, saying it could be a godsend for hospitals and workplaces because it “delivers lightning-fast results in as little as five minutes.”

Invigilator
Invigilator

This seems to be increasingly common world wide. Whether ventilators that do more harm than good (apparently sleep apnoea machines are often more useful) or dodgy PPE. Now we have this machine giving false results yet still out in the marketplace. Could it be relaunched as a machine for testing 'snake oil'?

Anda
Anda

Sidenote on Spanish figures, a very rough overview:

270 000 PCR cases

27 000 fatalities (all the above cases not closed) giving CFR of somewhere 10 to 15 %

2 000 000 (something over that amount) estimated to have been infected after immunological study, which is 5% of population.

That gives an IFR of 1 to 2 %

These are rough figures subject to all the usual errors, and without segregating age or any other parameter which might provide a better understanding. They just narrow down the frame of reference somewhat to what is occuring.

Sechel
Sechel

My friend does genetic testing. He's on the stats side. Apparently these are common problems. It's par for the course when products are rushed to market. Doesn't make Abbot a bunch of idiots. Speaks to the downside of the Administration plan to get testing out at warp speed.

Russell J
Russell J

Choices are few but I still wouldn't buy it if I were in charge.

Wouldn't buy a car or toaster oven that didn't work 25-50% of the time either.

Embarrassing and pathetic.

Montana33
Montana33

This is very depressing. China is testing 11 million people in Wuhan this week and we can’t get tests that work and we still lack testing supplies. A killer virus is paralyzing our society with fear and our economy will never recover while the virus is still spreading. We have already spent Trillions on this virus and we still lack testing. My friend tried to get a test yesterday and was told she can’t unless she has severe symptoms or is a front line worker.

Herkie
Herkie

I was charged $49 for a finger stick antibody test that came back negative on May 1st but really question it's accuracy. I was sick for 15 days in March with what on paper pretty much had to be Covid, and as of last night am sick again, same list of symptoms. This country cannot get it's collective shit together and we are going to fail to manage this to the point where we will not have to choose between an economic collapse/depression and a major pandemic with needless death and suffering because we are going to get BOTH!

lasttwo
lasttwo

cheaper to flip a coin almost as accurate

nielsll
nielsll

Article fails to mention that the sample acquisition and handling was not in accordance with the test protocol procedure. This should be retracted until the test is performed in accordance with the test procedure instructions.

Realist
Realist

Medical testing is not perfect, whether by automated machinery or a lab technologist. There are, however, acceptable levels of inaccuracy, and 48% is no where close.

How odd that Trump always seems to promote the craziest things:

Anybody who wants a test, gets a test.

Hydroxychloroquine. What have you go to lose.

It will go away in April, like magic.

The US and Germany have the best death rates per 100k. (US 26, Germany 9, Singapore, Japan, South Korea, Australia, New Zealand and many others less than 1) He just pulls this stuff out of his ass, doesn’t he?

The Abbott kits are fast and accurate. (I guess he should have used the Costello kits instead.)

A drink of Clorox each day will keep the virus away.

A lot of countries keep phoning me to find out how we are doing so well!

Mr. Purple
Mr. Purple

Faith-based science. Perfectly normal in these Trump-Pence times.

Carl_R
Carl_R

Every swab test, from the beginning to the present, has has a high false negative rate. I remember early on cases from China where they tested people 7 times, very certain that they had Covid, before they finally tested positive. That said the Abbott test may be worse, if they have 48% false negatives. 25% is more normal.

The blood tests for antibodies are much more accurate (well, some of them). The antibody test from Quidel claims a 0.7% false positive ratio. Quidel also has a quick test for the swabs, too. I suspect that it has a 25% false negative ratio, just like everyone else.

The Abbott, test, by the way, is the one that Trump uses to test people before he will meet with them.

purple squish
purple squish

Sounds pretty bleak for the false negative rate in the current version of the Abbott test, but I wonder about the false positive rate in the comparator test and the overall rate of positives. If it's something like a 2% false positive rate and a true 1% COVID prevalence in the sample, then most of the reference test's positives are false positives. In such a case it would be a mark of a good test if it "missed" those.

Zardoz
Zardoz

Snake oil from the snake oil salesman in chief.

dr smock
dr smock

By themselves alone, many medical tests for specific diseases have poor accuracy rates, including the test for ordinary flu virus. Testing many times for these specific diseases together with other testing and imaging, plus noting the progression of clinical signs helps the doctor increase the accuracy of diagnosis.

thimk
thimk

MY head is ready to explode from covid19 data/narratives/correlations . Until a solid direction and methodology is implemented I'm laying low . What really torques me is why there aren't any masks at the retail brick and mortar stores . or am i missing something ? DO i have always have to give a sacrifice to the Amazon gods ??

QTPie
QTPie

The picture at the top of the article is from March 30th but you should know that Trump had the Abbot device up there next to him again just a couple of days ago in his press conference.

Also next to him during the press conference he had a giant sign which read “America leads the world in testing”. This of course is totally misleading because while yes, we may run the most tests in absolute numbers but that statistic is completely and utterly meaningless. What actually counts is per-capita testing in which the US is far behind on.

That was the same press conference BTW where he also told an American journalist of Chinese heritage to send her questions to China rather than him.

Oh my Lord, what a moron he is. Really, it would be good for him to just shut up and not say anything about anything.

Felix_Mish
Felix_Mish

Words get thrown around so loosely that one must wonder about the pitchers.

Using "accurate" colloquially: If Mish's take on the story is accurate, then this test is very accurate. Because we can suppose (for omission reasons) this test doesn't mislabel someone without C19 as having C19.

"Accuracy" of test such as this has a specific meaning: (tp + tn) / (tp + tn + fp + fn) where "f" is "false" and "t" is "true".

The WSJ article is behind a paywall, so I can't see the TP, TN, and FP numbers.

As @Sechel points out, a FN rate of 48% isn't necessarily a bad thing. It's not ideal, to be sure. But if TN is 0%, then you may have a very, very useful test - if it's cheap and/or quick and/or has some otherwise desirable quality, for instance.

anoop
anoop

Awesome choice of title for the post!

Herkie
Herkie

I was curious about the efficacy of Remdesivir now that they are using far more widely for several weeks. I found this story and it gives hope, if the bureaucratic mess in DC can just get out of the way.

To sum it up, Remdesivir is complicated to make, expensive, and toxic beyond relatively low doses, yet, most of it is converted into a compound called GS-441524 in the bloodstream anyway. This article points out that GS-441524 is the precursor to Remdesivir and is already long been shown to be up to 96% effective in animals with other coronaviruses. It is easier/cheaper to make, and can be safely given in doses up to 1,000 times as large as Remdesivir without any side effects or toxicity. And evidence says your body is changing the Remdesivir into GS-441524 anyway. That appears to be the active healing ingredient. While the other factors in Remdesivir are the toxic ones that limit dose.

And so how many more will die before they decide to use it on the population? Well of course the answer is Remdesivir is far more profitable.

NOTE: "The attractive profile of GS-441524 from both manufacturing and clinical perspectives raises this question: Why hasn't Gilead opted to advance this compound to the clinic? We would be remiss for not mentioning patents, and thus profits. The first patent on GS-441524 was issued in 2009, while the first patent for remdesivir was issued in 2017."

Drug patents are good for 20 years. Gilead would see the loss of it's monopoly on GS-441524 in 2029, where the patent on Remdesivir runs till 2037. Folks, they have given cats GS-441524 which was a death sentence for feline covid that attacked hearts and now more than 96% of infected cats recover. Maybe the answer to surviving this pandemic rests with making friends with a Mexican veterinarian.

MorrisWR
MorrisWR

Mish, I have discussed generally our SARS-CoV-2 testing in my department (molecular) through email a few times but this story is very close to my work. I run a Panther Fusion rtPCR assay for Cov2 and it is an excellent assay yet the accuracy comes at a cost in time (about 2.5 hours). However PCR is a standard for viral amplification/detection. Another department used the GeneXpert and it was very good also for a 45 minute test. You need time (cycles) to amplify RNA to a detectable level. That is critical if there is a low dose infection or the source of the collection is sub-optimal. A viral infection can easily be missed just on the collection site and how it is collected (following guidelines by manufacturer of swabs/VTM/UTM). I would not trust any 15 minute test and would expect a lot of missed positive just due to low a cycle times.

As for a catchy name, in a laboratory nobody cares. Cepheid sells analyzers based on performance and probably price. I would still prefer mine done on at least a 2 hour rtPCR (or standard PCR).

Note: I do not disclose my company as the opinions are mine and not those of my employer (so there is no point).

CA2020
CA2020

I had one of these tests. The techs that did the testing had training the day before. I tested negative, imagine that. I am pretty sure that I was negative as I have been vigilant in attempting to not get this virus.


Global Economics

FEATURED
COMMUNITY