Just tellin’ it like it is COVID-19 Series #2: Risk-benefit, intended use and important considerations
Note: This is a repost of my original LinkedIn article published on November 4, 2020.
Despite my previous rant that not enough is being done to address asymptomatic transmission (https://www.linkedin.com/pulse/why-arent-we-tackling-asymptomatic-spread-covid-19-more-james-w-kim), it is also true that pretty much every IVD developer and their cousin is trying to contribute towards addressing (and also capitalize on) the COVID-19 situation, partly just to survive. We have already seen the enormous impact South Korean and Chinese IVD developers have had in supplying the world with PCR and serology tests for laboratory use in the early days of the pandemic. Without these myriad developers and the incredible speed and scale at which these tests were made available, we would undoubtedly be in a much, much worse situation than what we are facing now.
The problem is that everyone was geared toward laboratory testing: the developers, the regulatory authorities issuing EUA guidelines, the healthcare infrastructure etc. And who can blame them? Nobody knew much about SARS-CoV-2 back in March and we could only fight with the knowledge and tools we had at that time. Time was of the essence and everyone stepped up to meet the occasion.
But despite this valiant effort, today we are left without a diagnostic method to address the new threat of asymptomatic spread which can only be fulfilled by a screening test. And by the looks of how diagnostic laboratories are drowning in backlogs right now, a test that can be performed at home by ordinary people would be the only logical option.
The way modern healthcare system works is that an individual must actively show up to a professional healthcare setting presenting discomfort – essentially a set of symptoms. The physician combines these symptoms with circumstantial evidence (e.g. familial history, police reports etc.) and orders diagnostic tests based on evidence-based probability and reasonable suspicion to gain as complete as picture as practicable before making a final diagnosis and prescribing a course of treatment. Because demand for healthcare is essentially limitless, this is the only practical framework to administer a certain quality level of healthcare with limited resources. This is the equilibrium of cost versus benefit that we have arrived at.
Striking the appropriate cost-benefit balance is also the core tenet of regulatory approvals for medical products, although in this case the more common phrase is “risk-benefit”. Chemotherapy-level side-effects would be unacceptable for treating a sore throat but are considered perfectly acceptable when one’s life is on the line. This is where the concept of “Intended Use” becomes important, since the entire risk-benefit equation is established with this statement which the US FDA defines as:
“…objective intent of the persons legally responsible for the labeling of devices. The intent is determined by such persons' expressions or may be shown by the circumstances surrounding the distribution of the article.” (21CFR801.4)
It is important to note that circumstances of actual use are as important as the explicit description in dictating the de facto intended use. So just because a manufacturer claims their “syringe” is to be used as a shovel substitute doesn’t make the syringe a really crappy shovel. It just makes the syringe a syringe still subject to all safety regulations expected of that class of medical device, and the makes the manufacturer an idiot.
Let’s take a look at molecular and serology tests, essentially the only two options for SARS-CoV-2 diagnostic testing available right now.
PCR (Polymerase Chain Reaction) fundamentally requires some type of powered instrumentation for thermal cycling and may also require additional equipment to read the results. Ditto for isothermal amplification. Doesn’t matter how small, cheap, quick or accurate – molecular tests simply need a powered device. That’s the trade-off for the incredible accuracy and precision of the technique. So here is the dilemma: Molecular tests have implicit established performance expectations, so a product with compromised performance is just a crappy product. Especially because molecular diagnostics always requires a powered device of some sort. Repeated testing (see interval testing below) will also require periodic calibration to maintain performance, which in turn requires specific training and introduces cost. Or one could choose to provide a brand-new device every few runs, but climate change and cost. Any way you slice or dice it, molecular diagnostics – the diva of infectious disease detection techniques - trades in unmatched sensitivity and specificity with the requirement for a permanent supporting cast.
Serology testing may or may not require instrumentation, but it almost always has to be run with a timed, multi-step process because that’s how the biochemistry works. But the saving grace is that there are innovators developing one- or two-step tests that just may be simple enough to use without any training (“dummy-proofing” in industry speak). And since antibody-based detection traditionally had a wide spectrum of performance, the expectations for exemplary sensitivity and specificity are not as great as that of molecular methods.
So as far as home testing is concerned, molecular methods fail the intended use test since it cannot realistically be expected to perform at levels worthy of the method (the “circumstance”) at a home setting without significant added cost, thereby defeating the entire purpose. (Somebody please prove me wrong) That leaves serology tests, which may or may not be simple enough to use at home. But the issue remains on how any home testing product that passes the intended use test can plug into the aforementioned modern healthcare system.
This all ties back into the risk-benefit balance that is concerning regulators and the reason why there still is no approved COVID-19 test for home use. Regardless of the actual performance characteristics of the test (and never mind the ease of use and cost aspects), psychological reaction to a given result by an untrained, ordinary individual is difficult to control, let alone manage. Overreaction to a false positive can potentially lead to some very bad results (people have taken their own lives for far, far less serious reasons) and false negatives can endanger the public. Trained staff in diagnostic laboratories of the modern healthcare system have the knowledge and discipline to evaluate results and re-test as circumstances dictate to confirm some questionable calls. Physicians make a diagnosis considering all available evidence, which a diagnostic test results constitutes only a part. But it is a much more clear-cut and simple thought process for normal people. Negative means negative, positive means positive. Duh. This is another “circumstance” that the developer has to work to mitigate and incorporate into the intended use. Never think consumer behavior can be molded because of fine print in the instructions for use that almost nobody will ever properly read.
There is also the little detail that this is a declared PANDEMIC. Public health authorities need to keep tabs on everything that is going on to keep their heads above the water. But besides the general approach of uploading results to a siloed cloud database, there really hasn’t been a good solution to this issue presented thus far. Think about it. If the CDC has to access 50 different cloud servers with 50 different reporting formats by 50 different manufacturers, all on top of the existing workload of collecting and processing daily diagnostic laboratory data on a national level - not to mention managing health data privacy compliance issues – that looks more like shifting risk rather than mitigating it. There has got to be a better way for diagnostic developers to come together to collectively solve this issue, akin to how multinational COVID-19 vaccine developers have (sort of) come together to collaborate.
Having established that, we also need to consider that things are not exactly good in the real world. As I previously laid bare, shit has hit the fan and the risk has realized in spades. There is very little a home test can do to make things worse when 1.22 million people have already perished and over 5,000 people are dying EVERY DAY. The equilibrium of COVID-19 risk-benefit right now is such that the risk from errant behavior by a sliver of a small percentage of false results will be far, far outweighed by decreased transmission and resulting indirect benefit of socioeconomic stability. It would actually be prudent (and more useful) to prepare for the happy day when daily case levels substantially and sustainably drop after introducing home tests and the risk-benefit point inevitably slides towards heavier weighting of the consequences of false results. Possibly a sustained public education program from the beginning to continuously train and engrain into the populace on how to discern and/or react to false results may work. (Most) people are capable of eventually learning new information, after all. And I’m sure as hell not tired of the celebrity COVID-19 PSA’s yet. (Shout out to @VancityReynolds, @Sethrogen)
The final consideration is the fact that interval testing must be the default if any screening program is to be effective. We are all painfully aware by now that a negative test result today does not guarantee negativity in a few days. You immediately start climbing the ladder of positive probability the moment you receive a negative test result. To maintain the balance of socioeconomic stability (i.e. going on with “new” normal life) and public health safety, repeated testing based on statistically pre-determined intervals is the only way to maximize suppression of viral transmission. Here again we must seek equilibrium on the risk-benefit spectrum, since testing every hour would be ideal but impractical, while testing every week would be only a bit better than useless.
Given these facts and reasons, a COVID-19 screening test should ideally be one that is simple enough to be conducted at home by an individual with a secondary education, without any extraneous powered device, with an intended use profile that sufficiently addresses the “normal person” risk, is provisioned to effectively report results to public health authorities, and cheap enough to eventually conduct tens if not hundreds of tests per individual without breaking the bank.
Hey, I never said it would be easy.
A side note: The thing about developing a screening test for transient infectious diseases like COVID-19 is that it is essentially a self-restricting model. Logically, an effective screening test and public program would quickly weed out most cases before transmission can occur, eventually leaving very little demand for additional tests (and revenue) if the product works as intended. The demand for screening transient infections is incredibly dynamic and thus the development of such is difficult to justify as a business. One also must consider the risk in terms of time and effort it typically takes to develop such a test and obtain approval, especially since intended use as screening would (rightly) require a larger sample size for clinical trials, further driving up cost. Demand is simply too volatile to consider this a sound investment decision. Vaccines for transient infections are the same way, and precisely why NGOs and philanthropic initiatives were the only way that successful vaccination programs had endured the way they did. This very special and specific economic curse is probably why conspiracy theories regarding the pharmaceutical industry abound, and a topic I would like to explore in depth later on.
Comments