A preliminary review of a lateral flow device (LFD) being used in mass Covid-19 screening in Liverpool, has led some scientists to express concerns that it is not fit for purpose. They have also taken issue with some of the government’s claims surrounding the data. The government claims the LFDs are ‘accurate’ and ‘sensitive enough’ to be used in the community, including for asymptomatic people.
A quick, easy-to-use test to detect Covid-19 is key to the government’s plan to ease lockdown restrictions. The ‘gold standard’ RT-PCR test currently used needs to be run in a lab and is not well-suited for mass testing.
The review by PHE Porton Down and the University of Oxford considered 40 devices but only nine met some of the performance criteria, and only diagnostics company Innova’s was advanced enough for assessment – eight are still being tested with no results released as yet. The team evaluated the LFD in the lab and community settings over three months. It estimates its sensitivity at 76.8%, and specificity at 99.68%. Sensitivity is the ability of a test to correctly identify patients with a disease, while specificity is a measure of a test’s accuracy at identifying those that do not have the disease tested for.
LFDs are routinely used to test for different protein targets because they are relatively cheap, easy to use and accurate – the home pregnancy test is an example. LFDs detect viral protein, rather than RNA as in the RT-PCR test, and can provide results in under half an hour.
A liquid sample is placed on a pad where the target proteins – in this case, Sars-CoV-2 proteins – are bound by conjugated antibodies. The protein–antibody mix moves along a membrane by capillary flow across both ‘test’ and ‘control’ strips. These strips are coated with antibodies that can detect the protein of interest. When a protein has been detected, a coloured line appears.
‘Entirely unsuitable’
Jon Deeks, a professor of biostatistics at the University of Birmingham, said that the test is ‘entirely unsuitable’ for community testing, such as that planned for university students in the run-up to Christmas. ‘As the test may miss up to half of cases, a negative test result indicates a reduced risk of Covid, but does not exclude Covid,’ Deeks added. ‘This evidence raises serious concerns that the benefits are likely to be few, with serious risk of harm if the public are misled by unjustified government claims of this test’s high performance.’
Deeks pointed out that the report’s headline figure of 76.8% sensitivity mixes data from different settings. One study of Innova’s device at a testing centre found that the device appeared to miss one out of every two Covid-19 cases (58%). Another study using nurses to test patients suggests it missed one out of every four cases (73%).
Sebastian Johnston, a professor of respiratory medicine and allergy at Imperial College London, agreed that the sensitivity of the test is ‘not good enough’, especially when used by the public. In one study, he noted, a sensitivity of 95% was reported only when the test was used by a professional and when viral loads were above 100,000 RNA copies/ml. ‘That’s a lot of virus. Between 1000–100,000 RNA copies/ml, the test picked up 78% of cases when used by a trained healthcare professional. Below 1000, it only picked up 30% even when used by a professional.’
Other problems
Another concern is that while the rate of false positives is very low (overall 0.32%, down to 0.06% when testing happens in a lab), the numbers of false positives can still outnumber the number of cases detected when mass screening areas with low infection rates. ‘Sensitivity and specificity are both percentages of people for whom it’s known whether they are infected or not,’ explained Kevin McConway, emeritus professor of applied statistics at The Open University. ‘But in real life testing, you don’t know [this]. Picking up a lot of false positives may not matter much, for example if the new test is being used generally to find areas with high rates. But it could matter if results are going to be used to make decisions like asking people to self-isolate. The issue could be improved perhaps by repeatedly testing people who test positive.’
Deeks and Johnston are also concerned by government claims that there was little difference in how well the test detected symptomatic and asymptomatic individuals. Only 43 tests were performed on asymptomatic people, making the result provisional. The government also asserts that the test can identify infectious individuals, even though the review didn’t present any evidence on infectivity.
However, Jonathan Ball, professor of molecular virology at the University of Nottingham, argued that while the LFD lacks the PCR test’s sensitivity, it is a ‘pragmatic’ test for community surveillance where quickly identifying infected people is important. ‘Even though it won’t detect as many infected individuals as the PCR test, it will identify those with the highest viral loads. It won’t replace PCR, but it’s a useful additional tool.’
John Bell, Regius professor of medicine at the University of Oxford, agreed. ‘The data demonstrates that these tests can play a major role. They identify those who are likely to spread the disease and when used systematically in mass testing could reduce transmissions by 90%. They will be detecting disease in large numbers of people who have never previously even received a test.’
No comments yet