Antibody tests are only accurate when used between three and four weeks after you've had Covid-19 and 'won't work' if used at the wrong time, major review warns
- Research found accuracy was below around 70 per cent within two weeks
- And there is not enough data to work out how well tests work after five weeks
- Experts said it is also not clear if the tests will work on non-hospital patients
- One scientist said tests are not good enough to be used outside of hospitals
Coronavirus antibody tests are only known to be accurate between three and four weeks after someone has had Covid-19, a scientific review has found.
They may also not work on people who have only had a mild illness, but researchers admitted they can't be sure because almost all of the studies have been carried out on patients who were so badly ill they were hospitalised.
One scientist said the evidence showed no antibody tests currently available on the market are good enough to be used outside of hospitals.
The 300-page independent review, led by the Cochrane institute and the University of Birmingham, analysed data from 54 scientific studies of antibody tests used on 16,000 blood samples.
The tests examine people's blood to look for antibodies — substances made by the immune system — that indicate whether they have had Covid-19 in the past.
In the UK the tests, which Boris Johnson once promised people would be able to take themselves at home, are not currently widely available.
The accuracy of them is a huge sticking point — they can detect fewer than 30 per cent of positive results if used at the wrong time, the review warned.
And scientists still aren't really sure what they mean. In usual medicine, the presence of certain types of antibodies means someone is almost guaranteed not to get an illness again — but there is still no proof Covid-19 survivors are immune.
Antibody tests examine someone's blood to look for signs that they have been infected with Covid-19 in the past. In the UK they are only available in hospitals or as part of government surveys (Pictured: A blood sample collected by West Midlands Ambulance Service)
Professor Jon Deeks, a medical tests expert at the University of Birmingham, was one of the scientists behind the international review.
He said: 'We’ve analyzed all available data from around the globe — discovering clear patterns telling us that timing is vital in using these tests.
'Use them at the wrong time and they don’t work.
'While these first Covid-19 antibody tests show potential, particularly when used two or three weeks after the onset of symptoms, the data are nearly all from hospitalized patients, so we don’t really know how accurately they identify Covid-19 in people with mild or no symptoms, or tested more than five weeks after symptoms started.'
The Cochrane review found that the third and fourth week after someone has been infected with the coronavirus are the optimum time to use them.
Too soon, and they are inaccurate, but too late and their accuracy is completely unknown.
If someone took one of the blood tests within two weeks of developing symptoms, studies found, only seven out of 10 Covid-positive people would receive a positive result (70 per cent test sensitivity).
Between 15 and 35 days after symptoms, this accuracy increased to more than 90 per cent, on average.
For patients who had symptoms 35 days ago or longer, there were 'insufficient studies' to estimate how well the tests could work.
The researchers looked at studies evaluating 27 different types of antibody test - out of approximately 200 on the market - and said there was not enough data available to show whether lab-based tests were definitely better than hand-held ones.
Explaining how the accuracy they saw in the studies would affect real-world results, the researchers gave the following breakdown:
Results from tests three weeks after symptoms started (the most accurate timing) would reveal the following if 50 out of 1,000 had had the virus already (five per cent - on par with the UK population estimate):
- 58 people would test positive for Covid-19. Of them, 12 people (21 per cent) would not have Covid-19 (false positive result)
- 942 people would test negative for Covid-19. Of these, four people (0.4 per cent) would actually have Covid-19 (false negative result).
Doing the same tests on a set of 1,000 healthcare workers, of whom 500 had had the virus already (50 per cent), antibody tests would produce these results:
- 464 people would test positive for Covid-19. Of these, 7 people (two per cent) would not have Covid-19 (false positive result).
- 537 people would test negative for Covid-19. Of these, 43 (eight per cent) would actually have Covid-19 (false negative result).
The scientists said it is impossible to know how well the antibody tests would work on someone who was not so seriously ill that they ended up in hospital.
The review said: 'We do not know how well the tests work for people who have milder disease or no symptoms, because the studies in the review were mainly done in people who were in hospital.'
Dr Jac Dinnes, a Birmingham researcher who worked on the review, said the studies that had been done on antibody tests so far were not of high quality.
She said: 'The design, execution and reporting of studies of the accuracy of Covid-19 tests requires considerable improvement.
'Studies must report data broken down by time since onset of symptoms.
'Action is needed to ensure that all results of test evaluations are available in the public domain to prevent selective reporting. This is a fast-moving field and we plan to update this review regularly as more studies are published.'
Scientists responding to the review said it showed that the tests should be treated with caution and limited weight given to the results they produce.
One said none of the currently-available tests are good enough to be used outside of hospitals.
The Government has been flummoxed by antibody tests since the early days of Britain's epidemic.
Boris Johnson initially promised to get ones that people could use 'like pregnancy tests' in their own homes and officials lost around £20million buying kits from China which later turned out to not be fit for purpose.
A MailOnline investigation this month revealed that in its 'buy first, test later' approach, the Department of Health had to cancel £70million worth of orders for tests it decided weren't good enough.
Professor Eleanor Riley, an immunology expert at the University of Edinburgh, said of today's review: 'It is already very clear from this analysis that antibody testing early in the course of disease is unreliable and thus antibody tests cannot (and should not) replace virus detection for diagnosis of acute cases except where the time-course of disease is already well advanced. This is a useful and important clarification.
'The data also make it very clear that most, if not all, commercially available tests are not sufficiently accurate to warrant their use outside of the healthcare setting.'
Dr Alexander Edwards, a biomedical technology professor at the University of Reading, added: 'There is no doubt that antibody tests are essential, and are already extremely useful. But what must be kept in mind is that there are many different ways that we already use, and will continue to use, antibody tests...
'For example, we do not yet have enough data to understand how antibody responses relate to the risk of re-infection or transmission- so we can’t use antibody tests for "immunity passports".
'In contrast, we do have plenty of data showing how antibody tests can be used to survey the number of people who have already been infected.'
The main way the UK is currently using antibody tests is for NHS staff - to see which ones of them have had coronavirus already, and move them to 'clean' wards - and for population surveillance to see how many members of the public have had the illness.
Both the Office for National Statistics and Public Health England are using antibody tests for this purpose, and have tested tens of thousands of people to date.
This blood testing on behalf of the government has found that around 5.4 per cent of people in England have had the virus already. The ballpark is somewhere between 4.3 and 6.5 per cent, according to the Office for National Statistics.
This means around 3million people have had the virus already, and considerably more people in London have had it than in other regions.
Testing by Public Health England has found that around 17.5 per cent of people living in the capital have got antibodies to the disease, meaning their immune system has fought it off already.
This varies wildly across the country, PHE data shows, with around 12 per cent of people exposed in the North West and 10 per cent in the East of England, but fewer than 10 per cent in every other region.
Lowest are the South East and North East, where only around four per cent of people appear to have had the illness already.
CDC WARNS EVEN GOOD TESTS MAY BE WRONG 50% OF THE TIME - SO HOW DO THE TESTS FALL DOWN?
The US's Centers for Disease Control & Prevention last month warned that coronavirus antibody tests may be wrong 50 per cent of the time even if they are high quality.
It warned that the antibody testing is not accurate enough for it to be used for any policy-making decisions, as even with 95 per cent test specificity (the accuracy of negative results), 'less than half of those testing positive will truly have antibodies'.
This is why:
Antibody tests with what could be considered a high level of accuracy can still produce large margins of error if only a small proportion of a population has been infected.
A 95 per cent specific test, for example, will always produce five false positive results from a group of 100 people.
Even if it is sensitive enough to detect all the people who have genuinely had the disease, it will still return five false positives, and the effect this has on the results of a survey can be large if the number of true positives is low.
If the prevalence of antibodies is low - for example, only five per cent of people in the group have had the illness - the results could end up half wrong. The 95 per cent test, in that situation, would be expected to return 10 positives - five of them right, five of them wrong.
This means the functional accuracy of the test, known as its true predictive value, is only around 50 per cent.
The effect of these false positives is magnified if the prevalence of the virus in the population is low, and less noticeable if the prevalence is high.
For example, if 30 per cent of the population have been infected, those five false positive results would be counter-balanced by 30 true positives, making the test more like 85 per cent accurate.
A more specific test can reduce this effect; by comparison a 99.9 per cent specific test would return one wrong result per thousand - 100 per million.
Warning over Covid-19 antibody tests as leading doctors say blood kits STILL haven't been properly assessed and truth on immunity remains a mystery
Coronavirus antibody tests should not be used to tell people whether they have had the infection already, scientists have warned.
In a joint letter published in the British Medical Journal last night, a dozen leading scientists have cautioned that the blood-based kits — once dubbed 'game-changers' — by top politicians may actually be more trouble than they're worth.
Antibody tests examine blood to look for signs of past infection, as opposed to swab tests that tell if someone is currently infected.
An antibody is a substance produced by the immune system to remember how to fight off a particular illness — a Covid-19 antibody shows someone has had Covid-19.
But the presence of them doesn't guarantee someone won't get the illness again, the experts warned, because there is no firm proof that long-term immunity exists.
And, despite the Government having bought more than 10million of the tests, the way they have been tested is not good enough to prove their quality, the letter added.
Rolling out the tests in the NHS to patients would use up so much time and money, to produce uncertain or potentially worthless results, that it isn't worthwhile, it said.
Antibody testing is currently only available to frontline NHS staff, who are given tests made by the pharmaceutical companies Roche and Abbott.
Antibody tests work by feeding someone's blood through a device containing a fragment of the coronavirus and an enzyme to encourage them to react if all are present (Pictured: A test being used in Russia)
The scientists' warning in the BMJ was triggered by an NHS letter in May that asked hospitals to prepare to roll out antibody testing en masse to their staff and patients.
This asked NHS England to test all staff who want one, as well as patients for whom it would be useful or desired to know if they had had the infection.
Antibody tests work by feeding someone's blood through a device containing a fragment of the coronavirus and an enzyme to encourage them to react if all three parts are present.
If there is a reaction when the blood is inserted, it indicates that someone's immune system has antibodies and has learned how to fight off Covid-19 in the past.
But the researchers writing in the BMJ, who work at universities including Liverpool, Oxford, Nottingham, and Queen Mary and UCL in London, said this did not mean anything more than that someone had had the virus already.
With other disease, a positive antibody test may prove that someone is immune to an illness. But because the coronavirus has only been around for six months, scientists do not know whether people's bodies protect themselves from it over the long-term.
Studies have suggested that antibodies last for months, at least, but it remains unclear how protective they are.
The scientists wrote: 'The concept of 'immune passports,' allowing healthcare workers or others to work, has not been established.
'Those with a positive antibody test should still consider themselves at risk and follow infection control policies designed to prevent [in-hospital] spread and risk of infection.
'There is, therefore, no benefit to healthcare organisations or to others in knowing the status of employees at present.'
Another concern of the scientists was that the antibody tests being used in the UK have not been evaluated well enough to prove that they are good.
They raised concerns that tests were not being done on enough people known to have the coronavirus, to check their accuracy.
Public Health England antibody testing results show the levels of exposure to Covid-19 vary wildly across the UK, from around 17.5 per cent in London to lower than five per cent of people in the South East and North East
And they were also not being tested on enough people with other illnesses, to check whether the tests cross-reacted and returned a positive result for a different type of infection.
The letter said: 'Those who are at highest risk of death from this infection are elderly people, those from black and minority ethnic groups, and immunocompromised people.
'There are currently no data showing the performance of the tests in these groups.
'The correct route to generating valid test performance data is well designed prospective clinical studies.
'The [test] is being rolled out at an unprecedented pace and scale without adequate assessment, potentially compromising public trust in pathology services in the future.'
Because of concerns about the quality of the tests, and uncertainty about what the results even show, the scientists said the NHS's plan to roll out antibody tests was a waste of time and money.
The service would require results to be produced within 24 hours, they said, which would cost laboratories time and money.
They added: 'Given that routine testing of patients is neither clinically urgent nor meets a clear public health need, this push to introduce a non-evidence based test for uncertain gains risks inefficient use of scarce resources.'
In conclusion the researchers admitted that antibody tests were useful for research, to track how far the virus has spread, but not for much more.
No comments: