In the NDR podcast with Germany’s Chief Virologist Christian Drosten last week, presenter Korinna Hennig gave the example of a headline “Does eating strawberries help with the Coronavirus?” to which, when you read the article, the answer was clearly “no“. However, by the time you have read the article, the idea has already been planted that strawberries may help with the Coronavirus – and that is the inherent risk in the headline.
It is always useful to apply our plausibility and relevance test!
(And no, it is not “plausible” that strawberries help in any way at all – other than, perhaps, steering you away from chocolate in the interests of your general health…)
A Daily Mirror headline ran yesterday that “19 million” people in the UK may already have been infected.
Technically, this is just about plausible; a death rate at the very lowest conceivable based on current research would just about allow that conclusion.
However, it is so close to the borderline of plausibility as to be irrelevant and therefore frankly irresponsible, as implicitly it suggests pre-vaccine “herd immunity” is a viable means out of the current crisis, when there is minimal evidence to suggest this is true (although, under London below, it is not to be ruled out completely).
In fact, later the same day, the UK Government released evidence that in a two week period from late April to early May fewer than 150,000 people in England were infected at any time. This would push the overall number since late February very low indeed.
The balance of the evidence is that around 3-4 million people across the UK have been infected. That is very plausible.
It is, of course, not much of a headline – but therein lies the problem with too much media reporting currently, particularly in the press.
A BBC headline ran yesterday “70% more Coronavirus deaths than official figures”.
How plausible is that? And how relevant is it?
Well, of course, it is plausible. Not only do official daily counts in Northern Ireland record only those dying having tested positive for (or, latterly, having been diagnosed with) COVID-19, but they are quite openly counts of registered deaths up until that point. A combination of deaths “assumed” to be COVID-19 (which only appears on the final death certificate or theoretically may even in the end be determined by a Coroner), excess deaths on which COVID-19 had some effect (even, say, a heart attack for which the patient did not seek immediate attention), and the simple lag in reporting deaths all mean that it is plausible that the total number as of this moment is around 70% higher than the official daily count figure.
There is an issue here, however, with the relevance of the way the headline is worded. Implicit to it is the suggestion that deaths are somehow being “hidden”, even that the record is being tampered with to make it look better than it is. On the absolute contrary, Northern Ireland is one of the quickest jurisdictions in the world to publish excess death figures and thus enable a full comparison of total excess deaths, with deaths assumed to be associated with COVID-19, and with the official daily count of those diagnosed with COVID-19 before death.
A very common error here, made even by one of the “experts” providing evidence to the Health Committee at Stormont on Thursday, is to compare the “excess death” figure in the UK (the publication of the overall figure for the UK is slightly slower than for Northern Ireland alone, but still relatively quick by global standards) with the “tested positive for COVID-19 figure” for Germany (which is the only one we have beyond early April, but is clearly a lower total – by definition, we do not know by how much). This is a totally flawed comparison, and anyone defining themselves as an “expert” needs to do better.
The headline was all the more curious because it ran with figures two weeks out of date.
The figures as they were published yesterday are interesting not because of the gap between daily count and excess deaths (which we have in fact known about for weeks), but because they provide a clear argument for lockdown, at least as an initially important policy intervention. That is the relevant point here.
The question raised by a blog post a few days ago was whether the excess deaths were being caused by COVID-19 (but just missed by medical professionals) or by lockdown itself. Either was plausible, but the figures provide evidence towards the former.
None of this is perfect, because we are trying to compare deaths by date of occurrence with deaths by date of registration, and in a relatively small jurisdiction even minor statistical biases or errors can cause significantly different apparent outcomes. Nevertheless, fundamentally, if lockdown itself were causing deaths (in the immediate short term), we would expect that number (not the share, the number) of excess deaths over those attributed to COVID-19 to remain relatively stable. The evidence had suggested that it was relatively stable over a three-week period to 1 May (though we cannot yet absolutely know for reasons aforementioned) but in the week to 8 May we see an apparently dramatic decline. In the five weeks from the last week in March (i.e. to 1 May) deaths attributed to COVID-19 accounted for only 71% of all excess deaths in Northern Ireland; but in the six weeks from that week (i.e. to 8 May) that figure rises to 76%.
It would not appear to be “good news” that more deaths are being and thus potentially can be attributed directly to COVID-19 than until now, but at least it means that lockdown itself has not had quite the effect we may initially have feared in terms of people not seeking treatment in emergencies. However, that does not mean it has had zero effect; and nor does it mean at all that the long-term effects of not screening for cancer or the psychiatric difficulties inevitably associated with lack of social contact will not be considerable.
Republic of Ireland
This can work the other way, too. On Wednesday, the Republic of Ireland initially announced just 33 new cases. This seemed just about plausible, given already by this stage those were the sorts of figures emerging from the whole of London (with a much higher population), although it seemed a little unlikely (we will come back to why below).
Yet the next day, it announced 426 new cases. Questioned about this, its Chief Medical Officer said that the 426 included an “accumulation” of cases from one hospital which were being announced at once. This seems relevant, but also stretches plausibility – why would you declare 33 one day when you are sitting on hundreds?
Few journalists thought to ask the obvious question but thankfully two did. One revealed that there had been a significant problem with testing (potentially relevant to when infections are actually being determined); another revealed that of the 426, 223 were “accumulated” at one hospital but this was a problem because it was unclear if there had been any contact tracing of those 223 (definitely relevant). There is also the obvious problem that this would still leave the daily count at 203, up from 33 initially announced (but as one commenter subsequently points out, also subsequently published as 159) the previous day, which is implausible.
The problem is, this whole thing led to a debate about whether figures were being tampered with ahead of the announcement on raising lockdown from Monday. This is probably not the case, but in some ways that is more worrying – the first rule of the plausibility and relevance test is that in the battle between conspiracy and cock-up, it is almost always the latter…
On Thursday, according to a Cambridge University study there were just 27 new cases in London (population over 8 million). Given that it had long been suggested that in London R<0.5, this is just about plausible (although the above does make you wonder).
However, is it really plausible? The assumption was that in London R<0.5 because there had been so many cases to begin with that even relatively high figures constituted a sufficient decline on earlier numbers to deem the R-number relatively low.
This is combined with the notion of “undercover spread”, widely discussed in German-speaking Europe but completely missed in the English-speaking world.
Essentially, even under lockdown, the virus will spread into areas it has not previously touched – be that regionally (say into previously less affected parts of England like the South West) or even socially (say, into care homes); therefore even if R=1 and the virus is not spreading exponentially overall, it will almost certainly be spreading exponentially in places not previously heavily hit – essentially because the same number of infections proportionately means previously less affected areas will see R>1 while previously more affected areas will see R<1.
This is one of many problems with relying purely on R!
However, only 27 new cases is an extraordinarily low number. Certainly relevant questions need to be asked:
- Is testing being properly carried out (there are serious questions around the quality of tests when testing is based around random targets rather than established outcomes)?
- How are confirmed cases actually being “confirmed”?
- Are there cases not being assigned to a particular region?
If it really is as low as that, it opens up two relevant policy options which are completely contrary to each other:
- Is it, in fact, plausible to go for outright eradication in London (and, by extension, the whole of the UK and Ireland) by extending lockdown?
- Or, almost completely conversely, is it plausible that this figure so low because some sort of “herd immunity” has already been built up in London, where the virus spread very quickly before lockdown (this is what Sweden has been banking on all along)?
This is where the quest for plausible figures becomes hugely relevant…