It Would Have Been Worse (part 3)

Having a peek under the bonnet of pro-vaccine claims

In part 2 of this series I used the ratio of MV to general beds as a proxy of case severity. Some interesting features emerged in which we observed a sharp rise in reliance on MV beds in preference to general beds shortly after vaccination roll-out began. Though it could be argued this was the 2020/21 seasonal peak waiting to happen we have to ask why vaccines did little to alleviate this. Then there was the matter of prolonged reliance on MV beds over the summer months of 2021 at a time when viral infection within the population would have been minimal and risk of severe symptoms lowered. Either vaccines were exacerbating the situation or something else was going on. We also noted that roll-out of the booster was coincident with a background decline in reliance on MV beds that no doubt was inappropriately taken as evidence of vaccine benefit alone. Then we observed a puzzling plummet whereby MV bed use for COVID cases suddenly and dramatically declined at the start of 2022, this being indicative of an abrupt policy change rather than the elixir of life suddenly and inexplicably working after months of not obviously doing so. Let us now view that data another way…

In Through The Out Door

What I’ve done here is express that jiggly-wiggly time series of the ratio of MV to general bed use for COVID cases as monthly means and 95% confidence intervals by pandemic year:

Yes indeed, it is another of those traffic light charts that folk are finding intuitively easy to grasp. The puzzling plummet (a.k.a. most mysterious change) now glares at us in lime green right at the bottom of the chart. The entire series can be consider to be an outlier and ideally somebody at the sharp end might provide illumination as to how and why management of COVID cases abruptly changed on 1st January 2022. I’d like to think that the NHS executive had realised it was high-time to consider all that is not COVID, or that some wonderful new treatments were now available enabling fast recovery under standard care, but the cynic in me wonders if they’re simply trying to make the vaccines look good.

What we have to do here is put 2022 aside as anomalous and eyeball the remaining two years. When we do that we find our traffic light upside down again in that ratios for a well-vaxxed 2021 exceed the ratios for a non-vaxxed 2020 for every month from April to December. This is the vaccine working is it?

Going Total Pro-Vax

We could go total pro-vax at this stage and state that the mutated virus was more deadly during 2021 and that the vaccine most certainly worked to reduce case severity arising from new super-strains to the levels of MV bed use observed in the above chart. This is entirely possible, of course, but such a statement needs to be supported by empirical evidence otherwise it is mere conjecture.

Whilst we’ve got our biscuit tins open we might as well provide that evidence ourselves. But there’s a snag…

A Snag

Ideally we’d ascertain whether 2021 strains were nastier by looking at the mortality rate but the mortality rate depends on robust estimates of certified COVID death (we can get these from here) and robust estimates of total COVID cases (whoops-a-daisy). There is a whoopsie in the ointment because total COVID cases is going to depend on total viral tests (no tests = no cases; loads of tests = loads of cases) and these rocketed from 1,466 per day back on 1st March 2020 to a staggering 2,349,928 tests per day on 5 Jan 2022; this representing a factor increase of x1602.95. If we undertook over a thousand times more tests per day during 2022 than 2020 do you think there’s a chance we might have found more cases across England in 2022 simply because of this surge in detection activity? You betcha! If we lower test rates will it look like the vaccines have been effective? You betcha!

Double Snag & Bother

An easy route out of this is to standardise case counts to, say, 580,000 tests per day (the series mean from 30 Jan 2020 - 26 Jul 2022 being 578,378.9 viral tests per day). When we do this the case count looks like this:

Quite a difference, innit? A monster peak now springs up back in Mar - Apr 2020 when nothing much was reported owing to few PCR tests being undertaken on those who were already hospitalised. In contrast the monster reported peak of Dec 2021 - a function of test frenzy - now shrinks back into something sensible. We also observe a boost to case counts for the second and third waves owing to their relatively modest test activity.

It is also worth noting that case counts in recent months have dropped because test activity is in decline. We could fall into the trap of assuming this was due to waning infection rates, weak new strains, building immunity, beneficial social policies, appropriate behaviours, expertise of experts etc; and no doubt there are those who will wish to ascribe such decline to vaccine benefit. All will be a pile of hot air and meringue unless we somehow account for varying amounts of testing over time.

Off To Neverland!

This method of standardisation, though rather crude, has enabled us to get a first glimpse of what was likely going on rather then what authorities claimed was going on. But I need to do better. For one thing the PCR test is likely to introduce a different case count bias to LFD test kits. Then there’s the differing risk profiles of the population subgroups who use differing types of tests in different settings: that monster peak for standardised first wave cases is almost certainly over-egging the batter owing to the highly selective population from whence the source data arises.

Yes indeed, we are off to modelling land again and this time I’m going to try and develop estimates of case count bias that do a better job of crude standardisation to 580,000 virus tests per day. Once bagged we can continue exploring the situation through the lens of case severity; for example, a handy index to squizz would be MV bed count per 100 detected cases - these being standardised case counts.

Kettle On!