Do COVID Vaccines Work? (part 9)

I utilise data from an unknown NHS Trust to forge a staged multivariate logistic regression model in the assessment of vaccine efficacy

Crikey, we’re up to part 9 of this series already and there’s no end in sight, with countless wrinkles and avenues still to explore! We’ve just pipped past the summer solstice and we’re now into that period of shortening day length once again. Periods and cycles of all manner whirled through my holofractal individuated consciousness unit as I munched on my cornflakes this morning whilst cogitating on the difficulty of modelling in-hospital death with umpteen confounding factors. I asked myself if there was anything more that could be done to smooth the way a little – rolling the wicket prior to play as it were – and so I decided to take another look at how death stacked-up over time, and vaccine death in particular. By ‘vaccine death’ I mean those unfortunates souls who were vaccinated prior to death, with death arising from any and all causes. I reckoned some crayoning was in order, and flipped right back to the beginning to look at the variation in vaccinated death over time. Not in absolute terms, mind, but as a percentage of all in-hospital deaths to account for the steady slog back to normal levels of service provision. What I found got me thinking:

That distinct tail-off for week 37 sure looks like an artefact arising from administrative delay so I decided to place an end point at week 36. We see vaccinated death rise with vaccination uptake as expected and I’ve plonked another dashed line down at week 10, for this is the week where the percentage of vaccinated death matched the percentage reached at week 36. That slice in the middle looked nicely balanced to me, being a plateau of sorts that should offer a more stable basis for logistic regression modelling. Starting the data sample at week 10 should also have given the elixir plenty of time get going (though there's a potential study refinement here concerning delays following treatment that I'll address in part 10 of this series).

Choice Of Dependent

With the time frame fixed I decided to wade straight in by setting COVID Dx as the dependent variable despite its many shortcomings. A positive test result – whether accompanied by a relevant supporting diagnosis or not - is the only thing the experts and authorities seem to be bothered about, so let’s model that!

Stage 1a: Demographics

In terms of demographics we’ve got age, sex, total diagnoses made and case detection rate (a.k.a. CDR, a.k.a. disease prevalence). Then we’ve got two pretty important medical factors to consider in relation to COVID status, these being acute respiratory and chronic respiratory conditions. All in all that’s six factors to put into the pot, these being called ‘main effects’ by stats bods. We now need to add their interactions, there being 15 two-way and 20 three-way interactions to consider. There are, of course, higher order interactions we might also consider but these are going to slice the data into impossibly tiny chunks and I’d like my casserole to be as robust as possible. Thus, we arrive at a logistic stage 1a model stuffed with 41 terms many of which are going to be irrelevant, so we shall use a forward selection procedure to weed out those that fail to reach the 95% level of confidence. Here’s what those 41 terms look like before weeding:

Here’s what came out in the wash:

All of these terms are highly statistically significant (p<0.001), so let’s have a look at the column headed Exp(B), which gives us an estimate of the odds ratio (OR). Standing out like a very sore thumb is an OR of 121.98 for the interaction term Acute Respiratory by Chronic Respiratory. This is telling us that folk who were suffering a chronic respiratory disease who also developed an acute respiratory condition prior to death were 122 times more likely to receive a positive test result for COVID than those not suffering from these conditions. We may compare this whopping great odds ratio with that for Acute Respiratory alone, which amounts to OR = 15.32 (p<0.001), and may conclude that underlying respiratory health issues greatly increase risk of severe COVID leading to death. This finding is totally in-keeping with clinical observation - so far, so good!

Two more terms make total sense in that case detection rate (CDR) is linked to increased likelihood of a positive test result (OR = 3.18, p<0.001), as are the total number of diagnoses made on the electronic patient record (OR = 1.27, p<0.001). Making less intuitive sense are the two odds ratios less than unity that adjust for interactions between positive test result, age and case detection rate (OR = 0.99, p<0.001) and a complex interaction for positive test result, age, acute respiratory conditions and chronic respiratory disease (OR = 0.93, p<0.001). These are small effects that have the effect of shaving a little off the likelihood for a positive test result for the elderly during the thick of the pandemic (survivor bias?), and for the elderly with complex medical conditions (e.g. emphysema is likely to be diagnosed in preference to COVID for a patient with a long history and ambiguous chest X-ray).

Stage 1b: Vaccine Distribution

We’ve now got to level the playing field in terms of vaccine distribution. Aside from the single main effect carried by the variable Vaccinated there are ten possible interactions with basic demographics that we now need to add to the stage 1a forward selection procedure, these being:

Here’s what came out in the wash:

Eyeballs will no doubt zip right to that stonking odds ratio of OR = 336.50 (p<0.001) for the interaction term Acute Respiratory by Chronic Respiratory – this is a more refined estimate of what we saw before because we’re now adjusting for vaccine distribution thus altering the covariance matrix, and vaccine distribution was prioritised on the basis of clinical need rather than a random affair.

The two terms we now need to carefully consider are the main effect Vaccinated with an odds ratio OR = 0.05 (p<0.001) and the interactive term Age by Vaccinated with an odds ratio OR = 1.03 (p=0.002). I say ‘carefully’ because the naïve among us would wade straight in and claim total vaccine success, since the main effect odds ratio of OR = 0.05 is far less than unity, indicating an overall 20-fold reduction in the likelihood of a positive test result for the vaccinated cohort!

There are many ways this result could come about. It could indeed come about because of genuine benefit, but then again it could come about simply because healthcare professionals were told not to test vaccinated inpatients, particularly if they were very sick (as these obviously were). It’s a result in the ‘right direction' but we have yet to determine cause - stages 2 and 3 are where we get the vaccine to walk the talk and not just talk the talk!

Before we move on to these critical model stages we need to consider Age by Vaccinated (OR = 1.03, p=0.002) because this is already pulling the rug from under any notion of genuine vaccine benefit. Let me explain why. This term, with an odds ratio greater than unity, is revealing an increase in the likelihood of a positive test result with age for those who have been vaccinated. This is exactly opposite to what we would expect! Older people are more susceptible to development of severe COVID-19 and thus they were the first to receive their doses, with high levels of uptake amongst this sector of the population. Ergo we should see a benefit with increasing age in terms of reduced incidence of COVID prior to death, yet the model indicates the reverse is true – older vaccinated and rather sickly folk were more likely to be diagnosed with COVID than their unvaccinated counterparts.

Stage 2: Vaccine Interactions

This is a big one. Up to now we’ve seen what appears to be an overall vaccine benefit in terms of reduced likelihood of a positive test result for the vaccinated cohort. Against this we noted an interaction between age and vaccination status such that older vaccinated inpatients were more likely to be diagnosed with COVID prior to death than unvaccinated. Neither of these model terms tell us what we really need to know and that is whether administration of the vaccine is associated with reduced incidence of chronic and acute respiratory conditions for COVID cases. To answer this question we have to throw the following 11 interactive terms into the pot:

Note that I’ve assigned the two 2-way interactions to the third and final model stage when these terms will be entered (forced) into the equation. This is because they are the two most critical terms of all and I want to give them the very best chance of shining through the stats. The remaining nine 3-way interactions between vaccination and respiratory illness are really just icing on the cake and may not hold value as sample sizes dwindle, so I desired to submit them for conditional forward selection at the second stage. A bit of light pruning as it were.

Here’s what came out in the wash:

We may note that nothing has changed in the top table and that all nine 3-way vaccine interactions were rejected at the 95% level of confidence. I’ve included the rejection listing in the bottom table so you can see that nothing got close to the magic value of p=0.05, with CDR by Acute Respiratory by Vaccinated managing p=0.153 at best and thus failing the 90% level of confidence threshold that some analysts prefer to adopt. But these 3-way interactions are not where the serious statistical action is, that is coming next…

Stage 3: Vaccine Efficacy

This is THE big one. In this third and final stage I force the two interactive terms Acute Respiratory by Vaccinated and Chronic Respiratory by Vaccinated to establish whether there is a relationship between COVID diagnosis, respiratory illness and vaccination status prior to death. If the vaccines work as advertised one or both of these should reach statistical significance at the 95% level of confidence and their associated odds ratios should be less than unity. Less than unity means reduced likelihood of a COVID diagnosis for vaccinated individuals suffering from a respiratory condition just prior to death, this being evidence of vaccine benefit. Odds ratios greater than unity will be indicative of vaccine harm.

Here’s what came out in the wash:

The interactive term Chronic Respiratory by Vaccinated manages an odds ratio a shade under unity (OR = 0.99) but the p-value is way, way beyond the pale at p=0.980. Thus, despite the odds ratio going in the ‘right direction’, there is no concrete statistical evidence of vaccine benefit for those in-patients with chronic respiratory disease. In sum, their likelihood of receiving a positive test result is no different to the unvaccinated cohort suffering the same. I shall mark this down as vaccine failure!

Acute Respiratory by Vaccinated really is the crux of the matter in that the vaccines were designed - above all else - to reduce onset of severe COVID-19 leading to acute respiratory illness. This is not what we observe in this sample of 5,553 in-hospital deaths. In fact, what is rather worrying, is that the odds ratio is above unity at OR = 1.33, which indicates increased likelihood of a positive test result for vaccinated individuals suffering from acute respiratory conditions prior to death. Fortunately, the p-value indicates this estimate is not statistically significant at the 95% level of confidence (p=0.300) despite an odds ratio indicative of vaccine harm.

Luncheon Is Served

Well that is quite something! If I had to sum all this up in plain English it would be to say that there is no solid evidence of vaccine benefit within this sample of 5,553 in-hospital deaths. Statistically speaking the vaccinated fare no better than their unvaccinated counterparts when it comes to COVID-induced respiratory illness after we level the playing field.

So what of that monster 20-fold benefit associated with the Vaccinated main effect? I shall suggest that this is akin to a meringue - all puffed sugar and no substance because benefit does not accrue for COVID-induced respiratory conditions. This illusion can come about in many ways and inappropriate use of the PCR test lies at the heart of the matter.

On top of this we may possibly have just witnessed a vague indication that vaccines might be responsible for inducing COVID-like acute respiratory conditions in some people. Vaccines that cause a COVID-like allergic reaction, eh? Well, who’d have thought that back in the steamy days of celebrities and suits selling us the magic elixir through their gyrations, grimaces and jokes.

In part 10 I’m going to refine the model by considering dosing and the time delay between dosing and death. Right now we have a hotch-potch of unvaccinated in-hospital death, dose 1 in-hospital death, and dose 2 in-hospital death with no regard to days elapsed between vaccination and vaccine-acquired immunity.

Kettle On!