Excess Death Figures: Further Considerations (part 1)
Weekly excess death has become a popular yardstick for assessing the impact of COVID, government policies and COVID therapies but this method is brimming with issues.
My recent five part series entitled Excess Deaths by Cause, England 2020/w1 – 2022/w46 packed quite a punch. Back in the very first article I rambled on about inadequacies of this widely studied data under the sub-heading A Quick Word About Excess Deaths and I’d like to draw this to people’s attention once more by quoting the most pertinent parts:
Subscribers will be familiar with my usual ranting about this commonly used but rather inadequate method of calculating excess death. I’m not going to bang on about this again other than to say disease doesn’t carry a diary, and neither does the weather or any pathogen that I know of. There is seasonality with certain disease groups, especially respiratory, but that seasonality is not rigid. Thus, something nasty arriving a week or two earlier or later than ‘normal’ is going to throw such a basic calculation and give rise to spurious spike. Aside from the sliding of seasonal effects we’ve got the issue of longer-term trends that will rest on a vast raft of factors. The presence of any long term trend makes a mockery of any baseline based on a 5-year mean.
Then there’s the issue of the population changing over time both in size and age profile. Normally we have to adjust for this to avoid bias but the ONS don’t bother. I would bother (and have done so for previous analyses) but my time pressures are immense right now and I want to present excess death using the same method as the ONS. If I can, I’ll scrape some time together to produce a set of revised slides that take into account the changing age profile of the nation. One thing I will say in defence of ONS’ inadequate method is most dying is done by the oldest age groups and these sub-populations haven’t changed much over the last 10 years. Comparisons I have made between age-adjusted and non age-adjusted excess (not published) reveal minor differences that are somewhat academic, but I may well pen an article that reveals this!
…so I’ve gone and scraped some time together to produce a set of revised slides that take into account the changing age profile of the nation, and what I shall do below is set out a selection of dual time series plots so we may see with our very own eyeballs what differences arise when we tackle the changing national age profile. This, I feel, is most timely given the impact that these types of analyses (including mine) are having, and especially so given Professor Carl Heneghan and his team drew our attention to these tricky issues in this Substack article.
You know the drill… get the kettle on, fire up that toaster and let’s get munching!
Estimating Sub-populations
Before we eyeball yet more slides I better point out that all of my age-adjusted calculations, including mortality rates, are derived using population estimates for each age band using ONS per year of age data, thus a mortality for the 18 – 29y band will be the number of deaths for that age group divided by the population estimate for that precise age group (for the nation of England).
Anybody wanting to derive appropriately-banded sub-populations for themselves can find a rather large (77Mb) but tremendously useful Excel spreadsheet called analysistool2020uk.xlsx right here on the ONS website. This table provides annual trends based on midyear estimates only, so it is necessary to infill data for higher resolution monthly or weekly data-files. There are a number of ways of achieving this but we always need to bear in mind that sophistication is lost on a dataset that starts out being a pile of ONS estimates churned out by models resting on assumptions. We’re not very good at counting heads and there’s no way round this!
My preference is to utilise a combination of time series modelling, missing value estimation and linear interpolation modules within my stats package to arrive at sensible-looking growth curves. My eye is thus the final judge.
Using Sub-populations
Using the sub-population counts by age group is another matter entirely and again there are a number of ways of going about this. One method is to adopt a reference year, thence to derive a series of multiplication factors to apply to counts of deaths with each age band for other years. By way of example, if there were 10% less 70 – 79 year-olds back in 2015 compared to 2019 then the multiplication factor applied to 70 – 79 year-old deaths in 2015 would be x1.11. This is a simple calculation to make and is intuitively easy to understand, so I shall proceed by selecting a few choice time series so we may compare weekly excess death curves for unadjusted and adjusted (standardised) sub-population counts.
If sub-populations are changing little over time then we are not going to see much difference, if any, between the unadjusted and adjusted time series. However, if sub-populations have been somewhat dynamic we’ll start to see the two time series diverge. If the sub-population has been growing in size over time then the curve for standardised excess will be tucked beneath the unadjusted series. With this in mind let us peruse a modest selection of slides to get a taste.
Keep reading with a 7-day free trial
Subscribe to John Dee's Almanac to keep reading this post and get 7 days of free access to the full post archives.