Claim: Vaccinated English adults under 60 are dying at twice the rate of unvaccinated people the same age

[skibandit]

This claim is going a bit viral on the anti-vax side today. The source appears to be Alex Berenson from here: https://alexberenson.substack.com/p/vaccinated-english-adults-under-60

He looked at some UK government data and made a scary looking chart from a table in an excel file posted on a government site. This is being spread by people who are convinced that the vaccines are actually poison and part of some global depopulation scheme. They are claiming this is proof that the vaccine might reduce Covid deaths, but dramatically increases overall mortality. I eyeballed the excel data and it appears the chart he made matches the published data. So, assuming the data is not a typo, my initial guess is that this is an artifact of the very broad age range, 10-59, with the older 50-somethings (who have a higher death rate in general) being much more likely to get the vaccine than healthy young teenagers. So the vaccinated among those age 10-59 would have a higher death rate simply because they were much older than the unvaccinated. I haven't seen any debunkings of this chart and data yet, though - just a lot of anti-vaxers posting it as proof that they were right all along and the vaccines really are a deadly poison. Thanks.

https://www.ons.gov.uk/peoplepopula...ths/datasets/deathsbyvaccinationstatusengland

 

[Mendel]

In short, the biggest problem with this is that the number of unvaccinated people used to determine the rate depends on two other numbers:
#unvaccinated = population - #vaccinated

The closer #vaccinated gets to the population, the more errors matter, and there appear to be quite a few sources of those; which means the number of unvaccinated peopld is probably overestimated.

In addition, people with higher risks of exposure may also be more likely to be vaccinated.

Berenson has jumped to this kind of conclusion before.

 

[Landru]

This claim is going a bit viral on the anti-vax side today. The source appears to be Alex Berenson from here: https://alexberenson.substack.com/p/vaccinated-english-adults-under-60

He looked at some UK government data and made a scary looking chart from a table in an excel file posted on a government site. This is being spread by people who are convinced that the vaccines are actually poison and part of some global depopulation scheme. They are claiming this is proof that the vaccine might reduce Covid deaths, but dramatically increases overall mortality. I eyeballed the excel data and it appears the chart he made matches the published data. So, assuming the data is not a typo, my initial guess is that this is an artifact of the very broad age range, 10-59, with the older 50-somethings (who have a higher death rate in general) being much more likely to get the vaccine than healthy young teenagers. So the vaccinated among those age 10-59 would have a higher death rate simply because they were much older than the unvaccinated. I haven't seen any debunkings of this chart and data yet, though - just a lot of anti-vaxers posting it as proof that they were right all along and the vaccines really are a deadly poison. Thanks.

https://www.ons.gov.uk/peoplepopula...ths/datasets/deathsbyvaccinationstatusengland

The authors of the data set came to a different conclusion.
http://www.ons.gov.uk/peoplepopulat...thsoccurringbetween2januaryand24september2021

• Between 2 January and 24 September 2021, the age-adjusted risk of deaths involving coronavirus (COVID-19) was 32 times greater in unvaccinated people than in fully vaccinated individuals.

 

[DavidB66]

I haven't looked very closely at the tables, but on looking at a few entries for different age groups I think the OP is right to suspect that the anomalous result is an artifact of the choice of 10-59 as the age group. As pointed out, this is very broad and covers ages that are diverse with respect to both vulnerability and vaccination rates. It is worth noting that for most of the period covered, children aged 10-17 inclusive were not eligible for Covid vaccination at all in the UK, due to doubts about the risk/benefit balance for the individuals in that age group, where the risk of death and serious illness from Covid is very low.

If we take a narrower age group, such as 60-69, the benefits of vaccination are clearer. To take one example, for week-ending 17 September, when cases and deaths were rising again thanks to Delta-plus, the unvaccinated in this age group had a Covid-related death rate of 8.9 per 100,000. For those with one dose (after at least 21 days) the rate was 6.8, while for those with 2 doses the rate was only 1.1: an 8-fold advantage for the double-vaxxed over unvaxxed. For the very vulnerable over-80s the advantage was only 4-fold, but still better than the alternative!

 

[skibandit]

The authors of the data set came to a different conclusion.
http://www.ons.gov.uk/peoplepopulat...thsoccurringbetween2januaryand24september2021

Yeah, I've pointed that out to the people who've been spamming the Berenson chart at me. Unfortunately, but unsurprisingly, that had no effect. They either ignore it or claim that, well, of course the authors had to lie about their findings, luckily they were careless and left the real data in the excel spreadsheet for Berenson to find.

 

[Rory]

my initial guess is that this is an artifact of the very broad age range, 10-59, with the older 50-somethings (who have a higher death rate in general) being much more likely to get the vaccine than healthy young teenagers. So the vaccinated among those age 10-59 would have a higher death rate simply because they were much older than the unvaccinated.

Seems about right.

Also, the excel file that contains all the original data summarises the mortality rates on Table 8:

Hard to imagine anyone could look at that and make a case that the vaccinated have a higher mortality rate than the non-vaccinated.

 

[dressic]

That ONS Age-standardised mortality rate chart seems to show that the rates are converging. Am I reading this correctly?

https://www.ons.gov.uk/visualisations/dvc1644/fig1/index.html

 

[Mendel]

That ONS Age-standardised mortality rate chart seems to show that the rates are converging. Am I reading this correctly?

You're comparing the green line and the blue line, and while technically true, "converging" is not the word I'd use since they seem to be set to maintain quite some distance for an unforeseeable time.

But we also know that vaccine protection gets worse with time (hence the booster shots), so for the vaccinated rates to move towards the unvaccinated rates somewhat (multiplied with incidence changes) is not surprising.

What we don't expect (and don't usually see) is for vaccinated people to do worse than unvaccinated.

 

[tadaaa]

In short, the biggest problem with this is that the number of unvaccinated people used to determine the rate depends on two other numbers:
#unvaccinated = population - #vaccinated

The closer #vaccinated gets to the population, the more errors matter, and there appear to be quite a few sources of those; which means the number of unvaccinated peopld is probably overestimated.

In addition, people with higher risks of exposure may also be more likely to be vaccinated.

Berenson has jumped to this kind of conclusion before.

Apparently they are (probably intentionaly) falling for the Simpsons Paradox

Explained here


Source: https://youtu.be/ebEkn-BiW5k

 

[Rory]

I recreated his original table with all groups included to check that he'd done it right:

He only included the second dose though, which gives the impression of a rising slope with a plateau at the end (perhaps open to interpretation as "temporary, with more rising to come").

With all groups included - as well as the full range of dates (his didn't start till March 19th - perhaps purposefully) - it's probably less impactful as far as an anti-vaccine agenda goes.

I think more clarifying, though, is showing what happens if we simulate vaccinating age groups in sequence, starting with the group with the highest mortality rate, as was the case in the UK.

This simulation shows how the overall mortality rate (MR) is affected as we vaccinate in stages from old to young:

I think this really clears up what is probably quite a tricky explanation for many people to visualise in their heads - plus it mirrors exactly what's happening in reality: initial increase in average mortality rate among the vaccinated followed by gradual decline and eventual return to the norm.

Likewise, we can see what effect this staggered, sequential program of vaccination would have on the unvaccinated:

That, of course, is assuming that the unvaxed are dying at a normal mortality rate - which, in reality, they're not. It seems like, actually, the anti-vaxxer in the OP's table should be showing much lower mortality rates for the unvaxed than it is, if he's wanting to support his cause.

 

[Mendel]

I had first learned learned about this in a very accessible blog post/twitter thread, which of course I can't find again. So here's a more technical and more thorough explanation by Mary Gregory, Deputy Director for Regulation in the Office for Statistics regulation (heavily excerpted, emphasis mine):

Article:

There has been a lot of talk about a UK Health Security Agency (UKHSA) technical report. It includes information on COVID-19 case rates in England for vaccinated and unvaccinated groups (Table 5). For some the immediate reaction to these data has been outright disbelief, others have used the data to support pre-existing, and incorrect, views that vaccines are not effective. Neither of these reactions is right. Understanding the issues properly is extremely complex, but what we do know with some certainty, is that while the vaccine will not stop the spread of the virus completely, it has been shown to help improve outcomes.

[..]

Are the two groups in question comparable? It is easy to see that there may be different behaviours in people who have had two vaccinations compared to those who have had none. One hypothesis is that those with two vaccinations are more likely to get tested, meaning the case rates will look relatively higher for this group compared to the unvaccinated group. There will also be different risks associated with each group, the vaccination programme prioritised vulnerable groups and frontline health and social care workers, so includes those who are more at risk of infection. We haven't seen evidence to quantify the impact of these risks and behaviours, but it's likely there will be an impact.

[..]

In the calculation of COVID-19 case rates the most significant choice is the decision on what population estimates to use to calculate the rates ("the denominator"). There are two obvious choices: the National Immunisation Management Service (NIMS) or the ONS mid-year population estimates. Each source has its strengths and limitations and we don't yet know the true figure for the denominator. [..]

There are many advantages to using NIMS, not least because it is consistent with international approaches to considering immunisations and allows for analysis which would not be possible using aggregate population estimates. However, we also know that NIMS overestimates the population. Similarly, there are strengths in using ONS mid-year estimates, but we know these can have particular difficulties for small geographic breakdowns. We also know that the time lag created by using mid-year 2020 estimates has a disproportionate impact in older age groups – for example, it means that in more granular age bands some older age groups show more people having been vaccinated than the ONS population suggests exist. [..]

Looking just at the adult population, Figure 1 shows the different results which come from using the two different denominator options for the population who have never had a COVID-19 vaccine.

[..] While we don't yet know the true figure for the unvaccinated population, this seemingly simple choice has a huge impact. It is particularly problematic in this circumstance because any error in the total population estimate is applied in its entirety to the unvaccinated population.

As an example, for the 70 to 79 population, the NIMS figure is just 4% higher than the ONS mid-year estimates (5.02 million and 4.82 million respectively). These figures can then be used in combination with the data on total people vaccinated from NIMS to estimate the total number of people not vaccinated. In doing this, the difference of nearly 200,000 in the total population estimates is applied entirely to the relatively small number of 70 to 79 year olds who are not vaccinated. It means the NIMS estimate for the unvaccinated population in the 70 to 79 age band is 363% higher than the estimate of those not vaccinated based on the ONS mid-year estimates. So, an estimate 4% higher at the 70 to 79 age band has led to an estimate 363% higher in the estimate of the unvaccinated population at that age band. This has a huge impact on the case rates for this group, and the conclusions drawn from the data.

Source: https://osr.statisticsauthority.gov.uk/communicating-data-is-more-than-just-presenting-the-numbers/

As I've already outlined in post #2, the biggest source of error is in figuring out how many people are unvaccinated, because obviously they're not registered anywhere.

 

[NoParty]

The authors of the data set came to a different conclusion.
http://www.ons.gov.uk/peoplepopulat...thsoccurringbetween2januaryand24september2021

That's absolutely true and really relevant.

I'd just add the reminder that that (alone) doesn't necessarily mean that it's definitive.

Just this morning I saw a Factcheck.org where they implied that Bernie Sanders (and AOC)
were misrepresenting a study done on the cost of "Medicare for All," by asserting that the
author of the study was not, in fact, drawing the same conclusion.

First, I love Factcheck.org & Politifact, etc., but that doesn't mean I'll always agree with
their conclusion...just as Snopes will occasionally deem something that is true as "Mixed"
or "Mostly True" because it wouldn't be fully true if some part were different (!) Ack!

The problem with the Factcheck story was that conclusions were dependent on future
(unknown) data. We all know that Sanders & AOC are at the far left of the Democratic Party,
but I don't think that Factcheck ever informed their readers that the person drawing
opposite conclusions, the author of the study, Charles Blahous, is a longtime staunch conservative Republican. It's a pretty safe bet that all parties are just viewing the same
(known) data through their own filters. Blahous probably understands the data a bit better
than the two politicians, but his track record would lead one to expect him to view the (unknown)
future data through a certain filter too. In other words, there's little dispute about the current
data, but what it will mean is a matter of opinion...and in this case the study author has a
slant, as they do. And his opinion should be weighed with that in mind, as their's should.

https://www.factcheck.org/2018/08/the-cost-of-medicare-for-all/

 

[Mendel]

As an example, for the 70 to 79 population, the NIMS figure is just 4% higher than the ONS mid-year estimates (5.02 million and 4.82 million respectively). These figures can then be used in combination with the data on total people vaccinated from NIMS to estimate the total number of people not vaccinated. In doing this, the difference of nearly 200,000 in the total population estimates is applied entirely to the relatively small number of 70 to 79 year olds who are not vaccinated.

Example (approximate numbers):

4.75 million people are vaccinated.
NIMS: 5.02-4.75=270 000 unvaccinated (2.7 × 100 000)
ONS: 4.82-4.75=70 000 unvaccinated (0.7 × 100 000)

1000 unvaccinated cases.
NIMS case rate 1000/2.7=370 per 100 000
ONS case rate 1000/0.7=1430 per 100 000

1430 is almost 4 times as much as 370, so at least one of these numbers is very wrong.
You need to be very careful when drawing conclusions here.

 

[Rory]

I had first learned learned about this in a very accessible blog post/twitter thread.

the biggest source of error is in figuring out how many people are unvaccinated, because obviously they're not registered anywhere.

That's referring to a different set of data - more about case rates than mortality rates - which has a different problem as far as accurate interpretation goes, and also a different solution (the NIMS/ONS issue you outlined, which isn't the problem for the table in the OP).

 

[Wade_UK]

Norman Fenton, Professor of Risk and Information Management at Queen Mary College, University of London, has made a video on Youtube on this subject.

EDITED after flag raised by moderator:

Prof. Fenton gives an example of Simpson's Paradox with Covid-19 deaths and cases data released from Public Health England for June 2021. These screenshots start from 12 minutes 49 seconds into the video.

The 'deaths per 100,000 cases' is calculated as (70/27192) * 100000 = 257 for vaccinated and (44/53822) * 100000 = 83. Then the ratio of 257/83 is approximately 3, or that 3 times more vaccinated people have died than unvaccinated people.

He goes on to say that the age ranges used in the totals can be a confounder, and so he gives the example of breaking up the age range into two ranges: those aged 50 and over:

This comparison between vaccinated and unvaccinated is now 907/3893 = 0.2, or to put it in terms of unvaccinated vs vaccinated, the unvaccinated deaths are 5 times the vaccinated.

He shows the under 50 age range as well:

This age range has a ratio of unvaccinated to vaccinated of 11/10 showing the unvaccinated deaths are 1.1 times higher than the vaccinated.

This illustrates the paradox in that taking the age ranges separately, the unvaccinated have more deaths per cases than the vaccinated, but using the totals across the whole age range the reverse appears to be the conclusion to draw.



Note that the body referred to by Prof Fenton, Public Health England, has been replaced by two separate bodies : UK Health Security Agency and Office for Health Improvement and Disparities.

 

[NoParty]

It's 26 minutes long...can you point to a couple of specific places that directly
address this discussion?

Is it safe to say that Mr. Fenton is interpreting things differently than most in his field?

I only ask because in the comments, below, someone asks him:
"Could you also please provide a list of the pre print servers which wont even accept your last 12 papers?"
and Mr. Fenton responds:
"I've put tweets up about most. MedrXiv and arXiv are now consistently rejecting our papers."

Since I've never heard of MedrXiv or arXiv, the exchange gives me more questions than answers.

Are you familiar with MedrXiv and/or arXiv? Thanks.

 

[Rory]

Norman Fenton, Professor of Risk and Information Management at Queen Mary College, University of London, has made a video on Youtube on this subject.

Ditto. Normal metabunk procedure would be to post relevant screenshots, transcriptions and timestamps so we know what we're looking at.

The Posting Guidelines are maybe helpful here.

(Not to worry: everyone gets hit by the PGs and no-click policy in the beginning.)

 

[Mendel]

That's referring to a different set of data - more about case rates than mortality rates - which has a different problem as far as accurate interpretation goes, and also a different solution (the NIMS/ONS issue you outlined, which isn't the problem for the table in the OP).

The mortality rate has the exact same problem.

The graph in the OP has a mortality on the order of 2 per 100 000; these 100 000 are population, not infections or cases, because otherwise the rate should exceed 500 per 100 000.

So when you compute (# of unvaccinated deaths) / (# of unvaccinated population), the errors in determining "# of unvaccinated population" have the same effect on that death rate as they do on the case rate in my example.

When you take studies where the numbers are known exactly (e.g. because they pair up vaccinated and unvaccinated people, ideally with the same sex, age, and from the same region), you get valid answers that show the vaccine benefits clearly.

 

[Rory]

That could have been an issue for the data in the OP - and clearly has been for other data - but Simpson's Paradox alone seems sufficient to explain this particular case.

 

[Mendel]

That could have been an issue for the data in the OP - and clearly has been for other data - but Simpson's Paradox alone seems sufficient to explain this particular case.

Wouldn't it be easiest to take the actual spreadsheet data then, and show the subgroup results? Or is that not possible?

 

[Wade_UK]

It's 26 minutes long...can you point to a couple of specific places that directly
address this discussion?

I've edited my post with screenshots from 12:49 in his video, where he describes a similar set of data to that of the claim

 

[Wade_UK]

Is it safe to say that Mr. Fenton is interpreting things differently than most in his field?

I don't know to be honest. His videos are on calculating risk with Bayesian methods.

 

[Wade_UK]

I only ask because in the comments, below, someone asks him:
"Could you also please provide a list of the pre print servers which wont even accept your last 12 papers?"
and Mr. Fenton responds:
"I've put tweets up about most. MedrXiv and arXiv are now consistently rejecting our papers."

Since I've never heard of MedrXiv or arXiv, the exchange gives me more questions than answers.

Are you familiar with MedrXiv and/or arXiv? Thanks.

Only in that they are pre-print servers in that they will publish online a manuscript before it gets peer reviewed and accepted by a journal. My understanding of this process comes from F. Perry Wilson's course on Coursera Understanding Medical Research: Your Facebook Friend is Wrong

 

[Rory]

Wouldn't it be easiest to take the actual spreadsheet data then, and show the subgroup results? Or is that not possible?

Yeah, not really possible - they didn't break the 10-59 year age range down into sub-groups. Though when they calculated the age-standardised mortality rates they used 5-year age groups (not shown) - hence why that displays higher mortality rates for unvaccinated.

Original spreadsheet here.

 

[Mendel]

Yeah, not really possible - they didn't break the 10-59 year age range down into sub-groups.

Probably for privacy reasons.
The paradox should be demonstrable with cases and hospitalizations, though.

 

[Rory]

The paradox should be demonstrable with cases and hospitalizations, though.

Yep, I believe it would be. I think the thing that's clear is that ONS population figures are more accurate than NIMS - and because the data in this thread uses ONS data the NIMS issue isn't a factor.

Probably for privacy reasons

Spot on. I emailed the people at Health Data and they replied:

We provide age-specific mortality rates for age groups 10-59, 60-69, 70-79 and 80+. The 10-59 age category is unfortunately too wide to infer what is happening. Vaccinated people are more likely to be older and unvaccinated people more likely to be younger, therefore increasing the all-cause mortality rates for the vaccinated.

Unfortunately we had to provide such a wider age band due to disclosure of vaccination status for younger ages for this weekly data, however in future publications we will produce these figures for smaller age bands but longer periods, so we can still adhere to disclosure rules, or age-standardised within the age bands. Unfortunately this was not possible here because we were publishing for weekly rates, where there are fewer deaths, therefore a higher risk of disclosure.

Our next update is scheduled for early December and will be added to the release calendar.

 

[Mendel]

Article:

Source: https://ourworldindata.org/covid-deaths-by-vaccination

Data source: https://www.ons.gov.uk/peoplepopula...ths/datasets/deathsbyvaccinationstatusengland

 

[FatPhil]

That'll be Delta data. Omicron "largely evades immunity from past infection or two vaccine doses" ( https://www.imperial.ac.uk/news/232698/omicron-largely-evades-immunity-from-past/ ) (and "fully vaccinated" in your charts corresponds to the same "two vaccine doses" (unless J&J obviously), not requiring a booster).

 

[Mendel]

That'll be Delta data.

The data in the OP goes through September, so this adresses the original claim neatly, even using the same data source, while avoiding the mathematical paradox that arises from not age-stratifying the data.

Omicron "largely evades immunity from past infection or two vaccine doses"

That claim is out of date; it is now known that cellular immunity persists against omicron. Compare the weekly data out of Chile and Switzerland on the same page (or see https://www.metabunk.org/threads/covid-19-coronavirus-current-events.11085/post-264367 ).

I know you are aware of this, because we discussed this 10 days ago, see your post and my reply at https://www.metabunk.org/threads/li...nt-to-get-the-covid-vaccine.12133/post-263776 .

 

[deirdre]

That claim is out of date; it is now known that cellular immunity persists against omicron. Compare the weekly data out of Chile and Switzerland on the same page (or see https://www.metabunk.org/threads/covid-19-coronavirus-current-events.11085/post-264367 ).

your link doesn't show what you seem to be claiming it shows.

you should have stuck with "the OP data only goes through september, so it is not looking at Omicron."