The Problem with Inadequately Reviewed Fringe Science

[Gary C]

Continuing to grapple with the demarcation problem, where does science separate itself from pseudo-science.

"Fringe claims sometimes do make it into peer reviewed publications as a result of flaws in the system. An example is cold fusion, which would not otherwise have been taken seriously. Nevertheless, it did not fool most experts.

Source - https://skepticalinquirer.org/exclusive/the-problem-with-inadequately-reviewed-fringe-science/

Full disclosure: I was unaware that the Younger Dryas hypothesis had been so thoroughly debunked. It's a problem of bad information that has gotten into wide circulation.

edited for grammar

 

[FatPhil]

Continuing to grapple with the demarcation problem, where does science separate itself from pseudo-science.

"Fringe claims sometimes do make it into peer reviewed publications as a result of flaws in the system. An example is cold fusion, which would not otherwise have been taken seriously. Nevertheless, it did not fool most experts.

Source - https://skepticalinquirer.org/exclusive/the-problem-with-inadequately-reviewed-fringe-science/

Full disclosure: I was unaware that the Younger Dryas hypothesis had been so thoroughly debunked. It's problem of bad information that has gotten into wide circulation.

Before I'd even reached the title of that article (because having titles way down the page seems to be stylish nowadays), I noticed in the side panel:

Other Articles from Special Articles
The Parallels between RFK Jr and Tofrim Lysenko: When Pseudoscience Infects National Leadership
https://skepticalinquirer.org/exclu...en-pseudoscience-infects-national-leadership/

which dovetails quite nicely - both are about bad science having more influence and reach than they ought to have. Alas the extra ingredient that gets added to the mix is that perhaps review was maybe irrelevant anyway, as it can simply be ignored if you're stubborn enough.

YDIH? (and for those who might read this with JS turned off - there's hidden text that follows)

Spoiler: don't encourage me...

The corresponding author was a fraud, and the paper was published with doctored images - you don't even need to pull out the "they believed in biblical inerrancy" argument to know that your expectations that the paper would survive scrutiny would be low.

[Allen] West is Allen Whitt — who, in 2002, was fined by California and convicted for masquerading as a state-licensed geologist when he charged small-town officials fat fees for water studies.

-- https://retractionwatch.com/2023/02/21/journal-investigating-sodom-comet-paper-for-data-problems/
In his favour, he fessed up when he'd been found out for bearing false witness:

Portions of five out of 53 images were published with minor, cosmetic alterations.

--- https://pubpeer.com/publications/37B87CAC48DE4BC98AD40E00330143#18

But when the paper was corrected, these were the 5 images that were fixed:

Some of the figure panels have been manipulated to remove the features irrelevant to the scientific content depicted in those (e.g. measuring tape, previous image labels, visible fingers etc.). The Authors recognise that this level of manipulation was inappropriate, and provide original images. As a result, the following panels were corrected and the original versions of the figures containing these panels are shown below for the record: Figure 3, Figures 4C & 4D, Figures 7C & 7D, Figures 8A & 8C, Figures 10A-C, Figures 11A-C, Figures 13D, 13F & 13H, Figure 14A & 14C, Figures 15A & 15B, Figures 18A & 18D, Figure 21A, Figures 22A, 22C & 22E, Figures 23A, 23C & 23F, all panels in Figure 25, Figures 26A & 26B, all panels in Figure 28, Figures 29B, 29D, and 29E, all panels in Figure 30, Figure 31A, Figure 32A, 32C, 32D & 32F, Figures 33A & 33B, all panels in Figure 34, Figures 38B-D, Figure 39A, Figure 40A, Figures 41A & 41C, Figures 42A & 42C, Figure 43, Figure 44C, Figure 45C, Figure 46C, and Figure 47B & 47C.

Additionally, scales were re-adjusted in the following panels: Figure 4C, Figures 11A, 11C & 11D, and Figure 26A. A number of panels was rotated 180 degrees so that original labels are not oriented upside down; this affects Figure 13H, Figure 22B, Figures 25C, 25E, 25F, & 25K, Figure 32A & 32B, Figure 34E, Figures 38C & 38F. Panels 14A & 14B had colour adjusted – these panels now show original images. Position of panels in Figure 51 was re-adjusted.

Finally, panel 15b was horizontally flipped in relation to the original and had the arrow pointing north obscured. It has now been replaced with a correct image.

/Ibid/ - https://pubpeer.com/publications/37B87CAC48DE4BC98AD40E00330143#29

And that's just the fraud bits - there's bad science with incorrect spelling too!

YDIH was righteously smote at the [EDIT-typo] hand of Bik, et al. . Which at least shows shows that post-publishing review can be as effective as the pre-publishing filter.

 

[JMartJr]

Fringe claims sometimes do make it into peer reviewed publications as a result of flaws in the system. An example is cold fusion, which would not otherwise have been taken seriously. Nevertheless, it did not fool most experts.

/

I'd look at that more positively. A paper was submitted and published claiming an exciting new discovery, attempts were made to replicate it, it could not be replicated and so everybody round-filed it and moved on.

That looks like the system working,rather than a flaw in the system!

(For those who do not speak colloquial English -- possibly even American colloquial English -- "round file" means "the trashcan/rubbish bin.")

 

[Mendel]

Source - https://skepticalinquirer.org/exclusive/the-problem-with-inadequately-reviewed-fringe-science/

Apart from specific bad methods that can be pointed out on a case-by-case basis (such as collecting material with bad provenience), the article mentions some red flags we've seen time and again:

Powell doubled down with another book in 2020 and an absurd accusation in 2022 that the YDIH was the victim of "premature rejection" by specialists in fields (such as impact physics and planetary science) that most of the YDIH proponents had never bothered to master but were attempting to revolutionize.

They did not cite any physics-based models by me or anyone else (Bunch et al. 2012), even to explain why they rejected this well-established approach and understanding.

I declined after discovering that his "university" was actually an unaccredited diploma mill.

They recently created a vanity journal, Airbursts and Cratering Impacts, where they self-edit, self-review and republish versions of their own retracted papers and manuscripts that have repeatedly been rejected by legitimate journals.

Science is about communication as much as it is about research, and when nobody in the field takes you seriously while you actively ignore pertinent contributions of that field, you're probably engaged in pseudoscience.

Everyone knows about Galileo. But you can only be Galileo if you're right.

 

[jdog]

Continuing to grapple with the demarcation problem, where does science separate itself from pseudo-science.

"Fringe claims sometimes do make it into peer reviewed publications as a result of flaws in the system. An example is cold fusion, which would not otherwise have been taken seriously. Nevertheless, it did not fool most experts.

Source - https://skepticalinquirer.org/exclusive/the-problem-with-inadequately-reviewed-fringe-science/

Full disclosure: I was unaware that the Younger Dryas hypothesis had been so thoroughly debunked. It's a problem of bad information that has gotten into wide circulation.

edited for grammar

I saw this blistering piece in the back-and-forth battle over the YDIH claims last year: https://www.sciencedirect.com/science/article/abs/pii/S0012825224002897?via=ihub

Evidence and arguments purported to support the YDIH involve flawed methodologies, inappropriate assumptions, incomplete comparisons, overgeneralizations, misstatements of fact, misleading information, unsupported claims, irreproducible observations, misinterpretation of fundamental data, logical fallacies, and selected omission of contrary information. These issues are discussed within broader themes in the conduct of scientific research. The burden of proof is on the developers and supporters of the YDIH to critically test their own hypothesis and to fully respond to a large, diverse body of critiques, observations and contradictory evidence. To date, they have failed to do this.

One of the challenges with these debates is that past a point they really do rely on domain expertise and math, applying the correct physics and chemistry to problems. (You see a ton of that in the climate science blogs, especially over CO2 absorption line claims.)

In the fields I pay attention to, what tends to happen is that every few years some senior scientist will right a review paper summing up the state of knowledge in a particular topic -- here's what we learned, here's what we rejected, here are the areas for promising research. Those review papers are part of the way our understanding of things gets solidified and the less-supported ideas get less traction. (For climate issues you also have national and international review committees.)

Some authors get frozen out from reviews and committees and sometimes publications and don't like it. I've also seen papers get published in legitimate journals that are only tangentially related to their topic, like a piece asserting an alternative set of facts for climate change that got published in a journal about flood control a few years ago (which seems to be because engineers tend not to like the implications of humans causing climate change).

 

[RTM]

Fringe claims sometimes do make it into peer reviewed publications as a result of flaws in the system. An example is cold fusion, which would not otherwise have been taken seriously. Nevertheless, it did not fool most experts.

/

I'd look at that more positively. A paper was submitted and published claiming an exciting new discovery, attempts were made to replicate it, it could not be replicated and so everybody round-filed it and moved on.

That looks like the system working,rather than a flaw in the system!

(For those who do not speak colloquial English -- possibly even American colloquial English -- "round file" means "the trashcan/rubbish bin.")

I think a problem is that something can be put forward and then rigorously tested and rejected within the scientific community; but the initial claim can enter into the realm of popular science and spread quickly, but the revised science doesn't get the same attention.

 

[JMartJr]

I think a problem is that something can be put forward and then rigorously tested and rejected within the scientific community; but the initial claim can enter into the realm of popular science and spread quickly, but the revised science doesn't get the same attention.

I agree, but that seems a problem with what happens after the science. Perhaps a problem with science communication?

 

[Ann K]

I think fringe claims can circulate widely among the fringe readers, the "QAnon" believers, the creationists, the rabidly political groups, whereas they gain no traction with scientists in the relevant fields. But once they are popularized, the cat is out of the bag, and debunkers are vilified among those who want to believe - I'm sure @Mick West can vouch for that! If a spurious report (perhaps an honest one which used too small a sample size, perhaps a deliberately "cooked" report) makes too sweeping a claim, it's likely to be shot down quickly, as cold fusion was, the same as for any "perpetual motion" machine that sounds too good to be true. . But I think there are probably numbers of papers out there that disagree with mainstream science in smaller and less significant ways, and those can linger for decades until they're discovered.

 

[FatPhil]

I think fringe claims can circulate widely among the fringe readers, the "QAnon" believers

Today's Q-Fringe news:

QAnon shaman sues Trump for $40 trillion and targets Musk, T-Mobile and Warner Bros in rambling lawsuit

https://www.independent.co.uk/news/...man-trump-lawsuit-capitol-riots-b2835709.html

Admins: yes, please feel free to delete this. I could have PM'd Ann, but I thought others might like a laugh too.

 

[Hevach]

Fringe ideas get into peer reviewed publications all the time, ans the peer review process shows them for what they are much of the time.

But when you hear an "expert" say they are excluded from the discourse, the thing they're usually leaving out is that they weren't left out, they were defeated in review and have offered nothing substantiative to merit a revisit.

In science, you absolutely *can* get a second turn at bat after failing in peer review, but you need to acknowledge that you did fail, and when you come back you're expected to have something new that really addresses why you failed, or everyone just puts the kettle on the floor and ignores you.*

The way it's expected to work, when everyone is engaging with the scientific method in good faith:

Theorist: "I have an idea!"
Peer: "I have found these problems with your idea."
Theorist: "I acknowledge this outcome and promise further study."
(Later)
Theorist: "I have compelling and substantiative new evidence!"
Peer: "This is different than last time but I still have these concerns..."
(Repeat for as long as the grant money holds out)

Egos do happen and sometimes this process gets personally contentious, but you almost never see an entire field unify against a theory whose proponents are engaging in good faith and bringing substantially new evidence. But usually if you look into fringe scientists who have had their theories excluded, what has happened is this:

Peer: "I have found these problems with your idea."
Theorist: Dies mad about it.

Fringe science has a surprisingly easy time sticking around for years when it is pursued by people who respect the process and genuinely desire scientific advancement and not personal book sales.

For examples:

Black Hole Stars were a crazy "the math technically works but..." sort of idea dating back decades in theoretical astrophysics, massive objects (potentially billions of solar masses) that look like a single star but are actually huge accretion structures where a large region of space is collapsing directly into a supermassive black hole. Some time back Hubble observed objects that one team argued were candidates. Peer review found issues and ultimately the paper was not supported. However new observations have been out forth that do address the concerns from the Hubble observations - these appear to be coherent single structures and not tight arrangements of many objects. There are still concerns like validating redshift assumptions and narrowing mass bounds but this is a case where a failed theory has come back with new information and is being given a real second consideration and regardless of how it plays out new information is the lifeblood of science.

Compare to Plasma Cosmology, which has never addressed fundamental issues with failed predictions that have existed since the idea was first proposed.

The theory that Velociraptor, Deinonychus, and Achilobator should be a single genus lasted for almost 30 years (and is the reason Chrichton's raptors are way too big). New fossils kept challenging the mainstream assumptions about dromaeosaurs and the clade kept being reorganized, and consolidation was always right there as a possible option. The theory has been abandoned but the old consensus was also thrown out in the process and a new one emerged, because everyone engaged in good faith and nobody was just backing their horse in the race. This repeats the Brontosaurus/Camarasaurus/Apatosaurus story kids' dinosaur books only tell half of - all three genuses still exist, just with none of the original member species in the genus they started in.

Compare to aquatic ape theory, which was never had evidence but instead used gaps in existing evidence to create a narrative from whole cloth. In this case those gaps have closed bit by bit with new finds and positively strangled by modern genetic evidence, until there just isn't a big enough gap for it to hide in anymore.


*-Old math joke I learned in college. Put an engineer and a mathematician in a room with a kettle on the floor and a stove. The engineer puts the kettle on the stove. Take the mathematician to a new room with a kettle on a table and a stove, and he puts the kettle on the floor, thus reducing the problem to one with a known solution.

 

[JMartJr]

Great post. (I remain a "lumper" when it comes to the dromaeosaurs. I suppose I'll have to "die mad about it! ^_^ But maybe if I hold out long enough the tide will run my way again!

 

[FatPhil]

I've had first-hand experience of a mathematical paper that failed to get published so many times the author nearly gave up hope. I did almost all of the "number crunching" for it, but the maths was completely way over my head and I turned down a co-authorship, it was his baby. It was rejected for many many reasons, which got more and more infuriating over time, as we did address the issues. After a year or so of trying to improve it in order to try and satisfy some reviewers, the criticism "it's too long" started to appear.

This was over 20 years ago, and I've mostly forgotten what it was about. In some ways, it was fringe, and maybe the initial reviewers were right - we'd taken a common algebraic structure, defined by several paramters, where, in order for the standard theories and operations to apply, some function of those parameters had to be non-zero, something somewhere depended on dividing by that discriminant. This new creation was an investigation into exactly what happens when you set that discriminant to zero. You can't do that - everything changes, it's a whole new structure with entirely different properties!?!? Erm, yes, that's why we think it's worth publishing as new maths.

Finally, after *several years*, one reviewer responded "this is such an important breakthrough, it will almost certainly become a new chapter in all the standard textbooks for the field", and we finally felt at least partly vindicated. However, the author was still told that it needed splitting into two or three papers. By that stage, my work was done, and I don't know what its final fate was.

Of course, pure mathematics isn't a science, so this isn't directly equivalent. However, it's just a warning that there will always be corner cases where things don't quite work how they were intended.

 

[Mauro]

*-Old math joke I learned in college. Put an engineer and a mathematician in a room with a kettle on the floor and a stove. The engineer puts the kettle on the stove. Take the mathematician to a new room with a kettle on a table and a stove, and he puts the kettle on the floor, thus reducing the problem to one with a known solution.

A better version was circulating in Italy

A mathematician and a physicist (or an engineer, as you prefer) are given a bottle of water and an empty kettle on a stove. The problem is: heat the water. Both the physicist and the mathematician pour the water in the kettle, turn on he stove and heat the water. They are now given a kettle filled with water on the stove and an empty bottle. The problem is: heat the water. The physicist turns on the stove, while the mathematician carefully pours the water from the kettle back into the bottle and reduces the problem to one with a known solution.

PS.: a cool joke, but I have an incredible respect for mathematicians, surely more than for engineers (and I was one), and even physicists.

 

[Mendel]

A better version was circulating in Italy

A mathematician and a physicist (or an engineer, as you prefer) are given a bottle of water and an empty kettle on a stove. The problem is: heat the water. Both the physicist and the mathematician pour the water in the kettle, turn on he stove and heat the water. They are now given a kettle filled with water on the stove and an empty bottle. The problem is: heat the water. The physicist turns on the stove, while the mathematician carefully pours the water from the kettle back into the bottle and reduces the problem to one with a known solution.

PS.: a cool joke, but I have an incredible respect for mathematicians, surely more than for engineers (and I was one), and even physicists.

The explanation is that mathematics is concerned with truth, not with doing things, and truth is method-agnostic. So the mathematician is really concerned with the question, "is it possible to heat this water", and pouring the water from the kettle back into the bottle (and then citing the previously acquired truth) is less effort than bringing the kettle to a boil.

That's why the really powerful laws of mathematics have as few specifics as possible, so they can be used as "truth shortcuts" in the widest possible set of circumstances.

(Though, if the solution to the first problem had been structured well, boiling water in a kettle would already have been proven as a lemma.)

 

[Ann K]

*-Old math joke I learned in college. Put an engineer and a mathematician in a room with a kettle on the floor and a stove. The engineer puts the kettle on the stove. Take the mathematician to a new room with a kettle on a table and a stove, and he puts the kettle on the floor, thus reducing the problem to one with a known solution.

I recall a thing I read years ago, a catalog of courses at an "alternative" educational place, mentioned that the course on people-piling had been cancelled, because the previous year they had mathematically determined the optimum number of people in a pile was two. They then continued: "A one-person people pile is, of course, degenerate".