Study: Assessing correspondence between subjective reports and visual targets during attempted out-of-body experiences

[yoshy]

Assessing correspondence between subjective reports and visual targets during attempted out-of-body experiences

Saw this being passed around. A peer reviewed study, with a solid methodology for the most part, on out-of-body experiences tested if the individuals could OBE themselves and identify a target in another room. Drum roll........ They couldn't. But that's not the part worth discussing in my opinion... see the bolded portion below.

Claims of veridical perception often accompany out-of-body experiences (OBEs), but systematic evaluation under controlled conditions remains difficult. We examined whether descriptions produced after attempted OBEs corresponded to concealed visual targets beyond chance using computational semantic similarity. Twenty-one individuals affiliated with an institute specializing in OBE training attempted to perceive a randomly selected target displayed in a separate, inaccessible room. Targets were drawn from a standardized database and remained unknown to participants and researchers. Ten participants reported an OBE, and 13 provided target descriptions for analysis. Participant descriptions and target captions were converted into high-dimensional vector embeddings using a pretrained language model. For each trial, cosine similarity between the participant's description and each of 100 candidate targets was computed; the correct target was evaluated against the within-trial decoy distribution using a rank-based metric (our primary analysis) and standardized z-scores (a secondary, descriptive analysis). Control analyses confirmed that the pipeline identified correct targets when semantic correspondence was present, though these controls do not establish sensitivity to naturalistic human language. No individual trial showed statistically significant above-chance correspondence, and a combined rank-based test across all 13 trials likewise showed no evidence of above-chance performance. Two participants spontaneously reported accurate details about experimenters' locations and activities in an adjacent room. Although the target-identification results did not support anomalous cognition, the spontaneous environmental reports may have methodological value and warrant systematic investigation.

Including this in the abstract is very inappropriate, and it shouldn't have passed peer review. It's a glaring misuse of an abstract. An abstract should summarize the main findings of your experiment, i.e. the thing you actually planned and studied. Putting in anecdotes is not the proper use of an abstract. If they wanted to include this tidbit, I suppose they could put it in their discussion section or supplementary material, but absolutely not the abstract. It gives the appearance of credibility by including it in an abstract about an actual experiment with valid methodology. Another issue is the complete lack of context about these interviews. Abstracts should briefly include how they gathered the data presented in the abstract. This two sentence description doesn't describe essential information on the methodology for these interviews. (I haven't been able to find the full study document yet, so hopefully there are some answers in there.) This article from UVA provides some information, but not a lot.

• Were they planned, formal interviews?
• Did everybody have an interview?
• Were questions predetermined and uniform across test subjects?
• What do they mean "spontaneously"? Did they randomly shout "AND THERE TWO RESEARCHERS SITTING NEXT TO EACH OTHER AND ONE OF THEM LOOKED AT MY SOUL" in the middle of questions about the actual exam? If they were being interviewed, it seems likely they asked a question that would lead to them discussing seeing the researchers.
• Are there transcripts?
• Those who are familiar with the Ariel school UFO case know how easily interviewees can be influenced to give certain answers. Interviewers can do this completely unintentionally and even subconsciously. Transcripts are essential for determining if there were leading questions.

The UVA article hints at there being some sort of formal interview with the quote "But during interviews afterward", which I read as implying everyone was interviewed, but that's just an assumption.

Assessing the "accuracy" of the descriptions can be extremely subjective. On remote viewing threads, I have seen extremely vague drawings of shapes being called a "hit" for a remote viewer. Again, we need more info here.

My biggest criticism of including this in the abstract is what I'm calling the "anecdotal version of p-hacking". I'll explain what I mean and how it applies. One way p-hacking occurs: A researcher collects data with a specific test in mind. They run the test and get non-significant results. Bummer. At that point, researchers need to realize, but often don't, that any further statistical tests are invalid unless multiple hypothesis testing corrections are applied. Unfortunately, researchers will often keep running new tests until they inevitably find a significant value. Then they publish that test, maybe the first one too if we're lucky, but then make no mention of all of the other tests with non-significant results. I'll stress again that researchers who do this generally have no idea they're breaking the stats law so I am not making accusations of any deliberate fraud or deception. These researchers are doing the anecdotal version of p-hacking because they are reporting results of something that wasn't planned to be part of the study. Their actually study failed to produce the results they hoped for. Their abstract would be seen by many and quickly forgotten. But attaching this story to their abstract instantly makes it notable! (Judging by seeing it spread throughout forums and social media.) When there are literally no constraints on where and what to look for, one can easily find something coincidental and then attach greater meaning to it. Again, we can see examples of this regarding claims of "synchronicity". Since the interview was not planned to be part of the actual experiment, none of the usual safeguards against mistakes are present. All the types of bias (selection, confirmation, etc), lack of objective criteria, and so on are not controlled for here.

The way they present this feels like a "you can have your cake and eat it too" moment to me. In the UVA article, they accurately state:

The participants' unexpected observations were not part of the trial's predefined outcome measures and therefore can't be considered evidence for out-of-body experiences, or OBEs, UVA Health investigators say.

They say that it can't be admitted as evidence, yet they put in the abstract! The most read part of any scientific paper! Peer review should have had that removed from the abstract. Like I mentioned above, feel free to write something in the paper about it, since they plan to use their experience when formulating methodologies and experiments in the future. But putting it in the abstract makes it seem like credible scientific data. They also reference it in their interviews

"I hope this study encourages researchers to think creatively about how we test these experiences. The unexpected findings give us a potentially useful new direction, but they also need to be tested prospectively and under rigorous conditions before we can know what they mean," said researcher Marina Weiler, PhD, of the University of Virginia School of Medicine's Division of Perceptual Studies. "For me, the most important takeaway is that unexpected findings can sometimes help us ask better questions. We now have a specific idea that can be tested in future experiments, and that is exactly what science should do."

They have successfully switched the focus from the actual experiment that showed no evidence for the things they believe in to the unscientific interviews, which they talk about as if they are valuable, paradigm shifting data.

"I hope these findings encourage us to think more deeply about what they might mean for our understanding of consciousness and, ultimately, the nature of reality. If studies demonstrate that people can obtain accurate information that they could not have accessed through their ordinary senses, we would have to reconsider some of our assumptions about the relationship between consciousness, perception and the physical world," Weiler said. "At its deepest level, this research is not only asking whether out-of-body experiences are real. It is asking what we mean by 'real' in the first place, and whether our current understanding of reality is broad enough to account for everything human consciousness can experience."

Their actual findings were null results! Grrrrrrrrr

Note again that I haven't found the full paper yet, so some of my questions may already be answered therein. If I do, I will correct whatever possible.

 

[FatPhil]

13 trials

WIth that N, any kind of highly significant result, rather than being more convincing, would scream to me "terribly designed experiment that permits cheating (by tester or testee, it matters not)" - such powers would be oft noticed and well documented if they actually existed. What kind of p values are they claiming?

And I agree, those final sentences have no business being in the abstract. Including them looks like fishing.

(...for science, or for funding, or does it even matter, either's bad)

 

[yoshy]

WIth that N, any kind of highly significant result, rather than being more convincing, would scream to me "terribly designed experiment that permits cheating (by tester or testee, it matters not)" - such powers would be oft noticed and well documented if they actually existed. What kind of p values are they claiming?

And I agree, those final sentences have no business being in the abstract. Including them looks like fishing.

(...for science, or for funding, or does it even matter, either's bad)

They didn't bother reporting p-values since no tests came up significant. Thanks to your response, I just noticed they didn't include any quantitative results in their abstract at all. Even with non-significant p-values, abstracts generally contain some summary of results along with the non-significant p-values values. Not good practice to exclude everything like that.

 

[jon_eir]

They didn't bother reporting p-values since no tests came up significant. Thanks to your response, I just noticed they didn't include any quantitative results in their abstract at all. Even with non-significant p-values, abstracts generally contain some summary of results along with the non-significant p-values values. Not good practice to exclude everything like that.

really enjoyed this thread as this is an alien field to me can i asked how would you rate this rate the publishing journal?

• "Those who are familiar with the Ariel school UFO case know how easily interviewees can be influenced to give certain answers. Interviewers can do this completely unintentionally and even subconsciously. Transcripts are essential for determining if there were leading questions."

on this point have you any suggestions of research or paper covering this issue- I think its something most spot without having either a qualitative or quantitative foundation to build on.

 

[yoshy]

really enjoyed this thread

That's wonderful to hear

rate the publishing journal?

The journal is EXPLORE: The Journal of Science & Healing, which is inherently controversial since it's explicitly for non-mainstream topics. Its Wikipedia article paints a picture of the journal that isn't good.

Dean Radin, the journal's co-editor-in-chief,[8] published a paper in Explore on the effect on mood of eating chocolate which had been imbued with positive intent

That's just one example of pseudoscience they've published. Note that article was submitted by the co-editor-in-chief. Not a good sign.

The OBE study that I posted at last used a valid experimental design, but the peer review and editorial process of the journal appear to be non-existent.

on this point have you any suggestions of research or paper covering this issue

There's a wonderful thread on it in fact. Our very own @Charlie Wiser has written a phenomenal series on it. Note there's 8 sections linked at the top. The problematic interviews are the ones done by Hind and then Mack. If videos are more your preference, this YouTuber presents the info in an entertaining way (not everyone will enjoy his swearing and insulting jokes though). His videos heavily cite Charlie's article.

 

[NorCal Dave]

The journal is EXPLORE: The Journal of Science & Healing, which is inherently controversial since it's explicitly for non-mainstream topics. Its Wikipedia article paints a picture of the journal that isn't good.

Dean Radin, the journal's co-editor-in-chief,[8] published a paper in Explore on the effect on mood of eating chocolate which had been imbued with positive intent.

With Dean Radin as the editor in chief, I wonder if Joseph McMoneagle was one of the subjects. He was one of Puthoff and Targ's "remote viewers" at SRI during the early phases of what became Project Stargate:

In 1972, physicists Harold E. Puthoff and Russell Targ undertook a series of investigations of psychic phenomena sponsored by the CIA, for which they coined the term remote viewing.[57][58][59][unreliable source?] Among other activities, the project encompassed the work of consulting "consciousness researchers" including artist/writer Ingo Swann, military intelligence officer Joseph McMoneagle, and psychic/illusionist Uri Geller.[60]

en.wikipedia.org/wiki/SRI_International

In 1997, Robert Bigelow of Skinwalker Ranch fame, endowed a chair at UNLV for Consciousness Studies:

A professor of psychology at the Institute for Transpersonal Psychology in Palo Alto, Calif., has been hired as UNLV's first Bigelow Chair of Consciousness Studies, President Carol C. Harter announced Thursday.

Charles T. Tart will assume his duties in UNLV's College of Sciences this fall.

The Bigelow Chair of Consciousness Studies was established and funded by a gift of $3.7 million from Robert T. and Diane Bigelow of Las Vegas. The chair will initially be filled on an annual basis by distinguished visiting science scholars.

www.unlv.edu/news/release/unlv-hires-consciousness-studies-chair

At that time Dean Radin was at the CRL that was also part of UNLV:

The Consciousness Research Laboratory (CRL), a private research facility in Palo Alto, California, was originally founded at the University of Nevada-Las Vegas in 1993. It dropped its university affiliation in 1998 at which time it moved to its present location.

/www.encyclopedia.com/science/encyclopedias-almanacs-transcripts-and-maps/consciousness-research-laboratory-crl

Tart, who took the Bigelow Chair job had been at been at the ITP, now called Sophia University, (also in Palo Alto like CRL and SRI) and he was a research fellow the Institute for Noetic Sciences:

Tart, who has been at the Institute for Transpersonal Psychology since 1994, also served as visiting professor at the California Institute of Integral Studies in San Francisco (1994-95) and professor of psychology at the University of California-Davis (1966-94), where he is now professor emeritus. He is a senior research fellow at the Institute for Noetic Sciences in Sausalito, Calif.

https://www.unlv.edu/news/release/unlv-hires-consciousness-studies-chair

The Institute of Noetic Sciences has as its Chief Scientist, Dean Radin, who also did time at SRI:

Dean Radin, MS, PhD, is Chief Scientist at the Institute of Noetic Sciences and Associated Distinguished Professor at the California Institute of Integral Studies. He was awarded an Honorary DSc in yoga research from S-VYASA. Before joining the research staff at IONS in 2001, he held appointments at AT&T Bell Labs, Princeton University, University of Edinburgh, and SRI International.

https://noetic.org/science/scientists-fellows/

McMoneagle claims he worked with Radin while he was at the CRL, UNLV:

He reports that he worked with Dean Radin at the Consciousness Research Laboratory, University of Nevada, Las Vegas to seek patentable ideas via remote viewing for a "future machine" Radin conceived.[22] McMoneagle also says he has worked on missing person cases in Washington, San Francisco, New York and Chicago,[13] as well as employing remote viewing as a time machine to make various observations such as the origin of the human species. According to McMoneagle, humans came from creatures somewhat like sea otters rather than primates and were created in a laboratory by creators who "seeded" the earth and then departed.[23]

en.wikipedia.org/wiki/Joseph_McMoneagle

Yes, we evolved from Otters.

It's rather circular.

Interesting, author Damon Abraham is not part of an academic institute, but from Entangled Laboratories. And he has an app:

Entangled is an exploration into the role of consciousness in the Universe, and the ability to access information across time and space by tapping into it. Entangled consists of a mobile app connected to quantum hardware, which follows a set of rules called the Entangled Protocol.

The app allows anyone in the world to join. The quantum hardware looks for consciousness (psi) effects, and the Protocol allows thousands of experiments to run on the network.

https://entangled.org/

EDIT: It seems there are snippets of the paper available that indicate the subjects were in Brazile, so likely not including McMoneagle:

Forty individuals affiliated with the International Institute of Projectiology and Conscientiology (IIPC, Brazil) were initially screened. The IIPC is a Brazilian institution whose training programs emphasize meditation, visualization, and techniques described by participants as involving energetic self-regulation, to facilitate OBEs. Participants were asked: "If I asked you to induce an out-of-body experience now, what do you think your likelihood of success would be?"

 

[jon_eir]

A bit of unpacking from the last post (what a post btw) seems to seriously undermine this work—at least as far as the study and psi can be "debunked":

• Journal: EXPLORE explicitly publishes non-mainstream/alternative research, and its editorial leadership includes Dean Radin, himself a prominent psi researcher. That raises legitimate questions about editorial independence and standards, though it doesn't invalidate the paper by itself.
• Author: Damon Abraham is affiliated with Entangled Laboratories, an organization explicitly investigating consciousness/psi effects, rather than a conventional academic research institution.
• Methodology: Participants weren't a representative sample. They were recruited from a Brazilian organization that trains people in OBE-related practices and were screened partly according to their own claimed ability to induce an OBE.

I may have overstep the mark and none of those points alone disproves the findings, but together they give substantial reasons to scrutinize the methodology and claimed implications of paper and journal as a whole.

 

[yoshy]

A bit of unpacking from the last post (what a post btw) seems to seriously undermine this work—at least as far as the study and psi can be "debunked":

• Journal: EXPLORE explicitly publishes non-mainstream/alternative research, and its editorial leadership includes Dean Radin, himself a prominent psi researcher. That raises legitimate questions about editorial independence and standards, though it doesn't invalidate the paper by itself.
• Author: Damon Abraham is affiliated with Entangled Laboratories, an organization explicitly investigating consciousness/psi effects, rather than a conventional academic research institution.
• Methodology: Participants weren't a representative sample. They were recruited from a Brazilian organization that trains people in OBE-related practices and were screened partly according to their own claimed ability to induce an OBE.

I may have overstep the mark and none of those points alone disproves the findings, but together they give substantial reasons to scrutinize the methodology and claimed implications of paper and journal as a whole.

These things demonstrate that a strong bias is present, but they don't necessarily undermine a study per se. They similarly don't disprove any findings without direct evidence that the findings are bunk. On the other hand, when such biases exist, I agree that researchers need to be extra cautious and really scrutinize their plan. I think it's wise for researchers to be extremely transparent and, very importantly, they should "preregister" their study. Preregistering a study means you publicize the exact methodology planned for a study, all the way down to describing every statistical test you run. Preregistering would significantly reduce the potential for their biases to affect their work. I'll reiterate that many researchers who make statistical design mistakes do it because they simply don't realize they're making mistakes; I am NOT saying it automatically means deliberately deceptive or fraudulent behavior.

All that said, I think your third bullet leads to some good questions and doubt. I have more to say, but due to the length, I'm going to continue in a separate response.

 

[yoshy]

A kind benefactor was able to send a copy of the paper to me. We can answer some of my questions now! I have attached the paper for interested folks. Note that some words are getting combined when I paste them in from the PDF. So if you see any mistakes like that, it was likely on my end and not in the paper.

• Were they planned, formal interviews? Kind of answered. They were planned, and all participants were given the same questions. However, the paper itself does not give the exact wording of the questions or any transcripts (unless there's supplemental material that I don't have).

Interviews were conducted to document participants' experiences. Participants were asked whether they believed they had experienced an OBE, to describe the phenomenology of their experience, and to report any perceptions they believed were related to the target image. Responses were provided in free-form narrative format. Participants were also asked to rate their confidence in any reported perceptions on a scale from 0 (no confidence) to 5 (maximum confidence).

• Did everybody have an interview? Yes but only 13 attempted to describe the target. I assume they answered the other questions.
• Were questions predetermined and uniform across test subjects? I believe so. I can't say for sure unless they have them somewhere, perhaps in supplementary material.
• What do they mean "spontaneously"? Did they randomly shout "AND THERE TWO RESEARCHERS SITTING NEXT TO EACH OTHER AND ONE OF THEM LOOKED AT MY SOUL" in the middle of questions about the actual exam? If they were being interviewed, it seems likely they asked a question that would lead to them discussing seeing the researchers. I'm still not sure what they mean spontaneously. Mentioning this seems to be directly applicable to some of the interview topics. Even the question described as "report any perceptions they believed were related to the target image" could lead to them bringing up their OBE. Something as simple as "I didn't see a target but I saw a room with 2 members of the staff...".
• Are there transcripts? Not that I can find as of the moment.
• Those who are familiar with the Ariel school UFO case know how easily interviewees can be influenced to give certain answers. Interviewers can do this completely unintentionally and even subconsciously. Transcripts are essential for determining if there were leading questions. Without transcripts, I have no way to comment on this.

Here's the full description of the 2 participants they talk about in the abstract.

In addition to the quantitative analyses, two participants reported spontaneous observations that did not pertain to the designated visual target but are noteworthy. These reports were not part of the predefined outcome measures and were volunteered during post-session interviews.

Participant 7 stated that during her OBE, she perceived the two experimenters in the adjacent monitoring room. She described Researcher 1 seated to the left, focused on a computer screen, and Researcher 2 seated farther back in a corner, reading a book. According to contemporaneous session records, this description corresponded to the actual configuration at that specific time—namely, while the participant was in the induction room with the doors closed. Immediately after the session ended and the participant was escorted out, the seating arrangement changed. Importantly, neither researcher had adopted that specific configuration nor engaged in reading books during prior sessions.

Participant 13 independently reported perceiving the experimenters in the adjacent room while attempting to view the target. She described Researcher 1 seated on the right, taking notes, and Researcher 2 positioned on the left at the computer. This configuration was also accurate for that session and occurred only while the participant was in the other room connected to the EEG equipment, with the doors closed. As with Participant 7, this arrangement differed from the seating positions typically used across sessions.

These observations are presented for completeness, to illustrate findings that motivate further investigation. They were anecdotal, not systematically elicited, and were not subjected to formal statistic analysis. No other participants provided spontaneous environmental descriptions.

As @jon_eir mentioned, everyone was recruited from the IIPC. These are people who are training to be able to induce and control OBEs. I don't mind the selection bias in this case since they need to establish the existence of true OBEs in the first place! Selecting people who train maximizes the chances of finding evidence, presuming it exists of course.

Forty individuals affiliated with the International Institute of Projectiology and Conscientiology (IIPC, Brazil) were initially screened. The IIPCis a Brazilian institution whose training programs emphasize meditation, visualization, and techniques described by participants asinvolving energetic self-regulation, to facilitate OBEs.

Over three weeks, a series of experimental sessions was conducted at the IIPC headquarters in Foz do Iguaçu, Brazil.

The important fact here is that they did their testing at the IIPC facility. I'm not alleging they set up a way to cheat; the reason it's important is because it means the participants were likely quite familiar with the layout of the offices. One aspect that the researchers say is noteworthy is that both participants discussed the arrangement of the researchers, both mentioning a researcher sitting on the left on a computer. Since the participants know the building well, they would know the general layout of offices after seeing them so much. Thus, it's not especially notable or impressive to say how the person on the left is on a computer since they have prior information regarding where computers typically are.

According to contemporaneous session records, this description corresponded to the actual configuration at that specific time

These "records" aren't described anywhere. The researchers were keeping records on what they doing during the sessions? And these records were so specific and thorough that they can say "this description corresponded to the actual configuration at that specific time"? They say elsewhere that

As soon as participants were told they could begin using their preferred techniques to induce an OBE (for OBE induction techniques used by the participants, see,16 the researchers moved to Room 2, where they remained throughout the session to monitor the physiological recordings.

How exactly were they keeping detailed records of what they themselves were doing while also monitoring the participants? The participants had EEG monitoring them, which means it's even more reasonable to think one of the researchers is on a computer.

Importantly, neither researcher had adopted that specific configuration nor engaged in reading books during prior sessions.

mmmhmmmmmm. The configuration ("She described Researcher 1 seated to the left, focused on a computer screen, and Researcher 2 seated farther back in a corner, reading a book.") is so mundane that I'm curious why it would be noted at all in the first place. The claim that they never sat in the same configuration at all other than that just seems so unlikely. They were in that room for 20+ hours of observations! And why are they even recording this information when they didn't even envision or plan for the possibility of an OBE coming into that room?!

Similar questions for the claims regarding Participant 13. The configuration was R1 on the right taking notes and R2 on the left on computer.

As with Participant 7, this arrangement differed from the seating positions typically used across sessions.

With this one, they actually couch their language a little bit with "typically". How many different ways are there to sit in this room?! I don't understand!!!!!

Ok to sum up my thoughts on the two participant OBEs into the researchers' room. This is even weaker anecdotal "evidence" than I originally thought. The participants barely described anything more than "2 researchers were sitting in a room like this". There are no compelling or noteworthy occurrences at all. If a participant was able to say "R1 was doing a headstand while R2 playing Solitaire with the queen of diamonds on top of the stack", now that would be impressive. Being on a computer, taking notes, and reading a book are all completely mundane and common things to do while sitting in an office for hours.

A few random tidbits to end this comment. I haven't even closely evaluated the rest of the paper because I was all caught up on these 2 participants.

Inclusion judgments were made by the first author under full blinding: the identity of the target image was not known to any member of the research team until the data analysis phase, conducted several months after data collection concluded.

This is the kind of thing that really needs to be determined before the experiment.

A custom MATLABscript randomly selected one image from a pool of 100 standardized stimuli, selected from the BOSS database

Why such a small dataset of possible targets? I don't see a reason to limit it so much.

 

[FatPhil]

As with Participant 7, this arrangement differed from the seating positions typically used across sessions.

There's a bit of a stench to this. Why were the seating arrangements different? Was seating arrangement part of the test protocol? If not, how can we be sure you remembered correctly, because you didn't write it down as it wasn't part of the protocol? If it was part of the protocol, why did you violate it for these two participants? Likewise with their memory of whether the door was closed or not - sure, they may have been supposed to close the door, but did they always remember to do that? This just sounds so sloppy as an experiment. Not as bad as the pencil-scratching-on-paper number-guessing experiment, but still no better than the usual Puthoff/Targ/Swann/Geller SRI garbage that Randi et al. debunked. Stop making the same sloppy mistakes.

[EDIT: I'd not read your whole post when I replied, I notice that this particular thing elicited five exclamation marks from you!]