The Scientific Weight of Eyewitness Testimony

How many observations of supposed armed robberies happen, say, in one year in the US? I have really no idea, let me use n=1000. How many of those observations are confirmed to be actual armed robberies? Yet again I have no idea, there are many possibilities for errors: a cinema set mistaken for a real robbery, a prank, an hallucination, someone boasting he witnessed a robbery while he didn't etc. etc. etc.
Most common, I would expect, is someone witnessing an unarmed robbery, and thinking it was an armed robbery.

With UFOs, people don't often hallucinate or get pranked: judging by what ends up here (or in the blue book report), people actually see something in the sky that they can't identify; and that observation then turns into a UFO story as people try to make sense of it (which is human nature). It takes more mental effort to make sense of it as a natural terrestrial event, e.g. because you need to learn how your ATFLIR rotates to track a target, to identify aircraft at night, or what planets might be visible brightly.

There is another area where misperceptions occur with deadly regularity, and that's aviation. Aviators have died at night because they mistook lights in the sky for lights on the ground; they have died because they took a prominent line on the ground for a horizon in reduced visibility and lost control of themselves. These people had all heard of these phenomena in flight school. There is a lot of subconscious processing that goes on in the mind that stands between an observation and what a witness thinks they saw. You need hours of actual training in IFC as a pilot to learn to not mis-process these cues, and to instead rely on better cues (like the instruments in the cockpit).

When you think about it, the people most likely to see a UFO are amateur astronomers, yet they are the least likely to report one: and that's because they actually spent hours learning how to identify things in the sky.
The occasional UAP witness will not be able to understand enough of their observation to escape making a false narrative as they try to make sense of it.

Might there be actual extraterrestrial UFOs? Maybe. Will an eyewitness report be reliable enough to prove it? Definitely not.
 
Last edited:
@Mendel: My `logic' is about memory retrieval, not about observational errors. It has a basis in psychology by the way, just google `gist memory'.
People remember the `gist' of an event well, while their memories tend to be inaccurate on the details.

Your example of someone thinking they witnessed an armed robbery is an example of an observational error.
The memory retrieval of that person will still say `I saw an armed robbery' while the details of this memory are incomplete or inconsistent (provided that person never found out it actually wasn't an armed robbery of course).

Your assumption that most people who saw an `otherworldly craft' up close actually made an observational error is just that, an assumption. It is based on our current scientific world view that it's virtually impossible to cross interstellar distances no matter how advanced a civilization is. Or on a world view that if they did, they would not evade us and stay at a distance, but would attempt to make contact instead (preferably with our scientists, of course).

Ironically, the only way to test these world views is to actually study the UFO data instead of evading it while kicking in open doors about the scientific value of witness testimony. So I guess that's what I will continue to do. After all, real science is all about testing hypotheses with observations, not about defending a-priori world views with sweeping statements about the reliability of human testimony.
It has been studied... extensively. People just don't know about it because it happened before the History of Man began... i.e. before they were born.
 
2. Good scientific observation will control for the look-elsewhere effect, that is, for the tendency of people to underestimate the frequency of extraordinary events because they underestimate how many lottery tickets are actually being bought, that is, the chance of you winning the lottery is tiny, but the chance of someone winning the lottery is actually quite high.

An excellent point. Sorry I missed it earlier as I didn't read it properly through. Hence the late response.

Fravor's description of the tic-tac encounter is a case in point. It's indeed a rare thing for a trained pilot to be misled by encountering something non-extraordinary in a highly extraordinary manner, owing to special angles, rare difficulties to gauge distance, featurelessness of the object, parallax effects and other circumstances uniquely coinciding to compound the strangest of visual effects. In fact, it's even understandable why some ufologists would blame Mick for indulging in "mental gymnastics" to explain away what clearly came across as amazing feats of flight to seasoned professionals of flight (experienced military pilots).

However, out of the thousands of hours of flight by Navy pilots over the course of 20+ years it is in fact likely that a handful of times such a complex coincidence of unique effects takes place, creating a rare optical illusion. It's precisely these few encounters that would obviously make the cut for UFO eyewitness testimonies by trained military professionals.

That these encounters aren't remembered consistently owing to the fickle nature of human perception and memory is also a high likelihood. But it doesn't take away the possibility that these encounters were genuinely the result of unique and rare effects. And yet they don't prove aliens.
 
When you think about it, the people most likely to see a UFO are amateur astronomers, yet they are the least likely to report one

That’s interesting about amateur astronomers being the least likely to report UFOs. Do you remember where you saw that?
 
That’s interesting about amateur astronomers being the least likely to report UFOs. Do you remember where you saw that?
The point is, I didn't see any. I don't recall a single UFO report by an astronomer.

Here's an anecdotal argument for why this might be (I enjoyed the whole article, which includes two "UFO sightings" by the author):
Article:
Truth be told, amateur astronomers are lousy UFO reporters. We know the night sky too well to be fooled by after-dark sights such as planetary apparitions, brilliant meteors, International Space Station flyovers, Iridium flares, and other events that confuse and alarm the general public. We see Venus; John and Jane Doe see a hovering UFO.


A shorter version was provided by reddit commenter pelirrojo:
Article:
When armed with a telescope, UFO sightings become FO sightings.

That same reddit page mentions astronomer Clyde Tombaugh, who did report seeing UFOs.
 
Joseph Allen Hynek was an astronomy professor who operated a "Center for UFO Studies" with a toll-free hotline for 15 years.
Article:
While at Northwestern Hynek continued to consult on the Air Force’s Project Blue Book. In March, 1966, he investigated a well-publicized UFO sighting in Michigan. His conclusion that the UFO was nothing more than swamp gas angered many in Michigan, including Rep. Gerald Ford, who hauled Hynek before a congressional committee. The fiasco must have had some impact on Hynek because by October he was publicly calling for serious study of the “persistent and disturbing phenomenon” of UFOs and criticizing his fellow scientists for dismissing all UFO spotters as “hysterics or crackpots or cranks.” Newspapers around the country picked up on Hynek’s apparent about-face. He intensified his message when the Air Force closed Project Blue Book after a 1969 report by Dr. Edward Condon of the University of Colorado concluded that UFOs did not merit further inquiry. After 1969, Hynek was virtually alone in the scientific community in supporting the continued study of UFOs.

My estimate is that he was best positioned at the time to receive astronomer reports of UFOs, and if we wanted to quantify their proportion, we might do well to look there.
 
The point is, I didn't see any. I don't recall a single UFO report by an astronomer.
Got it. I found a handful of astronomer UFO sightings online, but I can’t make any substantive claims about how common they are so I’m happy to assume your assertion that amateur astronomers are least likely to report UFOs could be accurate.

Given that amateur astronomers virtually never see UFOs because the vast majority of UFO reports are misidentified objects like Venus, the Moon, satellites, balloons, and these are things amateur astronomers have seen many times, why doesn’t this same logic apply to military aviators? Particularly interceptor pilots, whose core training includes identifying airborne objects?

It seems based on recent high-profile sightings, military aviators (particularly interceptor pilots) are among the most likely people to see UFOs. But this is obviously a tentative claim.
 
It seems based on recent high-profile sightings, military aviators (particularly interceptor pilots) are among the most likely people to see UFOs.
I'm not sure what you are calling "recent".
The very recent past has seen a UAP reporting system established, possibly motivated by the idea that adversaries could be using drones to spy on US operations, and so we expect a close to perfect coverage of UAP sightings by this profession made in the line of duty to be reported; and they'd also come pre-discussed with the more common false identifications already eliminated from the process. So for that reason, I expect a greater part of these reports to make it into reputable reports, and especially government reports.

In the not-so-recent past, military aircraft were the only ones flying around with video recording devices hooked up to telephoto cameras with tracking systems. That increased both their media appeal and evidentiary value, making them high-profile.

In the larger context of this discussion, my point was that it takes a trained observer to assess a situation correctly and completely, and that anyone else is likely to only possess partial information and a (possibly false) narrative that substitutes the missing details to create a coherent story.

Now I would trust an Interceptor pilot to distinguish between a F/A 18 and a Mig-29, because that's what they trained for: they'd be paying attention to specific details and correlate it with the proper knowledge that normal people wouldn't have in mind. But identifying astronomical objects, or identifying aircraft at ranges where they won't come close to being engageable, is likely not part of their training, and that makes them unreliable eye witnesses for UFO sightings.
 
I'm not sure what you are calling "recent".
The very recent past has seen a UAP reporting system established, possibly motivated by the idea that adversaries could be using drones to spy on US operations, and so we expect a close to perfect coverage of UAP sightings by this profession made in the line of duty to be reported; and they'd also come pre-discussed with the more common false identifications already eliminated from the process. So for that reason, I expect a greater part of these reports to make it into reputable reports, and especially government reports.

In the not-so-recent past, military aircraft were the only ones flying around with video recording devices hooked up to telephoto cameras with tracking systems. That increased both their media appeal and evidentiary value, making them high-profile.

In the larger context of this discussion, my point was that it takes a trained observer to assess a situation correctly and completely, and that anyone else is likely to only possess partial information and a (possibly false) narrative that substitutes the missing details to create a coherent story.

Now I would trust an Interceptor pilot to distinguish between a F/A 18 and a Mig-29, because that's what they trained for: they'd be paying attention to specific details and correlate it with the proper knowledge that normal people wouldn't have in mind. But identifying astronomical objects, or identifying aircraft at ranges where they won't come close to being engageable, is likely not part of their training, and that makes them unreliable eye witnesses for UFO sightings.
Okay. Thanks. I have no doubt you know intimately what amateur astronomers are trained to identify.

It sure would be useful to have a recent military pilot discuss on this site how exactly they’re trained to visually judge distance, size, speed and shape of airborne objects during engagements. And whether they’re ever specifically instructed on how to visually differentiate Venus or a satellite or a balloon from a potential target. It’s largely taken for granted here that the top military training schools don’t cover these topics in depth. This might be true—the training they receive might very well be inadequate in this regard. But I also don’t think it’s far-fetched to suppose this kind of identification might be a significant part of their curriculum. And I haven’t stumbled upon a post on this site from someone who could actually say with some authority one way or the other.

Unfortunately my searches online for any military aviation curriculum relating to visual identification have come up empty. Does anyone have a link to some pertinent info?
 
Last edited:
I have no doubt you know intimately what amateur astronomers are trained to identify.
I've quoted you an amateur astronomer writing about it, just a few posts ago.
They're not "trained" (else they'd be professional astronomers), they accumulate experience from looking at the sky for hours upon hours as a hobby. Looking at the sky and identifying objects in it is what they do, and what they own equipment to do.

For fighter pilots, on the other hand, identifying objects in the sky is just incidental to their job. While amateur astronomers can indulge their curiosity at their whimsy, fighter pilots need to handle the workload in their cockpit, and thus their identification process is geared to quickly determine whether an object is threat, and if it is likely to be one, to determine its nature, and to respond adequately.

The best argument that Luis Elizondo put forth in his talk with Mick West about the identification skills of fighter pilots was an aircraft identification training card deck (that I took to symbolize what these pilors are trained to identify). I'm sure if there was a program that trained fighter pilots to identify objects that are not threats, Elizondo would know about it, and he'd mention it everywhere.
 
Okay. Thanks. I have no doubt you know intimately what amateur astronomers are trained to identify.

It sure would be useful to have a recent military pilot discuss on this site how exactly they’re trained to visually judge distance, size, speed and shape of airborne objects during engagements. And whether they’re ever specifically instructed on how to visually differentiate Venus or a satellite or a balloon from a potential target. It’s largely taken for granted here that the top military training schools don’t cover these topics in depth. This might be true—the training they receive might very well be inadequate in this regard. But I also don’t think it’s far-fetched to suppose this kind of identification might be a significant part of their curriculum. And I haven’t stumbled upon a post on this site from someone who could actually say with some authority one way or the other.
That would surely be useful, and an interesting thing to know. Until we have more data I'd like to note, however, that astronomers are scientists and as such should be rather well trained not to immediately go for the 'extraordinary' hypothesis where they see something unusual (exceptions happen, of course!). I really doubt fighters pilots are trained in skeptical thinking, and indeed there have been always been lots of reports about UFOs from the military. As an example in the 60s there were many reports of UFOs from Strategic Air Command bases, while now they are concentrated with Navy pilots, which also suggests some social/cultural effect is in play, rather than real UFOs. On the contrary I cannot remember on the spot a case of an UFO reported by a professional astronomer (Mars channels don't qualify, I think), nor by an amateur one. So I may be wrong, but Mendel's position looks to me to have a good prior probability of being true (even discarding the anedoctical 'evidence' from Mendel's, the quoted amateur astronomer, and my own experience)
 
Last edited:
Okay. Thanks. I have no doubt you know intimately what amateur astronomers are trained to identify.

It sure would be useful to have a recent military pilot discuss on this site how exactly they’re trained to visually judge distance, size, speed and shape of airborne objects during engagements. And whether they’re ever specifically instructed on how to visually differentiate Venus or a satellite or a balloon from a potential target. It’s largely taken for granted here that the top military training schools don’t cover these topics in depth. This might be true—the training they receive might very well be inadequate in this regard. But I also don’t think it’s far-fetched to suppose this kind of identification might be a significant part of their curriculum. And I haven’t stumbled upon a post on this site from someone who could actually say with some authority one way or the other.

Unfortunately my searches online for any military aviation curriculum relating to visual identification have come up empty. Does anyone have a link to some pertinent info?
https://fas.org/irp/doddir/army/fm3-01-80.pdf
I imagine there are manuals like this for every branch.
 
For fighter pilots, on the other hand, identifying objects in the sky is just incidental to their job.
I appreciate your opinion but can’t say that I share it. I think identifying an object in the sky would be a prequalification to firing on it. David Fravor, to name a well known example, was routed specifically to identify an object in the sky. He did so without weapons.
The best argument that Luis Elizondo put forth in his talk with Mick West about the identification skills of fighter pilots was an aircraft identification training card deck (that I took to symbolize what these pilors are trained to identify). I'm sure if there was a program that trained fighter pilots to identify objects that are not threats, Elizondo would know about it, and he'd mention it everywhere.
I don’t trust Luis Elizondo’s expertise (he seems kinda sketchy) but if the cards are an actual military training device for identifying different types of aircraft, I wouldn’t assume they represent the full extent of a fighter pilot’s training as it pertains to visual identification. Though it’s certainly a possibility. I’ll check out the video.
 
That would surely be useful, and an interesting thing to know. Until we have more data I'd like to note, however, that astronomers are scientists and as such should be rather well trained not to immediately go for the 'extraordinary' hypothesis where they see something unusual (exceptions happen, of course!). I really doubt fighters pilots are trained in skeptical thinking, and indeed there have been always been lots of reports about UFOs from the military. As an example in the 60s there were many reports of UFOs from Strategic Air Command bases, while now they are concentrated with Navy pilots, which also suggests some social/cultural effect is in play, rather than real UFOs. On the contrary I cannot remember on the spot a case of an UFO reported by a professional astronomer (Mars channels don't qualify, I think), nor by an amateur one. So I may be wrong, but Mendel's position looks to me to have a good prior probability of being true (even discarding the anedoctical 'evidence' from Mendel's, the quoted amateur astronomer, and my own experience)
I'm an amateur astronomer (40+ years) and I saw the Phoenix lights back in the '90s. Maybe that qualifies? It was a kind of like "V", viewed from an angle. It was odd, so I watched a while. In my binoculars I saw the lights didn't seem to be holding formation and were blinking. I remember thinking "planes" and losing interest. I was pretty surprised later when it became a thing. If I'd known, I might have dragged my 8-inch dob out for a closer look. Maybe I would have cared to remember which direction they were going, and how fast they were moving, too. I don't. I vaguely remember prepping to go out to Table Mesa to see Hale-Bopp with a friend, but that might have been a few weeks earlier or later.
 
That would surely be useful, and an interesting thing to know. Until we have more data I'd like to note, however, that astronomers are scientists and as such should be rather well trained not to immediately go for the 'extraordinary' hypothesis where they see something unusual (exceptions happen, of course!). I really doubt fighters pilots are trained in skeptical thinking, and indeed there have been always been lots of reports about UFOs from the military. As an example in the 60s there were many reports of UFOs from Strategic Air Command bases, while now they are concentrated with Navy pilots, which also suggests some social/cultural effect is in play, rather than real UFOs. On the contrary I cannot remember on the spot a case of an UFO reported by a professional astronomer (Mars channels don't qualify, I think), nor by an amateur one. So I may be wrong, but Mendel's position looks to me to have a good prior probability of being true (even discarding the anedoctical 'evidence' from Mendel's, the quoted amateur astronomer, and my own experience)
The Martian canals are a great example, though, of even highly trained observers being convinced they saw something extraordinary when dealing with relatively low quality imagery. And I don’t believe there was anything insincere in their beliefs.
 
I appreciate your opinion but can’t say that I share it. I think identifying an object in the sky would be a prequalification to firing on it. David Fravor, to name a well known example, was routed specifically to identify an object in the sky. He did so without weapons.

I don’t trust Luis Elizondo’s expertise (he seems kinda sketchy) but if the cards are an actual military training device for identifying different types of aircraft, I wouldn’t assume they represent the full extent of a fighter pilot’s training as it pertains to visual identification. Though it’s certainly a possibility. I’ll check out the video.
You reinforce my points.
If the purpose of a fighter pilot is to fire on things, why would it be necessary for them to identify an object once they determine it isn't to be fired on?

I wrote (and you quoted it) that the deck is symbolic for the full extent; but it symbolizes that the focus is on identifying military aircraft.
The point up for research is whether they are also trained to identify other aerial or space phenomena.

The Martian canals are a great example, though, of even highly trained observers being convinced they saw something extraordinary when dealing with relatively low quality imagery. And I don’t believe there was anything insincere in their beliefs.
Sincerity does not come into it. The confident eyewitness criminal misidentifications were not insincere.

The misidentification of Martian canals stems from astronomers not having any experience in identifying planetary features from great distances; they lack experience, have incomplete information, and take refuge in a narrative that gives them confidence in their identification, even though it is wrong.

Like I said, human nature.
 
Actually, this sort of error is not confined to humans.
Article:

Oct. 5, 1960: The moon tricks a radar​

A false alarm came when an early warning radar in Greenland reported to North American Air Defense Command headquarters that it had detected dozens of inbound Soviet missiles.

The report thrust Norad to its maximum alert level, according to the Union of Concerned Scientists, but officials later determined that the radar had been fooled by the “moonrise over Norway.”

Computer systems are designed (or, nowadays, trained) to interpret data according to the narratives designed into them, and they frequently get things wrong as well.
 
The Martian canals are a great example, though, of even highly trained observers being convinced they saw something extraordinary when dealing with relatively low quality imagery. And I don’t believe there was anything insincere in their beliefs.
The history of the Planet Vulcan, which was presumed to exist between the Sun and Mercury, is also instructive. As Uranus's orbit did not act like it should according to Newton, leading to the discovery of another planet out there tugging it out of position, so too Mercury's orbit was not quite exactly as predicted, and so it was widely believed that another planet, dubbed Vulcan, must lurk very close to the Sun to pull Mercury off its calculated orbit. Attempts to spot it, either during a transit or during solar eclipses, met with mixed success -- it was spotted and reported by a number of astronomers, professional and amateur. Eventually, failure to build any sort of orbital model for the planet from the various sightings, which would have led to Vulcan being spotted regularly since we would then know where to look, led most astronomers to decide it was not there after all. When Einstein's equations for gravity predicted the motions of Mercury as observed, that was the final nail in Vulcan's coffin. Vulcan was no longer needed to explain Mercury, and in fact could not be there or it would be pulling Mercury off of the orbit which Einstein calculated and which had been observed!

A couple of relevant points emerge from the history of Vulcan. First, numerous astronomers, from diligent amateurs to some of the top professionals in the field, saw what they hoped and expected to see -- a planet where Newtonian gravity said one had to be -- the only problem being that there was no such planet! Worth keeping in mind when the "Trained Observer" argument is put in play. Also, some of them continued to insist that they saw what they thought they saw, long after the consensus view became "There is no Vulcan to observe." * They made their initial reports at a time when reporting a sighting of the planet would be perceived as a career-boosting move (or prestige-boosting, among the amateurs), so that is not analogous to a modern UFO report. But those who stuck with their claims when it was widely acknowledged that their sightings had to have been mistaken observations of normal stars, sunspots and the like indicates that at least some astronomers, like some of everybody else, would assert the accuracy of their observations in the face of some skepticism and even ridicule.

* This would have been well before Einstein' work, but during the years when Vulcan still fit the math and theories, and only suffered from appearing much more intermittently and irregularly than it would have if it was actually there!

.71bkbkMZVsL.jpg
 
Wikipedia
Hendry was hired for CUFOS by the organization's founder, J. Allen Hynek, who was seeking a full-time investigator with scientific expertise and an open-minded attitude, and who was neither a debunker nor a "UFO believer".

As the chief investigator for CUFOS during most of the 1970s, Hendry personally investigated over 1000 UFO reports. He was able to find mundane explanations for the vast majority of UFO cases, but he also judged a small percentage of cases to be unexplained. One of the most famous "unexplained" cases he investigated was the Val Johnson Incident in 1979, in which a deputy sheriff in Minnesota experienced a "collision" with an unknown object which damaged his patrol car and left him temporarily unconscious. Hendry was the primary ufologist to investigate the case; in 1980 he debated the incident with well-known UFO debunker Philip Klass at a symposium held at the Smithsonian Institution.

He was reluctant to speculate as to origins of the unexplained cases, and argued they might be explainable with further data, leading some researchers to label Hendry a "closet skeptic".[1] At the same time, a few noted skeptics and debunkers who had praised Hendry's scientific rigor subjected him to strong criticism for his conclusion that a handful of well-documented UFO reports seemed to defy analysis, and might represent genuine anomalies. Hendry suggested that the criticism from both camps were little more than ad hominem attacks, since they typically paid little or no attention to the substance of his research.
Hendry's magnum opus was The UFO Handbook,[2] a guide for other UFO investigators. In the book, Hendry castigates many mainstream scientists for what he sees as their neglecting UFO studies, but he also had strong criticism for many amateur UFO investigators, who he thought did the subject more harm than good. Clark characterized Hendry's appraisal of ufology in general as "deeply pessimistic", concluding that the subject was all but paralyzed by infighting, a lack of cooperation and standardization, and dubious claims. The UFO Handbook even earned the praise of arch-skeptic Philip J. Klass, who in a review published in The Skeptical Inquirer described the book as "one of the most significant and useful books on the subject ever published."

I wish I could scan the entire book into PDF form and present it to the world. It's a forgotten out of print book. People think UFOs are new stuff, but they aren't the research has just been forgotten

 
Sincerity does not come into it. The confident eyewitness criminal misidentifications were not insincere.

The misidentification of Martian canals stems from astronomers not having any experience in identifying planetary features from great distances; they lack experience, have incomplete information, and take refuge in a narrative that gives them confidence in their identification, even though it is wrong.

Like I said, human nature.
I think we are in agreement. Even “trained observers” in their field can be fooled when information content is low, so we shouldn’t rely on an argument from authority and should strive to take observations scientifically with minimization of sources of systematic error, such as human perception.
 
Do you think, Mr. Wolf, that it also happens in reverse? "I saw something strange, not sure what" Later as the memory is blended with outside influence it becomes "I saw a UFO".

Not to be repetitive, but going back to 9/11, my wife was convinced that we saw the planes hitting the towers in real time that morning. She didn't have a misperception about what she saw, planes hitting the towers; her memory of exactly when she saw that image was convoluted. We didn't have the news on, our kids were young and watching Sponge Bob. Only after someone called us, did we switch to the news.

Just thinking that sometimes, it's the way the initial observation becomes clouded by memory.
After the Zapruder film was released, years later people claimed to remember seeing the JFK assassination live on TV. As we all know, it wasn't. You can take that with a grain of salt because I can't document that. I just remember reading about it or seeing it on a documentary program.
 
Last edited:
Back
Top