Case Analysis: Data: (Representation, Bases, Sheets, Maps, Mining) and Methodologies/Workflows)

David Esp

New Member
Just throwing this out there...

Is anyone (else) thinking about or looking into "Case Analysis: Data: (Representation, Bases, Sheets, Maps, Mining) and Methodologies/Workflows)" ? Any recommended OSINT apps/workflows?
Not expecting anyone to have all the answers, but how about we swap notes here?

What exists, or could be assembled, to improve the quality, productivity and pleasure of correlating information from multiple sources? Example source types include witness statements, interviews, talks, rumours, ideas. Is there anything to assist the correlation of such information in terms of one or more story "universes", eg each "universe" representing a different hypothesis and most likely differing in terms of timelines. Tools to simplify their comparison. Mega-DIFF.

What is a good and practical methodology/procedure/workflow and app (software/service) based solution to support such an approach? Integrated AI-based solutions tend to be prohibitively expensive. Can something sensible be assembled out of readily available (though possibly specialised or even open source) components (apps or coding libraries) ? Does such a project/solution already exist? Do other people (eg you) already do that? Without falling down an informal research & development rabbit-hole?

Some possible angles:
  • Big picture: OSINT Resources
  • Standard databases and spreadsheets (with dynamic macros/functions/computations/dependencies)
  • Specialised timeline-supporting apps/tools.
  • Hyper note supporting apps/services such as (geeky, advancing) ROAM and (popular-appeal, stable) Notion.
  • Witness statement notation systems (useful as shorthand and as possible input to dynamic display and reasoning systems)
  • Any related data standards (eg in XML).
  • Coding from scratch (eg in a simple/dynamic/flexible object-oriented programming language such as Smalltalk).
  • Audio (eg Podcast) and Video (eg YT) analysis/mark-up strategy/tools (eg CinePlay, ELAN, FCPX...)
I think in most cases there's not enough information to make it worth having a tool-set to analyze it. Two counter-examples might be the Nimitz UFO case, where there's a variety of events, differing accounts, and overlapping timelines, and then UFOs in general.

People sometimes think that setting up a database will give some useful information, but it often turns into a collection of junk. Take MUFON, for example. It's a clunky database (now semi-private) that contains multiple user stories and photos that have obvious explanations. If there are any gems in there, they get lost. Significant expert input and continued curation is required.

I think a timeline comparison tool might be useful, and I'm surprised there isn't such a thing.
I wasn't proposing a MUFON-scale thing, just a kind of "smart documentation" process and tool to help cope with any investigation project involving multiple accounts (sometimes from the same witness at different times, maybe in different contexts) for a single incident/event (eg Nimitz, RFI). Also with a map or network to represent various kinds of association (ideally varying overt time eg years) between people.

For the latter purpose (associations mapping) have been considering CASOS ORA Lite, the repurposable Cytoscape and the more general-purpose yEd.

I once knocked up something in an AI language (Prolog) to help me get my head around the structure and details of an important complex engineering system (and made an earlier development in BASIC to help cope with house-buying legalese). But development takes time and in any case I wonder if anything better (and likely more standardised) now exists. At a reasonable cost!

Simplest for timelines would be to use tables (in databases or spreadsheets or even documents) where each column was a timeline. I have found various timeline representation/presentation apps but none so far that can be dynamically linked to a table (as above).

I wonder what other people do. Or where else to ask.
Mick - In the Comments following your YT video debunking of internal reflection artefacts in a program on the History Channel, I discovered your recommendation of Kyle Polich, who is a data scientist working with AI (as I have also been). I will try asking him.
The relevant YT link:
Several diagrammatic systems exist to represent the structure and inference flow of reasoned arguments.

The following paper presents one such. My reference to that system is not intended as any kind of favourite or recommendation but purely as one particular and accessible example, to illustrate the kind of thing I have been considering (and am wondering if others have also, along with any supporting standards, software libraries or applications).

The paper:
“Using Computer-Aided Argument Mapping
to Teach Reasoning”, by Martin Davies, Ashley Barnett, and Tim van Gelder.
This paper states: “Argument mapping is a way of diagramming the logical structure of an argument to explicitly and concisely represent reasoning.”