Transients in the Palomar Observatory Sky Survey

[HoaxEye]

I got some of the code running and experimented with visualizing the filtering.

What the pipeline is doing is finding all the blobs of light in the planes, and then rejecting ones that are known stars or that don't look like points of light. The first step is filtering against the Gaia catalog. something like 90% of all detected blobs of light are rejected. Here I show the 15 most tenuous rejections, and five others at random.

Of note there WAS a cyan "spike", which was for rejecting blobs that were detected in things like diffraction effects from nearby bright stars. This had a couple of issues. Firstly, the range of a "nearby" star was coded as 90' when it should be 90". Correcting that surfaced some spike rejections where the star had simply moved a bit (the proper motion) and was detecting itself as a "spike" source. So I instructed my robot to add the Proper Motion calculations where possible to figure out where the star WAS in the 1950s.

There were a few other issues, but it think this has been a good exercise in validating and improving @HoaxEye's pipeline. I'm documenting everything I do. I'd like to get it into a trivially replicable Docker build.

Thank you.
"Firstly, the range of a "nearby" star was coded as 90' when it should be 90"
This is incorrect and a clear bug in MNRAS 2022. I have posted and commented about it. Correct range is 90' (arcsecs) and not 90" (arcmins). 90" is huge, size of a small continent, so it does not make any sense.

Quote from the paper, spikes' removal:

(a) For each SEXTRACTOR source, we look for counterparts in the USNO B-1.011 (Monet et al. 2003) in a circular region of 90-arcmin radius.

Source: https://academic.oup.com/mnras/article/515/1/1380/6607509

"So I instructed my robot to add the Proper Motion calculations where possible to figure out where the star WAS in the 1950s."
Proper motion part is addressed by the paper and implemented as-is in Vasco60:

For each one of the SExtractor sources fulfilling all the conditions described in the previous steps, we cross-matched them with Gaia EDR3 in a 180 arcmin radius, keeping all the counterparts in that radius. For these Gaia counterparts, we kept those having proper motion information, which was used to correct the position of the Gaia counterparts to the POSS I epoch. The adopted epoch for Gaia was J2016.0. SExtractor sources having a Gaia counterpart (corrected at POSS I epoch) at less of 5 arcsec were flagged as high proper motion sources and, therefore, removed from the list of candidates

 

[Mick West]

"Firstly, the range of a "nearby" star was coded as 90' when it should be 90"
This is incorrect and a clear bug in MNRAS 2022. I have posted and commented about it. Correct range is 90' (arcsecs) and not 90" (arcmins). 90" is huge, size of a small continent, so it does not make any sense.

Your code also uses 90', is this to replicate their code?

https://github.com/jannefi/vasco60/blob/5dbf452/vasco/mnras/spikes.py#L117

class SpikeConfig:
    rmag_key: str = "rMeanPSFMag"
    rules: List[Any] | None = None
    search_radius_arcmin: float = 90.0
    rmag_max_catalog: float = 16.0
...
        # If none within search radius (convert arcmin->arcsec), keep
        if not (dmin_arcsec <= cfg.search_radius_arcmin * 60.0 and m_near is not None):
            r2 = dict(r); r2["spike_reason"] = ""
            kept.append(r2)
            continue

 

[HoaxEye]

Your code also uses 90', is this to replicate their code?

https://github.com/jannefi/vasco60/blob/5dbf452/vasco/mnras/spikes.py#L117

class SpikeConfig:
    rmag_key: str = "rMeanPSFMag"
    rules: List[Any] | None = None
    search_radius_arcmin: float = 90.0
    rmag_max_catalog: float = 16.0
...
        # If none within search radius (convert arcmin->arcsec), keep
        if not (dmin_arcsec <= cfg.search_radius_arcmin * 60.0 and m_near is not None):
            r2 = dict(r); r2["spike_reason"] = ""
            kept.append(r2)
            continue

Yes. I fixed it in the calling code without touching the original SpikeConfig.

https://github.com/jannefi/vasco60/commit/6c9e393c3b79406ec3734661b44dce485994f3e8 - this is the fix commit

 

[Mick West]

Proper motion part is addressed by the paper and implemented as-is in Vasco60:

But it seems in your code, the 5-arcsecond Gaia cone match is done before the PM back-propagation, and the back-propagation is then only applied to the sources that matched — i.e., the ones already eliminated as known Gaia stars. The sources that drifted more than 5″ from their Gaia J2016 positions (the high-PM ones that actually needed epoch correction) never reach the back-propagation step, because they failed the upstream cone match.

My change was to do the back-propagation before the 5" check.

This came up because of:

some spike rejections where the star had simply moved a bit (the proper motion) and was detecting itself as a "spike" source

In those cases, the PM was more than 5", so they were not filtered by the 5" check.

 

[Mick West]

4. Astroquery Vizier class has an issue where setting ROW_LIMIT as class attribute doesn't propagate to instances. If your cone queries are only returning 50 rows, the veto is effectively not running. This will inflate survivor counts.

That's fixed in my version of the code, vasco/external_fetch_usnob_vizier.py

  Hunk 1 — default cap bumped:                                                                                                     
  -    row_limit: int = 20000,          
  +    row_limit: int = 200000,                                                                                                    
                              
  Hunk 2 — the actual row_limit bug fix:
  -    # Configure Vizier                                                                                                          
  -    Vizier.ROW_LIMIT = int(row_limit) if row_limit and row_limit > 0 else -1
  +    # row_limit must be passed to the Vizier() ctor — setting Vizier.ROW_LIMIT as a class attr is shadowed by the instance      
  default (50).                                                                                                                    
       cols = columns or _USNOB_COLUMNS                                                                                            
  -    viz = Vizier(columns=cols)                                                                                                  
  +    rl = int(row_limit) if row_limit and row_limit > 0 else -1                                                                  
  +    viz = Vizier(columns=cols, row_limit=rl)

Before that, it was returning 50 rows, after 32383 on my test tile

 

[Mick West]

Yes. I fixed it in the calling code without touching the original SpikeConfig.

https://github.com/jannefi/vasco60/commit/6c9e393c3b79406ec3734661b44dce485994f3e8 - this is the fix commit

No, it's not. It's reducing the PS1 bright star fetch from 35' to 3.0' (which should actually be 45' to cover the 60x60' tile). The 90' is still there.

 

[HoaxEye]

No, it's not. It's reducing the PS1 bright star fetch from 35' to 3.0' (which should actually be 45' to cover the 60x60' tile). The 90' is still there.

You're right to question the radii here, my bad. I said that this commit "fixed the MNRAS 2022 60 arcmin/sec bug" without re‑reviewing the actual spike code. That unit bug (arcmin vs arcsec) is real, but it was already addressed earlier in the MNRAS spike logic (the spike rejection itself is arcsecond‑based). I might have fixed it back in "Vasco30" project already and that commit history isn't visible in "Vasco60".

The commit I posted does not change the spike physics or the arcsec‑scale rejection rule. It only affects how bright stars are queried/filtered around candidates, and it mixes legacy tile‑level fetch radii with candidate‑level proximity checks. I should have documented this.

The 3′ value is intended as a per‑candidate proximity limit, not a tile‑coverage radius. The 90′ value is a legacy catalog prefetch radius. They serve different purposes. I should have explained that more clearly. The commit message/comment was misleading, thanks for noticing this.

 

[Mick West]

The commit I posted does not change the spike physics or the arcsec‑scale rejection rule. It only affects how bright stars are queried/filtered around candidates, and it mixes legacy tile‑level fetch radii with candidate‑level proximity checks.

It seems to be a query around the tile center, not stars.

That unit bug (arcmin vs arcsec) is real, but it was already addressed earlier in the MNRAS spike logic (the spike rejection itself is arcsecond‑based). I might have fixed it back in "Vasco30" project already and that commit history isn't visible in "Vasco60".

Vasco60 use the 90' as you have a default

search_radius_arcmin: float = 90.0

in class SpikeConfig: and cli_pipeline.py:610 does not override it to 1.5

 

[HoaxEye]

It seems to be a query around the tile center, not stars.


Vasco60 use the 90' as you have a default

search_radius_arcmin: float = 90.0

in class SpikeConfig: and cli_pipeline.py:610 does not override it to 1.5

search_radius_arcmin: float = 90.0 - yes, that's in the code. But:
- It is not used as the spike cutoff
- It is not the distance used in the rejection rule

Spike rejection is governed by arcsecond‑scale rules. Arcminute radii only decide which stars are even considered.

I think we're talking past each other because there are three different radii involved, and they serve different purposes.

• Spike physics is arcsecond‑scale (the MNRAS rule uses d_arcsec). That part is correct in the code and was already fixed earlier; the commit I linked earlier does not touch it.
• The 90′ value in SpikeConfig is a matching gate, a "don't bother evaluating the spike rules if the nearest bright star is farther than this". It is never reached in practice because the upstream fetch (radius_arcmin=3.0) only ever supplies stars within 3′. The gate is dead code relative to the fetch constraint.
• The 3′ value introduced here is a per‑candidate proximity prefilter: stars farther away physically cannot generate spikes affecting that detection, so they are ignored for efficiency.

So: 3′ is not intended to "cover a 60×60′ tile", and it does not replace the arcsecond‑scale spike rule. I realise my earlier comment was misleading about this.

I tried to clarify all this with a new code comment:
https://github.com/jannefi/vasco60/blob/main/vasco/cli_pipeline.py#L585

# 3) Bright-star spike removal via PS1 (within ~3′, r<=16)
# The spike physics operates on arcsecond separations
# the "90 arcsec" mentioned in MNRAS 2022 is the intended scale (the "90 arcmin" wording in the paper is a confirmed typo).
# The 3 arcmin value here is a per-candidate prefilter to limit which bright stars
# are even considered; it does NOT replace or redefine the arcsecond-scale spike rule.
# A small margin is used while staying physically motivated for Schmidt-plate spikes.

Edit: clarifying 90' value in SpikeConfig more precisely

 

[FatPhil]

Green stars are those with proper motion taken into account; yellow stars are others.

If that's your choice, can you please take into consideration the ~7% of males who simply cannot detect any difference between such yellows and greens.

 

[HoaxEye]

A new paper has been submitted to ArXiv:

Independent Recovery of Vanishing Sources on POSS-I Photographic Plates Using Automated Source Detection and Cross-Epoch Matching by Zachary Hayes
Source:
https://arxiv.org/abs/2604.04810
PDF attached. I haven't had the time to review it yet.

 

[John J.]

A new paper has been submitted to ArXiv:

Independent Recovery of Vanishing Sources on POSS-I Photographic Plates Using Automated Source Detection and Cross-Epoch Matching by Zachary Hayes
Source:
https://arxiv.org/abs/2604.04810
PDF attached.

I have a question concerning section 3.4., Temporal Correlation with Nuclear Tests (page 4 of paper and PDF), apologies in advance if it is naïve or a non-issue for you guys who have been following this subject:

The baseline rate—days more than 3 days from any test—is 13.3% (271/2,038 days).

Why is the baseline rate 3 days from any test, not 2 (i.e. excluding days -1 and +1)?

-Unrelated to this question, if the times of the relevant tests and plate exposures were known (I'm not sure they are), would it have been more methodologically elegant to use a window 36 hours each side of each test, as opposed to calendar days?
(This still takes it as axiomatic that it makes sense to include a span of time before successful tests, indicating a very high level of understanding of human affairs, or precognition, on the part of ETI).

 

[jdog]

I have a question concerning section 3.4., Temporal Correlation with Nuclear Tests (page 4 of paper and PDF), apologies in advance if it is naïve or a non-issue for you guys who have been following this subject:

The baseline rate—days more than 3 days from any test—is 13.3% (271/2,038 days).

Why is the baseline rate 3 days from any test, not 2 (i.e. excluding days -1 and +1)?

-Unrelated to this question, if the times of the relevant tests and plate exposures were known (I'm not sure they are), would it have been more methodologically elegant to use a window 36 hours each side of each test, as opposed to calendar days?
(This still takes it as axiomatic that it makes sense to include a span of time before successful tests, indicating a very high level of understanding of human affairs, or precognition, on the part of ETI).

I recall there being a comment in the initial nuclear-association paper that the authors blindly chose the +/- 1 day date range to register their target ahead of running the numbers, presumably to avoid p-hacking -- picking data after the fact to bring the association up to something statistically meaningful.

The problem, as you note, is that using dates is wildly imprecise, as all the observations are clustered around midnight Pacific time (and some plates where the 40-minute period crosses midnight are effectively both days) and the British and Soviet tests were in time zones offset by about 12 hours. I don't want to repeat the discussion from the other thread -- but we have both the timestamps for when the dates were exposed and the precise times of all the nuclear tests, so the analysis could have been done with regularized UTC times. However, having to pick an interval of hours rather than days for testing for associations would highlight the arbitrariness of the interval given the lack of any underlying physical mechanism.