Transients in the Palomar Observatory Sky Survey

[HoaxEye]

I got some of the code running and experimented with visualizing the filtering.

What the pipeline is doing is finding all the blobs of light in the planes, and then rejecting ones that are known stars or that don't look like points of light. The first step is filtering against the Gaia catalog. something like 90% of all detected blobs of light are rejected. Here I show the 15 most tenuous rejections, and five others at random.

Of note there WAS a cyan "spike", which was for rejecting blobs that were detected in things like diffraction effects from nearby bright stars. This had a couple of issues. Firstly, the range of a "nearby" star was coded as 90' when it should be 90". Correcting that surfaced some spike rejections where the star had simply moved a bit (the proper motion) and was detecting itself as a "spike" source. So I instructed my robot to add the Proper Motion calculations where possible to figure out where the star WAS in the 1950s.

There were a few other issues, but it think this has been a good exercise in validating and improving @HoaxEye's pipeline. I'm documenting everything I do. I'd like to get it into a trivially replicable Docker build.

Thank you.
"Firstly, the range of a "nearby" star was coded as 90' when it should be 90"
This is incorrect and a clear bug in MNRAS 2022. I have posted and commented about it. Correct range is 90' (arcsecs) and not 90" (arcmins). 90" is huge, size of a small continent, so it does not make any sense.

Quote from the paper, spikes' removal:

(a) For each SEXTRACTOR source, we look for counterparts in the USNO B-1.011 (Monet et al. 2003) in a circular region of 90-arcmin radius.

Source: https://academic.oup.com/mnras/article/515/1/1380/6607509

"So I instructed my robot to add the Proper Motion calculations where possible to figure out where the star WAS in the 1950s."
Proper motion part is addressed by the paper and implemented as-is in Vasco60:

For each one of the SExtractor sources fulfilling all the conditions described in the previous steps, we cross-matched them with Gaia EDR3 in a 180 arcmin radius, keeping all the counterparts in that radius. For these Gaia counterparts, we kept those having proper motion information, which was used to correct the position of the Gaia counterparts to the POSS I epoch. The adopted epoch for Gaia was J2016.0. SExtractor sources having a Gaia counterpart (corrected at POSS I epoch) at less of 5 arcsec were flagged as high proper motion sources and, therefore, removed from the list of candidates

 

[Mick West]

"Firstly, the range of a "nearby" star was coded as 90' when it should be 90"
This is incorrect and a clear bug in MNRAS 2022. I have posted and commented about it. Correct range is 90' (arcsecs) and not 90" (arcmins). 90" is huge, size of a small continent, so it does not make any sense.

Your code also uses 90', is this to replicate their code?

https://github.com/jannefi/vasco60/blob/5dbf452/vasco/mnras/spikes.py#L117

class SpikeConfig:
    rmag_key: str = "rMeanPSFMag"
    rules: List[Any] | None = None
    search_radius_arcmin: float = 90.0
    rmag_max_catalog: float = 16.0
...
        # If none within search radius (convert arcmin->arcsec), keep
        if not (dmin_arcsec <= cfg.search_radius_arcmin * 60.0 and m_near is not None):
            r2 = dict(r); r2["spike_reason"] = ""
            kept.append(r2)
            continue

 

[HoaxEye]

Your code also uses 90', is this to replicate their code?

https://github.com/jannefi/vasco60/blob/5dbf452/vasco/mnras/spikes.py#L117

class SpikeConfig:
    rmag_key: str = "rMeanPSFMag"
    rules: List[Any] | None = None
    search_radius_arcmin: float = 90.0
    rmag_max_catalog: float = 16.0
...
        # If none within search radius (convert arcmin->arcsec), keep
        if not (dmin_arcsec <= cfg.search_radius_arcmin * 60.0 and m_near is not None):
            r2 = dict(r); r2["spike_reason"] = ""
            kept.append(r2)
            continue

Yes. I fixed it in the calling code without touching the original SpikeConfig.

https://github.com/jannefi/vasco60/commit/6c9e393c3b79406ec3734661b44dce485994f3e8 - this is the fix commit

 

[Mick West]

Proper motion part is addressed by the paper and implemented as-is in Vasco60:

But it seems in your code, the 5-arcsecond Gaia cone match is done before the PM back-propagation, and the back-propagation is then only applied to the sources that matched — i.e., the ones already eliminated as known Gaia stars. The sources that drifted more than 5″ from their Gaia J2016 positions (the high-PM ones that actually needed epoch correction) never reach the back-propagation step, because they failed the upstream cone match.

My change was to do the back-propagation before the 5" check.

This came up because of:

some spike rejections where the star had simply moved a bit (the proper motion) and was detecting itself as a "spike" source

In those cases, the PM was more than 5", so they were not filtered by the 5" check.

 

[Mick West]

4. Astroquery Vizier class has an issue where setting ROW_LIMIT as class attribute doesn't propagate to instances. If your cone queries are only returning 50 rows, the veto is effectively not running. This will inflate survivor counts.

That's fixed in my version of the code, vasco/external_fetch_usnob_vizier.py

  Hunk 1 — default cap bumped:                                                                                                     
  -    row_limit: int = 20000,          
  +    row_limit: int = 200000,                                                                                                    
                              
  Hunk 2 — the actual row_limit bug fix:
  -    # Configure Vizier                                                                                                          
  -    Vizier.ROW_LIMIT = int(row_limit) if row_limit and row_limit > 0 else -1
  +    # row_limit must be passed to the Vizier() ctor — setting Vizier.ROW_LIMIT as a class attr is shadowed by the instance      
  default (50).                                                                                                                    
       cols = columns or _USNOB_COLUMNS                                                                                            
  -    viz = Vizier(columns=cols)                                                                                                  
  +    rl = int(row_limit) if row_limit and row_limit > 0 else -1                                                                  
  +    viz = Vizier(columns=cols, row_limit=rl)

Before that, it was returning 50 rows, after 32383 on my test tile

 

[Mick West]

Yes. I fixed it in the calling code without touching the original SpikeConfig.

https://github.com/jannefi/vasco60/commit/6c9e393c3b79406ec3734661b44dce485994f3e8 - this is the fix commit

No, it's not. It's reducing the PS1 bright star fetch from 35' to 3.0' (which should actually be 45' to cover the 60x60' tile). The 90' is still there.

 

[HoaxEye]

No, it's not. It's reducing the PS1 bright star fetch from 35' to 3.0' (which should actually be 45' to cover the 60x60' tile). The 90' is still there.

You're right to question the radii here, my bad. I said that this commit "fixed the MNRAS 2022 60 arcmin/sec bug" without re‑reviewing the actual spike code. That unit bug (arcmin vs arcsec) is real, but it was already addressed earlier in the MNRAS spike logic (the spike rejection itself is arcsecond‑based). I might have fixed it back in "Vasco30" project already and that commit history isn't visible in "Vasco60".

The commit I posted does not change the spike physics or the arcsec‑scale rejection rule. It only affects how bright stars are queried/filtered around candidates, and it mixes legacy tile‑level fetch radii with candidate‑level proximity checks. I should have documented this.

The 3′ value is intended as a per‑candidate proximity limit, not a tile‑coverage radius. The 90′ value is a legacy catalog prefetch radius. They serve different purposes. I should have explained that more clearly. The commit message/comment was misleading, thanks for noticing this.

 

[Mick West]

The commit I posted does not change the spike physics or the arcsec‑scale rejection rule. It only affects how bright stars are queried/filtered around candidates, and it mixes legacy tile‑level fetch radii with candidate‑level proximity checks.

It seems to be a query around the tile center, not stars.

That unit bug (arcmin vs arcsec) is real, but it was already addressed earlier in the MNRAS spike logic (the spike rejection itself is arcsecond‑based). I might have fixed it back in "Vasco30" project already and that commit history isn't visible in "Vasco60".

Vasco60 use the 90' as you have a default

search_radius_arcmin: float = 90.0

in class SpikeConfig: and cli_pipeline.py:610 does not override it to 1.5

 

[HoaxEye]

It seems to be a query around the tile center, not stars.


Vasco60 use the 90' as you have a default

search_radius_arcmin: float = 90.0

in class SpikeConfig: and cli_pipeline.py:610 does not override it to 1.5

search_radius_arcmin: float = 90.0 - yes, that's in the code. But:
- It is not used as the spike cutoff
- It is not the distance used in the rejection rule

Spike rejection is governed by arcsecond‑scale rules. Arcminute radii only decide which stars are even considered.

I think we're talking past each other because there are three different radii involved, and they serve different purposes.

• Spike physics is arcsecond‑scale (the MNRAS rule uses d_arcsec). That part is correct in the code and was already fixed earlier; the commit I linked earlier does not touch it.
• The 90′ value in SpikeConfig is a legacy catalog prefetch radius. It only controls how many bright PS1 stars are downloaded, not whether a candidate is rejected.
• The 3′ value introduced here is a per‑candidate proximity prefilter: stars farther away physically cannot generate spikes affecting that detection, so they are ignored for efficiency.

So: 3′ is not intended to "cover a 60×60′ tile", and it does not replace the arcsecond‑scale spike rule. I realise my earlier comment was misleading about this.

I tried to clarify all this with a new code comment:
https://github.com/jannefi/vasco60/blob/main/vasco/cli_pipeline.py#L585

# 3) Bright-star spike removal via PS1 (within ~3′, r<=16)
# The spike physics operates on arcsecond separations
# the "90 arcsec" mentioned in MNRAS 2022 is the intended scale (the "90 arcmin" wording in the paper is a confirmed typo).
# The 3 arcmin value here is a per-candidate prefilter to limit which bright stars
# are even considered; it does NOT replace or redefine the arcsecond-scale spike rule.
# A small margin is used while staying physically motivated for Schmidt-plate spikes.

 

[FatPhil]

Green stars are those with proper motion taken into account; yellow stars are others.

If that's your choice, can you please take into consideration the ~7% of males who simply cannot detect any difference between such yellows and greens.