Cygnus Astraea · repository

Repositoryproject-specificationANALYSIS_SUITE.md

Document · ANALYSIS_SUITE.md

Reanalysis suite: bounded research pilots

Scope and limits of the offline analysis utilities in src/cygnus/analysis and the Colab pilot notebook.

Project-authored Public copy · paths redacted

Status: implemented in src/cygnus/analysis/ as in-memory, offline research utilities with synthetic tests. This is not an automated discovery engine, an end-to-end sky survey, a validated prior-art gate, or a calibrated false-alarm estimator. The candidate-facing dossier and ledger rules in AGENTS.md remain authoritative.

Purpose and inventory#

The suite asks whether reductions, measurements and source models disagree in a reproducible, falsifiable way. It supports five pilot investigations:

  1. timeseries.py: reduction-disagreement triage, requiring matched time coordinates and declared masks; ranking does not equal detection significance. Adjacent nonzero disagreements can form a long group under noisy trends: this is not a transit/event detector, and common-mode signals are invisible to it.
  2. pixels.py: compare independent apertures/background controls and event-minus-control centroid when available; localization, quality flags and independent epochs remain mandatory.
  3. imaging.py: inspect model-subtraction residuals with explicit coverage/masks and a caller-declared nominal per-pixel screening threshold; a residual alone is not a new source, and the threshold is not a calibrated discovery significance.
  4. crossmatch.py: epoch-aware optical–infrared positional/quality comparisons; absence of a match is a statement about the queried catalogs and depth only.
  5. nss.py: Gaia non-single-star orbit triage and alternative-hypothesis checks; a photocentre orbit alone does not establish a dark companion.

An additional CPU-only, NumPy-based ml.py helper ranks target feature vectors by their distance to a separate caller-defined control sample. This is unsupervised anomaly triage, not a novelty classifier or significance calibration; document feature definitions/units and split controls by target or sector to avoid leakage. It cannot find events common to every reduction or correct poorly chosen controls.

io.py reads a master CSV and checks a product's SHA-256 and optional byte length before returning its existing path. It refuses traversal, symlink escapes, missing files, non-retained rows and unrecorded SHA-256. SPOC light-curve FITS loading and bounded TESScut cadence-window loading are opt-in via cygnus[analysis]; timing keywords are returned as recorded rather than silently assuming BJD_TDB. Pixel windows enforce an element cap before copying the cube. Nothing in this package mounts Drive, invokes network/archive APIs, uploads data, deletes staging files, changes the ledger or labels an astronomical candidate automatically.

The recorded, not live-verified Tier-1 inventory (docs/tier1_pack/SEARCH_LOG.md) consists of selected TESS/Kepler LCs, six TESScut field/sector cutouts, Gaia DR3 cones and NSS extracts, limited AllWISE and ZTF extracts, selected Legacy Survey bricks and cutouts, two ESO HARPS example spectra and smaller archive samples. These are intentionally targeted examples, not an unbiased or unsurveyed sample. Some older drive_only manifest rows have SHA-256 but no MD5 or byte length; public snapshot JSON can be older than the current master CSV. Confirm the actual Drive inventory, integrity and catalog release before analysis. Never infer that the two HARPS samples belong to a selected Gaia NSS source.

Colab workflow and access#

notebooks/cygnus_reanalysis_colab.ipynb is a proposed interactive entry point. Review code before granting Colab Drive access. The Colab Google identity may not have the same visibility as the rclone [private Drive store] connection (which has drive.file scope); mount success is not pack availability. Identify the actual mounted Cygnus/data/tier1/ location explicitly and never assume that the rclone remote prefix itself is a filesystem path.

Install the project from a user-supplied, reviewed checkout or wheel in the runtime (pip install -e '.[analysis]'); the notebook does not silently clone arbitrary code or ingest a remote manifest. Use docs/tier1_pack/MASTER_MANIFEST.csv from that checkout and confirm product IDs against the current Drive copy. For any large FITS product, confirm disk/RAM first, copy only one selected product to Colab's temporary disk, verify its SHA-256 again after copying, and read the local copy. The notebook should persist only derived, non-secret output under a separately specified location, with no overwrite of originals. Colab runtimes are temporary and Drive I/O is rate- and quota-limited; prefer CPU for the small baseline, reserve GPU for a validated model with measured benefit.

API composition, one target at a time#

The functions are importable from cygnus.analysis; none queries a catalog or operates on a Drive path by itself. The current Colab notebook demonstrates only the light-curve pilot. To use the other four routes, provide and independently audit their inputs:

  • Pixels: select a checksum-verified TESScut product, pass a small cadence interval to read_tesscut_window(start=..., stop=...), exclude nonzero QUALITY cadences, specify nonoverlapping aperture and background masks, then call aperture_counterfactuals and event_control_centroid using predeclared disjoint event/control masks. Outputs are detector-cutout pixel offsets, not on-sky identifications; WCS/pointing and blended-neighbor checks remain external.
  • Optical residuals: obtain independently validated aligned image/model/noise arrays and true exclusion mask (not merely a JPEG cutout). Call triage_residual(..., band=..., threshold_sigma=...) with a threshold fixed on controls, inspect returned edge flags and repeat with alternate background/model. independent_band_consistency takes an externally justified independence flag and common pixel grid; neither function performs ZOGY or PSF matching.
  • Optical–IR: construct Source records using documented ICRS positions, actual Julian-year epochs, proper-motion conventions, uncertainty, magnitude system and quality flags. match_optical_infrared returns all pairs inside the specified radius and marks missing PM/uncertainty or ambiguous neighbors unknown. It omits parallax and full covariance and refuses polar/large-motion regimes; use Astropy or a full ephemeris for those cases. A nonmatch is not an upper limit.
  • Gaia NSS: populate NEGATIVE_CONTROLS from actual catalog/literature and solution-quality checks; absent checks remain unknown in triage_nss. Only construct SpectroscopicOrbit(primary_rv_is_measured=True) when a genuine primary-star RV semiamplitude and uncertainties exist. spectroscopic_mass_constraint returns a mass-function envelope and conditional edge-on minimum, not a mass measurement or posterior; Gaia photocentre orbit coefficients alone are insufficient.
  • Optional unsupervised ML: compute a small documented feature matrix for controls and separate targets, then call rank_control_trained_anomalies (NumPy nearest-control distance, capped at 1000 controls and targets). Control-target ID overlap is refused; field/sector-level leakage still requires human design. Keep missing features as unknown rather than filling them with arbitrary zeros. No score threshold or sigma significance is supplied.

Scientific protocol#

  • For each dataset record archive/release/product ID, manifest checksum, query, retrieval date, FITS time standard/reference, frame/epoch, units, masks, software versions, random seed and all investigated targets/controls. A name/coordinate match is not an identification without epoch propagation and coverage limits.
  • Freeze an analysis recipe and include positive controls, negative controls, instrumental/artifact controls and held-out sector/field when possible. Inject signal into small synthetic or justified real inputs, recover it through the complete search, estimate empirical null and selection efficiency; never turn a score into a sigma claim without calibration.
  • Event vetoes/alternatives: background/scattered light, pointing, CCD defects/cosmic rays, dilution and background eclipsing binaries for TESS; sky gradients, bright-star wings, masks and coadd seams for Legacy; artifact flags and blending for WISE; NSS known issues, photocentre/light ratio and luminous companions for Gaia.
  • Escalate only a reproducible lead surviving available tests; mark missing tests not_tested, not passed. The current src/cygnus/priorart.py mostly contains adapter stubs and is not a complete novelty screen. Search current release watchlists and literature before suggesting an unidentified source. Do not submit externally without authorization.

Validation and next requirements#

Run python -m pytest -m 'not network' from the checkout with test dependencies. In this development pass, python -m pytest -q -m 'not network' returned 132 passed, 2 deselected. Synthetic unit tests validate calculations and negative cases; they do not validate real sky false alarms, source completeness, archive coverage or physical interpretations. Before a science campaign, add checksum-pinned real controls, independent epoch/imaging, pipeline-wide injection/recovery, an epoch-aware known-object gate and review by domain experts. Report negative results and the actual region/epochs/selections searched.

Background references: Astropy BLS documentation, Lightkurve TESS background example, Legacy Survey DR10 processing, ESA Gaia DR3 NSS overview, ESA DR3 known issues, Google Colab FAQ.