# Project Cygnus/Astraea: test-architecture plan

> **Working draft — not finalized.** Saved for further review and refinement. Sections 1–9 below are the pre-implementation repository assessment; consult the incremental implementation note at the end for subsequently added research utilities. Do not read the original gaps as a fresh inventory or as certification of the new code.

## 1. Executive assessment

**Assessment from source inspection, not a fresh test run:** this is a credible verification scaffold, with unusually strong synthetic publication-boundary tests. It is not yet a scientifically calibrated or consistently fail-closed verification system.

The most consequential gap is between the **documented provenance and evidence policy** and what the current interfaces enforce. `AGENTS.md:13-19,63-76,89-119` requires traceable measurements, explicit “not tested” states, falsification, and restrained claims. `ANALYSIS_STACK.md:19-43,87-129` describes stronger ledger and prior-art guarantees. Yet `src/cygnus/ledger.py:54-86,249-284` permits measurement references without referential checks, and `src/cygnus/priorart.py:98-133` records mostly stub results without itself controlling promotion.

**Safety comes first.** The opt-in live MAST tests isolate their SQLite file but not their downloads: `tests/test_ingest_mast.py:28-79` passes a temporary ledger, while `src/cygnus/ingest/mast.py:149-168` resolves the configured scratch root. Pack retirement deserves the same priority because it deletes staged files based on manifest claims (`src/cygnus/ingest/pack.py:547-585`).

The target state is a small, fast, hermetic offline gate; explicit subsystem and security contracts; synthetic scientific truth and null calibration tied to each *implemented* method; and separately reported, bounded archive and release checks. Passing those gates would support confidence in specified behavior—not certify a whole-sky search, a detection significance, or an astronomical discovery.

**Scope of verification:** I inspected repository instructions, configuration, production and test sources, campaign material, and publication documentation. I did not edit files, execute tests, contact archives or Drive, or start services during the assessment. Historical pass counts in `ANALYSIS_STACK.md:132-142` and `docs/PUBLISHING.md:175-181` are documentation records, **not current results**.

## 2. Current-state map

| Subsystem and exact files | Contracts exercised now | Important gaps and classification |
|---|---|---|
| **Ledger:** `tests/test_ledger.py:6-67`; `src/cygnus/ledger.py:29-86,160-204,225-284,295-407` | Offline SQLite tests for product upsert/idempotency, run lifecycle, one measurement retrieval, and prior-art upsert. | The measurement test cites `proj-1` without registering it (`test_ledger.py:38-56`): it demonstrates storage, **not** product integrity. Schema has no relevant foreign keys and stores product IDs as JSON. Missing-reference, ambiguous-archive, invalid-state, rollback, read-only, reopen and schema-upgrade contracts are untested. There is no explicit migration/version mechanism to test yet; design it before changing the schema. |
| **Paths, instruments, basic I/O:** `tests/test_scratch.py:9-32`, `test_instruments.py:6-29`, `test_netio.py:13-23`; `src/cygnus/config.py:24-41`, `ingest/scratch.py:21-45`, `ingest/netio.py:66-185` | Offline root containment and age cleanup, instrument lookup and absent universal PSF cutoff, known-byte hashes and User-Agent. | No symlink/absolute-subdirectory containment matrix, clock-boundary cases, or fake-response tests for resume, ignored Range, caps and retries. `test_ingest_pack.py:64-75` uses a closed **loopback** port: no external traffic, but still a local socket and a possible port-reuse race. |
| **MAST:** `tests/test_ingest_mast.py:17-79`; `src/cygnus/ingest/mast.py:37-168` | Optional-dependency check plus two marked **live-network** TESScut smoke cases: nonempty result and unavailable sector. | No fake-client offline tests of zero observations/products, download rows, checksum, duplicate handling, failure, or filesystem boundaries. The live tests do not redirect scratch. `search_and_download_lightcurves()` searches `target_name` and takes limited products; `src/cygnus/ingest/tier1.py:285-384` separately documents why its TESS collector uses cone/SPOC/nearest-target selection. Do not assume these two paths have identical semantics. |
| **Pack and Tier-1 ingest:** `tests/test_ingest_pack.py:25-169`, `test_tier1_config.py:11-39`; `src/cygnus/ingest/pack.py:378-585`, `ingest/tier1.py:1355-1412` | Offline register/budget/manifest basics; failed fetch through loopback; an optional local-`rclone` mirror verify→retire cycle and unverified refusal. Tier-1 registry, budget and name-cache checks. | Assertions such as `b"mast," in ...` and `"n1" in ...` do not establish exact records. Malformed manifests, missing/changed remote files, subprocess failure, spaces in paths, stale verification, absent hashes, and traversal lack tests. Source inspection identifies particularly urgent checks: verifier can report `ok` with **zero checked rows** (`pack.py:515-531`); retirement’s guard excludes hashless local rows (`:555-571`); it trusts a note rather than freshly validating the remote; and `PackBuilder.delete_local()` looks for `verified_remote` while the verifier writes `verified_md5` (`:379-430,529`). Those are behavioral risks requiring tests **and** reviewed hardening, not merely more assertions. `tier1.py:1373-1385` includes generation time in its config hash, contrary to the intended stable resumability in `ANALYSIS_STACK.md:79-83`. |
| **Prior-art gate:** `tests/test_priorart.py:12-38`; `src/cygnus/priorart.py:23-133` | Offline enumeration and explicit `not_tested` defaults, including missing SkyBoT epoch inputs. | SkyBoT is the sole implemented service adapter; other services remain stubs even with network allowed. Results record `domain=...`, not the full query/epoch/radius/catalog version. `apply_gate()` neither changes candidate audit/evidence nor appears wired into a production promotion path. Tests must not imply that the designed five gates in `ANALYSIS_STACK.md:87-119` already operate. |
| **Candidates and reporting:** `tests/test_candidate_record.py:17-54`, `test_dossier.py:21-59`; `src/cygnus/candidate_record.py:43-105,130-202`, `reporting/dossier.py:14-99`, `reporting/leads_board.py:20-43` | Offline validation of audit-state vocabulary, non-passed demotion, minimal provenance, rendering, and evidence-level ordering. | Several checks are substrings or relative positions, not parsed fields/rows. `validate_strict()` establishes nonempty provenance and audit, **not** completeness, authentic supporting observations, prior-art clearance, or independent corroboration. Leads ranking is expressly evidence-level-only scaffold (`leads_board.py:1-5`). |
| **CLI:** `tests/test_cli.py:6-30`; `src/cygnus/cli.py:25-93` | Healthy in-process `doctor --full` JSON and dossier stdout. | Missing malformed/missing record, unwritable scratch, ledger-open failure, argument/exit/stderr, and subprocess `python -m cygnus.cli` tests. `doctor` writes a probe and opens a writable ledger; it is not a harmless read-only inspection command. `pyproject.toml:5-26` declares no installed `console_scripts` entry point, so test the **module** entry point now; add/test an installed command only if one is introduced. |
| **Publication:** `tests/test_publish.py:1-470`; `src/cygnus/publish/{schema,safety,build,sources,snapshot,serve,markdown}.py`; `docs/PUBLISHING.md` | The strongest area: synthetic worktree, schema and path confinement, redaction, draft/withdrawal and restricted-content boundaries, read-only ledger build, checksum/catalog consistency, claim disagreement, generated links/assets, and local preview HTTP. These mix offline units, subsystem build integration, and a **loopback local-service** test. | One 470-line module spans distinct contracts and skips at module import if `jinja2` or YAML is absent (`test_publish.py:20-21`). Split by boundary while retaining shared fixtures and adversarial cases. Test failed-build atomicity and additional publication inputs; do not equate text leak scanning or an operator-set redistribution flag with proof that a binary is secret-free or licensed. Tests do not validate a deployed host. |
| **Scientific and campaign work:** `campaigns/run_wasp12_sector20.py:1-122`, `campaigns/wasp12_sector20/REPORT.md:21-68`; planned modules in `ANALYSIS_STACK.md:60-72` | An existing **campaign-specific, checksum-pinned known-signal recovery script**, not a pytest calibration suite. Its report explicitly limits the 20-permutation diagnostic and records unperformed artifact checks. | Do not say all scientific work is absent: this bounded validation exists. But the planned `domains/...` algorithms and injection/null modules are not present under `src/cygnus/`; their test packages are roadmap work contingent on implementation. A single known recovery cannot establish detection efficiency or a calibrated false-alarm rate. |
| **Manual probes and discovery:** `tests/probe_*.py`; `pyproject.toml:14-32`; `tests/conftest.py:10-38` | Three top-level executable probes are not named `test_*.py` and are outside normal configured pytest discovery. `conftest.py` adds `src` to imports; scratch and ledger fixtures are opt-in. Default pytest excludes the `network` marker. | `probe_data_check.py:23-32` explicitly uses the default worktree ledger and TESScut scratch: keep it manual, prominently unsafe for normal CI, and preferably relocate/document it. No repository CI workflow or lint/type configuration was found. Optional publication tests can all skip unless the site extra is installed. These are source/config observations, not a freshly executed collection report. |

A concrete provenance warning already recorded by the project is the unresolved measurement/product ID mismatch in `docs/PUBLISHING.md:194-200`; the probe constructs a shorter ID at `tests/probe_data_check.py:64-73`, while TESScut registers camera/CCD in its ID at `src/cygnus/ingest/mast.py:157-165`. Treat the document as a recorded observation, not as a substitute for independently inspecting the live ledger.

## 3. Target architecture and invocation

| Layer | Contents and permitted resources | Failure meaning |
|---|---|---|
| **Fast offline unit** | Ledger, records, renderers, path rules, parsers and fake HTTP/archive clients; only per-test temporary filesystem/SQLite; no external sockets, credentials, real scratch, Drive, or production `publish/`. Default local/PR gate. | Deterministic product-contract regression. Unexpected network or real-path access is a test failure. |
| **Offline subsystem contracts** | Complete synthetic pack, publication, CLI-subprocess and local SQLite workflows; disposable subprocesses and temporary trees; optional bounded **loopback** HTTP tests marked `local_service`. | Cross-component or security regression; distinguish absent optional executable from an actual assertion failure. |
| **Synthetic scientific calibration** | Small, versioned signal/null/artifact generators and seeded parameter grids for each **implemented** analysis method; no archive requirement. Separate fast examples from `slow` calibration grids. | Method failed a specified numerical/robustness contract; not evidence that real sky false alarms are calibrated. |
| **Pinned archival calibration** | A few justified observations identified by archive/product/release, SHA-256 and acquisition record; a separately provisioned local cache, never implicit download in default pytest. | Failure after checksum verification is analysis/integration evidence; unavailable fixture is explicitly unrun. |
| **Opt-in live archive smoke** | Narrow `network` cases with temporary ledger **and** scratch, bounded requests/size/time, service-aware status, no Drive or bulk Tier-1 run. | Separate outage, authentication/rate limit, archive-interface drift and product-code failure in reports. Neither an outage skip nor an outage failure proves the code passed. |
| **Manual operations; release/nightly** | Move/document `probe_*.py` away from routine tests. Run broader synthetic grids, optional local-`rclone` mirror, pinned fixtures and publication/security checks as explicit release jobs. | Operational evidence recorded with environment/version and unrun checks; not folded into a misleading single green badge. |

Retain default offline discovery; add markers only for meaningful resource or interpretation boundaries (`network`, `local_service`, `slow`, `archival`, possibly `security`). Do not mark every unit test. A test using loopback is offline with respect to archives, but should not be described as “no sockets.”

## 4. Risk-ranked roadmap

Each package below names its **test work** separately from likely **production behavior changes**. Runtime estimates are targets to validate, not measured timings.

| Phase / package | Precise work and acceptance | Likely production change; dependency; profile and false-confidence risk |
|---|---|---|
| **S0-A — make isolation fail closed** | Make ledger, scratch and publication roots temporary by default in relevant tests; add a sentinel outside the test tree and assert it remains untouched. Fix both marked MAST tests to inject temporary scratch; bound their request and output. Replace the closed-port race with a fake transport where testing request behavior. Audit every test invoking defaults or cleanup. | May require explicit destination/config injection in TESScut and a guarded scratch resolver (`config.py`, `mast.py`, `scratch.py`). First dependency for all other gates. Fast offline; live cases separately opt-in. Do not infer isolation merely because a ledger is temporary. |
| **S0-B — verify before any retirement** | Synthetic manifests covering malformed JSON/rows, duplicate or empty paths/hashes, zero checked rows, remote missing/mismatch, stale note, changed local file, filenames containing spaces, rclone missing/timeout/nonzero/malformed output, repeated retirement, and `../`, absolute, Windows-drive/UNC and symlink escapes. Assert *all* local files and ledger rows remain unchanged on failure. Exercise both `retire_local_copies()` and `PackBuilder.delete_local()`; retain one local-mirror `rclone` contract test. | Introduce validated manifest model, canonical containment, explicit verification evidence bound to manifest content/remote and a nonempty checked set; reconcile verification tokens or retire the duplicate delete API. Make deletion transactional or preflight every row before unlink. `pack.py:378-430,478-585`; `tools/upload_verify_retire.py` workflow. Offline fake-process tests fast; mirror optional. Avoid “note contains verified” as a security oracle. |
| **S1-A — ledger provenance and schema evolution** | Test missing run/candidate/product, same product ID in two archives, product ID resolution, duplicate/upsert semantics, checksum change, read-only writes, close nonexistent/already-closed run, rollback on partial failure, reopen persistence, old-schema upgrade and incompatible future schema. Specify whether candidate-less measurements and unchecksummed metadata records are legitimate; assert those exceptions explicitly. | Decide normalized product references (`archive`, ID or row FK), foreign keys with `PRAGMA foreign_keys=ON`, constraints, schema version and migrations, preserving existing ledgers through backup/upgrade rehearsal. `ledger.py`, `test_ledger.py`; depends on approved compatibility policy. Fast local SQLite. Avoid imposing a false requirement that *every* measurement has a candidate when probes legitimately do not. |
| **S1-B — deterministic ingest contracts** | Fake `Observations`/`Tesscut` results: zero coverage, wrong sector with **no get_cutouts/write/ledger request**, duplicate sector, download failure/missing path, idempotent reuse, file-header and checksum registration, differing camera/CCD/size/product, and path-safe target names. Fake streamed HTTP 206/200, interrupted `.part`, cap, retries and malformed bytes; freeze backoff clock. Add targeted TAP byte-normalization/sync→async fake-session tests, since `ingest/tap.py:33-192` has substantial retry/parsing behavior but no `test_tap.py`. | Reconcile or document `mast.py` exact-name selection versus Tier-1 cone selection, tighten download destination/identity and response validation if tests expose collisions or silent drops. Ensure retries have bounded budgets. Fast offline; optional `mast`/`astropy` parser tests isolated. Do not hard-code today's MAST search results into units. |
| **S1-C — Tier-1 and manifest exactness** | Parse CSV/JSON and assert exact key sets, row states, bytes, checksums, query, exclusions and search-log counts; test unresolved names create no guessed coordinates or downstream query. Test stable hash for equal configuration across timestamps, changed hash for meaningful config/version change, and failed-service state reflected in the run outcome. | Separate `generated_utc` from the hashed payload (`tier1.py:1365-1385`); agree what “completed with exclusions” versus “partial/failed” means (`:1396-1405`). No bulk collector call in CI. Fast fake collectors. A stable hash alone does not make the proposed resumable DAG exist. |
| **S1-D — prior-art and evidence promotion** | Exact offline service→`not_tested` mapping and ledger query fields; unknown domain rejection rather than silent empty coverage; SkyBoT missing epoch, zero/match, API error, radius/frame/epoch forwarding via fake adapter; partial match, unavailable catalog and stale version must not be interpreted as clear. Test later mutation of audit state demotes or rejects promotion. | Introduce structured outcomes (`passed/match/inconclusive/not_tested` plus query, epoch, radius, release, retrieval date and source IDs), then wire an explicit promotion gate across `priorart.py`, `CandidateRecord` and ledger. New adapters are separate, service-specific work; most are currently stubs. Fast fake adapters; no claim that a “no match” proves novelty. |
| **S1-E — dossier, board and CLI contracts** | Parse headings, audit table and ledger-linked references rather than substring checks. Test missing units/uncertainty as explicitly absent, all four audit states, untested catalog gate, contradictory ledger level, empty and tie-ordered leads, pipe/markup escaping, bad JSON/ID, absent input, doctor scratch/ledger failure and exact JSON/exit/stderr through `python -m cygnus.cli`. | Centralize evidence validation at mutation and publication boundaries; consider a typed dossier view rather than duplicating render logic. CLI should return documented, stable error codes without printing private paths/secrets. Offline and fast. Avoid snapshotting entire markdown pages where semantic assertions suffice. |
| **S1-F — organize publication without weakening it** | Split `test_publish.py` by schema/sources, safety, build boundary, candidate claims, rendered site and preview service, retaining shared synthetic fixtures. Parameterize traversal, symlink, secret, draft/restricted and failed-build cases; assert prior good site is preserved or partial tree removed on any build error. Add subprocess publication CLI tests in temporary worktrees; check skip reporting with/without `[site]`. | Consider staging-plus-atomic publish rather than deleting output before rendering (`src/cygnus/publish/build.py:160-179`); retain read-only ledger and allowlist. Offline build, with marked loopback server test. Do not assume binary leak scans, redistribution truth, or deployed-host headers from local tests. |
| **S2 — scientific contracts before campaign use** | When each planned method actually lands, specify synthetic truth at known units/time standard/epoch; test noise-free recovery, seeded noise, gaps/masks, red noise, blends/cosmic rays/jitter, null populations, parameter boundaries, alternate detrending, multiple-trial accounting, and injection→recovery curves with uncertainty. Require sensitivity and false-positive behavior across a *predeclared* grid, not one favorable example. | Implement methods listed as **designed**, not implemented, in `ANALYSIS_STACK.md:60-72`, with injectable RNG/config and provenance output. Begin with an extracted, testable component of `campaigns/run_wasp12_sector20.py`, not wholesale promotion of its limited permutation FAP. Fast small cases; slower grids nightly. Simulations validate a model of noise, not the sky's true noise distribution. |
| **S3 — archival and release corroboration** | Use a tiny pinned known-object and null/control set; check checksum, metadata, mask/time interpretation, recovery within justified published-data tolerances, SAP/PDC and independent epoch where available, artifact checks and recorded exclusions. Release gate rehearses migrations, pack non-deletion adversarial cases, publication build and optional mirror. | Fetch/cache procedure and legal provenance reviewed separately; no default downloads. Dependencies are S0–S2 and justified reference sources. Bounded `archival`/`slow` jobs plus a separate `network` smoke. A recovered known transit or passing publication build is not a calibrated discovery pipeline. |

## 5. High-value Given/When/Then contracts

- **Ledger provenance.** Given a logged run and a registered `(archive, product_id)`, when a measurement is stored with that product reference, then a query resolves exactly that product, its checksum and run metadata. Given a nonexistent run, nonexistent product, or ambiguous unqualified ID, insertion must fail with a specified error and **zero new rows**. The current free-text `product_ids_json` behavior does not meet this proposed contract.
- **Retirement.** Given two staged files and a manifest with one verified upload and one missing, mismatched or hashless local row, when retirement is requested, then it fails before unlinking **either** file; manifest states and ledger local paths are unchanged. Given `../sentinel`, a drive-letter path or a symlink escape, no outside path is read or deleted. Given zero eligible verified rows, verification cannot return actionable success.
- **MAST isolation.** Given a temporary ledger and scratch plus a fake Tesscut response, when a sector is unavailable, then `get_cutouts` is not called, no FITS file exists, and the product count remains zero. In the marked live counterpart, every returned path must resolve beneath that test's temporary scratch root.
- **Prior-art and claims.** Given `not_tested`, an API outage, stale watchlist or ambiguous match, when promotion is attempted, then the candidate remains an unverified lead—or raises the documented claim error—and the ledger preserves the exact gate outcome. “No match” must include searched source/release, epoch, criterion and date; it never asserts “uncataloged.”
- **CLI and publication.** Given malformed dossier JSON or an unwritable scratch location, when the module CLI runs in a child process, then exit code and stderr have defined shapes and no production ledger/scratch changes occur. Given a secret-bearing publication input, a failed build must not expose a partially built site or replace a previously valid one.
- **Scientific extraction, once implemented.** Given a seeded synthetic signal with declared time scale, units, gaps and injection parameters, when the method searches a fixed predeclared grid, then recovered parameter error and uncertainty coverage meet prechosen tolerances; on predeclared null/artifact controls its false-positive behavior meets a separately justified bound. Report failed and untested audits, not merely the top-scoring detection.

## 6. Test data and reproducibility policy

1. **Synthetic first.** Maintain small generator functions and human-readable fixture manifests recording model, units, time standard, coordinate frame/epoch where relevant, masks, seeds, parameter grid and expected *contract*, not guessed astronomical measurements. Fix RNG algorithm/version where exact draws matter; prefer statistical bounds over bit-for-bit fitting results.
2. **Archived data by exception.** For each retained real product record archive, release, URL/query and selection, product ID, retrieval date, SHA-256, license/access and size, FITS metadata relied upon, and reference publication or ephemeris. The existing WASP-12 script pins a product checksum (`campaigns/run_wasp12_sector20.py:20-23`); justify any future archived fixture and do not silently download it in ordinary pytest.
3. **Numerical tolerance is derived, not decorative.** Set absolute/relative tolerance from cadence or pixel sampling, search-grid resolution, injection parameters and numerical/library variability. Record the derivation alongside each scientific case; compare units and uncertainty coverage as well as point estimates. A nominal sigma or 20 shuffled trials is not a calibrated tail probability (`campaigns/wasp12_sector20/REPORT.md:32-52`).
4. **Environment provenance.** Record Python/OS and science-package versions, configuration hash excluding generation timestamp, archive release and data checksum in calibration output. Keep a constraints/lock strategy for CI and test the supported Python range rather than assuming the historical environment is current.
5. **Artifacts and privacy.** Generate only under `tmp_path` or an explicitly provisioned checksum-verified fixture cache. Clean `.part`, SQLite sidecars, local mirrors and generated sites; assert sentinels outside the temporary root survive. Never load real credentials, user ledger, Drive remote, production output or private absolute paths into public CI logs.
6. **Flakes are evidence to classify.** Preserve seed, response fixture, request IDs, service status and bounded logs. A deterministic regression fails the gate; a live outage is reported as **unrun/inconclusive** and retried in its separate lane. Time-bound quarantine needs an owner, issue, expiry and continuing visible reporting—not an unconditional skip.

## 7. CI and developer workflow

These are **proposed commands**, not commands executed for this assessment. Install the package in a disposable environment for release checks; current `tests/conftest.py:10-12` otherwise makes an uninstalled `src` checkout importable.

| Lane | Proposed invocation and reporting |
|---|---|
| Fast local / ordinary PR | `python -m pytest -m "not network and not local_service and not slow and not archival" -ra` after installing `.[test]`; default `pyproject.toml:27-32` already excludes `network`. Show deselection and skips; enforce a no-external-network guard in this lane. |
| Full offline | Install `.[test,site,mast]` where supported; `python -m pytest -m "not network and not archival and not slow" -ra`, including marked loopback tests. Pin optional tool availability or report local-`rclone` tests as skipped—not passed. |
| Optional science/nightly | Install relevant extras; run `-m "slow or archival"` against a deliberately provisioned, checksum-verified fixture cache, with no implicit fetch. Report unavailable fixtures and numerical calibration separately. |
| Archive smoke | Explicit operator job, e.g. `python -m pytest -o addopts= -m network -ra`, with temporary ledger/scratch, small budgets and no secrets. Report service outage distinctly; never make this public PR CI's deterministic oracle. |
| Release/security | Disposable installed-package subprocess CLI tests; ledger migration rehearsal; pack adversarial and publication tests; static build/leak gate. Run proposed `ruff check ...`, formatting check and a **gradually scoped** type check after adopting/configuring those tools. No lint/type tool is configured in the inspected `pyproject.toml:1-32`; do not present those checks as existing gates. |

Make optional extras explicit in the matrix. In particular, module-level `pytest.importorskip` in `tests/test_publish.py:20-21` can skip the entire publication module; the release/security lane must **require `[site]` and fail if that suite is not collected**, while a core-only local lane may legitimately report it skipped. Move manual probes to a documented `tools/probes/`-style location or label them unmistakably as operational scripts; `probe_data_check.py` must never be invoked by routine CI.

## 8. Prioritized acceptance matrix

| Priority | Measurable outcome / affected tests | Campaign blocker? | Public-release blocker? |
|---|---|---:|---:|
| **P0** | Every test download and ledger write stays in its temporary root; MAST live paths are asserted beneath temporary scratch; offline gate detects forbidden external access. `conftest.py`, `test_ingest_mast.py`, config/MAST tests. | **Yes** | **Yes** |
| **P0** | All malformed, missing, mismatched, hashless, zero-row and traversal pack cases refuse deletion; every failed case preserves file, manifest and ledger state. `test_ingest_pack.py`, `pack.py`. | **Yes**, if packs are retained/retired | **Yes**, if pack tooling is released |
| **P1** | No measurement with an unresolved required run/product reference can be committed; existing-ledger migration and read-only behavior are exercised without data loss. `test_ledger.py`. | **Yes** | **Yes** |
| **P1** | Offline prior-art returns exact `not_tested` records; incomplete/ambiguous gates cannot promote a claim; publication and ledger agree on evidence. `test_priorart.py`, candidate/publication contracts. | **Yes** for candidate claims | **Yes** for candidate pages |
| **P1** | Fake MAST/TAP/HTTP cases prove bounded requests, accurate product identity, checksum and failure semantics; no archive call in offline CI. Ingest tests. | **Yes** for ingest-dependent campaign use | Recommended |
| **P1** | Installed module subprocess CLI errors have documented exit/output; publication security tests cannot silently all skip and failed builds preserve the prior site. CLI and reorganized publication tests. | Recommended | **Yes** |
| **P2** | Each newly deployed scientific method passes predeclared synthetic recovery, null, robustness and artifact matrices, with completeness/uncertainty recorded. New domain/validation tests. | **Yes** for that method's scientific campaign | **Yes** before publishing its scientific claims |
| **P2** | A justified checksum-pinned archival case and independent or held-out check reproduce documented, limited conclusions; outages are separately reported. Campaign calibration jobs. | **Yes** for claims requiring those checks | **Yes** for such claims |
| **P3** | Fast/full/nightly/network/release lanes have explicit markers, optional dependencies, skip counts, supported-platform results, and maintained lint/type policy. CI configuration. | Recommended | **Yes** for a dependable release gate |

“Blocker” is conditional on the stated use: an engineering-only static site need not wait for an unimplemented monotransit algorithm, but it must not publish that algorithm's unearned scientific claim.

## 9. Explicit non-goals

- Do not equate passing units, one known-planet recovery, or a healthy live archive response with calibrated discovery sensitivity or scientific certification.
- Do not treat nominal sigma, the campaign's limited permutation diagnostic, an absent catalog match, or a stub prior-art result as proof of novelty.
- Do not invent universal PSF-width, transit-depth, proper-motion or false-alarm thresholds. The no-universal-PSF contract already appears in `src/cygnus/instruments.py` tests and `ANALYSIS_STACK.md:29-32`.
- Do not build exhaustive tests for designed-only `domains/...` modules before they exist; write each scientific contract as the corresponding implementation is reviewed.
- Do not make real Google Drive, bulk Tier-1 retrieval, production scratch/ledger, a deployed host, or archive uptime part of deterministic public CI.
- Do not raise test counts by duplicating rendering snapshots or testing implementation-private call order without a security, provenance or compatibility reason.

### Recommended first sprint

1. **Contain all test writes:** temporary ledger **and** scratch in both live MAST tests, with an outside-root sentinel and explicit path assertions.
2. **Make retirement fail closed:** add a compact adversarial manifest/fake-`rclone` matrix for zero checks, missing/mismatched/hashless uploads, subprocess failure and traversal; harden the deletion path revealed by those tests.
3. **Establish one end-to-end provenance contract:** register product→run→measurement in temporary SQLite, and make an unresolved reference fail without altering the ledger.

That sequence addresses unintended writes, irreversible deletion and scientifically consequential broken provenance before expanding the suite or implementing new astronomy algorithms.

## Incremental implementation note — research reanalysis pilot (draft)

**Subsequent to the assessment above:** `src/cygnus/analysis/` now contains small in-memory implementations for aligned light-curve reduction disagreement, aperture/background and centroid counterfactuals, single-band residual-island triage, bounded optical/infrared epoch-aware matching, Gaia NSS control bookkeeping and a measured-RV-only mass-function bound. `analysis/ml.py` provides NumPy nearest-control anomaly ranking; `analysis/io.py` provides read-only manifest SHA-256 verification and bounded FITS adapters. See `docs/ANALYSIS_SUITE.md` for scope and `notebooks/cygnus_reanalysis_colab.ipynb` for a reviewed-path, user-mounted TESS pilot. The notebook has **not** been run against Drive or real products. The scientific tests are small synthetic contracts; their availability does not prove survey completeness, significance calibration, prior-art clearance or detection of any unidentified object.

Add the new tests to the ordinary offline CI layer and separately document an optional science-stack environment (`pip install -e '.[analysis,test]'`). The five modules are **not replacements** for the larger designed `domains/...` pipelines in `ANALYSIS_STACK.md`: no full image subtraction, proper-motion/parallax orbit fitting, injection/recovery engine, automated prior-art adapters or classifier validated on archival controls exists yet. The original S0 safety priorities above (live-MAST scratch isolation, pack retirement and ledger referential integrity) remain outstanding; they must not be marked complete because a research utility was added. This working plan should be revisited after archival pilot results and independent review. Development verification for this addition: `python -m pytest -q -m 'not network'` returned **132 passed, 2 deselected**; no notebook/Drive/real-product science run was conducted.