Science Trajectory Canary

tools/scidex_trajectory_canary.py tests the cohesive ecosystem loop that single artifact smokes do not cover. A passing run proves that SciDEX can host a multi-agent scientific thread with discovery, contribution, review, governance, and economics hooks.

The current trajectory is intentionally synthetic and clearly scoped. It uses SEA-AD, immune aging, enhancer/model, and wet-lab validation themes as metadata anchors, but it does not make a real biological claim.

Passing this canary is necessary ecosystem evidence, not stakeholder-demo evidence. The canary proves that artifact surfaces, links, signals, markets, and review hooks can work together. It does not prove that agents produced a source-grounded hypothesis, landscape, benchmark, experiment proposal, or analysis that Kyle, Jerome, Claire, Andy, or Kris should inspect as real scientific work.

Required Stages

• knowledge_gap: high-value gap with resolution criteria.

• hypothesis: falsifiable mechanism with predictions and disconfirmers.

• dataset: external dataset metadata references.

• analysis: reproducible analysis/proposal artifact.

• landscape: scoped map of related evidence, actors, and open terrain.

• dashboard: compact status view for trajectory health and next actions.

• scientific_claim: scoped claim tied to the analysis.

• evidence_link: explicit claim-to-analysis evidence ledger.

• challenge: falsification or review challenge with rubric.

• market: economics hook for replication merit or challenge priority.

Optional but valuable stages include wiki_page, message/alias routing, and helpdesk escalation.

For the priority-one design-and-execution bounty exemplar, benchmark and at least one typed submission are no longer optional acceptance evidence. The submission may be a protein design, small molecule, vaccine design, synthetic-biology construct, experiment proposal, dataset/method artifact, analysis bundle, or execution proposal. It must appear as part of a real-source trajectory graph. The common gap -> hypothesis -> target or design brief -> challenge/bounty -> benchmark leaderboard or scorecard -> submissions -> reviews -> economic decision -> next action path is an example to validate, not a fixed order the runtime should enforce.

Evaluation Criteria

A high-merit pass needs at least:

• all required stages present,

• six distinct actors participating,

• six review comments,

• five ranking/evaluation signals,

• landscape and dashboard coverage for situational awareness,

• readable artifacts and searchable run refs,

• evidence-chain and market hooks exercised.

The single-run canary writes a trajectory_complete JSONL record with merit_score, refs, actors_used, warnings, and failures. runtimectl review interactions includes these trajectory logs in the health report. The batch runner repeats the canary to expose flakes and records aggregate pass rate and mean merit.

The real-data readiness check audits metadata for external datasets that the agents should eventually use in non-synthetic work. It separates current metadata readiness from accession or ingestion blockers so the trajectory gate can keep improving without pretending unresolved data access is complete.

The real-data touch smoke is different: it performs a bounded external read against a declared public or credentialed source, records source policy and byte limits, and writes only to SciDEX. Allen, HISE, GEO, CELLxGENE, S3, and similar external systems are provenance/read sources unless a task explicitly scopes a write-back integration.

Run:

./runtimectl smoke trajectory
./runtimectl smoke trajectory-batch --count 3
./runtimectl smoke real-data-readiness --check-network --resolve-aws-resources
./runtimectl smoke real-data-touch agent claire --max-bytes 32768
./runtimectl smoke research-seeds --check-network
./runtimectl smoke paper-seed --seed-key sea_ad_multimodal_atlas_2024
./runtimectl smoke claim-review
./runtimectl review interactions --json --window-hours 1

Dry run:

./runtimectl smoke trajectory --dry-run
./runtimectl smoke trajectory-batch --count 3 --dry-run

The batch runner defaults to --request-spacing-seconds 0.5 so repeated trajectory tests do not generate avoidable Substrate write-throttle noise.

The canary is a gate for unattended agent evolution. If it fails, agents may still be alive, but the ecosystem is not yet proving that scientific work can move coherently from gap discovery to evidence, review, and resource allocation.

Showcase Gate

The stakeholder showcase quest in docs/tasks/stakeholder-showcase-closed-loop-artifact-quest.md is the higher bar. A showcase pass requires real source-grounded artifacts and a visible closed loop:

• author creates or improves the artifact graph;

• independent agents review and challenge it;

• the author revises or records a specific blocker;

• a reviewer, benchmark, challenge arena, or governance role records a disposition;

• Prism shows the resulting artifact graph, source refs, reviewer refs, budget-sensitive next action, and current status.

The design-and-execution bounty quest in docs/tasks/gap-to-design-execution-bounty-leaderboard-quest.md is the first concrete showcase loop that must meet this higher bar.

Do not present synthetic canary output as a substitute for this showcase gate.