Operon Credential Keeper

`tools/scidex_operon_credential_keeper.py` keeps the VM-local Operon OAuth store usable for the unified SciDEX agent

Source: docs/operon-credential-keeper.md

Operon Credential Keeper

tools/scidex_operon_credential_keeper.py keeps the VM-local Operon OAuth store usable for the unified SciDEX agent runtime.

Canonical persona deployments are runtimectl targets named agent-<persona> and Docker containers named scidex-agent-<persona>. For example, agent-jerome maps to scidex-agent-jerome. Provider-specific scidex-operon-*, scidex-minimax-*, and scidex-rosalind-* containers are compatibility harnesses. Operon still runs as a shared local harness service at 127.0.0.1:18765 until that harness is embedded in the unified runtime image.

What the keeper does:

  • Verifies operon-runtime health.

  • Periodically runs a tiny authenticated Operon prompt so OAuth refresh can happen.

  • Copies rotated OAuth token material from the running Docker volume back to /home/ubuntu/operon-container-data/.operon so later operon-runtime recreates do not revert to a stale token.

  • Ensures the canonical scidex-agent-* containers are running and have OPERON_RUNTIME_URL available.

  • Fails the service run if required canonical agents cannot be started.

  • Never logs token contents.

Manual check:

cd /home/ubuntu/scidex-agents
/home/ubuntu/miniconda3/bin/python tools/scidex_operon_credential_keeper.py --json --force-smoke

Health-only check:

/home/ubuntu/miniconda3/bin/python tools/scidex_operon_credential_keeper.py --json --no-smoke

Systemd install on sandbox-02:

sudo cp ops/systemd/scidex-operon-credential-keeper.service /etc/systemd/system/
sudo cp ops/systemd/scidex-operon-credential-keeper.timer /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now scidex-operon-credential-keeper.timer

Operational checks:

systemctl list-timers scidex-operon-credential-keeper.timer
systemctl status scidex-operon-credential-keeper.timer
journalctl -u scidex-operon-credential-keeper.service -n 100 --no-pager

Non-secret state and audit files live under:

  • state/operon-credential-keeper.json

  • logs/operon-credential-keeper-*.jsonl

The timer runs every 15 minutes. The default authenticated smoke interval is 30 minutes via SCIDEX_OPERON_KEEPER_SMOKE_INTERVAL_SECONDS=1800; cheaper health-only runs happen between smokes.

Limitations and follow-up work:

  • If another machine uses the same rotating OAuth refresh token and invalidates the VM copy first, the keeper reports an auth failure but cannot mint a new token by itself. Resync a fresh token from the active logged-in Operon client or log in directly on the VM.

  • The keeper intentionally does not start scidex-operon-* compatibility persona containers unless invoked with --ensure-compat-operon-personas.

  • Embed the Operon harness into the unified runtime image or agent runtime manager so operon-runtime can be retired.

  • Add repeated-auth-failure alerting and a token-resync runbook that never prints token material.