SPEC-002 — Tool Artifacts & Pluggable Runtimes

Field	Value
Status	Draft v2 (rewritten 2026-04-29)
Owner	kris.ganjam@gmail.com
Date	2026-04-29
Depends on	SPEC-001, SPEC-013
Pillar	Tool Runtime
Supersedes	v1 “Forge as standalone tool runtime” framing

TL;DR

Tools are artifacts. Each tool declares a runtime — bwrap, mcp_server, http_post, in_process, or skill_call — that determines how the substrate executes it. There is no separate “Forge service.” Execution dispatches by runtime through a pluggable handler registry. Adding a new runtime kind = registering one Python function. v1’s scidex_tools/ directory and /api/forge/* routes are retired entirely; tools become first-class artifacts that compose with skills (SPEC-013).

0. 2026-05-23 Tool Forge audit

The standalone kganjam/Forge repository is an import source and temporary execution harness, not the target architecture. A code review found that it mixes catalog metadata, execution routing, telemetry, stale “agent” concepts, and hundreds of wrappers in one repo/process. The split is:

Substrate owns metadata/control-plane state: tool artifacts, function schemas, runtime descriptors, cache/rate-limit policy, cost models, lifecycle/deprecation state, result provenance, tool-call ledgers, usage stats, rankings, and review gates.
Agents own judgment: tool-selection policy, prompt/persona semantics, active-learning and budget-spend decisions, science trajectories, and closed-loop artifact creation/review/improvement.
Execution remains isolated: wrappers and heavy package environments run behind http_post, mcp_server, bwrap, remote_vm, or a future agent-owned tool-runtime service. Do not fold hundreds of wrappers into persona runtime logic or substrate business logic.

Concrete audit risks from the current Forge repo:

engine.py metrics persistence can deadlock because record_call holds a non-reentrant lock and then calls _persist_metrics, which reacquires it.
The successful-result cache condition has an operator-precedence bug that can call cache.set while caching is disabled.
config/rate_limits.json uses max_calls / window_seconds, while rate_limiter.py expects rate / burst.
Tool discovery is filesystem/docstring based, not substrate registry based.
Many wrappers contain mock, placeholder, synthetic, or access-limited fallbacks; they must import as draft/experimental, not active trusted tools.

Retirement criterion: Forge can be deleted or archived once no active substrate tool artifact points at it, no agent runtime imports it, all valid metadata has been registered in substrate, and any surviving execution code lives behind a substrate-registered external runtime or isolated agent-owned tool-runtime.

1. Why rethink Forge

v1 Forge was a single coupled HTTP service that wrapped ~58 tool implementations. Problems:

One service for all tool kinds — overdesigned for low-cost API wrappers (PubMed search), under-isolated for code-execution tools.
Tight coupling between substrate and Forge codepaths.
Hard to add a tool category without touching service code.
Standalone-service ops surface for operations that should sometimes be a function call.
Tools were not artifacts — they couldn’t evolve, accumulate signals, or be debated.

Per the user’s call to rethink Forge for v2: drop the service, lift tools to artifacts, dispatch by runtime.

2. The new model

┌─────────────────────────────────────────────────────────┐
│ Agent: scidex.tool.invoke("alphafold_predict", args)     │
└────────────────┬────────────────────────────────────────┘
                 ▼
┌─────────────────────────────────────────────────────────┐
│ Substrate                                                │
│  1. Load `tool` artifact (schema, runtime, args check)   │
│  2. Validate args against tool.input_schema              │
│  3. Mint job_id; emit `tool_call.started`                │
│  4. Dispatch by runtime:                                 │
│     - bwrap         → bwrap'd subprocess                 │
│     - mcp_server    → MCP tool call to registered server │
│     - http_post     → POST to declared URL              │
│     - in_process    → Python function call (trusted)     │
│     - skill_call    → load skill, run as instructions    │
│  5. Validate output against tool.output_schema           │
│  6. Persist `tool_call` artifact with result + provenance│
│  7. Emit `tool_call.completed`                           │
└─────────────────────────────────────────────────────────┘

2.1. Result-artifact submission (preferred integration path)

Per SPEC-017 §1, in-substrate scidex.tool.invoke is now an optional execution path, not the primary one. Most agents — humans, ai_personas, external services — execute tools in their own runtime (Docker, claude-code instance, browser, wet-lab notebook) and submit the result as an artifact:

agent picks a tool from substrate (or runs an off-substrate procedure)
  ↓
agent runs it in its own runtime, with its own credentials and resources
  ↓
agent calls scidex.create(type='analysis' | 'hypothesis' | 'figure' | …,
                          content={...result, produced_by: {actor, skill, tool}, ...})
  ↓
substrate stamps provenance edges; the artifact is now in the market

Why this is preferred:

The substrate is downstream of execution. It records, scores, and ranks; it does not need to host runners for every tool an agent might use.
Trust at the artifact, not the call. Reproducibility, dispute, and replication score artifacts directly — the substrate doesn’t have to vouch for arbitrary code paths.
Lower ops surface. No per-runtime ops burden for tools that are essentially someone else’s job to operate.

The substrate-side dispatcher in §2 above remains supported for tools that benefit from substrate co-location (cost-controlled API calls, in-process Python helpers, MCP-backed tools the substrate already trusts). It is no longer the only way to attribute a result to a tool.

2.2. Provenance edges

When a result artifact is created with content.produced_by, the substrate stamps typed edges into the substrate_provenance_links table (introduced PR 48):

Predicate	Target	Use
`produced_by_actor`	`target_agent_id`	The agent (human, ai_persona, ai_local, external_agent) that produced the artifact
`produced_by_skill`	`target_artifact_type='skill'`, `target_artifact_id=<skill-id>`	Skill artifact that drove the production
`produced_by_tool`	`target_artifact_type='tool'`, `target_artifact_id=<tool-id>`	Tool artifact used to produce the result

Properties:

Idempotent: a UNIQUE index on (source, predicate, target) collapses re-stamps. Replaying the same submission is safe.
Best-effort: an individual edge failure logs and continues; partial provenance beats none.
Read-back: scidex.get?include_provenance=true (and the helper fetch_provenance(...)) returns the edges; the agent /agents/<id> page can list “produced N artifacts via skill X.”
Substrate-owned: lives in substrate_provenance_links, not v1’s artifact_links (v1 is read-only post-cutover and its CHECK constraint rejects these predicates).

The helper module is scidex_substrate.skill.provenance. Direct callers (persona-runner, operator backfills) call stamp_provenance(...). Wiring scidex.create to auto-stamp from content.produced_by is a follow-up: it touches a verb shared by ~all artifact creation paths, so it lands behind a feature flag once submission flows are exercised in production.

3. Tool artifact type

{
  "type": "tool",
  "schema_version": 1,
  "id_strategy": { "format": "tool-{name}", "stable": true },
  "lock_mode": "content_hash",
  "mutability": "mutable_with_history",
  "content_schema": {
    "type": "object",
    "required": ["name", "version", "runtime", "input_schema", "output_schema"],
    "properties": {
      "name":          { "type": "string", "pattern": "^[a-z][a-z0-9_-]*$" },
      "version":       { "type": "string" },
      "summary":       { "type": "string" },
      "runtime":       { "enum": ["bwrap", "mcp_server", "http_post", "in_process", "skill_call"] },
      "runtime_config": {
        "type": "object",
        "description": "Per-runtime config; shape varies by runtime kind"
      },
      "input_schema":  { "type": "object", "description": "JSON Schema for args" },
      "output_schema": { "type": "object", "description": "JSON Schema for result" },
      "resource_caps": {
        "type": "object",
        "properties": {
          "vram_gb":       { "type": "number" },
          "wall_time_min": { "type": "number" },
          "memory_gb":     { "type": "number" },
          "network":       { "enum": ["allow", "deny", "specific"] },
          "cpu_count":     { "type": "number", "description": "Logical CPU count limit" }
        }
      },
      "sandbox_backend": {
        "type": "string",
        "enum": ["bwrap", "docker", "batch", "k8s"],
        "default": "bwrap",
        "description": "SPEC-081 §3.6: sandbox backend selection. Operators flip a tool to docker/batch/k8s without changing tool code. Switching backends produces a content-hash-bumped version of the tool artifact."
      },
      "sandbox_image": {
        "type": ["string", "null"],
        "description": "SPEC-081 §3.6: container image tag for docker/batch/k8s backends (e.g. scidex-analysis:v1.2). Required when sandbox_backend is docker, batch, or k8s; ignored for bwrap."
      },
      "render_hints": {
        "type": "object",
        "description": "SPEC-082 §3.4: default rendering metadata for the ToolResult envelope consumed by Prism (SPEC-003). Guides how the tool's output is displayed in the playground UI without per-tool bespoke HTML.",
        "properties": {
          "primary_view": { "type": "string", "enum": ["table", "json", "gallery", "markdown"] },
          "columns": { "type": "array", "items": { "type": "string" } },
          "order_by": { "type": "string" },
          "template": { "type": "string" }
        },
        "additionalProperties": false
      },
      "examples": { "type": "array" },
      "cost_model": {
        "type": "object",
        "properties": {
          "vram_hour_usd": { "type": "number" },
          "fixed_usd":     { "type": "number" }
        }
      },
      "examples": { "type": "array" },
      "files": {
        "type": "array",
        "items": { "properties": { "path": { "type": "string" }, "sha256": { "type": "string" } } }
      }
    }
  },
  "links": {
    "documents":   { "to_types": ["skill"], "description": "Skills documenting this tool" },
    "implements":  { "to_types": ["schema"] },
    "tested_by":   { "to_types": ["test_suite"] }
  },
  "signals": {
    "vote":        { "values": [-1, 1], "aggregation": "replace" },
    "rank":        { "dimensions": ["reliability", "speed", "cost"], "aggregation": "replace" },
    "usage_count": { "aggregation": "sum", "auto": true }
  },
  "lifecycle": {
    "states": ["draft", "active", "deprecated", "archived"]
  },
  "validators": ["tool_runtime_handler_exists", "tool_input_schema_valid", "tool_sandbox_image_required_when_not_bwrap", "tool_render_hints_valid"]
}

Tools live in the substrate repo at tools/<name>/ alongside skills, with an optional manifest + executable bundle (similar to skill bundles per SPEC-013 §4).

4. Runtime handlers

A runtime is a registered Python function: (tool_artifact, args, ctx) -> result. The registry lives in src/scidex_substrate/tool/runtimes.py. Adding a runtime = registering a handler.

from scidex_substrate.tool.framework import runtime

@runtime("bwrap")
async def run_bwrap(tool, args, ctx):
    config = tool.content["runtime_config"]
    cmd = ["bwrap", *config["bwrap_args"], config["binary"], *flatten(args)]
    proc = await asyncio.create_subprocess_exec(*cmd, ...)
    out = await proc.communicate()
    return parse_output(out, tool.content["output_schema"])

4.1 `bwrap` runtime

For sandboxed CPU/GPU jobs.

runtime_config:

{
  "binary": "/usr/local/bin/alphafold_predict",
  "bwrap_args": ["--ro-bind", "/", "/", "--tmpfs", "/home", "--unshare-all"],
  "stdin_format": "json",
  "stdout_format": "json"
}

Substrate launches the binary inside bwrap, captures stdout, validates against output_schema. Resource caps enforced via bwrap’s --memory, plus our own watchdog.

4.2 `mcp_server` runtime

For tools exposed by external MCP servers (third-party MCPs the substrate trusts).

runtime_config:

{
  "server_name": "pubmed_mcp",
  "tool_name":   "search"
}

Substrate’s MCP client dispatches; the external server runs the tool. Useful for CPU-only API wrappers that don’t need bwrap.

4.3 `http_post` runtime

For tools exposed as HTTP endpoints (internal microservices, external APIs).

runtime_config:

{
  "url":      "https://api.uniprot.org/uniprotkb/search",
  "method":   "POST",
  "headers":  { "Content-Type": "application/json" },
  "auth":     { "kind": "bearer", "secret_ref": "uniprot_token" }
}

Substrate POSTs JSON, parses response. No isolation; trust based on the URL.

4.4 `in_process` runtime

For pure Python tools (no isolation needed).

runtime_config:

{
  "module":   "scidex_substrate.tools.simple.compute_md5",
  "function": "compute"
}

Substrate imports and calls. Use only for trusted, low-blast-radius code. Never accept user-controlled runtime_config of this kind.

4.5 `skill_call` runtime

For “tools” that are really agent-level workflows. The skill_call runtime loads the named substrate skill and emits a first-class agent_work_packet instead of running an agent in-process. Agent runtimes in scidex-agents claim the packet, decide how to spend budget/steps, and write durable deliverables back through substrate verbs.

runtime_config:

{
  "skill":      "skill-find-similar-failed-hypotheses",
  "persona":    "synthesizer",
  "budget_usd": 0.50,
  "max_steps":  8
}

Substrate returns the queued work-packet ref and records the tool call audit row. Substrate does not import the agent runner or provider SDK on this path.

4.6 Adding a new runtime

# src/scidex_substrate/tool/runtimes/docker.py
from scidex_substrate.tool.framework import runtime

@runtime("docker")
async def run_docker(tool, args, ctx):
    ...

Importing this module registers the handler. runtime field in tool schemas accepts the new kind on next substrate restart.

5. Sandbox properties

Runtime	Network	Filesystem	Process iso	Use for
`bwrap`	configurable	sandboxed (rw mounts only)	yes	GPU jobs, untrusted code
`mcp_server`	depends on server	depends on server	depends	Vendor MCPs
`http_post`	only target URL	none	n/a	API wrappers
`in_process`	host	host	none	Trusted Python utilities
`skill_call`	substrate API only	n/a	queued agent runtime	Agent workflows

in_process is the highest-trust runtime. The substrate itself controls which functions are registered; user-submitted tool artifacts cannot specify arbitrary modules.

6. Cost & auth

Each tool_call artifact records:

runtime used.
duration_ms (wall time).
Peak resources (vram_peak_gb, memory_peak_mb where applicable).
cost_usd (computed from cost_model + actuals).
actor_id (debiting).

Cost flows through to the agent’s budget per SPEC-004. The substrate emits agent.budget_debited events.

Auth: tool invocation requires the agent’s standard JWT. No per-tool auth; trust scopes are the agent’s permissions. Tools requiring elevated trust (runtime: in_process) are gated by Senate-controlled tool-registration policies.

7. Tool registration

Tools live in tools/<name>/ in the substrate repo:

tools/alphafold_predict/
├── tool.json           (the tool artifact content)
├── README.md           (human docs)
└── examples/           (optional example inputs/outputs)

scidex tool register tools/alphafold_predict/ reads tool.json, computes file hashes, creates the tool artifact. CI re-registers on changes (file hash mismatch → version bump).

External tools (vendor MCPs, third-party HTTP endpoints) register via scidex tool register --runtime mcp_server --config '{...}'.

8. Skills compose with tools

Per SPEC-013, a skill can declare dependencies.tools: [...]. When the agent uses the skill:

Substrate verifies declared tools exist (validator: skill_dependencies_resolve).
Skill instructions tell the agent how/when to call tools.
Agent invokes via scidex.tool.invoke(name, args).
Substrate dispatches by runtime.
Result feeds back to agent via the tool_call artifact.

A skill can also document a tool: links.documents_artifact: tool:alphafold_predict. The skill explains when to use the tool, with what args, in what order. This documentation is itself a discoverable artifact.

9. Migration from v1 Forge tools

Each v1 tool maps to a new tool artifact:

v1 tool kind	v1 location	v2 runtime
`pubmed_search`, `string_protein_interactions`, … (CPU API wrappers)	`scidex_tools/` Python imports	`http_post` if external API; `in_process` if trusted local function
`alphafold_predict`, `diffdock`, … (GPU bwrap)	`scidex_tools/model_*.py`	`bwrap` with GPU resource caps
`figure_generator`, `notebook_executor`	`scidex/agora/`	`in_process` (trusted)
Vendor MCPs (k-dense MCPs, agentskills.io tools)	external	`mcp_server`

Most tools are http_post or in_process. Few need bwrap. None need a “Forge service.”

10. Implementation plan

#	Title	Scope	Risk
F1	`tool` artifact type schema in `default_schemas/`	Per §3.	Low
F2	Runtime framework: `@runtime` decorator, registry, dispatcher	Per §4. ~200 LoC.	Low
F3	`scidex.tool.invoke` verb	Loads tool, validates args, dispatches, persists tool_call.	Medium
F4	`bwrap` runtime handler	First runtime; lifts pieces from v1 `forge_tools.py`.	Medium
F5	`http_post` runtime handler	Stateless POST + response parsing.	Low
F6	`mcp_server` runtime handler	MCP client wrapper.	Medium
F7	`in_process` runtime handler + Senate gate for module registration	Trust-gated.	Medium
F8	`skill_call` runtime handler	Queue `agent_work_packet` for agent-owned execution.	Medium
F9	`scidex tool register` CLI	Reads `tools/<name>/tool.json`, creates artifact.	Low
F10	Migrate v1 tools batch by batch	Most via `http_post` or `in_process`; bwrap for GPU.	Medium per batch

Cadence: F1-F3 in week 1. F4-F8 weeks 2-3. F9-F10 over 4-6 weeks (parallelizable).

11. Open questions

Async tool calls — bwrap GPU jobs take minutes. scidex.tool.invoke should support async with progress events. Default: synchronous for fast tools, async + WebSocket/SSE progress for long-running. Schema declares.
Tool versioning at call time — should agents specify tool_version for reproducibility, or always latest? Default latest; allow pin via scidex.tool.invoke(name, version, args).
Resource cap enforcement — bwrap supports memory caps natively; wall-time via watchdog; vram via NVIDIA MIG or similar. Detail per-runtime in implementation.
Trust escalation for in_process — should the runtime require Senate approval per registered module, or is the entire registry curated at substrate-build-time? Curated at build for v1; Senate gate for community-submitted modules later.
Tool discoverability for agents — same as skill discovery (semantic search over tool.summary + tool.examples); top-K loaded into context.

12. What v1 Forge code becomes

v1 location	v2 fate
`/api/forge/*` HTTP routes	Deleted at cutover — agents call `scidex.tool.invoke(...)` instead
`scidex_tools/` Python modules	Adapted: most become `in_process` runtime modules in `src/scidex_substrate/tools/`; some become standalone `http_post` services if they need network isolation
bwrap launch logic in `scidex/forge/forge_tools.py`	Lifted into `bwrap` runtime handler
Forge cost-model code	Lifted into the `cost_model` field of tool artifacts + budget-debiting in the runtime dispatcher

13. Interaction with other specs

SPEC-001: tools are a registered artifact type. Standard verbs (get/list/search/signals) work uniformly.
SPEC-006: cutover migrates v1 tool registrations into v2 tool artifacts.
SPEC-013: skills compose with tools; tools may be documented by skills.
SPEC-008: tool_call artifacts feed audit logs, cost dashboards, runtime metrics.
SPEC-004: tool invocations debit agent budgets per cost_model.

SPEC-002 — Tool Artifacts & Pluggable Runtimes

TL;DR

0. 2026-05-23 Tool Forge audit

1. Why rethink Forge

2. The new model

2.1. Result-artifact submission (preferred integration path)

2.2. Provenance edges

3. Tool artifact type

4. Runtime handlers

4.1 bwrap runtime

4.2 mcp_server runtime

4.3 http_post runtime

4.4 in_process runtime

4.5 skill_call runtime

4.6 Adding a new runtime

5. Sandbox properties

6. Cost & auth

7. Tool registration

8. Skills compose with tools

9. Migration from v1 Forge tools

10. Implementation plan

11. Open questions

12. What v1 Forge code becomes

13. Interaction with other specs

4.1 `bwrap` runtime

4.2 `mcp_server` runtime

4.3 `http_post` runtime

4.4 `in_process` runtime

4.5 `skill_call` runtime