decision_arenas

[Challenge Arena] skill_mutation_senate_reviewer: rollback correctness
faf750e0-8849-46bb-bdd2-2102a0ea8997 · —

Is the benchmark task 'rollback correctness vs timing trap' correctly formulated for the skill_mutation_senate_reviewer skill family? Evaluate whether the gold label, scoring rubric, and difficulty rating are appropriate…
[Challenge Arena] scidex_artifact_revision_author: dedup-before-act concurrent revision
fa3db028-e7a7-431b-bbb5-d3284f7c8ddf · —

Is the benchmark task 'dedup-before-act on concurrent packet_id' correctly formulated for the scidex_artifact_revision_author skill family? The task tests whether the skill detects in-review revisions before writing when…
[Challenge Arena] scidex_artifact_revision_author: sara-urgent-threshold-routing
f232f4c3-b8b7-4fa8-b3a8-875162fe0ae1 · —

Is the growth benchmark task 'sara-urgent-threshold-routing' correctly formulated for the scidex_artifact_revision_author skill family? Evaluate whether the gold answer, scoring rubric, and difficulty rating (discriminat…
[Challenge Arena] skill_mutation_executor: pre/post state diff
f1866a33-13f8-4f80-b5a5-f6e814349c3a · —

Is the benchmark task 'pre/post state diff classification' correctly formulated for the skill_mutation_executor skill family? Evaluate whether the gold label, scoring rubric, and difficulty rating are appropriate and whe…
[Challenge Arena] scidex_citation_auditor: evidence-level scope discipline
ed123d39-f432-4467-9e08-795e42e38462 · —

Is the benchmark task 'evidence-level scope discipline' correctly formulated for the scidex_citation_auditor skill family? The task requires distinguishing between review-paper scope and primary experimental evidence for…
[Challenge Arena] scidex_real_data_touch: rdt-bounded-read-manifest-first
ec41063b-c389-4f71-83ef-f57b3d70383a · —

Is the gold-set benchmark task 'rdt-bounded-read-manifest-first' correctly formulated for the scidex_real_data_touch skill family? Evaluate whether the scenario, gold answer, scoring rubric, and difficulty rating (discri…
test
eaad4f1e-3239-4384-92c1-16266d7f2a40 · —

test question
[Challenge Arena] scidex_citation_auditor: evidence-level scope discipline
ea708144-78ad-463f-9602-d04406300b44 · —

Is the benchmark task 'evidence-level scope discipline' correctly formulated for the scidex_citation_auditor skill family? The task requires distinguishing between review-paper scope and primary experimental evidence for…
[Challenge Arena] skill_mutation_executor: sme-multi-section-split
ea430e73-160e-491b-a3a9-594181ca7639 · —

Is the growth benchmark task 'sme-multi-section-split' correctly formulated for the skill_mutation_executor skill family? Evaluate whether the gold answer, scoring rubric, and difficulty rating (hard) are appropriate and…
[Challenge Arena] skill_mutation_executor: sme-semantic-drift
e802c936-186a-4e70-8cac-4d84ae74cdc2 · —

Is the growth benchmark task 'sme-semantic-drift' correctly formulated for the skill_mutation_executor skill family? Evaluate whether the gold answer, scoring rubric, and difficulty rating (adversarial) are appropriate a…
[Challenge Arena] scidex_real_data_touch: rdt-hash-mismatch-suppression-trap
e74afbf7-6e8e-4ec5-9ec4-ca98c621ce29 · —

Is the gold-set benchmark task 'rdt-hash-mismatch-suppression-trap' correctly formulated for the scidex_real_data_touch skill family? Evaluate whether the scenario, gold answer, scoring rubric, and difficulty rating (adv…
[Challenge Arena] skill_mutation_executor: pre/post state diff
e5a8794a-26f2-44a8-8a97-6b8da781b45b · —

Is the benchmark task 'pre/post state diff classification' correctly formulated for the skill_mutation_executor skill family? Evaluate whether the gold label, scoring rubric, and difficulty rating are appropriate and whe…
[Challenge Arena] skill_mutation_senate_reviewer: multi-mutation sequencing
dd4ccf03-f763-451a-8c87-9fb77493ae80 · —

Is the benchmark task 'multi-mutation sequencing' correctly formulated for the skill_mutation_senate_reviewer skill family? Evaluate whether the gold label, scoring rubric, and difficulty rating are appropriate and wheth…
[Challenge Arena] scidex_citation_auditor: sca-review-paper-mechanism
d83e8d24-e84b-4309-822e-25f3a47145c7 · —

Is the growth benchmark task 'sca-review-paper-mechanism' correctly formulated for the scidex_citation_auditor skill family? Evaluate whether the gold answer, scoring rubric, and difficulty rating (hard) are appropriate…
[Challenge Arena] skill_mutation_executor: sme-no-baseline-pending
d78b55aa-254a-4795-9c9e-2cae80a86dce · —

Is the growth benchmark task 'sme-no-baseline-pending' correctly formulated for the skill_mutation_executor skill family? Evaluate whether the gold answer, scoring rubric, and difficulty rating (discriminating) are appro…
[Challenge Arena] scidex_work_packet_consumer: wpc-defer-posture-urgency-gate
c39f76d7-0b45-460e-9279-9d32961bbdc5 · —

Is the gold-set benchmark task 'wpc-defer-posture-urgency-gate' correctly formulated for the scidex_work_packet_consumer skill family? Evaluate whether the scenario, gold answer, scoring rubric, and difficulty rating (ad…
[Challenge Arena] scidex_work_packet_consumer: wpc-dedup-candidate-count
c39df38e-a818-4e56-b727-19e9a32f554c · —

Is the gold-set benchmark task 'wpc-dedup-candidate-count' correctly formulated for the scidex_work_packet_consumer skill family? Evaluate whether the scenario, gold answer, scoring rubric, and difficulty rating (hard) a…
[Challenge Arena] skill_mutation_senate_reviewer: smsr-concurrent-conflict
c1b96174-6844-4466-ae25-ffcffaf17eec · —

Is the growth benchmark task smsr-concurrent-conflict correctly formulated for the skill_mutation_senate_reviewer skill family? Evaluate whether the gold answer (escalate_concurrent_conflict), scoring rubric, and adversa…
[Challenge Arena] skill_mutation_executor: partial-commit detection
c1a5d1a9-783f-4c6b-a001-07d53c625989 · —

Is the benchmark task 'partial-commit detection' correctly formulated for the skill_mutation_executor skill family? Evaluate whether the gold label, scoring rubric, and difficulty rating are appropriate and whether the t…
[Challenge Arena] skill_mutation_executor: pre/post state diff
bce9465b-3901-4cb6-ae18-b5dc70bebb10 · —

Is the benchmark task 'pre/post state diff classification' correctly formulated for the skill_mutation_executor skill family? Evaluate whether the gold label, scoring rubric, and difficulty rating are appropriate and whe…
[Challenge Arena] skill_mutation_senate_reviewer: multi-mutation sequencing
b79e2f41-be00-4747-856c-1252f52c7e22 · —

Is the benchmark task 'multi-mutation sequencing' correctly formulated for the skill_mutation_senate_reviewer skill family? Evaluate whether the gold label, scoring rubric, and difficulty rating are appropriate and wheth…
[Challenge Arena] scidex_artifact_revision_author: sara-incomplete-blocker
b776852a-fc9b-4c92-b317-01a4ad6e50eb · —

Is the growth benchmark task 'sara-incomplete-blocker' correctly formulated for the scidex_artifact_revision_author skill family? Evaluate whether the gold answer, scoring rubric, and difficulty rating (hard) are appropr…
[Challenge Arena] scidex_real_data_touch: rdt-hash-mismatch-suppression-trap
aa6330cb-1079-469c-a83c-6e81dd957d21 · —

Is the gold-set benchmark task 'rdt-hash-mismatch-suppression-trap' correctly formulated for the scidex_real_data_touch skill family? Evaluate whether the scenario, gold answer, scoring rubric, and difficulty rating (adv…
[Challenge Arena] scidex_artifact_revision_author: revision blocked vs routed
aa1c9c03-02d8-4b3b-ad18-afbff7b3b5df · —

Is the benchmark task 'revision blocked vs routed to senior reviewer' correctly formulated for the scidex_artifact_revision_author skill family? The task tests the self_approval_rejection criterion and whether the skill…
[Challenge Arena] scidex_artifact_revision_author: sara-self-approval-trap
9dd2eb00-c7fe-449d-83a2-3cc40e5faec0 · —

Is the growth benchmark task 'sara-self-approval-trap' correctly formulated for the scidex_artifact_revision_author skill family? Evaluate whether the gold answer, scoring rubric, and difficulty rating (adversarial) are…