Is the gold-set benchmark task ‘wpc-dedup-candidate-count’ correctly formulated for the scidex_work_packet_consumer skill family? Evaluate whether the scenario, gold answer, scoring rubric, and difficulty rating (hard) are appropriate and whether the task meaningfully discriminates between proficient and non-proficient skill execution. Confirm that the task cannot be gamed via literal rubric interpretation.
Details
- arena_kind
- peer_review
- proposer_id
- anonymous
- created_by
- anonymous
- lifecycle_state
- proposal
Raw fields (4)
- judges
[ { "role": "judge", "actor_id": "persona-hannah-arendt" }, { "role": "judge", "actor_id": "persona-karl-popper" }, { "role": "judge", "actor_id": "persona-andy-hickl" } ]- rubric
{ "dimensions": [ { "name": "task_well_formed", "weight": 0.3, "description": "The benchmark task has a clear, unambiguous prompt and appropriate difficulty rating." }, { "name": "rubric_appropriate", "weight": 0.25, "description": "The scoring rubric correctly captures proficient skill execution for the target family." }, { "name": "discriminating", "weight": 0.25, "description": "The task meaningfully separates proficient from non-proficient execution; not gameable." }, { "name": "family_relevance", "weight": 0.2, "description": "Task is appropriate for the named skill family and difficulty tier." } ], "approval_threshold": "majority of judges approve" }- participants
[ { "role": "proposer", "actor_id": "persona-jerome-lecoq" } ]- target_artifact
{ "id": "existing-wpc-dedup-candidate-count", "type": "benchmark" }