la-bench-procedure-generator

dakesan 的头像作者 dakesan

当从LA-Bench格式JSONL文件生成详细实验程序时,应使用此技能。它编排多个子代理来解析输入数据、获取参考资料、生成程序、验证输出、完善结果并产生最终格式化的输出。通过处理LA-Bench数据或从data/public_test.jsonl或data/private_test_input.jsonl文件生成实验协议的请求触发。

📖 中文导读:当从LA-Bench格式JSONL文件生成详细实验程序时,应使用此技能。它编排多个子代理来解析输入数据、获取参考资料、生成程序、验证输出、完善结果并产生最终格式化的输出。通过处理LA-Bench数据或从data/public_test.jsonl或data/private_test_input.jsonl文件生成实验协议的请求触发。 以下正文为该技能的原始 SKILL.md 内容(英文),上方安装命令可直接使用。

name: la-bench-procedure-generator description: This skill should be used when generating detailed experimental procedures from LA-Bench format JSONL files. It orchestrates multiple subagents to parse input data, fetch reference materials, generate procedures, validate outputs, refine results, and produce final formatted outputs. Triggered by requests to process LA-Bench data or generate experimental protocols from data/public_test.jsonl or data/private_test_input.jsonl files.

LA-Bench Procedure Generator

Overview

This skill orchestrates a multi-agent workflow to generate detailed experimental procedures from LA-Bench format JSONL input files. Instead of generating procedures in a single step, it coordinates specialized subagents for parsing, reference fetching, generation, validation, refinement, and final output creation.

When to Use This Skill

  • When the user requests to generate experimental procedures from LA-Bench data
  • When processing files like data/public_test.jsonl or data/private_test_input.jsonl
  • When the user asks to "process LA-Bench format" or "generate detailed experimental protocols"

Core Workflow

This skill follows a workflow-based orchestration pattern with six distinct phases:

Phase 0: Initialize

  1. Create TODO list using TodoWrite tool to track all phases
  2. Verify input/output paths:
    • Input: data/public_test.jsonl or data/private_test_input.jsonl
    • Output: outputs/runs/generated_YYYYMMDD_HHMMSS.jsonl
  3. Set up workspace for intermediate results if needed

Phase 1: Data Acquisition (Parallel Execution)

Launch multiple Task tools in parallel to maximize efficiency:

Task 1: JSONL Parser Agent

Prompt: "Parse the JSONL file at [path] and extract all entries.
Return a list of all entries with their id, input, and output fields."

Task 2: Reference Fetcher Agent (uses web-reference-fetcher skill)

Prompt: "Use the web-reference-fetcher skill to fetch content from
all reference URLs found in the JSONL entries."

Task 3: Procedure Generator Agent (one per entry or batched)

Prompt: "Generate detailed procedure_steps for entry [id] using:
- instruction
- mandatory_objects
- source_protocol_steps
- fetched reference content
Output format: List of {id: int, text: str} objects"

Phase 2: Quality Validation

Task 4: Checker Agent

Prompt: "Validate the generated procedures against quality criteria
in references/quality_criteria.md. Check:
- Output format compliance
- Logical consistency
- Completeness
Report any issues found."

Phase 3: Refinement (Conditional)

If validation finds issues:

Task 5: Refiner Agent

Prompt: "Address the following validation issues: [issues].
Regenerate or fix the affected procedure_steps."

Phase 4: Final Output

Task 6: Output Generator Agent

Prompt: "Format all validated procedure_steps into LA-Bench output format
and save to outputs/runs/generated_[timestamp].jsonl.
Each line should be: {id: string, output: {procedure_steps: [...]}}
Use assets/output_schema.json as reference."

Important Notes

Data Flow

  • All entries in the JSONL are processed (loop through all IDs)
  • Data passes between agents through shared workspace or direct handoff
  • See references/data_flow.md for detailed inter-agent communication patterns

TODO Management

  • Update TODO status after each phase completion
  • Mark agents as in_progress when launching
  • Mark as completed only when phase is fully done

Parallel vs Sequential

  • Phase 1 agents run in parallel (use single message with multiple Task calls)
  • Phases 2-4 run sequentially (each depends on previous completion)

Error Handling

  • If any agent fails, document the failure and retry with adjusted prompt
  • If persistent failures occur, consult references/agent_specs.md for troubleshooting

Example Session

See references/example_session.md for a complete walkthrough of a typical execution.

Resources

references/

Documentation loaded into context as needed:

  • agent_specs.md: Detailed specifications for each subagent (prompts, inputs, outputs, implementation guidelines)
  • data_flow.md: How data passes between agents, workspace structure, and file formats
  • example_session.md: Real example of a complete workflow execution with agent interactions

assets/

Files used in final output:

  • output_schema.json: JSON schema for the final output format, ensures compliance with LA-Bench expected format