determinism

ingpoc's avatarfrom ingpoc

Use when verifying outcomes with code instead of LLM judgment, versioning prompts with hashes, or ensuring reproducible agent behavior. Load for any critical verification. Scripts return boolean exit codes, not subjective assessments. Prompts use semantic versioning with SHA256 validation.

⭐5stars🔀0forks📁View on GitHub🕐Updated Jan 2, 2026
[Evaluation and Benchmarks]

When & Why to Use This Skill

This Claude skill enhances agent reliability by replacing subjective LLM judgments with deterministic code verification and semantic prompt versioning. By leveraging SHA256 validation and binary exit codes, it ensures reproducible agent behavior and consistent outcomes, making it an essential tool for developers building production-grade, mission-critical AI workflows.

Use Cases

  • Automated Testing: Using scripts like pytest to verify code changes with binary exit codes instead of relying on the LLM's subjective assessment of code quality.
  • Data Integrity Validation: Programmatically checking JSON structures or schema compliance using Python scripts to prevent hallucinations in data extraction tasks.
  • Prompt Version Control: Implementing semantic versioning and hash-validation for critical prompts to ensure that agent behavior remains consistent across different environments.
  • Infrastructure Health Checks: Verifying server status or API availability via curl commands to provide objective, real-time system monitoring within an agentic workflow.
  • Reproducible Research: Ensuring that complex agent tasks can be audited and repeated by enforcing strict code-based verification steps rather than probabilistic LLM checks.
namedeterminism
description"Use when verifying outcomes with code instead of LLM judgment, versioning prompts with hashes, or ensuring reproducible agent behavior. Load for any critical verification. Scripts return boolean exit codes, not subjective assessments. Prompts use semantic versioning with SHA256 validation."
keywordsverification, deterministic, hash, version, reproducible, code-not-judgment

Determinism

Reproducible outcomes through code verification and prompt versioning.

Core Principle

"Claude can run scripts without loading either the script or the PDF into context. And because code is deterministic, this workflow is consistent and repeatable." - Anthropic Engineering

Instructions

  1. Replace LLM judgment with script verification
  2. Version prompts with semantic versioning
  3. Hash-validate critical prompts: scripts/validate-prompt.sh
  4. Use exit codes (0 = pass, 1 = fail), not text

LLM Judgment vs Code Verification

Task LLM (Bad) Code (Good)
Tests passed? "The tests appear to pass" pytest; echo $? → 0 or 1
Valid JSON? "This looks like valid JSON" python -c "json.load(f)"
Server running? "The server should be up" curl -s localhost/health

References

File Load When
references/code-verification.md Writing verification scripts
references/prompt-versioning.md Versioning/hashing prompts