This Claude skill provides a robust framework for evaluating document relevance within RAG (Retrieval-Augmented Generation) pipelines. It utilizes binary scoring and structured reasoning to filter retrieved content, ensuring that downstream generation is based solely on pertinent information, thereby minimizing hallucinations and significantly improving the accuracy of AI-generated responses.

When should I use grading?

grading is useful in the following scenarios: • RAG Pipeline Optimization: Automatically grade and filter documents retrieved from vector databases like ChromaDB to ensure only high-quality, relevant context is used for answer generation. • Adaptive Query Rewriting: Implement a self-correcting search loop that triggers query rewriting and retries when initial retrieval results fail to meet relevance thresholds. • Knowledge Base Auditing: Systematically evaluate the topical alignment of documents within a repository to identify gaps or low-quality content. • Automated Content Filtering: Streamline document processing workflows by programmatically identifying and discarding irrelevant text chunks based on specific user queries.

name	grading
description	Evaluates document relevance to queries using binary scoring (yes/no) with reasoning. Use when filtering retrieved documents, determining relevance for answer generation, or when the user mentions grading, filtering, or document quality.

Document Grading

Instructions

Grade documents for relevance using the functions in components/grader.py. The grading uses Claude API with structured outputs for consistent binary decisions.

Default workflow:

Retrieve documents from ChromaDB or web search
Call grade_documents(query, documents) to grade all documents
Call filter_relevant_documents(graded_docs) to get only relevant ones
Check if len(relevant_docs) == 0 to trigger retry logic

Key functions:

# Grade multiple documents (preferred)
graded_docs = grade_documents(query, documents)

# Filter to relevant only
relevant_docs = filter_relevant_documents(graded_docs)

# Grade single document (if needed)
result = grade_document(query, document)

Grading returns:

relevant: Boolean (True/False)
reasoning: Explanation of decision

Critical for adaptive retry: When ALL documents are graded as not relevant (len(relevant_docs) == 0), the pipeline triggers query rewriting and retry.

Implementation: components/grader.py, uses GRADING_MODEL from config.py (default: Haiku 4.5), temperature 0.

Examples

Example 1: Standard grading workflow

from components.grader import grade_documents, filter_relevant_documents
from tools.chromadb_tool import query_chromadb

# Input
docs = query_chromadb("gaming laptops", top_k=5)
# Retrieved: 5 documents from catalog

# Grade
graded = grade_documents("gaming laptops", docs)
relevant = filter_relevant_documents(graded)

# Output
# Found 3 relevant documents out of 5 total
# Each has grading_result: {relevant: True, reasoning: "..."}

Example 2: Triggering retry

# Input: Vague query
docs = query_chromadb("fast computer", top_k=5)
graded = grade_documents("fast computer", docs)
relevant = filter_relevant_documents(graded)

# Output
# len(relevant) == 0
# → Triggers query rewrite and retry

Example 3: In adaptive RAG pipeline

# After retrieval
graded_documents = grade_documents(current_query, retrieved_documents)
relevant_documents = filter_relevant_documents(graded_documents)

if len(relevant_documents) == 0:
    # No relevant docs - rewrite query and retry
    if num_retries < RETRY_LIMIT:
        rewrite_result = rewrite_query(current_query, previous_context)
        current_query = rewrite_result['rewritten_query']
        num_retries += 1
        # Loop back to retrieval
else:
    # Found relevant docs - proceed to generation
    generate_answer(original_query, relevant_documents)

grading

When & Why to Use This Skill

Use Cases