model-interaction-api

aghyad-deeb's avatarfrom aghyad-deeb

Simple API for interacting with locally hosted models via vLLM. Use when you need to chat with a model, send messages, or generate responses from a local model server.

⭐1stars🔀1forks📁View on GitHub🕐Updated Dec 28, 2025
[Developer Tools]

When & Why to Use This Skill

The Model Interaction API is a specialized Claude skill designed to streamline communication with locally hosted Large Language Models (LLMs) via vLLM. It provides a comprehensive CLI for managing multi-turn chat sessions, handling system/user/assistant messages, and maintaining session state through robust checkpointing. By offering built-in logging compatible with the Inspect framework, it bridges the gap between local model deployment and rigorous performance evaluation.

Use Cases

  • Local Model Prototyping: Quickly test and interact with vLLM-hosted models (like Llama or Mistral) directly from the command line without building a custom interface.
  • Conversation State Management: Save and resume complex, multi-step AI interactions using session checkpoints to debug specific failure points or branch conversations.
  • Model Evaluation & Benchmarking: Generate detailed interaction logs in Inspect format to analyze model behavior, accuracy, and response consistency during development.
  • Prompt Engineering Sandbox: Iteratively refine system prompts and tool-use sequences in a controlled local environment to optimize model performance before production deployment.
namemodel-interaction-api
descriptionSimple API for interacting with locally hosted models via vLLM. Use when you need to chat with a model, send messages, or generate responses from a local model server.

Model Interaction API

A simple CLI for interacting with locally hosted models via vLLM.

CLI: python3 .claude/skills/model-interaction-api/scripts/eval_cli.py

Find model ID: curl -s http://localhost:8000/v1/models | python3 -c "import sys,json; print(json.load(sys.stdin)['data'][0]['id'])"

Commands

Command Description
CLI start MODEL_ID --label NAME Start a new session
CLI system "msg" Add a system message
CLI user "msg" Add a user message
CLI assistant "msg" Add an assistant message
CLI tool "output" Add tool output (auto-wrapped in <output> tags)
CLI generate Generate model response (auto-adds to conversation)
CLI show Print full conversation
CLI status Show session info
CLI clear Reset session

Session Checkpointing

Command Description
CLI save [PATH] Save session to checkpoint file (default: sessions/{label}/{date}/)
CLI load PATH Load a previously saved session checkpoint
CLI export PATH Copy the .eval log file to a destination path

Example

CLI=".claude/skills/model-interaction-api/scripts/eval_cli.py"

# Start session
python3 $CLI start "model-name" --label "my_chat"

# Add messages
python3 $CLI system "You are a helpful assistant."
python3 $CLI user "Hello!"

# Generate response
python3 $CLI generate

# Add tool output if model requested a command
python3 $CLI tool "command output here"

# Continue
python3 $CLI generate

# Save checkpoint (for later resumption)
python3 $CLI save

# Or save to specific path
python3 $CLI save /path/to/my_session.json

# Later: load a saved session
python3 $CLI load sessions/my_chat/2025-12-17/session_12-30-00.json

# Export .eval log to another location
python3 $CLI export /path/to/destination/

Logs

Conversations are saved in Inspect format to logs/. View with:

inspect view --log-dir logs/

Session Checkpoints

Session checkpoints are JSON files saved to sessions/ by default. They contain:

  • Full conversation history
  • Model ID and session metadata
  • Path to the associated .eval log file

Use checkpoints to:

  • Resume interrupted sessions
  • Branch conversations from a specific point
  • Archive sessions for later reference