llm-inference

Use when wanting to interact with any LLM - Explains available inference endpoints so the agent selects suitable models.

⭐2stars🔀0forks📁View on GitHub🕐Updated Dec 11, 2025⬇️2downloads

When & Why to Use This Skill

This Claude skill enables seamless interaction with multiple Large Language Models (LLMs) through OpenAI-compatible inference endpoints. It empowers agents to intelligently select the most appropriate model—ranging from high-speed options like Llama 3.1 to powerful reasoning models like GLM-4—based on specific task requirements, context window constraints, and performance needs.

Use Cases

  • Dynamic Model Selection: Automatically routing simple text processing tasks to fast, efficient models like Llama-3.1-8b while reserving complex reasoning or creative tasks for high-parameter models like GPT-OSS-120B.
  • Large-Scale Document Processing: Utilizing models with 65,536 token context windows to analyze extensive datasets, long-form reports, or entire codebases without losing context.
  • Cross-Model Format Standardization: Leveraging diverse inference providers to transform arbitrary strings into structured formats (JSON, Markdown) by applying explicit system prompts across different model architectures.
  • Performance Optimization: Balancing request-per-minute limits and token throughput by switching between available endpoints to maintain agent responsiveness during high-load periods.
namellm-inference
description"Use when wanting to interact with any LLM - Explains available inference endpoints so the agent selects suitable models."

LLM Inference

The Cloudflare Pages function functions/cerebras-chat.ts provides OpenAI-compatible LLM inference. See tools/cerebras-llm-inference/index.html for a working example.

Available models

Model Max context tokens Requests / minute Tokens / minute
gpt-oss-120b 65,536 30 64,000
llama-3.3-70b 65,536 30 64,000
llama3.1-8b 8,192 30 60,000
qwen-3-235b-a22b-instruct-2507 65,536 30 64,000
qwen-3-235b-a22b-thinking-2507 65,536 30 60,000
qwen-3-32b 65,536 30 64,000
zai-glm-4.6 64,000 10 150,000
  • llama3.1-8b is the fastest option.
  • zai-glm-4.6 is the most powerful option.
  • gpt-oss-120b remains the best all rounder.

LLMs are not just for chat: they can be used to process any string in any arbitrary way. If making a tool that requires the LLM to respond in a specific way or format then be very clear and explicit in its system prompt; eg what to include/exclude, plain/markdown formatting, length, etc.