What does this skill do?

Cost Optimization for LLM Skill provides a composable architecture to control and reduce spending on LLM APIs without sacrificing response quality. It combines intelligent model routing based on task complexity, an immutable cost tracker, selective retries in response to transient errors, and a prompt cache to minimize latency and cost per token in repeated iterations.

Dynamic Routing
It automatically selects the most cost-effective model for simple tasks and the most powerful one for complex tasks based on thresholds for length and number of elements.
Budget Control
Keep API spending within a strict limit using an immutable tracker that logs every call and halts processing if the budget is exceeded.
Selective retries
Retry only transient errors, such as rate limits or connection drops, using exponential backoff. Fail immediately in the event of authentication errors or invalid requests.
Prompt Cache
Mark long system prompts with `cache_control` to avoid resending them with every request, thereby reducing latency and the cost per token in repeated iterations.

Usage examples

🔀 Routing by Complexity
I have 50 short texts to categorize and 3 long documents of 15,000 words each to summarize. Use dynamic routing to assign the appropriate model to each task.
📊 Budget Monitoring
Process this batch of 200 items with a maximum budget of 5€. Stop processing if the limit is exceeded, and provide me with the CostTracker showing the cumulative cost.
🔄 Retry with backoff
Execute this Claude API call with selective retries. If a rate limit error occurs, retry up to 3 times using exponential backoff.
💾 System prompt cache
I'm going to make 10 requests using the same 2,000-word system prompt. Enable the prompt cache to reduce the cost and latency for each iteration.

Features

Intelligent Model Routing Automatically selects between economical and powerful models based on the length of the text and the number of elements, with the option to specify a particular model.
Immutable Cost Tracker Use frozen dataclasses to record cumulative spending without altering the previous state, ensuring full budget traceability.
Selective retries with backoff Retry only temporary errors (rate limit, connection, server) using exponential backoff. Fails immediately in the event of authentication errors or malformed requests.
Ephemeral prompt cache Mark system prompts with `cache_control` to prevent redundant re-sends and significantly reduce latency and cost per token.
Composable pipeline It integrates routing, budget control, retries, and caching into a single pipeline that processes text, verifies the budget, and records the cost of each operation.

Frequently asked questions

The skill is primarily designed for the Anthropic (Claude) API, but the architecture is modular and can be adapted to other providers, such as OpenAI, by adjusting the model names and handling transient errors.
Yes, you need the Anthropic Python SDK (anthropic) and Python 3.10+ with support for dataclasses with slots. Install it with: pip install anthropic.
An immutable CostTracker based on frozen dataclasses is used. Each API call returns a new tracker with the updated cumulative cost. If the total exceeds the budget_limit, the system throws an exception and stops processing.
Yes. The thresholds for text length and number of elements are configurable in the `select_model` function. You can also force a specific model using the `force_model` parameter.

Related skills

More skills in the same category:

Cost Optimization for LLMs — Reduce Spending on AI APIs

¿Prefieres escuchar el contenido? Genera la narración de audio con un clic.