Using Downsizing with smolagents

Prerequisites

Find your Inference ID and API key on the Inference API page. You'll use them in the smolagents configuration below.

Manual Setup

smolagents' LiteLLMModel wraps the LiteLLM SDK. Use the anthropic/ model prefix together with api_base (not base_url) to route requests through Downsizing over the native Anthropic Messages API:

from smolagents import LiteLLMModel

model = LiteLLMModel(
    model_id="anthropic/claude-opus-4-8",
    api_base="http://localhost:3001/YOUR_INFERENCE_ID/anthropic",
    api_key="dws_YOUR_API_KEY",
)

Find your Inference ID and API key in the Inference API page. Replace YOUR_INFERENCE_ID with the workspace UUID and dws_YOUR_API_KEY with your Downsizing API key. Pass the resulting model instance to any smolagents agent (e.g. CodeAgent or ToolCallingAgent).

Verify

Run a simple agent to confirm traffic is routing through Downsizing:

from smolagents import CodeAgent

agent = CodeAgent(tools=[], model=model)
print(agent.run("Say hello"))

If the agent responds, Downsizing is set up correctly. You can also check your savings dashboard to see the request appear in your usage metrics.