Using Downsizing with LangGraph

Prerequisites

Find your Inference ID and API key on the Inference API page. You'll use them in the LangGraph configuration below.

Manual Setup

LangGraph uses langchain-anthropic's ChatAnthropic class as its model, which accepts a base_url (aliased as anthropic_api_url) pointing at a custom Anthropic-compatible endpoint:

from langchain_anthropic import ChatAnthropic

model = ChatAnthropic(
    model="claude-opus-4-8",
    base_url="http://localhost:3001/YOUR_INFERENCE_ID/anthropic",
    anthropic_api_key="dws_YOUR_API_KEY",
)

Find your Inference ID and API key in the Inference API page. Replace YOUR_INFERENCE_ID with the workspace UUID and dws_YOUR_API_KEY with your Downsizing API key. Pass the resulting model instance to any LangGraph node or prebuilt agent.

Verify

Run a simple prebuilt agent to confirm traffic is routing through Downsizing:

from langgraph.prebuilt import create_react_agent

agent = create_react_agent(model, tools=[])
result = agent.invoke({"messages": [{"role": "user", "content": "Say hello"}]})
print(result["messages"][-1].content)

If the agent responds, Downsizing is set up correctly. You can also check your savings dashboard to see the request appear in your usage metrics.