Add once. Keep your calls.

JavaScript

Install the SDK and initialize it before creating model clients.

npm install @smallermodels/node

import { init } from "@smallermodels/node";

init({
  apiKey: process.env.SMALLERMODELS_API_KEY,
});

Leave the rest of the application alone. Calls through supported clients are detected automatically, including calls that use different providers, models, or fallback configurations.

Python

Python uses the same one-time initialization.

pip install smallermodels

import os
import smallermodels

smallermodels.init(
    api_key=os.environ["SMALLERMODELS_API_KEY"],
)

No decorators, replacement clients, or changes to individual calls are required.

Run the preview locally

The local harness includes the monitoring SDK, a disposable SQLite control plane, a mock inference request, and the live operations portal. Production telemetry is handled by the Cloudflare Worker and Postgres control plane.

npm install
npm run control-plane:dev

npm run demo:observe

Open the SDK demo project to inspect the captured operation and recent call. The local database is stored under .smallermodels/.

Interactive test app

Start the instrumented playground to submit support drafting, restaurant text-to-JSON, and summarization requests while watching its monitoring panel update.

npm run demo-app:dev

Open the SDK test app.

1. Deploy and monitor

Deploy normally. Every user response still comes from the provider and model your application already calls. SmallerModels identifies supported inference traffic and measures its request shape, latency, output structure, and estimated cost.

2. Review the portal

Calls appear in the portal as they are observed. SmallerModels groups similar traffic into operations such as support drafting, search-query parsing, or text-to-JSON extraction. Each operation shows volume, current models, latency, cost, and representative examples.

3. Run discovery

Start discovery for an operation from the portal. SmallerModels evaluates cheaper models against captured traffic and measures whether they preserve the behavior your application needs. Exact match and schema checks can run automatically; semantic tasks can use an evaluator configured for the operation.

The result is a proposed policy with evidence, not a model recommendation in isolation.

4. Enable the policy

Review the measured quality, coverage, latency, and savings, then select Enable. The SDK receives the approved policy automatically. Eligible requests use the smaller model; every other request continues through its original call path.

1. 1
   Monitor

   Existing providers serve all production responses.

2. 2
   Validate

   Discovery measures candidate models on representative traffic.

3. 3
   Enable

   The approved policy routes only requests inside its validated region.

4. 4
   Monitor

   The portal tracks quality, fallbacks, latency, coverage, and savings.

Fallback behavior

SmallerModels retains the original request path for every supported call. That means different parts of one application can keep using different providers, endpoints, models, credentials, and settings. When a request is outside the approved policy or the smaller path cannot run safely, the original call executes unchanged.

Once response streaming begins, the serving path is fixed. The SDK never splices output from two models into one response.

Compatibility

Automatic monitoring is available for explicitly supported SDKs and versions. The first release targets the official OpenAI and Anthropic clients for Node and Python. Unsupported traffic passes through unchanged rather than changing application behavior.

Data and security

• Existing provider credentials stay in the customer process.
• Unsupported calls pass through without being modified.
• Request and response fields can be redacted before telemetry leaves the process.
• Capture, retention, and evaluator settings are configured per project.
• Production routing begins only after a policy is explicitly enabled.