LLM Operations
Cost, evaluation, routing and monitoring of language models.
LLM Operations
Small vs large language models: choosing the right size
Use a small language model for narrow, repetitive tasks you can test, and a large one for open-ended reasoning and broad knowledge. Decide on your own data.
LLM OperationsHow to evaluate LLM applications: test sets, judges, regressions
Evaluate an LLM app with a test set built from real cases, cheap code checks first, an LLM judge you have checked against people, and a rerun on every change.
LLM OperationsControlling LLM costs: gateways, budgets, caching and routing
Control LLM costs by routing all calls through one gateway, caching repeated prompts, sending routine work to small models and tracking cost per task.