How to Reduce LLM Costs in Node.js Apps (2026)

Most Node.js apps overspend on LLMs for boring reasons: they send too much context, use a premium model for trivial work, regenerate identical answers, and only look at cost after the bill lands. I’ve made all four mistakes.

If you want to reduce LLM cost in Node.js, the biggest wins are not exotic. Route requests by difficulty, trim inputs before the API call, cache stable work, batch background jobs, and put a dollar figure on every route in production.

My default pattern in 2026: use a cheap fast model for classification and extraction, escalate only the hard cases, and treat output tokens like the expensive part of the meal — because they usually are.

Start with a per-request cost budget

Before I touch prompts, I set a target cost per user action. Not monthly spend. Per action. A chat turn, a document summary, a support-ticket triage, a code review comment — each gets a budget in cents or fractions of a cent.

The formula is simple:

cost = inputTokens / 1,000,000 × inputPrice + outputTokens / 1,000,000 × outputPrice

Take a support assistant sending 2,000 input tokens and receiving 400 output tokens. On gpt-4.1 at $2.00 / 1M input, $8.00 / 1M output, that is $0.0072 per turn. At 100,000 turns per day, you are spending about $720/day. On gpt-4o-mini at $0.15 / 1M input, $0.60 / 1M output, the same shape costs $0.00054 per turn, or $54/day.

In Node.js, I usually encode this as metadata next to each LLM call: route name, model, input token estimate, max output tokens, and budget. In Express, put it in middleware. In NestJS, put it in an interceptor around your LLM service. In Next.js route handlers, attach it before returning the stream. Boring plumbing, massive payoff.

Route models by difficulty, not vibes

Using one model everywhere is the fastest way to burn money. I almost never start with the most capable model. I start with the cheapest model that can reliably do the job, then escalate.

For production Node.js apps, my practical routing looks like this: gpt-4o-mini, gpt-4.1-nano, or gemini-2.0-flash for classification, rewrite, tags, short extraction, and guardrail checks; gpt-4.1-mini or gemini-2.5-flash for heavier structured extraction; gpt-5.1, gpt-4.1, claude-sonnet-4, or gemini-2.5-pro when quality actually matters; o3, o4-mini, or o3-pro only for reasoning tasks that justify latency and price.

Here’s the table I’d keep next to the codebase:

Model	Pricing	Context window	Best use in Node.js apps
gpt-4o-mini	$0.15 / 1M input, $0.60 / 1M output	128k	Cheap chat, summarization, routing, lightweight agents
gpt-4.1-mini	$0.40 / 1M input, $1.60 / 1M output	1M	Long-context extraction, code-aware tasks, reliable JSON
gpt-4.1	$2.00 / 1M input, $8.00 / 1M output	1M	High-quality generation, complex edits, long docs
gpt-5.1	$1.25 / 1M input, $10.00 / 1M output	Large context	Premium reasoning and generation when output quality matters
o4-mini	$1.10 / 1M input, $4.40 / 1M output	Large context	Affordable reasoning, planning, multi-step tool use
claude-sonnet-4	$3.00 / 1M input, $15.00 / 1M output	200k	Writing, analysis, agentic workflows with strong instruction following
gemini-2.0-flash	$0.10 / 1M input, $0.40 / 1M output	1M	Very cheap bulk work, document scanning, fast classification
gemini-2.5-pro	$1.25 / 1M input, $10.00 / 1M output	1M	Long-context reasoning and multimodal workloads

Blunt rule: if your first call is premium, your router is probably missing.

Trim prompts and context before the API call

Context windows got huge, which tricked people into sending everything. Don’t. A 1M-token window is a capability, not an invitation.

In Node.js, I trim in layers. First, keep the system prompt short and stable. Second, summarize conversation history after a few turns instead of replaying the whole transcript. Third, retrieve only the top chunks you need, not the top 20 because the vector database made it easy. Fourth, strip HTML, markdown boilerplate, base64 blobs, tracking text, and repeated navigation before the model sees the input.

For RAG in Next.js or Express, I like this flow: fetch candidate chunks, rerank them, apply a hard token budget, then build the prompt. If the prompt builder cannot fit the budget, it should drop low-value chunks automatically. Not throw an error at runtime. Not quietly exceed the budget. Drop.

A practical target: keep most interactive requests under 2,000 to 6,000 input tokens. For long documents, summarize sections locally or with a cheap model first, then send the compressed version to the expensive model. I’ve seen this cut input spend by 60–85% without hurting answer quality.

Cache stable LLM work aggressively

LLM caching is not just for chatbots. It works beautifully for classification, normalization, translation, product enrichment, title generation, embeddings-adjacent preprocessing, and “explain this error” features. If the same input and prompt produce a reusable answer, cache it.

In Node.js, I use a content-addressed key: hash the model name, versioned prompt, normalized input, temperature, response schema, and relevant options. Store the result in Redis, Postgres, or your job database. The prompt version matters. Skip it and you’ll serve stale answers after a prompt change. Ask me how I learned that one.

For deterministic tasks, set temperature low and cache exact inputs. For user-facing chat, cache only sub-steps: intent detection, policy checks, retrieved summaries, extracted entities, and tool results. Semantic caching can work, but I treat it as an optimization after exact caching because false positives are expensive in a different way.

Here is the math. If 35% of a 100,000-request/day workload hits cache and the uncached request costs $0.00054 on gpt-4o-mini, you save about $18.90/day. If the same workload runs on gpt-4.1 at $0.0072/request, the same cache hit rate saves $252/day. Same engineering. Very different bill.

Batch background jobs and cap output tokens

Interactive traffic and background traffic should not use the same execution pattern. User-facing requests need low latency. Backfills, enrichment, report generation, moderation sweeps, and nightly summaries should be batched, rate-limited, and queued.

In Node.js, I usually put BullMQ, Cloud Tasks, SQS, or a simple Postgres queue between the app and the provider. The worker groups small tasks by model and prompt type, sends batches where the API supports it, and retries with idempotency keys. This also protects you from a deploy that accidentally starts 30,000 expensive calls at once. Yes, that happens.

Output caps are the underrated lever. Developers obsess over prompt length, then allow 4,000 output tokens for a response that should be 120 words. Since output tokens are often several times more expensive than input tokens, this is wasteful. Set max_output_tokens per route. For classification, 20–80 tokens. For JSON extraction, 200–800. For summaries, 300–1,200. For long-form generation, make the user ask for it explicitly.

I also prefer structured outputs for cost control. A strict JSON schema reduces rambling, retry loops, parser failures, and “almost correct” responses. Less text, fewer retries, lower cost.

Monitor cost at the route, user, and feature level

If your dashboard only shows total LLM spend, it is nearly useless. You need cost by route, model, tenant, user, feature flag, prompt version, cache status, and retry count. That is how you find the endpoint quietly burning half the bill.

For Node.js, wrap every provider call in one internal function. Do not let teams call OpenAI, Anthropic, or Google clients directly from random route handlers. The wrapper should record input tokens, output tokens, model, latency, status, error type, cache hit, request ID, and estimated cost. In Express, add request context with AsyncLocalStorage. In NestJS, use dependency injection and interceptors. In Next.js, pass metadata from the route handler into the LLM client wrapper.

Set hard alerts: cost per request above budget, daily spend pacing above forecast, retry rate above normal, output tokens above expected range, and sudden model mix changes. I built Tokenwise because I wanted this level of visibility without stitching five dashboards together, but the principle matters more than the tool: measure cost where engineering decisions are made.

My favorite alert is simple: “This route spent 2× more per successful request than yesterday.” It catches prompt bloat, model regressions, cache misses, and runaway retries fast.

Verdict

If I were optimizing a Node.js app today, I would not start with provider negotiations or a rewrite. I’d put all LLM calls behind one wrapper, add per-route cost tracking, route 70–90% of traffic to cheap models, trim context hard, cache deterministic work, and cap outputs. That usually cuts spend before anyone has to argue about product scope.

My actual default stack: gpt-4o-mini or gemini-2.0-flash for cheap high-volume work, gpt-4.1-mini for long-context structured extraction, gpt-4.1 or claude-sonnet-4 for quality-sensitive user-visible responses, and o4-mini or o3 only when the task is genuinely reasoning-heavy. Simple, measurable, and much cheaper than pretending every request deserves the flagship model.