How to Secure LLM API Keys and Endpoints in Production (2026)

LLM API keys are the most expensive secrets in your codebase. This guide covers key rotation, endpoint hardening, proxy-layer security, and automated vulnerability scanning for AI-powered apps.

By Theo · Maker of Tokenwise
LLM Security

Key takeaways

  • LLM API keys are higher-value targets than most secrets because they guard compute budgets, not just data — a stolen key can cost hundreds per hour.
  • The five common leak vectors in production are client bundles, source maps, error responses, debug endpoints, and deploy artifacts.
  • Proxy-layer security requires workspace-scoped keys, rate limiting, credential isolation, and TLS on every hop.
  • Treat model output as untrusted input: never render raw HTML from completions, and validate prompt inputs server-side.
  • Automated security scanning catches exposed keys and misconfigurations that manual review misses — run it after every deploy.
  • Key rotation should be quarterly at minimum, with post-rotation scanning to confirm the old key no longer appears in production.

LLM API keys are expensive secrets. A leaked OpenAI or Anthropic key does not just expose data — it opens a direct line to your billing account. At scale, a compromised key can burn through thousands of dollars in hours before anyone notices.

This guide walks through the security model for apps that call LLM providers, the specific risks introduced by proxies and observability layers, and the checks that catch exposed keys before an attacker does.

Why LLM API keys are higher-value targets than most secrets

Traditional API keys guard data. LLM keys guard compute budgets. A stolen database key lets an attacker read records. A stolen LLM key lets them generate millions of tokens at your expense — and there is no row-level access control to limit the damage.

The economics make these keys particularly attractive:

  • A single GPT-4.1 or Claude Opus key can cost the owner $50–500 per hour of sustained abuse.
  • Stolen keys are resold on forums and Telegram channels within minutes of exfiltration.
  • Most providers rate-limit by account, not by key — so abuse degrades service for your real users.
  • Unlike database credentials, LLM keys often have no IP allowlist or scope restriction by default.

The Keybrum security scanner checks for exposed LLM API keys as part of its standard scan. It tests 80 secret patterns across your deployed app — JavaScript bundles, source maps, API responses, environment files, and debug endpoints — and verifies whether each key is still active by calling the provider's own validation endpoint.

The five places LLM keys leak in production

Leaked keys rarely come from a developer pasting them into a public repo (though that still happens). In production apps, the common leak vectors are subtler:

  1. Client-side JavaScript bundles. Framework build systems inline environment variables at build time. A NEXT_PUBLIC_OPENAI_KEY or a Vite import.meta.env reference ships the key to every browser.
  2. Source maps. Even when the bundle is minified, the source map often contains the original variable assignment. Source maps are served by default in many frameworks.
  3. Error responses. An unhandled exception in a server route can serialize the request context — including headers with Authorization: Bearer sk-... — into a 500 response body.
  4. Debug and admin endpoints. /admin/queues, /_debug, /api/health with verbose mode — these management interfaces often dump environment variables or configuration objects that include secrets.
  5. Git history and deploy artifacts. A key removed from the current code may still live in a previous commit, a Docker layer, or a CI artifact.

An automated security scan catches these before an attacker does. Keybrum's free check runs 338 attack checks against your live URL — including dedicated secret detectors for OpenAI, Anthropic, Google AI, Mistral, Cohere, Groq, DeepSeek, xAI, and OpenRouter keys — and reports what a stranger can see from the outside.

Proxy-layer security: what your observability stack exposes

If you route LLM traffic through a proxy for observability and cost tracking, the proxy itself becomes a high-value target. It holds every provider key, sees every request payload, and often stores full prompt and completion text.

The security checklist for any LLM proxy deployment:

  • Authentication on the proxy endpoint. Your /v1/proxy or /v1/chat/completions route must require a workspace-scoped API key. An unauthenticated proxy is an open relay for LLM abuse.
  • Key isolation. Each workspace should have its own proxy key. A compromised key should not give access to other workspaces' traffic or provider credentials.
  • Rate limiting. Enforce per-key request limits. Without them, a leaked proxy key enables the same abuse as a leaked provider key — but through your infrastructure.
  • No provider keys in responses. The proxy should never echo provider credentials in error messages, headers, or response bodies. Strip them at the edge.
  • TLS everywhere. Proxy traffic carries full prompts and completions. Any unencrypted hop exposes user data and potentially PII.

Tokenwise's proxy enforces all five by default: workspace-scoped tw_api_* keys with per-tier rate limits, isolated provider credentials, and TLS termination at the edge. But every proxy deployment should be tested from the outside — which is what a web app security scanner like Keybrum does. It probes your deployed app the way a stranger would: no source code access, no credentials, just the same HTTP requests an attacker sends.

Hardening your LLM-powered app: a practical checklist

Beyond key management, LLM-powered apps have a broader attack surface than traditional CRUD apps. The model endpoint itself is a vector:

Input validation

  • Validate and sanitize prompt inputs on the server before they reach the model. Prompt injection is real, and the model cannot reliably defend itself.
  • Set max_tokens on every call. An unbounded completion can generate megabytes of text — and you pay for every token.
  • Reject requests that exceed a sane input size. A 200K-token context window does not mean every request should use it.

Output handling

  • Never render model output as raw HTML. Treat completions as untrusted user input — because attackers can influence them through prompt injection in retrieved context.
  • Strip or escape any HTML, JavaScript, or SQL in model responses before they reach the database or the browser.
  • Log and monitor output for unexpected patterns — model responses that contain URLs, scripts, or system commands may indicate injection.

Access control

  • Put LLM endpoints behind the same authentication as the rest of your API. A /api/chat route that accepts anonymous requests is an open invitation.
  • Scope model access by user role. Not every user needs access to every model or every feature.
  • Implement budget caps per user or workspace. Cost optimization and security overlap here: a runaway prompt loop is both a cost and a security event.

Automated security testing for AI apps

Manual security reviews miss things. The app changes with every deploy, and the attack surface grows with every new API route, every new model integration, every new framework upgrade.

Automated scanning catches the regressions that code review misses:

  • Exposed secrets in production. Keys that were safe in staging but leaked through a build misconfiguration.
  • Unprotected admin endpoints. A queue dashboard, a debug route, a health check that dumps configuration.
  • Missing authentication. An API route added in a rush that skipped the auth middleware.
  • CORS misconfigurations. An overly permissive Access-Control-Allow-Origin that lets any site call your API.
  • Supabase and Firebase misconfigurations. Row-level security policies that look correct but fail under specific conditions.

Keybrum runs 338 security checks against your live app URL — covering OWASP Top 10, exposed secrets, infrastructure misconfigurations, and framework-specific vulnerabilities for Next.js, Supabase, Firebase, and other common stacks. Each finding comes with a one-click fix prompt you can paste into Cursor, Claude Code, or Lovable. The free security check takes a URL and returns results in minutes.

Key rotation: the security habit most teams skip

Rotating LLM API keys should be as routine as rotating database passwords. The reality: most teams create a key once, paste it into production, and never touch it again.

A rotation plan for LLM keys:

  1. Generate a new key in the provider dashboard. Most providers support multiple active keys per project.
  2. Deploy the new key to production alongside the old one. If your proxy or app supports key arrays, add the new key first.
  3. Verify traffic flows through the new key. Check your observability dashboard — Tokenwise shows which key served each request.
  4. Revoke the old key in the provider dashboard.
  5. Scan your app to confirm the old key no longer appears anywhere in the deployed output. A Keybrum scan verifies this automatically — it checks whether previously leaked keys are still active after rotation.

Try this week

  • Run a free security check on your app to see what a stranger can find — exposed keys, open admin panels, missing authentication, CORS issues.
  • Audit your environment variable naming: anything prefixed with NEXT_PUBLIC_, VITE_, or EXPO_PUBLIC_ ships to the client.
  • Check that your LLM proxy endpoint requires authentication. Try calling it without credentials from outside your network.
  • Review your error handling: trigger a 500 on a route that uses LLM keys and check whether the error response leaks the key or its headers.
  • Set a calendar reminder to rotate your LLM API keys quarterly, or after any team member leaves.

Frequently asked questions

How do I check if my LLM API keys are exposed in production?
Run an automated security scan against your live URL. Keybrum checks 80 secret patterns across JavaScript bundles, source maps, API responses, and debug endpoints, then verifies whether each found key is still active.
What makes LLM API keys more dangerous than other secrets?
LLM keys guard compute budgets. A stolen key lets an attacker generate millions of tokens at your expense — often $50–500 per hour — with no row-level access control to limit the damage.
Should I use a proxy for LLM API security?
A proxy adds observability and lets you enforce rate limits, key isolation, and budget caps at a single point. But the proxy itself becomes a high-value target, so it needs authentication, TLS, and regular security testing.
How often should I rotate LLM API keys?
At minimum quarterly, and immediately after any team member leaves or any security scan reports an exposed key. Most providers support multiple active keys, so rotation can happen without downtime.
What is prompt injection and why does it matter for security?
Prompt injection is when an attacker manipulates model inputs — through user fields, retrieved documents, or URL parameters — to override the system prompt and extract data, bypass restrictions, or trigger unintended actions.
Can I use a free tool to test my app for LLM key exposure?
Yes. Keybrum offers a free security check at keybrum.com/check that scans your live URL for exposed secrets, missing authentication, and 338 other security checks.

More guides

Add this to your app in one line

Point your OpenAI baseURL at Tokenwise and every call is logged, priced, and optimizable — no SDK rewrite, no LangChain required.