Unbounded AI endpoints: paying for someone else's prompts

Why this keeps happening in AI-generated code

Code assistants optimise for the happy path: a request comes in, the model is called, a result goes back. Authentication, rate limiting and quotas are separate concerns that the prompt rarely asks for, so they are either skipped or added as optional scaffolding that silently turns itself off when an env var is missing.

Starter templates make it worse. Several popular ones guard paid calls with if (ratelimit) where ratelimit is only constructed when Redis env vars exist. The README calls those vars optional, the deploy button does not prompt for them, and the production deploy runs with zero throttling. Generated code copies that pattern without noticing what it implies.

A second source is framework semantics the model does not reason about. In Next.js, every exported async function in a "use server" file becomes a public HTTP endpoint. A helper that calls generateText for an internal title gets exported for convenience and is now an unauthenticated LLM API. Similarly, a Supabase service-role client bypasses row-level security, so a route that reads a stored api_key by an id from the request body will happily return another user's key.

• Auth, rate limiting and input caps are cross-cutting; they are not in the feature prompt, so they are not in the code.
• Optional rate limiting (if (ratelimit)) means no rate limiting on the default deploy.
• "use server" exports and service-role clients turn internal helpers into public, privileged endpoints.
• Shared env API keys (OPENAI_API_KEY, REPLICATE_API_KEY) plus open signup means every account spends from one wallet.

How to recognise it in your own repo

Start from the paid calls and walk backwards to the request. For each call site, answer three questions: who can reach this, how often, and how big can the input be. If any answer is "anyone", "unlimited" or "whatever the client sends", you have an unbounded endpoint.

Concrete things to grep for, based on the patterns seen across audited starters:

• Paid call sites: generateText, streamText, chat.completions.create, embeddings.create, replicate.com/v1/predictions, predictions. List every file that hits one.
• Missing auth in those files: no getServerProfile(), auth(), getServerSession, or equivalent before the paid call. In chatbot-ui, app/api/chat/custom/route.ts never calls getServerProfile().
• Optional limiting: if (ratelimit), ratelimit &&, or a limiter that returns early when REDIS_URL / UPSTASH_REDIS_REST_URL is unset or the client is not isReady.
• Bad identifiers: ratelimit.limit(ip ?? ""), headers.get("x-real-ip") with no fallback, or any limiter keyed on a value the client can set.
• "use server" files: check every exported function. Anything that calls a model, sends email, or touches billing should not be exported from one.
• Service-role usage: SUPABASE_SERVICE_ROLE_KEY, supabaseAdmin, createClient(... service_role ...) in a route that also reads an id from req.json() or the body.
• Env keys applied to all users: functions like addApiKeysToProfile that override per-user keys with process.env.OPENAI_API_KEY.
• Unbounded input: messages, text, prompt, chatSettings.model passed straight through with no .slice, no length check, no allowlist. No export const maxDuration on routes that download or embed files.
• Guest or anonymous identities created per request (createGuestUser on every cookieless hit) combined with a quota keyed on user id.

Real examples from public starters

mckaywrigley/chatbot-ui, app/api/chat/custom/route.ts:20. The route creates a service-role Supabase client, loads a model by customModelId taken from the request body, and uses that model's stored api_key and base_url to run completions with whatever messages the caller sends. There is no session check and no rate limit. Anyone who knows or guesses a models.id can run unlimited completions billed to another user's key, and the server will POST to an attacker-controlled base_url. The same repo, in lib/server/server-chat-helpers.ts:60, overrides every user's profile keys with server env keys when set, signup is open with no email confirmation, and none of /api/chat/*, /api/command, /api/retrieval/* or /api/assistants/openai apply rate limits or validate chatSettings.model or message size.

vercel/ai-chatbot, app/(chat)/actions.ts:23. generateTitleFromUserMessage is exported from a "use server" module, so it is a registered server action endpoint. It calls generateText with a client-controlled message, has no auth(), no rate limit and no length cap. It was only meant to be called from app/(chat)/api/chat/route.ts:132. Anyone who obtains the action ID can run unlimited AI Gateway requests with arbitrarily long prompts, bypassing both the per-user message quota and BotID, which only protects POST /api/chat. In the same repo, proxy.ts:24 redirects every cookieless request to /api/auth/guest, which inserts a new User row and issues a fresh identity with the same maxMessagesPerHour: 10 as registered users. Clearing cookies resets the quota. The only cross-identity control, checkIpRateLimit in lib/ratelimit.ts:22, returns early when REDIS_URL is unset, when the client is not yet isReady, or on any Redis error.

Nutlope/roomGPT, app/generate/route.ts:17. POST /generate has no auth and every request creates a paid Replicate prediction. Rate limiting only runs if (ratelimit), which is undefined whenever UPSTASH_REDIS_REST_URL and UPSTASH_REDIS_REST_TOKEN are missing; the README and .example.env call them optional and the Vercel deploy button only prompts for REPLICATE_API_KEY. At app/generate/route.ts:21 the identifier is ratelimit.limit(ipIdentifier ?? "") from x-real-ip alone. Where the header is missing, every user shares one 5/day bucket and the whole site locks after five generations. Where the header is passed through from the client, a spoofed value gets a fresh bucket per request.

How to fix it

The fix is the same shape in every case: authenticate before the paid call, key the rate limit on something the client cannot rotate for free, cap the input, and put a global spend ceiling in front of the provider so a leak is bounded even if everything else fails. Make the limiter fail closed in production. If Redis is not configured, return 503 rather than proceeding.

Specifics from the audited repos. For chatbot-ui, call getServerProfile() at the top of app/api/chat/custom/route.ts and add .eq("user_id", profile.user_id) to the models query, or better, replace the service-role client with createServerClient using NEXT_PUBLIC_SUPABASE_ANON_KEY and the request cookies so RLS applies. Add a lib/server/rate-limit.ts with @upstash/ratelimit and call it after getServerProfile() in every route under app/api; validate chatSettings.model against lib/models/llm/llm-list.ts. For vercel/ai-chatbot, move generateTitleFromUserMessage into lib/ai/title.ts starting with import "server-only" and import it from the chat route; cap the prompt with .slice(0, 2000); rate-limit guest creation per IP in app/(auth)/api/auth/guest/route.ts; lower guest.maxMessagesPerHour in lib/ai/entitlements.ts. For roomGPT, return 503 when redis is undefined, add a second Ratelimit.fixedWindow(500, '1440 m') keyed on the constant 'global', and resolve the IP from x-real-ip then the first entry of x-forwarded-for, returning 400 if neither exists.

The prompt below is written to be pasted into Cursor or Claude Code against your own repo. It asks the assistant to find every paid call and apply the four controls rather than fixing one file.

Audit and harden every server-side endpoint that makes a paid AI or model call (generateText, streamText, chat.completions.create, embeddings.create, Replicate predictions, or any HTTP call to an AI provider). For each one:

1. Authentication. Require a verified session before the paid call (getServerProfile(), auth(), or the project's equivalent). Return 401 JSON if missing. If the route looks up a resource by an id from the request body, also verify the resource belongs to the session user; prefer a user-scoped DB client so RLS applies instead of a service-role/admin client. Return 403/404 if not owned.

2. Rate limiting that fails closed. Add a shared helper (e.g. lib/server/rate-limit.ts using @upstash/ratelimit) with a sliding window per user id (e.g. 30/min) and per IP (e.g. 60/min). Resolve the IP from x-real-ip, then the first entry of x-forwarded-for, and return 400 if neither exists; never fall back to an empty string or a shared constant. Call the limiter right after auth and return 429 JSON when exceeded. If the Redis client is not configured or errors in production, return 503 JSON and console.error, do not skip the check. Add a second limiter keyed on the constant string 'global' with a daily cap so total provider spend is bounded.

3. Input caps. Enforce a max request body size (~1MB), truncate or reject over-long prompts/messages/text, and validate the requested model against an explicit allowlist instead of passing it through from the client.

4. Exposure. Search every file starting with "use server" and move any function that performs a paid call into a plain module that begins with `import "server-only"` and is not marked "use server"; update imports. Remove exports that no client component calls.

Also: set `export const maxDuration` on routes that download or embed files, enforce file size server-side, batch embeddings calls. If a guest/anonymous identity is created per request, rate-limit its creation per IP and give guests a lower quota. Update .env.example and the README so rate-limiter env vars are marked required.

List every file you changed and, for each paid call site, state which of the four controls now applies and where. Do not stub or skip any of them.

Pre-launch checklist

Run through this once against production config, not local. Most of these failures only appear when an env var is absent in the deployed environment.

• Every route or server action that calls a model requires a verified session before the call.
• No paid function is exported from a "use server" file. Internal helpers live in server-only modules.
• Rate limiting is mandatory: with UPSTASH_REDIS_REST_URL / REDIS_URL removed from the deployment, the paid routes return 503, not success.
• Limits are keyed on user id plus IP, the IP is resolved from x-real-ip then x-forwarded-for, and a missing IP returns 400 rather than an empty-string bucket.
• A global daily cap exists on top of per-user limits so a bypass is bounded in money.
• Guest or anonymous identity creation is itself rate-limited per IP, and guests have a lower quota than registered users.
• model, messages, text and prompt are length-checked and the model is validated against an allowlist server-side.
• No route reads a user-owned secret (api_key, base_url) via a service-role client using an id from the request body without an ownership check.
• If server env keys are shared across accounts, signup requires email confirmation or an allowlist, and a per-user daily budget is enforced.
• File processing routes set maxDuration, enforce size server-side, and batch embeddings; embeddingsProvider === "local" is rejected on serverless.
• Provider errors (401, 402, 422, 429) are checked and returned as JSON so the client can show a message instead of hanging.
• Provider dashboards have a hard spend limit and an alert configured; this is the last line of defence when the code is wrong.