Demo, all content is generated
Question

Someone called my /api/generate 20,000 times overnight

Solved · 1352 views · asked by harriet_g · edited

Small writing assistant, Next.js on Vercel, calls Anthropic API. Logged in users only get the button, but the route itself... apparently anyone can call it. Bill for one night is $85. How do I rate limit this properly? Claude Code suggested an in-memory Map counter but I read that doesn't work on serverless.

What I’ve tried

Added a check that the request has a Referer from my domain. Lowered max_tokens. Set a spend limit in the Anthropic console, which at least stopped the bleeding.

Comment
Ouch. At least the spend limit stopped it at one night. sergio_ruiz · edited

3 answers

Marked as helpful by the asker
lena_ops · edited

Correct, an in-memory Map on serverless is a counter per instance, and instances come and go. And the Referer header is set by the caller, so a script just sends your domain.

Order of fixes:

  1. Auth first. In the route, get the user from the session and return 401 if there's none. That alone kills anonymous abuse.
  2. Limit per user, not per IP. Use a shared store. The common option is Upstash Redis with @upstash/ratelimit (one new dependency, free tier is plenty):
const ratelimit = new Ratelimit({
  redis: Redis.fromEnv(),
  limiter: Ratelimit.slidingWindow(20, '1 h'),
})
const { success } = await ratelimit.limit(user.id)
if (!success) return new Response('Too many requests', { status: 429 })
  1. Keep the provider spend limit as the last line of defense.

If you already use Supabase/Postgres you can also count rows in a generations table for the last hour instead of adding Redis. Slower, but zero new services.

Comment
The route had no auth check at all. Added that + the Postgres count version since I already have Supabase. harriet_g · edited
Also turn on Vercel's firewall rate limiting for that path if you're on Pro. Stops it before your function even runs, so you don't pay the invocations either. deploydan · edited
chidi_eze · edited

Check your logs for the IPs/user agents of those 20k calls. If it was one script, your key was probably shared somewhere as 'free AI endpoint'. Happens a lot with unauthenticated AI routes.

Comment
zainab_a · edited

Product angle: instead of only a per-hour limit, give each user a daily credit count stored in the DB and show it in the UI ('14 generations left today'). Abuse is capped, and honest users understand why they stopped instead of seeing a 429.

Comment
Love this, turning the limit into a feature. harriet_g · edited