Demo, all content is generated
Question

Should I switch my API routes to the edge runtime to avoid cold starts?

Open · 895 views · asked by dan_the_intern · edited

Some requests on my app take 1-2s randomly, I read it's cold starts. Cursor says I can add export const runtime = 'edge' to my route handlers and they'll be instant. They talk to Supabase and one calls OpenAI. Any downside? Feels too easy.

What I’ve tried

Added it to one route as a test. It works but I can't tell if it's faster, sometimes it seems slower.

Comment
Before switching anything, which requests are slow? All of them or specific routes? pawel_z · edited

3 answers

rafa_dev · edited

I wouldn't. Edge functions run close to the user, your database is in one place. So a user in Sydney hits an edge function in Sydney, which then makes every query to your Supabase in Europe. That's likely why it sometimes feels slower.

Also, cold starts on Vercel's Node runtime are much less of a thing than they used to be (fluid compute reuses instances). Before changing runtimes, find out what the slow requests actually are: check the function duration in the logs. Random 1-2s is often a slow OpenAI call or a region mismatch, not a cold start.

Comment
the slow ones are all the OpenAI route. so, not cold starts? dan_the_intern · edited
Right. That's the model thinking. Stream the response so the user sees tokens immediately, that fixes the perceived wait. rafa_dev · edited
lena_ops · edited

Slight disagreement: edge is fine for routes that don't touch your database. Redirects, geolocation, A/B cookie logic, OG images, or a thin proxy that streams from OpenAI are all good edge candidates. The rule I use: if it talks to your database more than once, run it next to the database.

Comment
Fair, agreed on that split. The OpenAI proxy on edge is a reasonable case since there's no DB hop. rafa_dev · edited
oksana_k · edited

Measure before and after with a fixed test: 20 requests to the route with curl -w "%{time_total}\n". One click in the browser is too noisy to tell edge from node.

Comment
node: median 1.9s, edge: median 2.1s for the OpenAI one. Reverted to node and added streaming. dan_the_intern · edited