OpenAI streaming works locally, on Vercel the whole answer appears at once after 15s

It's not Vercel, it's the loop. You read the entire stream into text on the server and only then return it. Locally it probably only looked like streaming because the client animates the text.

Pass the stream through instead:

export async function POST(req: Request) {
  const { messages } = await req.json();
  const stream = await openai.chat.completions.create({ model, messages, stream: true });
  const encoder = new TextEncoder();
  const body = new ReadableStream({
    async start(controller) {
      for await (const chunk of stream) {
        controller.enqueue(encoder.encode(chunk.choices[0]?.delta?.content ?? ""));
      }
      controller.close();
    },
  });
  return new Response(body, { headers: { "Content-Type": "text/plain; charset=utf-8" } });
}

And on the client, read res.body.getReader() instead of await res.text(), otherwise the browser waits for the end as well.

If it still buffers after that fix, look at what sits in front of the app. On Vercel it streams fine, but if you ever move to a VPS behind nginx, add the header X-Accel-Buffering: no or nginx collects the whole response first. Same idea with some Cloudflare settings.

Also send Cache-Control: no-cache, no-transform on streamed responses. Some proxies and CDNs compress the response, and compression can make them buffer it first.