Demo, all content is generated
Question

SQLAlchemy QueuePool limit of size 5 overflow 10 reached after a few hours

Solved · 877 views · asked by ines_data · edited

FastAPI + SQLAlchemy + Postgres. Runs fine after restart, then after a few hours every request fails with:

sqlalchemy.exc.TimeoutError: QueuePool limit of size 5 overflow 10 reached, connection timed out, timeout 30.00

Cursor wrote this helper that every route uses:

def get_db():
    return SessionLocal()
What I’ve tried

Increased pool_size to 20. Now it takes longer before it breaks, but it still breaks.

Comment
Show get_db please, 9 out of 10 times it's the session not being closed. marco_py · edited

3 answers

Marked as helpful by the asker
olu_backend · edited

Your sessions are never closed, so each request keeps a connection until the pool runs dry. Bigger pool = slower leak.

Make it a dependency with yield so FastAPI closes it after the response:

def get_db():
    db = SessionLocal()
    try:
        yield db
    finally:
        db.close()

@app.get("/items")
def list_items(db: Session = Depends(get_db)):
    ...

Then put pool_size back to the default. If you run on a host that drops idle connections, add pool_pre_ping=True to create_engine too.

Comment
Running for 2 days now without the error. Put pool_size back to 5 like you said. ines_data · edited
postgres_pete · edited

You can watch the leak from the Postgres side: select state, count(*) from pg_stat_activity group by 1;. Lots of idle in transaction = sessions opened and never closed/committed.

Comment
marco_py · edited

If you ever move to async SQLAlchemy, same pattern with async with AsyncSessionLocal() as session: yield session. The context manager closes it for you.

Comment