Match function filters by workspaceid and orders by distance, limit 10: Since I added an HNSW index, small workspaces get only 2-4 results even though they have

Demo, all content is generated

Question

pgvector search with a filter returns 3 results instead of 10 after adding HNSW index

Open · 23 views · asked last week by dirk_vl · edited

Match function filters by workspace_id and orders by distance, limit 10:

select id, content from documents
where workspace_id = $1
order by embedding <=> $2
limit 10;

Since I added an HNSW index, small workspaces get only 2-4 results even though they have 50+ documents. Big workspaces are fine. Without the index I always get 10.

What I’ve tried

Claude Code suggested raising hnsw.ef_search to 200, that helps a bit but not for the smallest workspaces. Don't want to remove the index, it made search 20x faster.

Comment

The index returns its nearest candidates first and the workspace filter is applied after that, so small workspaces get filtered away. I know newer pgvector versions have an iterative scan option for this but haven't used it myself, hoping someone who has can answer properly. hannah_reyes last week · edited

Thanks, at least I understand why now. Will read up on iterative scans. dirk_vl last week · edited

Activity

20 helpers notified

Keyboard shortcuts

pgvector search with a filter returns 3 results instead of 10 after adding HNSW index

What I’ve tried

Activity

Your answer

pgvector search with a filter returns 3 results instead of 10 after adding HNSW index

What I’ve tried

Activity

Related

Your answer