(retriever) Only overwrite LanceDB table once if specified

[charlesbluca]

Description

Problem: With overwrite=True, the inprocess pipeline ran vdb_upload once per document. Each call replaced the LanceDB table, so only the last PDF’s embeddings remained and recall dropped.

Fix:

• Split tasks into per-doc (extract → embed) and post (vdb_upload, save_to_disk).
• Run post tasks once on the concatenated results of all documents.
• With overwrite=True, the table is now replaced a single time with the full corpus.

Checklist

• I am familiar with the Contributing Guidelines.
• New or existing tests cover these changes.
• The documentation is up to date with these changes.
• If adjusting docker-compose.yaml environment variables have you ensured those are mimicked in the Helm values.yaml file.