fix: prevent HTTP client connection leaks in embedders

[choutos]

Fixes #919

Problem

The AsyncOpenAI, Ollama, and VoyageAI embedders create HTTP clients that are never explicitly closed. This causes connections to accumulate in CLOSE_WAIT state and eventually exhaust file descriptors.

Investigation Details

In production, we observed:

• ~35,000 file descriptors held by pgai-vectorizer-worker
• All were sockets in CLOSE_WAIT state
• Connections were to the OpenAI API (via Cloudflare)
• The systemd error: Failed to allocate manager object: Too many open files

Root Cause

The _embedder property in openai.py creates an AsyncOpenAI client that holds an httpx.AsyncClient. When the embedder is garbage collected, the HTTP client's connections are not properly closed, leaving them in CLOSE_WAIT until the OS times them out.

Similar patterns exist in ollama.py and voyageai.py.

Solution

1. Add cleanup() method to the Embedder base class
2. Implement cleanup() in OpenAI, Ollama, and VoyageAI embedders to close underlying HTTP clients
3. Call cleanup() in Executor.run()'s finally block to ensure resources are released
4. Reuse client instances instead of creating new ones for each request (Ollama, VoyageAI)

Changes

• embeddings.py: Add cleanup() method to base class
• openai.py: Store client reference, implement cleanup()
• ollama.py: Add _get_client() for client reuse, implement cleanup()
• voyageai.py: Add _get_client() for client reuse, implement cleanup()
• vectorizer.py: Call cleanup() in finally block

Testing

This fix was developed in response to a production issue. We recommend:

• Unit tests for cleanup() methods
• Integration test that verifies no file descriptor leak after multiple embedding batches

[CLAassistant]

All committers have signed the CLA.