Retrieval

[joelpaulkoch]

We should support websearch and fetch url.

Techniques:

• hybrid search (with simple concatenation of multiple retrievals in d3c5ca4 and reciprocal rank fusion in be2ee59)
• deduplication in d3c5ca4
• reranking
• query routing/rewriting/expansion
• small-to-big retrieval (implictly supported)

More advanced:

• Contextual Retrieval
• adaptive retrieval
• embedding align
• Hypothetical Document Embeddings (HyDE)