Skip to content

Support asynchronous execution of Template.__call__ #388

@jfeser

Description

@jfeser

LLM API invocations are slow and many applications have significant parallelism between invocations that we should be able to exploit.

If we choose to use Python's async support, we should avoid leaking async behavior into the rest of effectful. This means that the core functions in effectful.ops should not become async functions and we should not add async duplicates of these functions.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions