feat: add chat-completions/ and responses/ provider prefixes #335
+574
−4
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Adds
chat-completions/andresponses/prefixes, providing a universal entrypoint for LLM APIs that support Chat Completions or Responses. This allows openbench to support LLM providers that have not been explictly implemented in openbench. Users add a base URL, API key, and prefix their model ID withchat-completions/orresponses/. This is similar to Inspect AI'sopenai-api/prefix.What are you adding?
Changes Made
Testing
pytest)pre-commit run --all-files)Checklist
Related Issues
Closes #
Additional Context
Note
Introduces generic Chat Completions and Responses format providers with new
chat-completions/<provider>/<model>andresponses/<provider>/<model>prefixes, updates registry/docs/config, and bumps core to 0.5.3.ChatCompletionsFormatAPI(src/openbench/model/_providers/chat_completions.py) andResponsesFormatAPI(src/openbench/model/_providers/responses.py)._registry.pyaschat-completionsandresponses.ProviderTypeandPROVIDER_CONFIGSwith generic entries (env varsCHAT_COMPLETIONS_BASE_URL,RESPONSES_BASE_URL, etc.).chat-completions/<provider>/<model-name>orresponses/<provider>/<model-name>and adjust example command.openbench-coreversion to0.5.3.Written by Cursor Bugbot for commit 1eb0705. This will update automatically on new commits. Configure here.