Skip to content

Add Ollama executor support for local model inference#2

Open
Darktex wants to merge 4 commits intomasterfrom
claude/llmvm-agentic-harness-01CjPeSmufwQPaMTMtmM4usF
Open

Add Ollama executor support for local model inference#2
Darktex wants to merge 4 commits intomasterfrom
claude/llmvm-agentic-harness-01CjPeSmufwQPaMTMtmM4usF

Conversation

@Darktex
Copy link
Owner

@Darktex Darktex commented Nov 26, 2025

This commit adds full Ollama integration to LLMVM, enabling users to run
local language models with complete tool calling support using the
block pattern.

Key Features:

  • New OllamaExecutor that inherits from OpenAIExecutor
  • Leverages Ollama's OpenAI-compatible API endpoint
  • Full conversation mode support
  • Tool calling via blocks for local code execution
  • Supports popular models: llama3.1, qwen2.5, mistral, etc.

Implementation Details:

  • llmvm/common/ollama_executor.py: Main executor implementation

    • Inherits from OpenAIExecutor for OpenAI compatibility
    • Configurable endpoint (default: http://localhost:11434/v1)
    • No API key validation (Ollama runs locally)
    • Supports streaming responses and stop tokens
  • llmvm/common/helpers.py: Executor registration

    • Added 'ollama' executor to get_executor() method
    • Environment variable support: OLLAMA_API_BASE
    • Config variable: ollama_api_base, default_ollama_model

Testing & Documentation:

  • scripts/test_ollama.py: Comprehensive test suite

    • Tests conversation mode
    • Tests tool calling with blocks
    • Validates model availability
    • Supports multiple models and endpoints
  • docs/OLLAMA.md: Complete setup and usage guide

    • Installation instructions
    • Configuration options
    • Model recommendations
    • Troubleshooting guide
    • Performance tips
  • README.md: Updated to mention Ollama support

Usage:
LLMVM_EXECUTOR='ollama' LLMVM_MODEL='llama3.1' python -m llmvm.client

Testing:
python scripts/test_ollama.py

Requirements:

  • Ollama installed and running (https://ollama.ai)
  • A model pulled (e.g., ollama pull llama3.1)

Closes requirement: Enable Ollama compatibility with conversation mode
and full tool calling support using blocks.

claude and others added 4 commits November 26, 2025 02:34
This commit adds full Ollama integration to LLMVM, enabling users to run
local language models with complete tool calling support using the
<helpers> block pattern.

Key Features:
- New OllamaExecutor that inherits from OpenAIExecutor
- Leverages Ollama's OpenAI-compatible API endpoint
- Full conversation mode support
- Tool calling via <helpers> blocks for local code execution
- Supports popular models: llama3.1, qwen2.5, mistral, etc.

Implementation Details:
- llmvm/common/ollama_executor.py: Main executor implementation
  - Inherits from OpenAIExecutor for OpenAI compatibility
  - Configurable endpoint (default: http://localhost:11434/v1)
  - No API key validation (Ollama runs locally)
  - Supports streaming responses and stop tokens

- llmvm/common/helpers.py: Executor registration
  - Added 'ollama' executor to get_executor() method
  - Environment variable support: OLLAMA_API_BASE
  - Config variable: ollama_api_base, default_ollama_model

Testing & Documentation:
- scripts/test_ollama.py: Comprehensive test suite
  - Tests conversation mode
  - Tests tool calling with <helpers> blocks
  - Validates model availability
  - Supports multiple models and endpoints

- docs/OLLAMA.md: Complete setup and usage guide
  - Installation instructions
  - Configuration options
  - Model recommendations
  - Troubleshooting guide
  - Performance tips

- README.md: Updated to mention Ollama support

Usage:
  LLMVM_EXECUTOR='ollama' LLMVM_MODEL='llama3.1' python -m llmvm.client

Testing:
  python scripts/test_ollama.py

Requirements:
- Ollama installed and running (https://ollama.ai)
- A model pulled (e.g., ollama pull llama3.1)

Closes requirement: Enable Ollama compatibility with conversation mode
and full tool calling support using <helpers> blocks.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants