Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
278af27
Add ideal database schema for ruby_llm-agents with normalized tables and
adham90 Feb 1, 2026
36102a1
Update ideal_database_schema.md
adham90 Feb 1, 2026
d018e53
Split large execution details into separate table and model
adham90 Feb 2, 2026
3ba1b7b
Remove API configuration DB table and related code
adham90 Feb 4, 2026
1cc6b1a
Remove workflow orchestration subsystem
adham90 Feb 4, 2026
b95bac8
Simplify alert system by removing built-in notifiers
adham90 Feb 4, 2026
0b7dcf0
Refactor alert configuration to use single on_alert handler
adham90 Feb 4, 2026
932f1b4
Add before_call and after_call hooks; remove built-in moderation and
adham90 Feb 4, 2026
6c03544
Remove PII redaction and content moderation features and docs
adham90 Feb 4, 2026
2d28dff
Rename execution_metadata to metadata throughout the codebase and docs
adham90 Feb 4, 2026
bc1509d
Remove agent version support from DSL and execution tracking
adham90 Feb 4, 2026
f559672
Remove version comparison feature and related UI from agents view and
adham90 Feb 4, 2026
915b7b8
Remove execution type filter from agent executions UI and export link
adham90 Feb 4, 2026
64e3f0c
Remove workflow columns and agent_version from executions
adham90 Feb 4, 2026
0215f31
Add redaction configuration for PII and sensitive data
adham90 Feb 4, 2026
469af36
Introduce simplified DSL for agents with prompt-centric syntax
adham90 Feb 4, 2026
ecb0f53
Refactor agents to use simplified DSL syntax
adham90 Feb 5, 2026
f6c3b7b
Simplify agent DSL with inline prompts and updated reliability rules
adham90 Feb 5, 2026
3f01854
Simplify dashboard ranges and remove unused components
adham90 Feb 5, 2026
4b0b632
Remove dashboard partials for agent, model, and error comparisons
adham90 Feb 5, 2026
4c261f3
Redesign dashboard with updated styles and layouts
adham90 Feb 6, 2026
80aa145
Add sorting controls to agents index and adjust dashboard spacing
adham90 Feb 6, 2026
8c836bc
Refactor agent config partials and show view for consistent style
adham90 Feb 6, 2026
ef25fea
Refactor agent config partial to use grid layout and improve styling
adham90 Feb 6, 2026
4b9c4db
Update agent views with badge styling and enhanced stats display
adham90 Feb 6, 2026
67184ec
Simplify agent executions view by removing filters and adding link to
adham90 Feb 6, 2026
80225bd
Refactor executions list UI with compact layout and inline sorting
adham90 Feb 6, 2026
678ef13
Improve table layout with flex and min-width styling
adham90 Feb 6, 2026
a4d255f
Refactor execution show view for clean layout and styling
adham90 Feb 6, 2026
cbabf84
Update show.html.erb
adham90 Feb 6, 2026
8f7c6a2
Update show.html.erb
adham90 Feb 6, 2026
2c81fc9
Update show.html.erb
adham90 Feb 6, 2026
87821fc
Update show.html.erb
adham90 Feb 6, 2026
3d306da
Update show.html.erb
adham90 Feb 6, 2026
139a7c2
Refactor tenant views with updated styling and layout
adham90 Feb 6, 2026
6f1d9d7
Add search and sortable columns to tenants list view
adham90 Feb 6, 2026
20c3188
Refactor system config view for clarity and style
adham90 Feb 6, 2026
5cebb0d
Refactor UI badges and remove deprecated components
adham90 Feb 6, 2026
ad29762
Normalize case in specs for tool calls, arguments, and errors messages
adham90 Feb 6, 2026
380a522
Create v2.0.0 upgrade migrations for schema changes
adham90 Feb 7, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
87 changes: 87 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,93 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Added

- **`before_call` and `after_call` callbacks for conversation agents** - Agent-level hooks that run before and after LLM calls. Use method names or blocks. Callbacks can mutate context, raise to block execution, or inspect responses. Follows the same pattern as image pipeline's `before_pipeline`/`after_pipeline` hooks.

### Removed

- **BREAKING: Removed `version` DSL method** - The `version` DSL method has been removed from all agent types. This method was originally intended for cache invalidation but added complexity without significant benefit. Cache keys are now content-based, automatically generated from a hash of your prompts and parameters. This means caches invalidate automatically when you change your prompts—no manual version bumping required. If you need traceability, use `execution_metadata` instead (see migration guide below). The `agent_version` column is no longer written to; existing data will remain. The `by_version` scope and version filtering in the dashboard have also been removed.

- **BREAKING: Removed ApiConfiguration table and model** - The `ruby_llm_agents_api_configurations` table has been removed entirely. API keys should now be configured via environment variables and the `ruby_llm` gem configuration, following 12-factor app principles. Per-tenant API keys can still be provided via the `llm_tenant` DSL's `api_keys:` option on your model.

- **BREAKING: Removed built-in moderation system** - The `moderation` DSL, `Moderator` class, `ModerationResult`, and `ModerationError` have been removed. Use the new `before_call` hook to implement custom moderation logic with your preferred moderation service.

- **BREAKING: Removed built-in PII redaction** - The `Redactor` utility and redaction configuration options have been removed. Use the new `before_call` hook to implement custom redaction logic.

- **BREAKING: Removed image content policy** - The `content_policy` DSL for image generators, editors, and transformers has been removed. Implement custom content filtering in your application layer if needed.

### Migration Guide

**If you were using the `version` DSL:**

1. Remove `version "X.Y"` calls from your agent classes—they'll now raise an error
2. Cache invalidation is now automatic (content-based). When you change prompts, the cache key changes automatically
3. If you need traceability (e.g., to track which "version" of an agent produced a result), use `execution_metadata`:
```ruby
class ApplicationAgent < RubyLLM::Agents::BaseAgent
def execution_metadata
{
git_sha: ENV['GIT_SHA'] || `git rev-parse --short HEAD 2>/dev/null`.strip.presence,
deploy_version: ENV['DEPLOY_VERSION']
}.compact
end
end
```
4. The `agent_version` column can optionally be removed (safe to leave in place):
```ruby
class RemoveAgentVersionFromExecutions < ActiveRecord::Migration[7.1]
def change
safety_assured do
remove_index :ruby_llm_agents_executions, [:agent_type, :agent_version], if_exists: true
remove_column :ruby_llm_agents_executions, :agent_version, :string
end
end
end
```

**If you were using the `ApiConfiguration` model:**

1. Export any API keys stored in the database
2. Set them as environment variables instead:
```bash
export OPENAI_API_KEY="sk-..."
export ANTHROPIC_API_KEY="sk-ant-..."
```
3. Configure in your initializer:
```ruby
# config/initializers/ruby_llm.rb
RubyLLM.configure do |config|
config.openai_api_key = ENV["OPENAI_API_KEY"]
config.anthropic_api_key = ENV["ANTHROPIC_API_KEY"]
end
```
4. Run the migration to drop the table:
```ruby
class RemoveApiConfigurations < ActiveRecord::Migration[7.1]
def up
drop_table :ruby_llm_agents_api_configurations, if_exists: true
end
end
```

For per-tenant API keys, use the `llm_tenant` DSL:
```ruby
class Organization < ApplicationRecord
include RubyLLM::Agents::LLMTenant

encrypts :openai_api_key, :anthropic_api_key

llm_tenant(
id: :slug,
api_keys: {
openai: :openai_api_key,
anthropic: :anthropic_api_key
}
)
end
```

## [1.3.4] - 2026-01-29

### Improved
Expand Down
3 changes: 1 addition & 2 deletions LLMS.txt
Original file line number Diff line number Diff line change
Expand Up @@ -95,8 +95,7 @@ ruby_llm-agents brings structure, observability, and control to LLM-based Rails

- [Multi-Tenancy Guide](wiki/Multi-Tenancy.md): Setting up multi-tenant support
- [LLM Tenant](lib/ruby_llm/agents/llm_tenant.rb): Tenant concern for models
- [API Configuration](app/models/ruby_llm/agents/api_configuration.rb): Per-tenant API keys
- [Tenant Budget](app/models/ruby_llm/agents/tenant_budget.rb): Per-tenant spending limits
- [Tenant](app/models/ruby_llm/agents/tenant.rb): Per-tenant budgets and tracking

## Caching

Expand Down
68 changes: 11 additions & 57 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@ Build intelligent AI agents in Ruby with a clean DSL, automatic execution tracki
- **Rails-Native** - Seamlessly integrates with your Rails app: models, jobs, caching, and Hotwire
- **Production-Ready** - Built-in retries, model fallbacks, circuit breakers, and budget limits
- **Full Observability** - Track every execution with costs, tokens, duration, and errors
- **Workflow Orchestration** - Compose agents into pipelines, parallel tasks, and conditional routers
- **Zero Lock-in** - Works with any LLM provider supported by RubyLLM

## Show Me the Code
Expand All @@ -32,13 +31,12 @@ class SearchIntentAgent < ApplicationAgent
model "gpt-4o"
temperature 0.0

param :query, required: true
# Prompts with {placeholder} syntax - params auto-registered
system "You are a search intent analyzer. Extract structured data from queries."
prompt "Extract search intent from: {query}"

def user_prompt
"Extract search intent from: #{query}"
end

schema do
# Structured output with returns DSL
returns do
string :refined_query, description: "Cleaned search query"
array :filters, of: :string, description: "Extracted filters"
end
Expand Down Expand Up @@ -69,17 +67,13 @@ result = ChatAgent.call(
class ReliableAgent < ApplicationAgent
model "gpt-4o"

reliability do
retries max: 3, backoff: :exponential
fallback_models "gpt-4o-mini", "claude-3-5-sonnet"
circuit_breaker errors: 10, within: 60, cooldown: 300
total_timeout 30
end

param :query, required: true
prompt "{query}"

def user_prompt
query
on_failure do
retries times: 3, backoff: :exponential
fallback to: ["gpt-4o-mini", "claude-3-5-sonnet"]
circuit_breaker after: 10, within: 60, cooldown: 5.minutes
timeout 30
end
end
```
Expand Down Expand Up @@ -122,41 +116,6 @@ result.url # => "https://..."
result.save("logo.png")
```

```ruby
# Workflow orchestration - sequential, parallel, routing in one DSL
class OrderWorkflow < RubyLLM::Agents::Workflow
description "End-to-end order processing"
timeout 60.seconds
max_cost 1.50

input do
required :order_id, String
optional :priority, String, default: "normal"
end

step :validate, ValidatorAgent
step :enrich, EnricherAgent, input: -> { { data: validate.content } }

parallel :analysis do
step :sentiment, SentimentAgent, optional: true
step :classify, ClassifierAgent
end

step :handle, on: -> { classify.category } do |route|
route.billing BillingAgent
route.technical TechnicalAgent
route.default GeneralAgent
end

step :format, FormatterAgent, optional: true
end

result = OrderWorkflow.call(order_id: "123")
result.steps[:classify].content # Individual step result
result.total_cost # Sum of all steps
result.success? # true/false
```

## Features

| Feature | Description | Docs |
Expand All @@ -167,14 +126,11 @@ result.success? # true/false
| **Reliability** | Automatic retries, model fallbacks, circuit breakers with block DSL | [Reliability](https://github.com/adham90/ruby_llm-agents/wiki/Reliability) |
| **Budget Controls** | Daily/monthly limits with hard and soft enforcement | [Budgets](https://github.com/adham90/ruby_llm-agents/wiki/Budget-Controls) |
| **Multi-Tenancy** | Per-tenant API keys, budgets, circuit breakers, and execution isolation | [Multi-Tenancy](https://github.com/adham90/ruby_llm-agents/wiki/Multi-Tenancy) |
| **Workflows** | Pipelines, parallel execution, conditional routers | [Workflows](https://github.com/adham90/ruby_llm-agents/wiki/Workflows) |
| **Async/Fiber** | Concurrent execution with Ruby fibers for high-throughput workloads | [Async](https://github.com/adham90/ruby_llm-agents/wiki/Async-Fiber) |
| **Dashboard** | Real-time Turbo-powered monitoring UI | [Dashboard](https://github.com/adham90/ruby_llm-agents/wiki/Dashboard) |
| **Streaming** | Real-time response streaming with TTFT tracking | [Streaming](https://github.com/adham90/ruby_llm-agents/wiki/Streaming) |
| **Conversation History** | Multi-turn conversations with message history | [Conversation History](https://github.com/adham90/ruby_llm-agents/wiki/Conversation-History) |
| **Attachments** | Images, PDFs, and multimodal support | [Attachments](https://github.com/adham90/ruby_llm-agents/wiki/Attachments) |
| **PII Redaction** | Automatic sensitive data protection | [Security](https://github.com/adham90/ruby_llm-agents/wiki/PII-Redaction) |
| **Content Moderation** | Input/output safety checks with OpenAI moderation API | [Moderation](https://github.com/adham90/ruby_llm-agents/wiki/Moderation) |
| **Embeddings** | Vector embeddings with batching, caching, and preprocessing | [Embeddings](https://github.com/adham90/ruby_llm-agents/wiki/Embeddings) |
| **Image Operations** | Generation, analysis, editing, pipelines with cost tracking | [Images](https://github.com/adham90/ruby_llm-agents/wiki/Image-Generation) |
| **Alerts** | Slack, webhook, and custom notifications | [Alerts](https://github.com/adham90/ruby_llm-agents/wiki/Alerts) |
Expand Down Expand Up @@ -231,13 +187,11 @@ mount RubyLLM::Agents::Engine => "/agents"
| [Getting Started](https://github.com/adham90/ruby_llm-agents/wiki/Getting-Started) | Installation, configuration, first agent |
| [Agent DSL](https://github.com/adham90/ruby_llm-agents/wiki/Agent-DSL) | All DSL options: model, temperature, params, caching, description |
| [Reliability](https://github.com/adham90/ruby_llm-agents/wiki/Reliability) | Retries, fallbacks, circuit breakers, timeouts, reliability block |
| [Workflows](https://github.com/adham90/ruby_llm-agents/wiki/Workflows) | Pipelines, parallel execution, routers |
| [Budget Controls](https://github.com/adham90/ruby_llm-agents/wiki/Budget-Controls) | Spending limits, alerts, enforcement |
| [Multi-Tenancy](https://github.com/adham90/ruby_llm-agents/wiki/Multi-Tenancy) | Per-tenant budgets, isolation, configuration |
| [Async/Fiber](https://github.com/adham90/ruby_llm-agents/wiki/Async-Fiber) | Concurrent execution with Ruby fibers |
| [Testing Agents](https://github.com/adham90/ruby_llm-agents/wiki/Testing-Agents) | RSpec patterns, mocking, dry_run mode |
| [Error Handling](https://github.com/adham90/ruby_llm-agents/wiki/Error-Handling) | Error types, recovery patterns |
| [Moderation](https://github.com/adham90/ruby_llm-agents/wiki/Moderation) | Content moderation for input/output safety |
| [Embeddings](https://github.com/adham90/ruby_llm-agents/wiki/Embeddings) | Vector embeddings, batching, caching, preprocessing |
| [Image Generation](https://github.com/adham90/ruby_llm-agents/wiki/Image-Generation) | Text-to-image, templates, content policy, cost tracking |
| [Dashboard](https://github.com/adham90/ruby_llm-agents/wiki/Dashboard) | Setup, authentication, analytics |
Expand Down
1 change: 0 additions & 1 deletion app/controllers/concerns/ruby_llm/agents/sortable.rb
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,6 @@ module Sortable
"agent_type" => "agent_type",
"status" => "status",
"model_id" => "model_id",
"agent_version" => "agent_version",
"total_tokens" => "total_tokens",
"total_cost" => "total_cost",
"duration_ms" => "duration_ms",
Expand Down
61 changes: 5 additions & 56 deletions app/controllers/ruby_llm/agents/agents_controller.rb
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,6 @@ def index
@agents_by_type = {
agent: @agents.select { |a| a[:agent_type] == "agent" },
embedder: @agents.select { |a| a[:agent_type] == "embedder" },
moderator: @agents.select { |a| a[:agent_type] == "moderator" },
speaker: @agents.select { |a| a[:agent_type] == "speaker" },
transcriber: @agents.select { |a| a[:agent_type] == "transcriber" },
image_generator: @agents.select { |a| a[:agent_type] == "image_generator" }
Expand All @@ -60,7 +59,7 @@ def index
Rails.logger.error("[RubyLLM::Agents] Error loading agents: #{e.message}")
@agents = []
@deleted_agents = []
@agents_by_type = { agent: [], embedder: [], moderator: [], speaker: [], transcriber: [], image_generator: [] }
@agents_by_type = { agent: [], embedder: [], speaker: [], transcriber: [], image_generator: [] }
@agent_count = 0
@deleted_count = 0
@sort_params = { column: DEFAULT_AGENT_SORT_COLUMN, direction: DEFAULT_AGENT_SORT_DIRECTION }
Expand Down Expand Up @@ -119,13 +118,11 @@ def load_agent_stats
def load_filter_options
# Single query to get all filter options (fixes N+1)
filter_data = Execution.by_agent(@agent_type)
.where.not(agent_version: nil)
.or(Execution.by_agent(@agent_type).where.not(model_id: nil))
.where.not(model_id: nil)
.or(Execution.by_agent(@agent_type).where.not(temperature: nil))
.pluck(:agent_version, :model_id, :temperature)
.pluck(:model_id, :temperature)

@versions = filter_data.map(&:first).compact.uniq.sort.reverse
@models = filter_data.map { |d| d[1] }.compact.uniq.sort
@models = filter_data.map(&:first).compact.uniq.sort
@temperatures = filter_data.map(&:last).compact.uniq.sort
end

Expand All @@ -149,7 +146,7 @@ def load_filtered_executions

# Builds a filtered scope for the current agent's executions
#
# Applies filters in order: status, version, model, temperature, time.
# Applies filters in order: status, model, temperature, time.
# Each filter is optional and only applied if values are provided.
#
# @return [ActiveRecord::Relation] Filtered execution scope
Expand All @@ -160,10 +157,6 @@ def build_filtered_scope
statuses = parse_array_param(:statuses)
scope = apply_status_filter(scope, statuses) if statuses.any?

# Apply version filter
versions = parse_array_param(:versions)
scope = scope.where(agent_version: versions) if versions.any?

# Apply model filter
models = parse_array_param(:models)
scope = scope.where(model_id: models) if models.any?
Expand All @@ -188,37 +181,6 @@ def load_chart_data
@trend_data = Execution.trend_analysis(agent_type: @agent_type, days: 30)
@status_distribution = Execution.by_agent(@agent_type).group(:status).count
@finish_reason_distribution = Execution.by_agent(@agent_type).finish_reason_distribution
load_version_comparison
end

# Loads version comparison data if multiple versions exist
#
# Includes trend data for sparkline charts.
#
# @return [void]
def load_version_comparison
return unless @versions.size >= 2

# Default to comparing two most recent versions
v1 = params[:compare_v1] || @versions[0]
v2 = params[:compare_v2] || @versions[1]

comparison_data = Execution.compare_versions(@agent_type, v1, v2, period: :this_month)

# Fetch trend data for sparklines
v1_trend = Execution.version_trend_data(@agent_type, v1, days: 14)
v2_trend = Execution.version_trend_data(@agent_type, v2, days: 14)

@version_comparison = {
v1: v1,
v2: v2,
data: comparison_data,
v1_trend: v1_trend,
v2_trend: v2_trend
}
rescue StandardError => e
Rails.logger.debug("[RubyLLM::Agents] Version comparison error: #{e.message}")
@version_comparison = nil
end

# Loads the current agent class configuration
Expand All @@ -234,16 +196,13 @@ def load_agent_config
# Common config for all types
@config = {
model: safe_config_call(:model),
version: safe_config_call(:version) || "N/A",
description: safe_config_call(:description)
}

# Type-specific config
case @agent_type_kind
when "embedder"
load_embedder_config
when "moderator"
load_moderator_config
when "speaker"
load_speaker_config
when "transcriber"
Expand Down Expand Up @@ -284,16 +243,6 @@ def load_embedder_config
)
end

# Loads configuration specific to Moderators
#
# @return [void]
def load_moderator_config
@config.merge!(
threshold: safe_config_call(:threshold),
categories: safe_config_call(:categories)
)
end

# Loads configuration specific to Speakers
#
# @return [void]
Expand Down
Loading