-
Notifications
You must be signed in to change notification settings - Fork 11
Description
Summary
When you set values=[""] (a list with one empty string) in the list evaluator config, it accidentally blocks every single input. Any message — "Book a flight", "Tell me a joke", anything — gets denied. One typo in the config = your entire agent stops working. ested end-to-end on a running server (FastAPI + PostgreSQL). The server accepts values=[""] without complaint and stores it in the database. The bug triggers at evaluation time.
The existing guard (if not self._values) catches an empty list [] but misses a list containing an empty string [""] — because [""] has length 1, so it looks non-empty.
Motivation
This is an open-source, self-hosted project. Anyone can configure controls through the API or a UI form. It's really easy to accidentally end up with an empty string in the values list — a blank form field, a trailing comma, or just "" instead of []. When that happens, there's no error, no warning. The control silently starts blocking everything and you have no idea why your agent stopped responding.
Current behavior
You set values=[""] in a list control config.
The server accepts it — no validation error.
Under the hood, the evaluator builds a regex from that empty string: \b()\b.
That regex matches at every word boundary in every string — so it matches everything.
Every evaluation returns matched: true, every request gets denied.
The agent is completely shut down.
Expected behavior
Empty strings should be filtered out from the values list before the regex is built.
values=[""] should behave the same as values=[] — the evaluator should say "Empty control values - control ignored" and return matched: false.
Reproduction
Create a control: PUT /api/v1/controls → PUT /api/v1/controls/{id}/data with values=[""], match_mode="contains", action.decision="deny".
Attach it to an agent through a policy.
Send any message to POST /api/v1/evaluation.
Result: Every message is denied. 5 out of 5 normal user requests blocked.
Minimal reproduction (3 lines):
from agent_control_evaluators.list.evaluator import ListEvaluator
from agent_control_evaluators.list.config import ListEvaluatorConfig
import asyncio
evaluator = ListEvaluator(ListEvaluatorConfig(values=[""], match_mode="contains"))
result = asyncio.run(evaluator.evaluate("Tell me a joke"))
print(result.matched) # True — should be False
Proposed solution
One-line fix in evaluator.py line 38 — filter out empty strings:
# Before:
self._values = [str(v) for v in config.values]
# After:
self._values = [str(v) for v in config.values if str(v).strip()]
Additional context
Bug file: evaluators/builtin/src/agent_control_evaluators/list/evaluator.py, line 38-50.
Tested end-to-end on a running server (FastAPI + PostgreSQL). The server accepts values=[""] without complaint and stores it in the database. The bug triggers at evaluation time.
The existing guard (if not self._values) catches an empty list [] but misses a list containing an empty string [""] — because [""] has length 1, so it looks non-empty.
**_
_**