Skip to content

ListEvaluator: values=[""] builds match-all regex, can cause complete agent denial-of-service. Can be handled from UI/Backend. #120

@Ritvik777

Description

@Ritvik777

Summary

When you set values=[""] (a list with one empty string) in the list evaluator config, it accidentally blocks every single input. Any message — "Book a flight", "Tell me a joke", anything — gets denied. One typo in the config = your entire agent stops working. ested end-to-end on a running server (FastAPI + PostgreSQL). The server accepts values=[""] without complaint and stores it in the database. The bug triggers at evaluation time.
The existing guard (if not self._values) catches an empty list [] but misses a list containing an empty string [""] — because [""] has length 1, so it looks non-empty.

Motivation

This is an open-source, self-hosted project. Anyone can configure controls through the API or a UI form. It's really easy to accidentally end up with an empty string in the values list — a blank form field, a trailing comma, or just "" instead of []. When that happens, there's no error, no warning. The control silently starts blocking everything and you have no idea why your agent stopped responding.

Current behavior

You set values=[""] in a list control config.
The server accepts it — no validation error.
Under the hood, the evaluator builds a regex from that empty string: \b()\b.
That regex matches at every word boundary in every string — so it matches everything.
Every evaluation returns matched: true, every request gets denied.
The agent is completely shut down.

Expected behavior

Empty strings should be filtered out from the values list before the regex is built.
values=[""] should behave the same as values=[] — the evaluator should say "Empty control values - control ignored" and return matched: false.

Reproduction

Create a control: PUT /api/v1/controls → PUT /api/v1/controls/{id}/data with values=[""], match_mode="contains", action.decision="deny".
Attach it to an agent through a policy.
Send any message to POST /api/v1/evaluation.
Result: Every message is denied. 5 out of 5 normal user requests blocked.
Minimal reproduction (3 lines):

from agent_control_evaluators.list.evaluator import ListEvaluator
from agent_control_evaluators.list.config import ListEvaluatorConfig
import asyncio
evaluator = ListEvaluator(ListEvaluatorConfig(values=[""], match_mode="contains"))
result = asyncio.run(evaluator.evaluate("Tell me a joke"))
print(result.matched) # True — should be False

Proposed solution

One-line fix in evaluator.py line 38 — filter out empty strings:

# Before:

self._values = [str(v) for v in config.values]

# After:

self._values = [str(v) for v in config.values if str(v).strip()]

Additional context

Bug file: evaluators/builtin/src/agent_control_evaluators/list/evaluator.py, line 38-50.
Tested end-to-end on a running server (FastAPI + PostgreSQL). The server accepts values=[""] without complaint and stores it in the database. The bug triggers at evaluation time.
The existing guard (if not self._values) catches an empty list [] but misses a list containing an empty string [""] — because [""] has length 1, so it looks non-empty.

**_

Image

_**

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions