Skip to content

tests: fuzzing improvements#469

Open
qdeslandes wants to merge 1 commit intofacebook:mainfrom
qdeslandes:disable_fuzzing_on_all
Open

tests: fuzzing improvements#469
qdeslandes wants to merge 1 commit intofacebook:mainfrom
qdeslandes:disable_fuzzing_on_all

Conversation

@qdeslandes
Copy link
Contributor

Multiple improvements to the fuzzing logic:

  • Remove the fuzzing binary from the make all target
  • Rename fuzzing to fuzzing_quick for 60 seconds fuzzing
  • Add fuzzing target to fuzz the parser indefinitely

@meta-cla meta-cla bot added the cla signed label Mar 12, 2026
Multiple improvements to the fuzzing logic:
- Remove the fuzzing binary from the `make all` target
- Rename `fuzzing` to `fuzzing_quick` for 60 seconds fuzzing
- Add `fuzzing` target to fuzz the parser indefinitely
@qdeslandes qdeslandes force-pushed the disable_fuzzing_on_all branch from ff1541d to d9e57b2 Compare March 12, 2026 12:09

add_custom_target(fuzzing
COMMAND ${CMAKE_CTEST_COMMAND} --output-on-failure -L fuzzing
COMMAND ${CMAKE_CTEST_COMMAND} --verbose -R "fuzzing.parser"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude: ctest -R performs regex substring matching. The pattern fuzzing.parser matches any test name containing that substring, which includes both fuzzing.parser and fuzzing.parser.60 (. is also a regex wildcard matching any character). This means make fuzzing will run the 60-second test and the indefinite test, rather than just the indefinite fuzzer.

Consider anchoring the regex:

Suggested change
COMMAND ${CMAKE_CTEST_COMMAND} --verbose -R "fuzzing.parser"
COMMAND ${CMAKE_CTEST_COMMAND} --verbose -R "^fuzzing\\.parser$$"

($$ is needed because CMake interprets a single $; the shell receives $ which anchors end-of-string.)

Alternatively, rename the indefinite test to something that isn't a prefix of the other, e.g. fuzzing.parser.infinite.


- name: Run fuzzing tests
run: ctest --test-dir $GITHUB_WORKSPACE/build -L fuzzing --verbose
run: ctest --test-dir $GITHUB_WORKSPACE/build -R fuzzing.parser.60 --verbose

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude: Minor consistency note — the other CI test steps use label-based selection (-L unit, -L e2e), while this now uses regex (-R). This works in practice, but the unescaped . characters are regex wildcards. For robustness you could escape them or keep using -L:

Suggested change
run: ctest --test-dir $GITHUB_WORKSPACE/build -R fuzzing.parser.60 --verbose
run: ctest --test-dir $GITHUB_WORKSPACE/build -R 'fuzzing\.parser\.60$' --verbose

@github-actions
Copy link

Claude: review of facebook/bpfilter #469 (d9e57b2)

Must fix

  • fuzzing target regex matches both teststests/fuzz/CMakeLists.txt:102ctest -R "fuzzing.parser" performs a regex substring match, so it matches both fuzzing.parser and fuzzing.parser.60. make fuzzing will run both tests instead of just the indefinite fuzzer. Anchor the regex or rename the tests to avoid prefix collision.

Suggestions

  • CI regex consistency.github/workflows/ci.yaml:125 — Other CI steps use -L (label-based) test selection; this step switches to -R (regex) with unescaped dots. Consider escaping dots and anchoring, or staying with label-based selection.

Nits

  • fuzzing_quick could be its own doc entrydoc/developers/build.rst:106 — Every other make target gets its own bullet point. Since fuzzing_quick is a distinct target, listing it separately would be more consistent and easier to scan.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant