Skip to content

TEST: Cover Fuzzer MCTS search score, stop conditions and score normalization - #4320

Open
Lanre Shittu (Shizoqua) wants to merge 1 commit into
microsoft:mainfrom
Shizoqua:test/fuzzer-mcts-edge-cases
Open

Lanre Shittu (Shizoqua) wants to merge 1 commit into
microsoft:mainfrom
Shizoqua:test/fuzzer-mcts-edge-cases

Conversation

@Shizoqua

Copy link
Copy Markdown
Contributor

Description

Closes #4319.

Adds unit tests for FuzzerGenerator and _MCTSExplorer logic that had no direct coverage:

  • _calculate_uct_score: step 0 gives the same finite score as step 1 instead of taking log(0), a less visited node scores higher than an often visited one with equal rewards, and a more rewarded node scores higher with equal visits.
  • _should_stop_generation: generation stops when the next batch would exceed max_query_limit (but not when it fits exactly) and when target_jailbreak_goal_count is reached.
  • _normalize_score_to_float: booleans map to 1.0 and 0.0, numbers are clamped to [0, 1], and unexpected types are treated as 0.0 with a warning.

The tests live in a new file so they stay independent of the open jailbreak-counting change in #4231. No production code changes.

Tests and Documentation

Added tests/unit/executor/promptgen/fuzzer/test_fuzzer_search_edge_cases.py (15 tests). No docs or notebooks changed, so JupyText was not needed.

Copilot AI balanced review requested due to automatic review settings November 15, 2026 14:15

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues remain for the regression-test additions covering FuzzerGenerator and _MCTSExplorer edge cases.

No open issues 🎉

What changed in this PR

Adds focused regression tests for FuzzerGenerator and _MCTSExplorer, addressing #4319 without production changes. The tests remain separate from the jailbreak-counting work in #4231.

Changes:

  • Tests finite step-zero search scores and ordering by node visits and rewards.
  • Covers query-limit boundaries and the jailbreak-goal stop condition.
  • Verifies score normalization, clamping, and warnings for unexpected types.
File Description
tests/​unit/​executor/​promptgen/​fuzzer/​test_fuzzer_search_edge_cases.py Adds 15 test cases covering search scores, stopping conditions, and score normalization.

Review effort: Balanced

@hannahwestra25 hannahwestra25 self-assigned this Nov 20, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TEST: Fuzzer MCTS search score, stop conditions and score normalization lack unit tests

3 participants