pdd fix modifies prompt files but does not regenerate corresponding code

[jamesdlevine]

Description

When pdd fix (agentic mode) modifies a .prompt file to fix an issue, it does not subsequently regenerate the code from that updated prompt. This violates PDD's core principle that prompts are the source of truth and code is a regenerable artifact.

Steps to Reproduce

1. Run pdd fix https://github.com/promptdriven/pdd/issues/350
2. Observe that pdd/prompts/agentic_bug_step10_pr_LLM.prompt is modified
3. Note that the corresponding code file is not regenerated from the updated prompt

Expected Behavior

After modifying a prompt file, pdd fix should:

1. Detect that the prompt changed
2. Run pdd generate (or equivalent) to regenerate the code from the updated prompt
3. Re-run tests to verify the regenerated code works correctly

Actual Behavior

The prompt file is modified but the code file remains unchanged. The fix is incomplete because only half of the prompt→code relationship is updated.

Context

This occurred in the same session as issue #355, where pdd fix also failed to report the file changes. The prompt agentic_bug_step10_pr_LLM.prompt was modified to remove references to a non-existent log file parameter, but no code regeneration followed.

Impact

• Prompt and code become out of sync
• The fix may appear complete (tests pass) but the generated code doesn't reflect the prompt changes
• Future pdd generate runs could produce different results, causing confusion

[jamesdlevine]

Step 1: Duplicate Check

Status: No duplicates found

Search Performed

• Searched for: "pdd fix regenerate", "prompt code sync", "pdd generate", "agentic prompt modify", "source of truth prompt", "agentic fix code", "prompt file changed code not"
• Issues reviewed: 50+ issues scanned, 10 manually evaluated for similarity

Related Issues (Not Duplicates)

Issue	Title	Status	Relationship
#350	pdd bug Step 10 generates pdd fix command referencing non-existent log file	Open	Same workflow session - about invalid command parameters
#355	pdd fix reports "No changes to commit" despite actually modifying files	Open	Same workflow session - about change reporting
#154	generation overwrite a file doesn't always happen	Open	Different issue - about file overwrite failures, not prompt→code sync
#102	pdd bug + pdd test incremental to increase coverage should append tests	Open	Feature request - about test file handling, not prompt regeneration

Analysis

This issue describes a distinct architectural gap in the pdd fix workflow:

1. pdd bug Step 10 generates pdd fix command referencing non-existent log file #350 addresses invalid command generation (log file parameter)
2. pdd fix reports "No changes to commit" despite actually modifying files #355 addresses incorrect change reporting (missing file modifications in summary)
3. pdd fix modifies prompt files but does not regenerate corresponding code #356 addresses missing code regeneration (prompt updated but corresponding code not regenerated)

The three issues share a common workflow context but describe different failures:

• pdd bug Step 10 generates pdd fix command referencing non-existent log file #350: Command generation defect
• pdd fix reports "No changes to commit" despite actually modifying files #355: Change tracking defect
• pdd fix modifies prompt files but does not regenerate corresponding code #356: Workflow completeness defect (core PDD principle violation)

Issue #356 is unique because it highlights a violation of PDD's fundamental principle that prompts are the source of truth and code should always be regenerated from updated prompts.

Conclusion

This is a new, distinct issue that should remain open. It may be related to the same underlying workflow session as #350 and #355, but it identifies a separate problem that requires independent investigation and resolution.

---

Proceeding to Step 2: Documentation Check

[jamesdlevine]

Step 2: Documentation Check

Status: Confirmed Bug

Documentation Reviewed

• README.md - Command documentation for pdd fix, pdd sync, pdd change
• CONTRIBUTING.md - Core philosophy and dev unit concepts
• docs/whitepaper.md - Core PDD principles and workflow
• docs/prompt-driven-development-doctrine.md - PDD doctrine and principles
• docs/TUTORIALS.md - Issue-driven development workflow
• docs/faq.md - Usage patterns and mental model
• docs/prompting_guide.md - Prompt maintenance patterns
• pdd/prompts/agentic_fix_primary_LLM.prompt - Agentic fix agent behavior
• pdd/prompts/agentic_bug_step*.prompt - Agentic bug workflow steps

Findings

The reported behavior contradicts multiple documented PDD principles:

1. Prompts as Source of Truth

"Prompts are the source of truth; code is regenerated from prompts" - CONTRIBUTING.md
"Prompts authoritatively define the system's intended behavior. Code becomes a generated artifact derived from these prompts." - whitepaper.md

2. Regenerative Development Principle

"Changes are implemented by modifying the relevant prompt(s) and regenerating the affected code." - whitepaper.md
"Regenerate, Don't Patch: Change the prompt, then regenerate affected surfaces." - doctrine.md

3. Synchronization Loop

"Always back-propagate implementation learnings to prompts. Keep prompts, code, examples, and tests in continuous sync." - doctrine.md
"The fundamental unit is often considered the prompt and its generated code, example, and test file – all kept in sync." - whitepaper.md

4. Current Agentic Fix Behavior
The agentic_fix_primary_LLM.prompt instructs the agent to "use file editing tools to make minimal, targeted fixes" to code and test files. However, when the agent modifies a prompt file to fix a bug, there is no subsequent regeneration step documented in the agentic workflow.

The TUTORIALS.md even acknowledges this gap:

"Run pdd sync on modified prompts to regenerate code if needed"

This suggests the behavior is known but not automated. When pdd fix (agentic mode) modifies a prompt file, it should:

1. Detect the prompt was modified
2. Automatically run pdd generate or pdd sync to regenerate the code
3. Re-run tests to verify the regenerated code works

Conclusion: This is a confirmed bug where the agentic pdd fix workflow violates PDD's core synchronization principle. Prompt changes should trigger code regeneration to maintain the prompt→code invariant that PDD is built upon.

---

Proceeding to Step 3: Triage

[jamesdlevine]

Step 3: Triage

Status: Proceed

Summary

The issue is well-documented with sufficient information for reproduction:

What we understand:

• Expected behavior: When pdd fix (agentic mode) modifies a .prompt file, it should automatically regenerate the corresponding code file via pdd generate, maintaining the prompt→code synchronization that is PDD's core principle
• Actual behavior: pdd fix modifies prompt files (e.g., agentic_bug_step10_pr_LLM.prompt) but does NOT regenerate the corresponding code files
• Reproduction context: Occurred during pdd fix https://github.com/promptdriven/pdd/issues/350

Code area identified:

• pdd/agentic_e2e_fix_orchestrator.py - The orchestrator for the agentic fix workflow
• pdd/commands/fix.py - The fix command entry point
• The workflow steps (particularly Step 8: "Running pdd fix") appear to lack a regeneration phase after prompt modifications

Why we can proceed:

1. Clear reproduction steps are provided
2. The expected vs actual behavior is unambiguous
3. The code area is identifiable (agentic fix orchestrator)
4. Previous steps confirmed this is a valid bug, not a documentation misunderstanding

---

Proceeding to Step 4: Reproduction

[jamesdlevine]

Step 4: Reproduction

Status: Reproduced (via code analysis)

Environment

• Repository: promptdriven/pdd (local clone)
• Platform: Linux 6.8.0-87-generic
• Analysis Method: Static code analysis of workflow orchestrators

Steps Taken

1. Analyzed the e2e fix workflow in pdd/agentic_e2e_fix_orchestrator.py
2. Reviewed Step 8 prompt template (agentic_e2e_fix_step8_run_pdd_fix_LLM.prompt)
3. Reviewed Step 9 prompt template (agentic_e2e_fix_step9_verify_all_LLM.prompt)
4. Compared with the pdd change workflow in pdd/agentic_change_orchestrator.py which correctly handles prompt regeneration
5. Searched for any sync/regeneration logic in the e2e fix orchestrator

Observed Behavior

The e2e fix workflow has a clear gap between Step 8 and Step 9:

Step 8 (agentic_e2e_fix_orchestrator.py:434-444):

• Runs pdd fix --manual on failing dev units
• Parses FILES_MODIFIED from output (line 435)
• Tracks changed files but does NOT check for .prompt files
• No code regeneration step exists

Step 9 (agentic_e2e_fix_orchestrator.py:481-491):

• Immediately runs test verification
• Has no awareness of whether prompts were modified
• Tests stale code if prompts changed

Contrast with pdd change workflow (agentic_change_orchestrator.py:495-537):

# Parse files_to_stage to extract prompt files
for file_path in file_list:
    if file_path.startswith("prompts/") and file_path.endswith(".prompt"):
        module = extract_module_from_include(file_path)
        if module:
            modified_modules.add(module)

# Build dependency graph and compute sync order
graph = build_dependency_graph(prompts_dir)
sorted_modules, cycles = topological_sort(graph)
affected = get_affected_modules(sorted_modules, modified_modules, graph)
# ... generates sync_order.sh to regenerate code

The pdd change workflow correctly:

1. Detects modified .prompt files
2. Builds a dependency graph
3. Computes affected modules via topological sort
4. Generates a sync order script for code regeneration

The pdd fix e2e workflow lacks all of this.

Affected Files

• pdd/agentic_e2e_fix_orchestrator.py:434-444 - Parses changed files but doesn't detect prompts
• pdd/agentic_e2e_fix_orchestrator.py:481-499 - Jumps directly to Step 9 verification
• pdd/prompts/agentic_e2e_fix_step8_run_pdd_fix_LLM.prompt - No guidance to regenerate code
• pdd/prompts/agentic_e2e_fix_step9_verify_all_LLM.prompt - Runs tests without regeneration

Notes

• The issue is confirmed via static analysis - the code path clearly shows no regeneration step
• The reference implementation exists in agentic_change_orchestrator.py and could be adapted
• Supporting utilities already exist: sync_order.py provides build_dependency_graph(), topological_sort(), get_affected_modules(), and generate_sync_order_script()
• The fix would require adding prompt detection and regeneration logic between Step 8 and Step 9

---

Proceeding to Step 5: Root Cause Analysis

[jamesdlevine]

Step 5: Root Cause Analysis

Root Cause Identified: Yes

Summary

The bug occurs because agentic_e2e_fix_orchestrator.py (used by pdd fix in e2e agentic mode) is missing the prompt-to-code regeneration logic that exists in agentic_change_orchestrator.py. When Step 8 (run_pdd_fix) modifies a .prompt file via pdd fix --manual, the orchestrator parses the FILES_MODIFIED output (lines 434-438) but never checks if any modified files are prompts that need corresponding code regeneration. Step 9 (verify_all) immediately runs tests against the old (stale) code files.

Technical Details

• Location: pdd/agentic_e2e_fix_orchestrator.py:434-444 and the gap between Steps 8 and 9
• Problematic Code:

# Parse changed files (lines 434-438)
new_files = _parse_changed_files(step_output)
for f in new_files:
    if f not in changed_files:
        changed_files.append(f)

# Parse dev unit states (Step 8) (lines 440-444)
if step_num == 8:
    s5_out = step_outputs.get("5", "")
    dev_units_str = _parse_dev_units(s5_out)
    dev_unit_states = _update_dev_unit_states(step_output, dev_unit_states, dev_units_str)

# Step 9 runs immediately after with no regeneration check

• Why it fails: The code only tracks which files changed for reporting purposes. There is no logic to:
  1. Check if any FILES_MODIFIED entries are .prompt files
  2. Build a dependency graph to identify affected modules
  3. Run pdd generate or pdd sync on modified prompts before verification

External Research

• Search terms: "pdd regenerate prompt code sync", "prompt driven development sync workflow"
• Sources checked: GitHub issues, PRs in promptdriven/pdd repo
• Relevant findings:
  • PR feat(sync): auto-update tests when prompt changes (#203) #328 (feat: auto-update tests when prompt changes) addresses a similar sync issue for tests, confirming the pattern that prompt changes should trigger downstream regeneration
  • Issue Automatically update tests based on prompt changes during sync #203 (tracked by PR feat(sync): auto-update tests when prompt changes (#203) #328) shows the team's commitment to keeping prompts as source of truth
  • The agentic_change_orchestrator.py already implements correct sync logic at lines 495-537

Repository History

• Relevant commits: bef4caf introduced agentic_e2e_fix_orchestrator.py on 2026-01-15
• Related PRs:
  • PR Fix agentic bug workflow timeout configuration #258 (merged 2026-01-05): Fixed agentic bug workflow timeout configuration
  • PR feat(sync): auto-update tests when prompt changes (#203) #328 (draft): Adds auto-update tests when prompt changes - shows team awareness of sync issues
• Similar past issues: Issue Automatically update tests based on prompt changes during sync #203 (auto-update tests on prompt change) addresses a parallel sync gap
• Key observation: The e2e fix orchestrator was created from scratch without incorporating the sync logic that exists in the change orchestrator, likely because it was designed for a different workflow (fix existing code vs. implement new changes)

Experiments Performed

1. Code path comparison: Verified agentic_change_orchestrator.py has sync logic at lines 495-537 using sync_order.py functions (build_dependency_graph, topological_sort, get_affected_modules, generate_sync_order_script)
2. E2E fix orchestrator analysis: Confirmed no imports from sync_order.py and no calls to any regeneration/sync functions
3. Step 8 prompt analysis: Confirmed agentic_e2e_fix_step8_run_pdd_fix_LLM.prompt instructs the agent to run pdd fix --manual which can modify .prompt files
4. Step 9 prompt analysis: Confirmed agentic_e2e_fix_step9_verify_all_LLM.prompt immediately runs tests without any regeneration step

Fix Direction

The e2e fix orchestrator needs a regeneration step between Steps 8 and 9. Two approaches:

Option A: Add regeneration logic (like change orchestrator)

1. Import sync_order functions in agentic_e2e_fix_orchestrator.py
2. After Step 8 completes, check changed_files for .prompt files
3. If prompts modified, build dependency graph and run pdd generate on affected modules
4. Then proceed to Step 9 verification

Option B: Add a new Step 8.5

1. Create a new agentic_e2e_fix_step8_5_regenerate_LLM.prompt
2. Insert between Step 8 and Step 9 in the orchestrator
3. The step would run pdd sync or pdd generate on any modified prompt modules

Option A is preferred as it matches the change orchestrator pattern and doesn't require a new agentic step (saving cost/time).

---

Proceeding to Step 6: Test Plan

[jamesdlevine]

Step 6: Test Plan

Existing Test Coverage

• Test file: tests/test_agentic_e2e_fix_orchestrator.py
• Current coverage: Only tests prompt template formatting (KeyError prevention for placeholders). Tests that prompts can be formatted with orchestrator context variables. Does NOT test workflow logic, file handling, or regeneration.
• Gap: No tests for:
  • Post-Step 8 file modification handling
  • Detection of .prompt file changes
  • Triggering code regeneration from modified prompts
  • Integration with sync_order.py functions

Reference Implementation

The agentic_change_orchestrator.py correctly implements sync/regeneration at lines 495-537:

# Parse files_to_stage to extract prompt files
# Build dependency graph and compute sync order
# Generate sync_order_script for regeneration

Proposed Tests

Test 1: Detect prompt file in FILES_MODIFIED (Primary Bug)

• Input: Step 8 output contains FILES_MODIFIED: pdd/prompts/agentic_bug_step10_pr_LLM.prompt
• Expected: Orchestrator detects .prompt file change and triggers regeneration before Step 9
• Actual (before fix): Orchestrator proceeds directly to Step 9, running tests against stale code

Test 2: No regeneration needed for code-only changes

• Input: Step 8 output contains FILES_MODIFIED: pdd/agentic_e2e_fix_orchestrator.py
• Expected: No regeneration triggered (only .py file modified, not a prompt)
• Rationale: Ensures regeneration logic only activates for prompt changes

Test 3: Multiple file types including prompts

• Input: Step 8 output contains FILES_MODIFIED: pdd/prompts/foo_python.prompt, pdd/some_module.py, tests/test_foo.py
• Expected: Regeneration triggered for foo module; other files unchanged
• Rationale: Mixed file changes should correctly identify prompts needing regeneration

Test 4: LLM runtime prompts should NOT trigger regeneration

• Input: Step 8 output contains FILES_MODIFIED: pdd/prompts/agentic_step9_LLM.prompt
• Expected: No regeneration triggered (LLM prompts are runtime prompts, not code-generation prompts)
• Rationale: Per sync_order.py:74, _LLM suffix indicates runtime prompt

Test 5: Sync order respects dependency graph

• Input: Prompt file bar_python.prompt depends on foo_python.prompt; both modified
• Expected: foo synced before bar (topological order)
• Rationale: Dependencies must regenerate first to ensure correct cascading

Test 6: Context includes sync_order variables before Step 9

• Input: Step 8 modifies a code-generation prompt
• Expected: sync_order_script and sync_order_list added to context before Step 9
• Rationale: Mirror the change orchestrator's approach at lines 535-537

Test Location

• File: tests/test_agentic_e2e_fix_orchestrator.py (append new test class)
• Framework: pytest with unittest.mock
• Pattern: Follow test_agentic_change_orchestrator.py mocking patterns

Test Implementation Notes

1. Use @pytest.fixture for mock_dependencies (mock run_agentic_task, load_prompt_template, subprocess)
2. Create temp directory with mock prompt files for dependency graph tests
3. Capture context passed to template.format() to verify sync variables
4. Mock Step 8 output to include different FILES_MODIFIED patterns

Code Changes Required for Tests to Pass

The fix will need to add logic between Step 8 and Step 9:

# After Step 8, before Step 9:
new_files = _parse_changed_files(step8_output)
prompt_files = [f for f in new_files if f.endswith('.prompt') and not f.endswith('_LLM.prompt')]
if prompt_files:
    # Use sync_order.py functions to:
    # 1. Extract module names from prompt files
    # 2. Build dependency graph
    # 3. Get affected modules (topological order)
    # 4. Generate and execute sync script OR call pdd sync directly

---

Proceeding to Step 7: Generate Test

[jamesdlevine]

Step 7: Generated Test

Test File

tests/test_agentic_e2e_fix_orchestrator.py

Test Code

# -----------------------------------------------------------------------------
# Fixtures for Workflow Logic Tests
# -----------------------------------------------------------------------------

@pytest.fixture
def temp_cwd(tmp_path):
    """Returns a temporary directory path to use as cwd."""
    return tmp_path


@pytest.fixture
def mock_e2e_fix_dependencies(temp_cwd):
    """
    Mocks the external dependencies for e2e fix orchestrator tests.
    """
    with patch("pdd.agentic_e2e_fix_orchestrator.run_agentic_task") as mock_run, \
         patch("pdd.agentic_e2e_fix_orchestrator.load_prompt_template") as mock_load, \
         patch("pdd.agentic_e2e_fix_orchestrator.console") as mock_console:

        # Default mock behaviors
        mock_run.return_value = (True, "Default Agent Output", 0.1, "gpt-4")

        mock_template = MagicMock()
        mock_template.format.return_value = "Formatted Prompt"
        mock_load.return_value = mock_template

        yield mock_run, mock_load, mock_console


# -----------------------------------------------------------------------------
# Issue #356: Prompt Regeneration Tests
# -----------------------------------------------------------------------------

def test_sync_order_context_populated_when_prompt_modified(mock_e2e_fix_dependencies, temp_cwd):
    """
    Test that when Step 8 modifies a .prompt file, sync_order_script and
    sync_order_list are added to context before Step 9 formatting.

    Issue #356: pdd fix modifies prompt files but does not regenerate
    corresponding code. This violates PDD's core principle that prompts
    are the source of truth.

    TDD: Before fix, this test fails because the orchestrator does not
    check for .prompt files in FILES_MODIFIED and does not set up
    sync_order_script/sync_order_list context variables.
    """
    mock_run, mock_load, mock_console = mock_e2e_fix_dependencies

    # Simulate Step 8 reporting a modified prompt file
    def side_effect_run(**kwargs):
        label = kwargs.get("label", "")
        instruction = kwargs.get("instruction", "")

        if "step8" in label:
            # Step 8 reports a modified .prompt file
            return (True, "FILES_MODIFIED: pdd/prompts/some_module_python.prompt", 0.1, "gpt-4")
        if "step9" in label:
            # Step 9 should have access to sync variables - return early exit
            return (True, "ALL_TESTS_PASS", 0.1, "gpt-4")
        # Default for steps 1-7
        return (True, f"Output for {label}", 0.1, "gpt-4")

    mock_run.side_effect = side_effect_run

    # Capture the context passed to Step 9 template formatting
    step9_context = {}
    call_count = [0]

    def capture_format(**kwargs):
        call_count[0] += 1
        # Step 9 is the 9th call to format
        if call_count[0] == 9:
            step9_context.clear()
            step9_context.update(kwargs)
        return "Formatted"

    mock_template = MagicMock()
    mock_template.format.side_effect = capture_format
    mock_load.return_value = mock_template

    # Create a git repo structure so orchestrator can find git root
    (temp_cwd / ".git").mkdir(parents=True)

    # Create prompts directory with a mock prompt file
    prompts_dir = temp_cwd / "pdd" / "prompts"
    prompts_dir.mkdir(parents=True)
    (prompts_dir / "some_module_python.prompt").write_text("% some_module prompt")

    # Run orchestrator
    run_agentic_e2e_fix_orchestrator(
        issue_url="https://github.com/test/repo/issues/356",
        issue_content="Test issue content",
        repo_owner="test",
        repo_name="repo",
        issue_number=356,
        issue_author="tester",
        issue_title="Test issue",
        cwd=temp_cwd,
        resume=False,
        quiet=True,
    )

    # Assert: Step 9 context must have sync_order variables when prompt was modified
    assert "sync_order_script" in step9_context, (
        f"sync_order_script missing from Step 9 context. "
        f"When a .prompt file is modified in Step 8, the orchestrator should "
        f"populate sync_order_script before Step 9. Keys present: {list(step9_context.keys())}"
    )
    assert "sync_order_list" in step9_context, (
        f"sync_order_list missing from Step 9 context. "
        f"When a .prompt file is modified in Step 8, the orchestrator should "
        f"populate sync_order_list before Step 9. Keys present: {list(step9_context.keys())}"
    )

What This Test Verifies

This test verifies that the e2e fix orchestrator properly handles prompt file modifications by:

1. Mocking Step 8 to report FILES_MODIFIED: pdd/prompts/some_module_python.prompt (a code generation prompt)
2. Capturing the context passed to Step 9's template formatting
3. Asserting that sync_order_script and sync_order_list are present in the Step 9 context

Why it fails on buggy code:

• The current agentic_e2e_fix_orchestrator.py parses FILES_MODIFIED at line 434-438 but does NOT:
  • Check if any modified files are .prompt files
  • Import or use the sync_order functions (build_dependency_graph, topological_sort, etc.)
  • Populate sync_order_script or sync_order_list context variables

The test clearly demonstrates the bug: when Step 8 modifies a prompt file, the context passed to Step 9 is missing the sync order variables that would trigger code regeneration.

Comparison with working code:

• agentic_change_orchestrator.py:495-537 correctly implements this logic by detecting .prompt files and generating sync order before Step 12

Running the Test

pytest tests/test_agentic_e2e_fix_orchestrator.py::test_sync_order_context_populated_when_prompt_modified -v

Test Results (Current Buggy Code)

FAILED tests/test_agentic_e2e_fix_orchestrator.py::test_sync_order_context_populated_when_prompt_modified
AssertionError: sync_order_script missing from Step 9 context. 
When a .prompt file is modified in Step 8, the orchestrator should 
populate sync_order_script before Step 9. 
Keys present: ['issue_url', 'repo_owner', 'repo_name', 'issue_number', 
'cycle_number', 'max_cycles', 'issue_content', 'protect_tests', 
'protect_tests_flag', 'step1_output', 'step2_output', 'step3_output', 
'step4_output', 'step5_output', 'step6_output', 'step7_output', 
'step8_output', 'dev_units_identified', 'next_cycle']

Additional Tests Added

Two supplementary tests were also added to document expected behavior:

1. test_no_sync_order_needed_when_only_code_files_modified - Verifies that when only .py files are modified (no prompts), sync_order should have default values
2. test_llm_prompt_files_should_not_trigger_regeneration - Verifies that _LLM.prompt files (agent instruction prompts) do NOT trigger code regeneration

---

Proceeding to Step 8: Verification

[jamesdlevine]

Step 8: Verification Complete

Test Execution

============================= test session starts ==============================
platform linux -- Python 3.12.12, pytest-8.3.5, pluggy-1.6.0
tests/test_agentic_e2e_fix_orchestrator.py::test_sync_order_context_populated_when_prompt_modified FAILED [100%]

=================================== FAILURES ===================================
____________ test_sync_order_context_populated_when_prompt_modified ____________

    # Assert: Step 9 context must have sync_order variables when prompt was modified
>   assert "sync_order_script" in step9_context, (
        f"sync_order_script missing from Step 9 context. "
        f"When a .prompt file is modified in Step 8, the orchestrator should "
        f"populate sync_order_script before Step 9. Keys present: {list(step9_context.keys())}"
    )
E   AssertionError: sync_order_script missing from Step 9 context. When a .prompt file is modified in Step 8, the orchestrator should populate sync_order_script before Step 9. Keys present: ['issue_url', 'repo_owner', 'repo_name', 'issue_number', 'cycle_number', 'max_cycles', 'issue_content', 'protect_tests', 'protect_tests_flag', 'step1_output', 'step2_output', 'step3_output', 'step4_output', 'step5_output', 'step6_output', 'step7_output', 'step8_output', 'dev_units_identified', 'next_cycle']

============================== 1 failed in 6.56s ===============================

Full test suite: 9 passed, 1 failed (only the bug-detection test fails)

Verification Status

PASS: Test correctly detects the bug

The test fails for exactly the right reason:

• Step 8 reports FILES_MODIFIED: pdd/prompts/some_module_python.prompt
• The orchestrator parses this but does not check for .prompt files
• Step 9 context is missing sync_order_script and sync_order_list
• The assertion message is clear and points to the exact issue

Summary

Step	Result
Duplicate Check	✅ No duplicates found
Documentation	✅ Confirmed bug
Triage	✅ Sufficient info to proceed
Reproduction	✅ Reproduced via code analysis
Root Cause	✅ Missing regeneration logic
Test Plan	✅ 6 test cases planned
Test Generation	✅ 3 tests added
Verification	✅ Test correctly fails

Bug Details

• Location: pdd/agentic_e2e_fix_orchestrator.py:434-444
• Root Cause: Orchestrator parses FILES_MODIFIED but doesn't check for .prompt files or trigger regeneration
• Test File: tests/test_agentic_e2e_fix_orchestrator.py
• Reference Fix: See agentic_change_orchestrator.py:495-537 for correct implementation

Next Steps

1. Fix the bug at the identified location by adding prompt file detection and sync logic
2. Run the test to confirm the fix (pytest tests/test_agentic_e2e_fix_orchestrator.py::test_sync_order_context_populated_when_prompt_modified -v)
3. Run full test suite to check for regressions
4. Submit PR with fix and test

---

Proceeding to Step 9: E2E Tests

[jamesdlevine]

Step 9: E2E Test

Environment Discovery

• Repository language: Python
• Test framework: pytest (8.3.5) with pytest-asyncio, pytest-mock, pytest-cov, pytest-xdist
• Existing E2E tests found: Yes - tests/test_e2e_issue_219_duplicate_includes.py, tests/test_operation_logging_e2e.py, tests/test_pddrc_true_e2e.py, tests/test_e2e_issue_295_openai_schema.py
• Test configuration: pytest.ini - defines markers including e2e, integration, slow
• Fixtures/helpers used: conftest.py - provides runner (CliRunner), preserve_pdd_path, restore_standard_streams fixtures
• Environment variables required: PDD_PATH, PDD_FORCE_LOCAL, API keys for real E2E tests
• Auth/credentials: Tests use @pytest.mark.e2e for tests requiring real LLM calls; mocked tests use fake API keys

E2E Test File

tests/test_e2e_issue_356_prompt_regeneration.py

Test Type

Integration (orchestrator workflow)

What This Test Verifies

When pdd fix (via the e2e fix orchestrator) modifies a .prompt file in Step 8, the orchestrator should:

1. Detect the prompt file in FILES_MODIFIED
2. Build a dependency graph using sync_order functions
3. Populate sync_order_script and sync_order_list context variables
4. Pass these to Step 9 for code regeneration

This is the core behavior that Issue #356 reports is missing - the orchestrator parses FILES_MODIFIED but ignores .prompt files and never triggers regeneration.

Test Code

"""
E2E Test for Issue #356: pdd fix modifies prompt files but does not regenerate code
"""
import pytest
from pathlib import Path
from unittest.mock import MagicMock, patch
from pdd.agentic_e2e_fix_orchestrator import run_agentic_e2e_fix_orchestrator

class TestE2EPromptRegeneration:
    @pytest.fixture
    def project_with_prompts(self, tmp_path):
        """Create realistic project with prompts and include dependencies."""
        (tmp_path / ".git").mkdir()
        prompts_dir = tmp_path / "prompts"
        prompts_dir.mkdir()
        
        # Base prompt that others depend on
        (prompts_dir / "python_preamble.prompt").write_text("% Python standards")
        
        # Module prompts with includes
        (prompts_dir / "cli_python.prompt").write_text(
            "<include>prompts/python_preamble.prompt</include>\nCLI module"
        )
        (prompts_dir / "sync_order_python.prompt").write_text(
            "<include>prompts/python_preamble.prompt</include>\nSync order module"
        )
        
        # _LLM.prompt (should NOT trigger regen)
        (prompts_dir / "agentic_step1_LLM.prompt").write_text("% Agent instruction")
        
        return tmp_path

    def test_e2e_orchestrator_adds_sync_order_when_prompt_modified(
        self, project_with_prompts
    ):
        """PRIMARY BUG TEST: sync_order variables must be populated when .prompt modified."""
        cwd = project_with_prompts
        step9_context_captured = {}
        format_call_count = [0]

        def capture_format(**kwargs):
            format_call_count[0] += 1
            if format_call_count[0] == 9:
                step9_context_captured.update(kwargs)
            return "Formatted"

        mock_template = MagicMock()
        mock_template.format.side_effect = capture_format

        def mock_run_agentic_task(**kwargs):
            label = kwargs.get("label", "")
            if "step8" in label:
                return (True, "FILES_MODIFIED: prompts/cli_python.prompt", 0.1, "gpt-4")
            if "step9" in label:
                return (True, "ALL_TESTS_PASS", 0.1, "gpt-4")
            return (True, f"Output for {label}", 0.1, "gpt-4")

        with patch("pdd.agentic_e2e_fix_orchestrator.run_agentic_task", 
                   side_effect=mock_run_agentic_task), \
             patch("pdd.agentic_e2e_fix_orchestrator.load_prompt_template", 
                   return_value=mock_template), \
             patch("pdd.agentic_e2e_fix_orchestrator.console"):
            run_agentic_e2e_fix_orchestrator(
                issue_url="https://github.com/test/repo/issues/356",
                issue_content="Test issue",
                repo_owner="test",
                repo_name="repo",
                issue_number=356,
                issue_author="tester",
                issue_title="Test",
                cwd=cwd,
                resume=False,
                quiet=True,
            )

        # THE KEY ASSERTIONS
        assert "sync_order_script" in step9_context_captured, \
            f"sync_order_script missing! Keys: {sorted(step9_context_captured.keys())}"
        assert "sync_order_list" in step9_context_captured, \
            f"sync_order_list missing! Keys: {sorted(step9_context_captured.keys())}"

Test Execution Result

============================= test session starts ==============================
platform linux -- Python 3.12.12, pytest-8.3.5

tests/test_e2e_issue_356_prompt_regeneration.py::TestE2EPromptRegeneration::test_e2e_orchestrator_adds_sync_order_when_prompt_modified FAILED [ 25%]
tests/test_e2e_issue_356_prompt_regeneration.py::TestE2EPromptRegeneration::test_e2e_orchestrator_no_regen_for_code_only_changes PASSED [ 50%]
tests/test_e2e_issue_356_prompt_regeneration.py::TestE2EPromptRegeneration::test_e2e_orchestrator_llm_prompt_excluded_from_regen PASSED [ 75%]
tests/test_e2e_issue_356_prompt_regeneration.py::TestE2EPromptRegeneration::test_e2e_orchestrator_dependency_chain_included PASSED [100%]

=================================== FAILURES ===================================
_ TestE2EPromptRegeneration.test_e2e_orchestrator_adds_sync_order_when_prompt_modified _

E       AssertionError: BUG DETECTED (Issue #356): sync_order_script missing from Step 9 context!
E         
E         When Step 8 modifies a .prompt file (prompts/cli_python.prompt),
E         the orchestrator should compute a sync order and add sync_order_script
E         to the context BEFORE formatting the Step 9 prompt.
E         
E         Keys present in Step 9 context: ['cycle_number', 'dev_units_identified', 'issue_content', 
E         'issue_number', 'issue_url', 'max_cycles', 'next_cycle', 'protect_tests', 
E         'protect_tests_flag', 'repo_name', 'repo_owner', 'step1_output', 'step2_output', 
E         'step3_output', 'step4_output', 'step5_output', 'step6_output', 'step7_output', 
E         'step8_output']
E         
E         Expected: sync_order_script and sync_order_list should be present.
E         This is how agentic_change_orchestrator.py handles it (lines 495-537).

============================== 1 failed, 3 passed in 5.38s ========================

E2E Test Status

PASS: E2E test correctly detects the bug

The primary test (test_e2e_orchestrator_adds_sync_order_when_prompt_modified) fails exactly as expected because the orchestrator does not populate sync_order_script or sync_order_list in the Step 9 context when a .prompt file is modified.

Test Suite Summary:

• 1 test FAILS (correctly detects bug): Primary bug detection test
• 3 tests PASS (document expected behavior): Code-only changes, LLM prompt exclusion, dependency chains

Key Finding:
The Step 9 context contains 19 keys but is missing the critical sync_order_script and sync_order_list variables that should be populated after Step 8 modifies a prompt file.

---

Proceeding to Step 10: Create Draft PR

[jamesdlevine]

Step 10: Draft PR Created

Pull Request

PR #41: Add failing tests for #356: pdd fix does not regenerate code after prompt changes

Branch

fix/issue-356

What's Included

• Failing unit test at tests/test_agentic_e2e_fix_orchestrator.py
• Failing E2E test at tests/test_e2e_issue_356_prompt_regeneration.py
• Commits: 1

Next Steps for Maintainers

1. Review the failing tests to understand the expected behavior
2. Implement the fix at the identified location (pdd/agentic_e2e_fix_orchestrator.py:434-444)
3. Verify both unit and E2E tests pass with your fix
4. Run full test suite to check for regressions
5. Mark the PR as ready for review

PDD Fix Command

To auto-fix this bug using PDD:

cd /home/james/pdd-cap-fixes/pdd_cap/.pdd/worktrees/fix-issue-356
pdd --force fix --loop --max-attempts 5 --verification-program context/agentic_e2e_fix_orchestrator_example.py pdd/prompts/agentic_e2e_fix_orchestrator_python.prompt pdd/agentic_e2e_fix_orchestrator.py tests/test_agentic_e2e_fix_orchestrator.py fix-issue-356.log

---

Investigation complete. A draft PR with failing tests has been created and linked to this issue.

[jamesdlevine]

Step 2: E2E Test Check (Cycle 1)

Status: 2 Tests Failing

E2E Tests Run

• test_e2e_issue_356_prompt_regeneration.py::TestE2EPromptRegeneration::test_e2e_orchestrator_adds_sync_order_when_prompt_modified ✗
• test_e2e_issue_356_prompt_regeneration.py::TestE2EPromptRegeneration::test_e2e_orchestrator_no_regen_for_code_only_changes ✓
• test_e2e_issue_356_prompt_regeneration.py::TestE2EPromptRegeneration::test_e2e_orchestrator_llm_prompt_excluded_from_regen ✓
• test_e2e_issue_356_prompt_regeneration.py::TestE2EPromptRegeneration::test_e2e_orchestrator_dependency_chain_included ✓
• test_agentic_e2e_fix_orchestrator.py::TestPromptFormatting::test_step1_prompt_formats_without_error ✓
• test_agentic_e2e_fix_orchestrator.py::TestPromptFormatting::test_step2_prompt_formats_without_error ✓
• test_agentic_e2e_fix_orchestrator.py::TestPromptFormatting::test_step3_prompt_formats_without_error ✓
• test_agentic_e2e_fix_orchestrator.py::TestPromptFormatting::test_step7_prompt_formats_without_error ✓
• test_agentic_e2e_fix_orchestrator.py::TestPromptFormatting::test_step1_prompt_formats_with_protect_tests_flag ✓
• test_agentic_e2e_fix_orchestrator.py::TestPromptFormatting::test_step8_prompt_formats_with_protect_tests_flag ✓
• test_agentic_e2e_fix_orchestrator.py::TestPromptFormatting::test_step9_prompt_formats_without_error ✓
• test_agentic_e2e_fix_orchestrator.py::test_run_agentic_e2e_fix_orchestrator_has_protect_tests_parameter ✓
• test_agentic_e2e_fix_orchestrator.py::TestLoopModeNoLogFile::test_step1_prompt_no_log_file_in_loop_mode ✓
• test_agentic_e2e_fix_orchestrator.py::TestLoopModeNoLogFile::test_step8_prompt_no_log_file_in_loop_mode ✓
• test_agentic_e2e_fix_orchestrator.py::TestLoopModeNoLogFile::test_no_touch_log_file_command_for_loop_mode ✓
• test_agentic_e2e_fix_orchestrator.py::TestLoopModeLogFileE2E::test_loop_mode_without_log_file_processes_single_test_file ✓
• test_agentic_e2e_fix_orchestrator.py::test_sync_order_context_populated_when_prompt_modified ✗
• test_agentic_e2e_fix_orchestrator.py::test_no_sync_order_needed_when_only_code_files_modified ✓
• test_agentic_e2e_fix_orchestrator.py::test_llm_prompt_files_should_not_trigger_regeneration ✓

Failures

test_e2e_orchestrator_adds_sync_order_when_prompt_modified

AssertionError: BUG DETECTED (Issue #356): sync_order_script missing from Step 9 context!

When Step 8 modifies a .prompt file (prompts/cli_python.prompt),
the orchestrator should compute a sync order and add sync_order_script
to the context BEFORE formatting the Step 9 prompt.

Keys present in Step 9 context: ['cycle_number', 'dev_units_identified', 'issue_content', 'issue_number', 'issue_url', 'max_cycles', 'next_cycle', 'protect_tests', 'protect_tests_flag', 'repo_name', 'repo_owner', 'step1_output', 'step2_output', 'step3_output', 'step4_output', 'step5_output', 'step6_output', 'step7_output', 'step8_output']

Expected: sync_order_script and sync_order_list should be present.
This is how agentic_change_orchestrator.py handles it (lines 495-537).

test_sync_order_context_populated_when_prompt_modified

AssertionError: sync_order_script missing from Step 9 context. When a .prompt file is modified in Step 8, the orchestrator should populate sync_order_script before Step 9. Keys present: ['issue_url', 'repo_owner', 'repo_name', 'issue_number', 'cycle_number', 'max_cycles', 'issue_content', 'protect_tests', 'protect_tests_flag', 'step1_output', 'step2_output', 'step3_output', 'step4_output', 'step5_output', 'step6_output', 'step7_output', 'step8_output', 'dev_units_identified', 'next_cycle']

Summary

• Total tests: 19
• Passed: 17
• Failed: 2

Both failing tests detect the same bug: when Step 8 modifies a .prompt file, the orchestrator does not populate sync_order_script and sync_order_list in the Step 9 context. This means prompt changes are not followed by code regeneration.

---

Proceeding to Step 3: Root Cause Analysis

[jamesdlevine]

Step 3: Root Cause Analysis (Cycle 1)

Status: CODE_BUG

Failure Analysis

Test: test_sync_order_context_populated_when_prompt_modified

• Expected behavior (per docs): According to CONTRIBUTING.md, "prompts are the source of truth; code is regenerated from prompts." When a .prompt file is modified, the corresponding code should be regenerated to maintain synchronization.
• Test expects: When Step 8 reports FILES_MODIFIED: pdd/prompts/some_module_python.prompt, the orchestrator should populate sync_order_script and sync_order_list in the Step 9 context before formatting the verification prompt.
• Code does: The orchestrator at pdd/agentic_e2e_fix_orchestrator.py:434-438 parses FILES_MODIFIED output but only adds files to changed_files list for tracking. It does NOT:
  1. Check if any modified files are .prompt files
  2. Import or use sync_order functions (build_dependency_graph, topological_sort, get_affected_modules)
  3. Populate sync_order_script or sync_order_list context variables for Step 9

Test: test_e2e_orchestrator_adds_sync_order_when_prompt_modified

• Expected behavior (per docs): Same as above - prompt changes should trigger code regeneration.
• Test expects: Same assertion - sync_order_script and sync_order_list should be present in Step 9 context.
• Code does: Same issue - no regeneration logic exists in the e2e fix orchestrator.

Root Cause

The agentic_e2e_fix_orchestrator.py is missing the prompt file detection and sync order logic that exists in agentic_change_orchestrator.py.

Reference implementation (from agentic_change_orchestrator.py:643-670):

# Initialize defaults
sync_order_script = ""; sync_order_list = "No modules to sync"

# Parse files_to_stage to extract prompt files
for file_path in file_list:
    if file_path.startswith("prompts/") and file_path.endswith(".prompt"):
        module = extract_module_from_include(file_path)
        if module: modified_modules.add(module)

# Build dependency graph and compute sync order
if prompts_dir.exists() and modified_modules:
    graph = build_dependency_graph(prompts_dir)
    sorted_modules, cycles = topological_sort(graph)
    affected = get_affected_modules(sorted_modules, modified_modules, graph, cyclic_modules)
    if affected:
        script_path = worktree_path / "sync_order.sh"
        sync_order_list = generate_sync_order_script(affected, script_path, worktree_path)
        sync_order_script = str(script_path)

# Add to context before Step 13
context["sync_order_script"] = sync_order_script
context["sync_order_list"] = sync_order_list

The e2e fix orchestrator lacks ALL of this logic.

Recommendation

Fix Required in: pdd/agentic_e2e_fix_orchestrator.py

The fix should:

1. Import sync_order functions at the top of the file:

   from pdd.sync_order import (
       build_dependency_graph,
       topological_sort,
       get_affected_modules,
       generate_sync_order_script,
       extract_module_from_include,
   )

2. After Step 8 completes (around line 444), add logic to:

   • Check changed_files for .prompt files (excluding _LLM.prompt files)
   • Build dependency graph if prompts were modified
   • Compute sync order using topological sort
   • Set sync_order_script and sync_order_list context variables

3. Before Step 9 formatting, add these variables to the context:

   context["sync_order_script"] = sync_order_script
   context["sync_order_list"] = sync_order_list

This mirrors the pattern established in agentic_change_orchestrator.py lines 643-670.

---

Proceeding to Step 4: Fix E2E Tests (if needed)

[jamesdlevine]

Step 4: Fix E2E Tests (Cycle 1)

Status: No Test Fixes Needed

Root cause analysis indicates CODE_BUG only. The failing tests correctly detect the bug described in Issue #356:

• test_sync_order_context_populated_when_prompt_modified - Unit test verifying sync_order context variables
• test_e2e_orchestrator_adds_sync_order_when_prompt_modified - E2E test verifying orchestrator behavior

Both tests properly assert that when Step 8 modifies a .prompt file, the orchestrator should populate sync_order_script and sync_order_list in the Step 9 context. This aligns with PDD's documented principle that "prompts are the source of truth; code is regenerated from prompts."

The tests are correct - the code needs to be fixed, not the tests.

---

Proceeding to Step 5: Identify Dev Units