pdd fix reports "No changes to commit" despite actually modifying files

[jamesdlevine]

Description

When running pdd fix against a GitHub issue, the command summary reports no files changed and "No changes to commit", but git status reveals that a prompt file was actually modified.

Steps to Reproduce

1. Run pdd fix https://github.com/promptdriven/pdd/issues/350
2. Observe the command output summary showing empty "Files changed:" and "Dev units fixed:"
3. Run git status to see actual file modifications

Expected Behavior

The summary should accurately report all files that were modified during the fix process:

Files changed: pdd/prompts/agentic_bug_step10_pr_LLM.prompt
Dev units fixed: agentic_bug_step10_pr_LLM

Actual Behavior

The summary shows:

Files changed:
Dev units fixed:
No changes to commit

But git status shows:

modified:   pdd/prompts/agentic_bug_step10_pr_LLM.prompt

And git diff confirms the file was substantively changed (log file parameter removed from the prompt template).

Session Log

[james@devk3-4 fix-issue-350]$ pdd fix https://github.com/promptdriven/pdd/issues/350
2026-01-20 20:17:13,877 - pdd.llm_invoke - INFO - Using user-specific LLM model CSV: /home/james/.pdd/llm_model.csv
2026-01-20 20:17:13,900 - pdd.llm_invoke - INFO - LiteLLM disk cache configured at /home/james/pdd-work/pdd_cap/litellm_cache.sqlite
Checking for updates...
Fetching issue #350 from promptdriven/pdd...
No worktree found for issue #350, using current directory
Starting Agentic E2E Fix for Issue #350
Working Directory: /home/james/pdd-cap-fixes/pdd_cap/.pdd/worktrees/fix-issue-350
Resuming from cycle 1 step 1...
Fixing e2e tests for issue #350: "pdd bug Step 10 generates pdd fix command referencing non-existent log file"

[Cycle 1/5] Starting fix cycle...
[Step 2/9] Running e2e tests...
Successfully loaded prompt: agentic_e2e_fix_step2_e2e_tests_LLM
  -> Step 2 complete. Cost: $0.6503
ALL_TESTS_PASS detected in Step 2. Exiting loop.

E2E fix complete
   Total cost: $1.3547
   Cycles used: 1/5
   Files changed:
   Dev units fixed:
   No changes to commit
Agentic fix completed: All tests passed during e2e check.

Actual Git State After Command

$ git status
On branch fix/issue-350
Changes not staged for commit:
    modified:   pdd/prompts/agentic_bug_step10_pr_LLM.prompt

$ git diff pdd/prompts/agentic_bug_step10_pr_LLM.prompt
-   - Use error log path: `fix-issue-{issue_number}.log` for the fix command output
...
-pdd --force fix ... fix-issue-{{issue_number}}.log
+pdd --force fix ...

Analysis

The change tracking mechanism appears to miss file modifications that occur during the fix process. This could be due to:

• The file change detection not scanning prompt files
• The change detection running before file modifications are written
• The "ALL_TESTS_PASS" early exit skipping the change collection logic

Environment

• Debug snapshot: .pdd/core_dumps/pdd-core-20260121T041914Z.json
• Working directory: /home/james/pdd-cap-fixes/pdd_cap/.pdd/worktrees/fix-issue-350
• Branch: fix/issue-350

[jamesdlevine]

Step 1: Duplicate Check

Status: Potential regression of #136, keeping #355 as canonical issue

Search Performed

• Searched for: "files changed", "no changes to commit", "change tracking", "pdd fix", "ALL_TESTS_PASS", "early exit", "change detection"
• Issues reviewed: ~15 potentially relevant issues

Findings

Related Issue Found: #136 - "Agentic fix doesn't output all files changed in the log"

• Status: Closed as "completed" (Nov 2025)
• Description: CLI agents edit files but changes aren't reported in the CLI log

Analysis:
Issue #136 describes the same general symptom that #355 reports:

• Files are modified during agentic fix
• The change summary doesn't reflect the actual modifications

However, #355 appears to be either:

1. A regression of Agentic fix doesn't output all files changed in the log #136 (the original fix didn't fully resolve the issue)
2. A different code path triggering the same symptom (specifically the ALL_TESTS_PASS early exit path)

Decision: Keeping #355 open as the canonical issue because:

• It provides significantly better documentation (reproducible steps, session logs, git state proof)
• It identifies a specific potential cause (ALL_TESTS_PASS early exit skipping change collection)
• It includes a debug snapshot for investigation
• The fact that Agentic fix doesn't output all files changed in the log #136 was closed but the problem persists suggests incomplete resolution

Other Related Issues:

• pdd fix incorrectly prints the name of an output code file when no new code file was generated #232 - "pdd fix incorrectly prints the name of an output code file when no new code file was generated" (closed) - Related but different symptom
• Log all PDD commands (not just sync) and update state files #277 - "Log all PDD commands and update state files" (closed) - Enhancement for logging infrastructure

---

Proceeding to Step 2: Documentation Check

[jamesdlevine]

Step 2: Documentation Check

Status: Confirmed Bug

Documentation Reviewed

• README.md - Core CLI documentation, workflow descriptions
• docs/TUTORIALS.md - User tutorials for pdd fix workflow
• docs/faq.md - FAQ on regeneration and file handling
• CONTRIBUTING.md - Development workflow guide
• CHANGELOG.md - Recent version changes related to file tracking

Findings

The documentation describes that pdd fix should report all files changed during the fix process, with no documented exceptions for the ALL_TESTS_PASS early exit scenario.

Root Cause Analysis:

After reviewing the source code in pdd/agentic_e2e_fix_orchestrator.py, I found a discrepancy between the summary display logic and the commit logic:

1. Commit logic (correct): Uses MD5 hash comparison (_get_file_hashes()) to detect files that actually changed during the workflow (lines 179-238). This correctly identifies files that were modified.

2. Summary display (buggy): Uses _parse_changed_files() (lines 79-90) which parses FILES_CREATED: and FILES_MODIFIED: tokens from agent output text. This relies on each step's LLM agent explicitly outputting these tokens.

3. The ALL_TESTS_PASS early exit issue: When the workflow detects ALL_TESTS_PASS in Step 2 (line 475-479), it exits before the summary gets populated with file changes from later steps. However, the critical issue is that Step 1 may have run pdd fix as a subprocess, which modifies files but doesn't add those changes to the changed_files list because:

   • The subprocess runs outside the LLM agent context
   • The agent output may not include FILES_MODIFIED: tokens for subprocess-modified files

Evidence supporting this is a bug:

• The CHANGELOG entry v0.0.121 explicitly states: "Hash-Based Commit Detection: E2E fix orchestrator (pdd fix) now commits only files that actually changed during the workflow using MD5 hash comparison"
• This implies the intent is to track ALL file changes, not just those explicitly reported by agents
• The user expectation (accurately report changed files) matches the documented behavior

The fix needed:
The summary display (lines 516-518) should use the actual file hash comparison to detect changes, similar to how _commit_and_push() already does, rather than relying on parsed agent output.

---

Proceeding to Step 3: Triage

[jamesdlevine]

Step 3: Triage

Status: Proceed

Summary

We have sufficient information to attempt reproduction and investigation:

What we understand:

• Expected: pdd fix summary should report all files modified during the fix process
• Actual: Summary shows empty "Files changed:" and "No changes to commit" while git status reveals actual modifications
• Root cause hypothesis: The summary uses agent output parsing (FILES_CREATED:/FILES_MODIFIED: tokens) rather than the hash-based detection used for commit logic. When ALL_TESTS_PASS triggers early exit at Step 2, files modified by subprocess commands (like pdd fix in Step 1) aren't captured in the changed_files list.

Reproduction context:

• Clear reproduction steps provided
• Session log shows exact command and output
• Git state documented showing the discrepancy
• Debug snapshot available at .pdd/core_dumps/pdd-core-20260121T041914Z.json
• Worktree preserved at .pdd/worktrees/fix-issue-350

Investigation targets:

• The code that aggregates changed_files for summary display
• The ALL_TESTS_PASS early exit path
• The hash-based comparison logic that correctly detects changes for commits (to understand why summary doesn't use it)

---

Proceeding to Step 4: Reproduction

[jamesdlevine]

Step 4: Reproduction

Status: Reproduced

Environment

• OS: Linux 6.8.0-87-generic
• Python: 3.12.12 (Anaconda)
• pdd version: 0.0.124
• Working directory: /home/james/pdd-cap-fixes/pdd_cap/.pdd/worktrees/fix-issue-350

Steps Taken

1. Examined the worktree for issue pdd bug Step 10 generates pdd fix command referencing non-existent log file #350 at /home/james/pdd-cap-fixes/pdd_cap/.pdd/worktrees/fix-issue-350
2. Ran git status to confirm file modification state
3. Ran git diff to see actual changes
4. Analyzed the orchestrator code to trace the change tracking flow

Observed Behavior

$ git status
On branch fix/issue-350
Your branch is up to date with 'origin/fix/issue-350'.

Changes not staged for commit:
    modified:   pdd/prompts/agentic_bug_step10_pr_LLM.prompt

$ git diff pdd/prompts/agentic_bug_step10_pr_LLM.prompt
-   - Use error log path: `fix-issue-{issue_number}.log` for the fix command output
-pdd --force fix ... fix-issue-{{issue_number}}.log
+pdd --force fix ...

The file was substantively modified (log file parameter removed), but the summary showed empty "Files changed:" and "Dev units fixed:".

Affected Files

• pdd/agentic_e2e_fix_orchestrator.py:79-90 - _parse_changed_files() only parses FILES_CREATED: and FILES_MODIFIED: tokens from LLM agent output
• pdd/agentic_e2e_fix_orchestrator.py:435-438 - Populates changed_files list by parsing agent output
• pdd/agentic_e2e_fix_orchestrator.py:516 - Summary display uses the agent-parsed changed_files list
• pdd/agentic_e2e_fix_orchestrator.py:521-527 - _commit_and_push() correctly uses hash-based detection but the result isn't reflected in the summary

Root Cause Analysis

The bug is in the dual tracking mechanism:

1. Hash-based detection (_get_file_hashes() at lines 179-195): Correctly detects all file changes by comparing MD5 hashes before and after the workflow. This is used for the commit logic.

2. Agent output parsing (_parse_changed_files() at lines 79-90): Parses FILES_CREATED: and FILES_MODIFIED: tokens from LLM output. This is used for the summary display.

When ALL_TESTS_PASS triggers early exit at Step 2 (line 475), files modified by Step 1's pdd fix subprocess are not in the changed_files list because:

• The subprocess modified files directly without emitting FILES_MODIFIED: tokens
• The summary display at line 516 uses the incomplete changed_files list

The fix should use the hash-based detection results for the summary display instead of the agent-parsed list.

Notes

• The commit logic (_commit_and_push) would correctly detect and commit the changed file
• The bug is cosmetic in that files ARE tracked for commits, but the user-facing summary is misleading
• This matches the hypothesis from Step 2 documentation check

---

Proceeding to Step 5: Root Cause Analysis

[jamesdlevine]

Step 5: Root Cause Analysis

Root Cause Identified: Yes

Summary

The bug occurs because agentic_e2e_fix_orchestrator.py uses two separate, unsynchronized mechanisms for file change tracking:

1. Agent output parsing (_parse_changed_files()) - parses FILES_CREATED: and FILES_MODIFIED: tokens from LLM agent output and stores them in the changed_files list. This list is used for the user-facing summary display.

2. Hash-based detection (_get_file_hashes() / _commit_and_push()) - compares MD5 hashes of files before and after the workflow. This is used for the commit logic.

When pdd fix runs as a subprocess in Step 1 (via agentic_e2e_fix_step1_unit_tests_LLM.prompt), it modifies files directly but the Step 1 agent does not emit FILES_MODIFIED: tokens. When ALL_TESTS_PASS triggers early exit at Step 2, the changed_files list remains empty because no agent reported the modifications. However, the hash-based commit mechanism would correctly detect the changes - but its result is not reflected in the summary.

Technical Details

• Location: pdd/agentic_e2e_fix_orchestrator.py:516
• Problematic Code:

# Line 516 - Uses agent-parsed list for display
console.print(f"   Files changed: {', '.join(changed_files)}")

The changed_files list is populated at lines 434-438:

# Parse changed files
new_files = _parse_changed_files(step_output)
for f in new_files:
    if f not in changed_files:
        changed_files.append(f)

But _parse_changed_files() only parses FILES_CREATED: and FILES_MODIFIED: tokens from agent output (lines 79-90). When pdd fix runs as a subprocess, it modifies files directly without emitting these tokens.

• Why it fails: The summary display at line 516 shows the changed_files list (agent-parsed), not the actual files detected by hash comparison. The hash-based mechanism correctly detects changes but is only used for commit logic, not the summary.

External Research

• Search terms: "pdd fix files changed", "agentic orchestrator file tracking"
• Sources checked: GitHub PRs, closed issues, CHANGELOG
• Relevant findings:
  • Issue Agentic fix doesn't output all files changed in the log #136 (closed): "Agentic fix doesn't output all files changed in the log" - identified as potential regression
  • PR Fix #232: Only print modified file paths in fix command output #326 (closed): "Fix pdd fix incorrectly prints the name of an output code file when no new code file was generated #232: Only print modified file paths in fix command output" - proposed content comparison approach for fix_error_loop.py but was rejected in favor of simpler fix
  • v0.0.121 CHANGELOG: Documents "Hash-Based Commit Detection" feature, confirming the design intent was to track ALL file changes, not just agent-reported ones

Repository History

• Relevant commits: The file has been modified mainly for version bumps; the dual-mechanism architecture appears to be original design
• Related PRs:
  • PR Fix #232: Only print modified file paths in fix command output #326 addressed similar issue in fix_error_loop.py but was closed - maintainer chose simpler truthy check approach
  • Issue pdd fix incorrectly prints the name of an output code file when no new code file was generated #232 (closed) was about similar symptom in non-agentic pdd fix
• Similar past issues:
  • Issue Agentic fix doesn't output all files changed in the log #136: Same symptom, likely same root cause, closed without clear fix
  • Issue pdd fix incorrectly prints the name of an output code file when no new code file was generated #232: Related but different context (loop mode file reporting)

Experiments Performed

1. Analyzed debug snapshot (.pdd/core_dumps/pdd-core-20260121T041914Z.json): Confirmed output shows empty "Files changed:" despite Step 1 running pdd fix which modified pdd/prompts/agentic_bug_step10_pr_LLM.prompt

2. Traced code flow:

   • Line 346: initial_file_hashes captures file state before workflow
   • Lines 434-438: changed_files populated from agent output parsing only
   • Lines 474-479: Early exit on ALL_TESTS_PASS in Step 2
   • Line 516: Summary displays agent-parsed changed_files (empty)
   • Lines 521-527: _commit_and_push() uses hash comparison (would detect changes)

3. Reviewed Step 1 prompt (agentic_e2e_fix_step1_unit_tests_LLM.prompt): Confirms prompt instructs agent to run pdd fix subprocess but does NOT require FILES_MODIFIED: token output

Fix Direction

The fix should use the hash-based detection for the summary display, not the agent-parsed list. Two approaches:

Option A: Compute changed files from hash comparison for summary
After the workflow completes (line 511), compute actual changes:

current_hashes = _get_file_hashes(cwd)
actual_changed = [f for f, h in current_hashes.items()
                  if f not in initial_file_hashes or initial_file_hashes[f] != h]
console.print(f"   Files changed: {', '.join(actual_changed)}")

Option B: Update changed_files list during _commit_and_push()
Have _commit_and_push() return the list of files it detected as changed, and use that for the summary.

Option A is simpler and more direct - it reuses existing infrastructure and ensures consistency between what's displayed and what's committed.

---

Proceeding to Step 6: Test Plan

[jamesdlevine]

Step 6: Test Plan

Existing Test Coverage

• Test file: tests/test_agentic_e2e_fix_orchestrator.py
• Current coverage: Only tests prompt template formatting (that templates can format without KeyError). Tests verify {step1_output} substitution, protect_tests_flag, and similar template variables.
• Gap: No tests for file change tracking logic. The orchestrator's core change detection mechanism (_parse_changed_files vs _get_file_hashes) is completely untested.

Root Cause Location

The bug is at pdd/agentic_e2e_fix_orchestrator.py:516:

console.print(f"   Files changed: {', '.join(changed_files)}")

This changed_files list is populated by _parse_changed_files() (lines 79-90), which only captures FILES_CREATED: and FILES_MODIFIED: tokens from agent output. When ALL_TESTS_PASS triggers early exit at Step 2, files modified by subprocesses (like pdd fix) aren't in this list.

The fix should use _get_file_hashes() comparison (lines 179-195), which correctly detects file changes via MD5 hash comparison - the same mechanism used for commits (lines 521-527).

Proposed Tests

Test 1: Summary displays files from hash-based detection (Primary Bug)

• Input: Mock scenario where ALL_TESTS_PASS detected in Step 2, no FILES_MODIFIED: tokens in output, but actual file modified on disk
• Expected: Summary should include the modified file in "Files changed:" output
• Actual (before fix): Summary shows empty "Files changed:" because changed_files list is empty

Test 2: Summary matches commit detection

• Input: Complete workflow run where files are modified
• Expected: Files shown in summary should exactly match files staged for commit
• Actual (before fix): Summary can show different files than what gets committed

Test 3: Hash-based detection captures subprocess modifications

• Input: Step 1 runs pdd fix as subprocess which modifies a file, Step 2 returns ALL_TESTS_PASS
• Expected: Modified file from subprocess should appear in "Files changed:" summary
• Actual (before fix): Subprocess-modified files are not tracked

Test 4: Agent-parsed files still included (no regression)

• Input: Agent output includes FILES_MODIFIED: some_file.py token
• Expected: some_file.py should still appear in summary (maintain backwards compatibility)
• Actual (before fix): This already works; test ensures fix doesn't break it

Test Location

• File: tests/test_agentic_e2e_fix_orchestrator.py (append new test class)
• Framework: pytest with unittest.mock
• Pattern: Follow existing pattern from test_agentic_bug_orchestrator.py:
  • Use mock_dependencies fixture to mock run_agentic_task, load_prompt_template, console
  • Use tmp_path fixture for isolated file operations
  • Create actual files in tmp_path to verify hash detection

Test Implementation Approach

class TestFileChangeTracking:
    """Tests for file change detection in summary display.
    
    Issue #355: Summary shows empty "Files changed:" even when files
    were modified during the workflow.
    """
    
    def test_summary_shows_hash_detected_changes_on_early_exit(
        self, mock_dependencies, default_args, tmp_path
    ):
        """
        Bug reproduction: When ALL_TESTS_PASS triggers early exit,
        summary should still show files modified by subprocesses.
        """
        # Setup: Create a file that will be "modified" during test
        test_file = tmp_path / "prompts" / "test.prompt"
        test_file.parent.mkdir(parents=True, exist_ok=True)
        test_file.write_text("original content")
        
        # Mock step outputs - Step 2 returns ALL_TESTS_PASS (early exit)
        # No FILES_MODIFIED tokens in output
        def side_effect_run(*args, **kwargs):
            label = kwargs.get('label', '')
            if label == 'cycle1_step2':
                # Simulate subprocess modifying file BEFORE returning
                test_file.write_text("modified content")
                return (True, "ALL_TESTS_PASS - Tests verified", 0.1, "model")
            return (True, "Step output", 0.1, "model")
        
        mock_run, _, mock_console = mock_dependencies
        mock_run.side_effect = side_effect_run
        
        # Run orchestrator
        success, msg, cost, model, changed_files = run_agentic_e2e_fix_orchestrator(
            **default_args
        )
        
        # ASSERTION: changed_files should include the hash-detected file
        # Currently fails: changed_files is empty because _parse_changed_files
        # found no FILES_MODIFIED: tokens
        assert "prompts/test.prompt" in changed_files or len(changed_files) > 0, \
            "Summary should show files detected via hash comparison"

Notes

• Tests require creating actual files in tmp_path because hash-based detection uses MD5 hashes of real files
• Need to mock the git commands in _get_modified_and_untracked() or use a real git repo in tmp_path
• The fix should unify the two tracking mechanisms: use hash detection for summary display (consistent with commit logic)
• Consider adding a helper method like _get_changed_files_from_hashes() to compute the diff between initial_file_hashes and current state

---

Proceeding to Step 7: Generate Test

[jamesdlevine]

Step 7: Generated Test

Test File

tests/test_agentic_e2e_fix_orchestrator.py

Test Code

class TestFileChangeTracking:
    """Tests for issue #355: Summary must show files from hash-based detection.

    The bug: When ALL_TESTS_PASS triggers early exit at Step 2, subprocess
    modifications (pdd fix) aren't reported because the summary uses
    agent-parsed FILES_MODIFIED: tokens instead of hash-based detection.

    Root cause: Line 516 displays `changed_files` (from _parse_changed_files)
    but commits use initial_file_hashes comparison (lines 521-527).
    """

    def test_summary_displays_hash_detected_files_not_agent_parsed(self, tmp_path):
        """Summary should display files detected via hash comparison.

        This is the primary bug reproduction test for issue #355.
        When a subprocess modifies files without emitting FILES_MODIFIED:,
        the summary should still show those files because hash detection
        captures them (for commit logic).

        Currently FAILS because line 516 uses changed_files (agent-parsed)
        instead of comparing initial_file_hashes with current state.
        """
        # ... creates git repo, mocks run_agentic_task to modify file
        # without emitting FILES_MODIFIED: token, triggers ALL_TESTS_PASS
        # Asserts: "test_module.py" in console output

    def test_changed_files_return_value_uses_hash_detection(self, tmp_path):
        """The changed_files return value should include hash-detected files."""
        # ... similar setup, asserts module.py in returned changed_files list

    def test_agent_parsed_files_still_included(self, tmp_path):
        """Files reported via FILES_MODIFIED: tokens should still be tracked.
        (Regression prevention test - currently PASSES)"""
        # ... uses FILES_MODIFIED: token, asserts file is included

    def test_summary_matches_commit_detection(self, tmp_path):
        """Summary displayed files should match what commit logic detects."""
        # ... creates two files, Step 1 reports file1.py, Step 2 modifies file2.py
        # Asserts both files appear in summary (currently only file1.py shown)

What This Test Verifies

The tests verify that the summary display at line 516 should use hash-based detection (like the commit logic) instead of agent-parsed FILES_MODIFIED: tokens:

Test	Expected	Actual (Bug)	Status
test_summary_displays_hash_detected_files	Shows test_module.py	Shows empty	FAILS
test_changed_files_return_value	Returns ['module.py']	Returns []	FAILS
test_agent_parsed_files_still_included	Includes reported.py	Includes reported.py	PASSES
test_summary_matches_commit_detection	Shows file1.py, file2.py	Shows file1.py only	FAILS

The bug occurs because when pdd fix runs as a subprocess, it modifies files directly without emitting FILES_MODIFIED: tokens. The summary uses the agent-parsed list (empty) while the commit logic correctly uses hash comparison (finds the modified file).

Running the Test

python -m pytest tests/test_agentic_e2e_fix_orchestrator.py::TestFileChangeTracking -v

---

Proceeding to Step 8: Verification

[jamesdlevine]

Step 8: Verification Complete

Test Execution

============================= test session starts ==============================
platform linux -- Python 3.12.12, pytest-8.3.5

pdd_cap/.../tests/test_agentic_e2e_fix_orchestrator.py::TestFileChangeTracking::test_summary_displays_hash_detected_files_not_agent_parsed FAILED [ 25%]
pdd_cap/.../tests/test_agentic_e2e_fix_orchestrator.py::TestFileChangeTracking::test_changed_files_return_value_uses_hash_detection FAILED [ 50%]
pdd_cap/.../tests/test_agentic_e2e_fix_orchestrator.py::TestFileChangeTracking::test_agent_parsed_files_still_included PASSED [ 75%]
pdd_cap/.../tests/test_agentic_e2e_fix_orchestrator.py::TestFileChangeTracking::test_summary_matches_commit_detection FAILED [100%]

=================================== FAILURES ===================================

test_summary_displays_hash_detected_files_not_agent_parsed:
  AssertionError: Summary should show test_module.py as changed (via hash detection), but got:
     Files changed: 
     Dev units fixed: 

test_changed_files_return_value_uses_hash_detection:
  AssertionError: changed_files should include 'module.py' (modified by subprocess), but got: []

test_summary_matches_commit_detection:
  AssertionError: Summary should show file2.py (detected by hash), but got:
     Files changed: file1.py  (only agent-parsed file shown)
     [green]Would commit 2 files[/green]  (hash detection found both)

================== 3 failed, 1 passed in 1.98s ===================

Verification Status

PASS: Tests correctly detect the bug

All 3 failing tests demonstrate the exact behavior described in issue #355:

• The summary shows empty or incomplete "Files changed:"
• But hash-based detection (used for commits) correctly identifies the modifications
• The test_summary_matches_commit_detection test clearly shows the discrepancy: summary says "file1.py" while commit logic found 2 files

Summary

Step	Result
Duplicate Check	No duplicate; regression of #136
Documentation	Confirmed Bug
Triage	Proceed
Reproduction	Reproduced
Root Cause	Identified (dual tracking mechanisms)
Test Plan	4 tests designed
Test Generation	4 tests written
Verification	PASS - Tests correctly detect bug

Bug Details

• Location: pdd/agentic_e2e_fix_orchestrator.py:516
• Root Cause: Summary display uses changed_files from _parse_changed_files() (agent-parsed tokens) instead of hash-based detection used for commit logic (lines 521-527)
• Test File: tests/test_agentic_e2e_fix_orchestrator.py (class TestFileChangeTracking)

Next Steps

1. Fix the bug at line 516 to use hash-based detection for summary display
2. Run the tests to confirm the fix (3 tests should now pass)
3. Run full test suite to check for regressions
4. Submit PR with fix and test

---

Proceeding to Step 9: E2E Tests

[jamesdlevine]

Step 9: E2E Test

Environment Discovery

• Repository language: Python
• Test framework: pytest 8.3.5
• Existing E2E tests found: Yes - tests/test_operation_logging_e2e.py, tests/test_pddrc_true_e2e.py, etc.
• Test configuration: pytest.ini and pyproject.toml
• Fixtures/helpers used: conftest.py (CliRunner fixture, autouse fixtures for PDD_PATH/stream restoration)
• Environment variables required: None for these tests (mocked GitHub API)
• Auth/credentials: Mocked via _fetch_issue_data patch

E2E Test File

tests/test_agentic_e2e_fix_orchestrator.py

Test Type

CLI Integration Tests

What This Test Verifies

These tests verify that the CLI command pdd fix <url> correctly reports all modified files in its output, including files modified by subprocess calls that don't emit FILES_MODIFIED: tokens.

The bug causes the CLI to show Files changed: with an empty or incomplete list, even though files were actually modified and would be committed by the hash-based commit logic.

Test Code

class TestE2EFileChangeReporting:
    """E2E tests for issue #355: CLI output must show all changed files.

    These tests verify the user-facing CLI behavior by invoking the fix command
    through Click's CliRunner and checking the actual terminal output.

    The bug: When `pdd fix <url>` runs and modifies files via subprocess
    (like nested `pdd fix` calls), those changes don't appear in the
    summary output ("Files changed:") even though git shows them as modified.
    """

    def test_cli_output_shows_hash_detected_files(self, tmp_path):
        """E2E: CLI should output 'Files changed: <file>' for hash-detected changes.

        Currently FAILS because the CLI displays changed_files from
        _parse_changed_files() (agent tokens only), not hash-based detection.
        """
        # Creates git repo, mocks GitHub API and run_agentic_task
        # Step 2 modifies file WITHOUT emitting FILES_MODIFIED: token
        # Verifies CLI output includes the modified file
        # FAILS: Shows "Files changed:" (empty)

    def test_cli_output_no_changes_message_accuracy(self, tmp_path):
        """E2E: Summary should show modified files, not empty list.

        Currently FAILS because the orchestrator only reports agent-parsed files.
        """
        # Verifies modified_file.py appears in "Files changed:" summary
        # FAILS: Shows "Files changed:" (empty)

    def test_cli_changed_files_return_includes_subprocess_modifications(self, tmp_path):
        """E2E: Both agent-reported and subprocess-modified files should appear.

        Currently FAILS because orchestrator returns only agent-parsed files.
        """
        # Step 1: Agent modifies file1 with FILES_MODIFIED: token
        # Step 2: Subprocess modifies file2 without token
        # Verifies BOTH appear in output
        # FAILS: Only shows agent_reported.py, missing subprocess_modified.py

Test Execution Result

$ pytest tests/test_agentic_e2e_fix_orchestrator.py -k "TestE2E" -v --tb=line

============================= test session starts ==============================
platform linux -- Python 3.12.12, pytest-8.3.5

collected 14 items / 11 deselected / 3 selected

test_cli_output_shows_hash_detected_files FAILED [ 33%]
test_cli_output_no_changes_message_accuracy FAILED [ 66%]
test_cli_changed_files_return_includes_subprocess_modifications FAILED [100%]

=================================== FAILURES ===================================
test_cli_output_shows_hash_detected_files:
    AssertionError: CLI output should show 'subprocess_modified.py' in Files changed, but got:
       Files changed: 
       Dev units fixed: 

test_cli_output_no_changes_message_accuracy:
    AssertionError: CLI should show 'modified_file.py' in Files changed summary

test_cli_changed_files_return_includes_subprocess_modifications:
    AssertionError: Output should include subprocess_modified.py (via hash detection), got:
       Files changed: agent_reported.py   # ONLY shows token-parsed file!
       Dev units fixed: 

======================= 3 failed, 11 deselected in 2.30s =======================

E2E Test Status

PASS: E2E tests correctly detect the bug

All 3 CLI-level E2E tests fail as expected, confirming:

1. The Files changed: summary only shows files parsed from FILES_MODIFIED: tokens
2. Files modified by subprocess (like nested pdd fix calls) are not shown
3. The summary display at line 516 uses the wrong data source (agent-parsed changed_files list instead of hash-based detection)

Combined with the 3 unit tests from Step 7 (also failing), we have 6 tests that will pass once the fix is implemented:

• 3 unit tests (TestFileChangeTracking class)
• 3 E2E tests (TestE2EFileChangeReporting class)

---

Proceeding to Step 10: Create Draft PR

[jamesdlevine]

Step 10: Draft PR Created

Pull Request

PR #38: Add failing tests for #355: pdd fix reports 'No changes to commit' despite modifying files (superseded)

Replacement PR: fix: Use git-based file change detection in e2e fix summary (#355)

Branch

fix/issue-355

[jamesdlevine]

Final tests should verify that any potentially modified file types are captured and displayed when modified - all dev unit parts, in addition to other files (preamble, test.prompt, csv files, markdown files, etc.)

[jamesdlevine]

[jamesdlevine]

Step 2: E2E Test Check (Cycle 1)

Status: 6 Tests Failing

E2E Tests Run

• TestFileChangeTracking::test_summary_displays_hash_detected_files_not_agent_parsed ✗
• TestFileChangeTracking::test_changed_files_return_value_uses_hash_detection ✗
• TestFileChangeTracking::test_agent_parsed_files_still_included ✓
• TestFileChangeTracking::test_summary_matches_commit_detection ✗
• TestE2EFileChangeReporting::test_cli_output_shows_hash_detected_files ✗
• TestE2EFileChangeReporting::test_cli_output_no_changes_message_accuracy ✗
• TestE2EFileChangeReporting::test_cli_changed_files_return_includes_subprocess_modifications ✗

Failures

test_summary_displays_hash_detected_files_not_agent_parsed

AssertionError: Summary should show test_module.py as changed (via hash detection), but got:
   Files changed: 
   Dev units fixed: 

The test modified test_module.py during Step 2 but no FILES_MODIFIED: token was emitted. The summary shows empty "Files changed:" instead of detecting the modification via hash comparison.

test_changed_files_return_value_uses_hash_detection

AssertionError: changed_files should include 'module.py' (modified by subprocess), but got: []

The returned changed_files list is empty because it relies on agent-parsed tokens only.

test_summary_matches_commit_detection

AssertionError: Summary should show file2.py (detected by hash), but got:
   Files changed: file1.py

Step 1 reported file1.py via token, Step 2 modified file2.py without token. Only file1.py appears in summary.

test_cli_output_shows_hash_detected_files

AssertionError: CLI output should show 'subprocess_modified.py' in Files changed, but got:
   Files changed: 

CLI invocation shows empty "Files changed:" despite file being modified.

test_cli_output_no_changes_message_accuracy

AssertionError: CLI should show 'modified_file.py' in Files changed summary, but got:
   Files changed: 

Same issue via CLI - modified file not appearing in summary.

test_cli_changed_files_return_includes_subprocess_modifications

AssertionError: Output should include subprocess_modified.py (via hash detection), got:
   Files changed: agent_reported.py

Shows only the agent-reported file, missing the subprocess-modified file.

Summary

All 6 failing tests confirm the bug described in issue #355:

• The summary display at line 516 uses changed_files from _parse_changed_files() (agent tokens only)
• Files modified by subprocesses without FILES_MODIFIED: tokens are not captured
• The hash-based detection used for commit logic is not used for the summary display

The one passing test (test_agent_parsed_files_still_included) confirms that files reported via FILES_MODIFIED: tokens are correctly tracked - this is expected behavior that should be preserved.

---

Proceeding to Step 3: Root Cause Analysis

[jamesdlevine]

Step 3: Root Cause Analysis (Cycle 1)

Status: CODE_BUG

Failure Analysis

The test failures clearly demonstrate the bug described in issue #355. I ran all 7 tests:

• 6 FAIL (detecting the bug)
• 1 PASS (regression prevention test)

Test: test_summary_displays_hash_detected_files_not_agent_parsed

• Expected behavior (per docs): CHANGELOG v0.0.121 states: "E2E fix orchestrator (pdd fix) now commits only files that actually changed during the workflow using MD5 hash comparison." The same hash-based detection should be used for the summary display.
• Test expects: When a file is modified during the workflow (without FILES_MODIFIED: token), the summary should show that file in "Files changed:"
• Code does: Shows "Files changed: " (empty) because line 516 uses changed_files which is populated by _parse_changed_files() (agent token parsing), not hash comparison

Test: test_changed_files_return_value_uses_hash_detection

• Expected behavior (per docs): The returned changed_files list should include all files that changed during the workflow
• Test expects: changed_files should contain 'module.py' (modified by subprocess)
• Code does: Returns [] because no FILES_MODIFIED: token was emitted

Test: test_summary_matches_commit_detection

• Expected behavior (per docs): Summary display should match what commit logic detects
• Test expects: Both file1.py (agent-reported) and file2.py (subprocess-modified) should appear
• Code does: Shows "Files changed: file1.py" but "Would commit 2 files" - proving the discrepancy between summary (agent-parsed) and commit (hash-based)

Test: test_agent_parsed_files_still_included (PASSES)

• This regression test confirms that files reported via FILES_MODIFIED: tokens are still tracked - this behavior should be preserved

Root Cause

The orchestrator uses two separate, unsynchronized mechanisms for file change tracking:

1. Agent output parsing (_parse_changed_files() lines 79-90): Parses FILES_CREATED: and FILES_MODIFIED: tokens from LLM agent output. Stores results in changed_files list. Used for summary display (line 516).

2. Hash-based detection (_get_file_hashes() lines 179-195): Compares MD5 hashes before and after workflow. Used for commit logic (lines 521-527).

When ALL_TESTS_PASS triggers early exit at Step 2, files modified by subprocess calls (like nested pdd fix) are correctly detected by hash comparison for commits, but not included in the summary or return value because:

• The subprocess modified files directly without emitting FILES_MODIFIED: tokens
• The summary display at line 516 uses the incomplete changed_files list (agent-parsed only)

Evidence from test output:

Files changed: file1.py        # Agent-parsed (only file1.py from token)
[green]Would commit 2 files[/green]  # Hash-based (both file1.py AND file2.py)

Recommendation

FIX THE CODE at pdd/agentic_e2e_fix_orchestrator.py:

The fix should unify the summary display with the commit detection by using hash-based detection for both:

1. Before the summary display (around line 511-516), compute actual changed files from hash comparison:

   current_hashes = _get_file_hashes(cwd)
   actual_changed = [f for f, h in current_hashes.items()
                     if f not in initial_file_hashes or initial_file_hashes[f] != h]
   console.print(f"   Files changed: {', '.join(actual_changed)}")

2. Update the return value to use actual_changed instead of changed_files

3. Preserve agent-parsed files (union of both sets) for backwards compatibility if needed

This ensures the summary displays the same files that would be committed, matching user expectations and documented behavior.

---

Proceeding to Step 4: Fix E2E Tests (if needed)