Add failing tests for #8: template injection bug#382
Draft
Serhan-Asad wants to merge 1 commit intopromptdriven:mainfrom
Draft
Add failing tests for #8: template injection bug#382Serhan-Asad wants to merge 1 commit intopromptdriven:mainfrom
Serhan-Asad wants to merge 1 commit intopromptdriven:mainfrom
Conversation
This commit adds comprehensive test coverage to reproduce and verify the template injection bug reported in issue promptdriven#8. Tests added: - tests/test_issue_8_template_injection.py: Unit tests verifying that load_prompt_template() should preprocess <include> directives - tests/test_e2e_issue_8_template_injection.py: E2E tests simulating the full user workflow at step 5 of pdd change Root cause: The load_prompt_template() function doesn't call preprocess() to expand <include> directives. When Step 5's template includes docs/prompting_guide.md containing JSON examples with braces, Python's .format(**context) interprets those braces as template variables, causing KeyError. All tests correctly fail on the current buggy code and will pass once load_prompt_template() is fixed to call preprocess(). Related to promptdriven#8 Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Contributor
Author
Step 10: Draft PR Created ✅What's IncludedThis PR adds comprehensive failing tests for the template injection bug:
Test ResultsAll 6 tests correctly fail on the current buggy code, detecting that:
Next Steps for Maintainers
PDD Fix CommandTo auto-fix this bug using PDD, first create a prompt file (if one doesn't already exist), then run: pdd --force fix --loop --max-attempts 5 --verification-program context/load_prompt_template_example.py <prompt_file> pdd/load_prompt_template.py tests/test_issue_8_template_injection.pyNote: A prompt file for
Impact Summary
Investigation complete. Generated by PDD agentic bug workflow (Steps 1-10) |
Contributor
|
Reopening: this PR was automatically closed by a force push to main on Feb 9, not intentionally. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds failing tests that detect the template injection bug reported in #8.
Test Files
tests/test_issue_8_template_injection.pytests/test_e2e_issue_8_template_injection.pyWhat This PR Contains
Root Cause
The
load_prompt_template()function inpdd/load_prompt_template.pydoesn't callpreprocess()to expand<include>directives. When Step 5's template contains<include>docs/prompting_guide.md</include>and the prompting guide has JSON examples with braces like{"type": "module"}, Python's.format(**context)at line 521 ofagentic_change_orchestrator.pyinterprets those braces as template variables, causingKeyError: '\n "type"'.Test Coverage
The test suite includes:
Unit Tests (
test_issue_8_template_injection.py):test_load_prompt_template_should_preprocess_includes: Verifies thatload_prompt_template()should expand<include>directives and escape bracestest_step5_template_has_unprocessed_include: Integration test reproducing the exact Issue Generating files from default preprocessed prompt name fails: could not determine language #8 scenariotest_load_prompt_template_should_call_preprocess: Root cause verification using mockingE2E Tests (
test_e2e_issue_8_template_injection.py):test_orchestrator_step5_with_include_directive: Primary E2E test simulating the exact user workflowtest_orchestrator_full_context_with_include: Tests with full orchestrator contexttest_multiple_includes_in_template: Regression test for multiple includesNext Steps
pdd/load_prompt_template.pyto callpreprocess()Fixes #8
Generated by PDD agentic bug workflow (Steps 1-10)