Skip to content

Conveniences for setting up worktrees #387

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jamesdlevine wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Conveniences for setting up worktrees #387

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
53e4f13
Add parameterized conda environment to Makefile
jamesdlevine Jan 20, 2026
ea41e5b
Add developer setup script and worktree-aware conda environment
jamesdlevine Jan 23, 2026
94ae8e5
Clarify PDD_CONDA_ENV precedence in Makefile
jamesdlevine Jan 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 15 additions & 2 deletions CONTRIBUTING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,13 +61,26 @@ Each complete change ideally includes four elements:
```
2. Create and activate a Conda environment (recommended):
```bash
conda create -n pdd python=3.12
conda activate pdd
conda create -n pdd-dev python=3.12
conda activate pdd-dev
```
To use a different environment name, set the `PDD_CONDA_ENV` environment variable before running make commands:
```bash
# Option 1: Export for the session
export PDD_CONDA_ENV=my-env
make <target>

# Option 2: Set inline for a single command
PDD_CONDA_ENV=my-env make build

# Option 3: Pass as a make argument
make test PDD_CONDA_ENV=my-env
```
3. Install dependencies in editable mode with dev extras:
```bash
pip install -e .[dev]
```
Note: Many Makefile targets (e.g., `make test`, `make coverage`, `make lint`) automatically install dev dependencies into the conda environment before each run.
4. Optional: Use Docker for reproducible environments, especially for tricky, environment‑dependent bugs.

## Running Tests
Expand Down
87 changes: 62 additions & 25 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,21 @@
# Detect if we're under a worktrees directory
_PARENT_DIR := $(notdir $(patsubst %/,%,$(dir $(CURDIR))))
_CURRENT_DIR := $(notdir $(CURDIR))

# Conda environment precedence:
# 1. PDD_CONDA_ENV from environment (highest priority)
# 2. Current directory name if under worktrees/
# 3. "pdd" (default)
ifndef PDD_CONDA_ENV
ifeq ($(_PARENT_DIR),worktrees)
PDD_CONDA_ENV := $(_CURRENT_DIR)
else
PDD_CONDA_ENV := pdd
endif
endif

PDD_PYTHON_VERSION := 3.12

# Directories
PROD_DIR := pdd
STAGING_DIR := .
Expand Down Expand Up @@ -58,7 +76,26 @@ help:
@echo " make staging - Copy files to staging"
@echo " make production - Copy files from staging to pdd"
@echo " make update-extension - Update VS Code extension"

@echo ""
@echo "Environment Setup:"
@echo " make dev-setup - Run full developer setup (interactive)"
@echo " make create-conda-env - Create conda env (requires worktrees dir)"

# Create conda environment (only valid if under worktrees directory)
.PHONY: create-conda-env
create-conda-env:
ifeq ($(_PARENT_DIR),worktrees)
@echo "Creating conda environment '$(_CURRENT_DIR)' with Python $(PDD_PYTHON_VERSION)"
@conda create -n $(_CURRENT_DIR) python=$(PDD_PYTHON_VERSION) -y
Comment on lines +88 to +89
Copy link

Copilot AI Jan 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The create-conda-env target uses _CURRENT_DIR directly instead of PDD_CONDA_ENV. This creates an inconsistency where this target bypasses the environment name logic defined at lines 9-15. Consider using $(PDD_CONDA_ENV) for consistency.

Suggested change
@echo "Creating conda environment '$(_CURRENT_DIR)' with Python $(PDD_PYTHON_VERSION)"
@conda create -n $(_CURRENT_DIR) python=$(PDD_PYTHON_VERSION) -y
@echo "Creating conda environment '$(PDD_CONDA_ENV)' with Python $(PDD_PYTHON_VERSION)"
@conda create -n $(PDD_CONDA_ENV) python=$(PDD_PYTHON_VERSION) -y

Copilot uses AI. Check for mistakes.
else
$(error Cannot create conda environment. Current directory must be immediately under a 'worktrees' directory. Current parent directory: $(_PARENT_DIR))
endif

# Run the full developer setup script
.PHONY: dev-setup
dev-setup:
@bash scripts/dev-setup.sh

# Public repo paths (override via env if needed)
PUBLIC_PDD_REPO_DIR ?= staging/public/pdd
PUBLIC_PDD_REMOTE ?= https://github.com/promptdriven/pdd.git
Expand Down Expand Up @@ -186,31 +223,31 @@ run-examples: $(EXAMPLE_FILES)
@echo "Running examples one by one:"
@$(foreach example_file,$(EXAMPLE_FILES), \
echo "Running $(example_file)"; \
conda run -n pdd --no-capture-output PYTHONPATH=$(STAGING_DIR):$(PDD_DIR):$$PYTHONPATH python $(example_file) || exit 1; \
conda run -n $(PDD_CONDA_ENV) --no-capture-output PYTHONPATH=$(STAGING_DIR):$(PDD_DIR):$$PYTHONPATH python $(example_file) || exit 1; \
)
@echo "All examples ran successfully"

# Ensure dev dependencies are installed before running tests
ensure-dev-deps:
@echo "Updating pdd conda environment with dev dependencies"
@conda run -n pdd --no-capture-output pip install -e '.[dev]'
@echo "Updating $(PDD_CONDA_ENV) conda environment with dev dependencies"
@conda run -n $(PDD_CONDA_ENV) --no-capture-output pip install -e '.[dev]'

# Run tests
test: ensure-dev-deps
@echo "Running staging tests"
@cd $(STAGING_DIR)
@conda run -n pdd --no-capture-output PDD_RUN_REAL_LLM_TESTS=1 PDD_RUN_LLM_TESTS=1 PDD_PATH=$(abspath $(PDD_DIR)) PYTHONPATH=$(PDD_DIR):$$PYTHONPATH python -m pytest -vv -n auto $(TESTS_DIR)
@conda run -n $(PDD_CONDA_ENV) --no-capture-output PDD_RUN_REAL_LLM_TESTS=1 PDD_RUN_LLM_TESTS=1 PDD_PATH=$(abspath $(PDD_DIR)) PYTHONPATH=$(PDD_DIR):$$PYTHONPATH python -m pytest -vv -n auto $(TESTS_DIR)

# Run tests with coverage
coverage: ensure-dev-deps
@echo "Running tests with coverage"
@cd $(STAGING_DIR)
@conda run -n pdd --no-capture-output PDD_PATH=$(STAGING_DIR) PYTHONPATH=$(PDD_DIR):$$PYTHONPATH python -m pytest --cov=$(PDD_DIR) --cov-report=term-missing --cov-report=html $(TESTS_DIR)
@conda run -n $(PDD_CONDA_ENV) --no-capture-output PDD_PATH=$(STAGING_DIR) PYTHONPATH=$(PDD_DIR):$$PYTHONPATH python -m pytest --cov=$(PDD_DIR) --cov-report=term-missing --cov-report=html $(TESTS_DIR)

# Run pylint
lint: ensure-dev-deps
@echo "Running pylint"
@conda run -n pdd --no-capture-output pylint pdd tests
@conda run -n $(PDD_CONDA_ENV) --no-capture-output pylint pdd tests

# Fix crashes in code
crash:
Expand Down Expand Up @@ -265,7 +302,7 @@ ifdef MODULE
$(eval RESULTS_FILE := $(MODULE)_verify_results.log)

@echo "Verifying $(PY_FILE) functionality..."
-conda run -n pdd --no-capture-output pdd --strength .9 --verbose verify --max-attempts 3 --budget 5.0 --output-code $(PDD_DIR)/$(MODULE)_verified.py --output-program $(CONTEXT_DIR)/$(MODULE)_example_verified.py --output-results $(RESULTS_FILE) $(PY_PROMPT) $(PY_FILE) $(PROGRAM_FILE)
-conda run -n $(PDD_CONDA_ENV) --no-capture-output pdd --strength .9 --verbose verify --max-attempts 3 --budget 5.0 --output-code $(PDD_DIR)/$(MODULE)_verified.py --output-program $(CONTEXT_DIR)/$(MODULE)_example_verified.py --output-results $(RESULTS_FILE) $(PY_PROMPT) $(PY_FILE) $(PROGRAM_FILE)
else
@echo "Please specify a MODULE to verify"
@echo "Usage: make verify MODULE=<module_name>"
Expand All @@ -292,7 +329,7 @@ ifdef CHANGE_FILE
$(eval CHANGE_FILE_BASENAME := $(basename $(notdir $(CHANGE_FILE))))
$(eval DETECT_OUTPUT_FILE := $(CHANGE_FILE_BASENAME)_detect.csv)
@echo "Output will be saved to $(DETECT_OUTPUT_FILE)"
@conda run -n pdd --no-capture-output pdd detect --output $(DETECT_OUTPUT_FILE) $(ALL_PROMPT_FILES) $(CHANGE_FILE)
@conda run -n $(PDD_CONDA_ENV) --no-capture-output pdd detect --output $(DETECT_OUTPUT_FILE) $(ALL_PROMPT_FILES) $(CHANGE_FILE)
@echo "Detection complete. Results in $(DETECT_OUTPUT_FILE)"
else
@echo "Please specify a CHANGE_FILE to detect changes against."
Expand Down Expand Up @@ -321,7 +358,7 @@ ifdef PROMPT_FILE
$(eval ACTUAL_OUTPUT_FILE := $(if $(OUTPUT_FILE),$(OUTPUT_FILE),$(DEFAULT_OUTPUT_FILE)))

@echo "Output will be saved to $(ACTUAL_OUTPUT_FILE)"
@conda run -n pdd --no-capture-output pdd change --output $(ACTUAL_OUTPUT_FILE) $(CHANGE_PROMPT) $(CODE_FILE) $(PROMPT_FILE)
@conda run -n $(PDD_CONDA_ENV) --no-capture-output pdd change --output $(ACTUAL_OUTPUT_FILE) $(CHANGE_PROMPT) $(CODE_FILE) $(PROMPT_FILE)
@echo "Single prompt modification complete. Output at $(ACTUAL_OUTPUT_FILE)"
else
@echo "Error: PROMPT_FILE must be specified for single prompt change mode."
Expand Down Expand Up @@ -371,8 +408,8 @@ ifdef CSV_FILE
)

@if [ ! -z "$(OUTPUT_LOCATION)" ]; then echo "Output location specified by user: $(OUTPUT_LOCATION)"; fi
@echo "Executing from $(PROMPTS_DIR): conda run -n pdd --no-capture-output pdd --force change --budget 10.0 --csv $(CMD_OUTPUT_ARG) $(REL_CSV_FILE) $(REL_CODE_DIR)"
@cd $(PROMPTS_DIR) && conda run -n pdd --no-capture-output pdd --force change --budget 10.0 --csv $(CMD_OUTPUT_ARG) $(REL_CSV_FILE) $(REL_CODE_DIR)
@echo "Executing from $(PROMPTS_DIR): conda run -n $(PDD_CONDA_ENV) --no-capture-output pdd --force change --budget 10.0 --csv $(CMD_OUTPUT_ARG) $(REL_CSV_FILE) $(REL_CODE_DIR)"
@cd $(PROMPTS_DIR) && conda run -n $(PDD_CONDA_ENV) --no-capture-output pdd --force change --budget 10.0 --csv $(CMD_OUTPUT_ARG) $(REL_CSV_FILE) $(REL_CODE_DIR)
@echo "CSV batch prompt modification complete."
else # This means CSV_FILE was not defined, which contradicts the outer ifeq logic. This branch likely won't be hit if CSV_FILE is the primary condition.
@echo "Error: CSV_FILE must be specified for CSV batch change mode." # This case should ideally not be reached due to outer ifeq
Expand All @@ -390,7 +427,7 @@ ifdef MODULE
prompt="$(PROMPTS_DIR)/$${name}_python.prompt"; \
echo "Fixing $$name"; \
if [ -f "$(CONTEXT_DIR)/$${name}_example.py" ]; then \
conda run -n pdd --no-capture-output python -m pdd.cli --time 1 --strength .9 --temperature 0 --verbose --force fix --loop --auto-submit --max-attempts 5 --output-test output/ --output-code output/ --verification-program $(CONTEXT_DIR)/$${name}_example.py $$prompt $(PDD_DIR)/$${name}.py $(TESTS_DIR)/test_$${name}.py $${name}.log; \
conda run -n $(PDD_CONDA_ENV) --no-capture-output python -m pdd.cli --time 1 --strength .9 --temperature 0 --verbose --force fix --loop --auto-submit --max-attempts 5 --output-test output/ --output-code output/ --verification-program $(CONTEXT_DIR)/$${name}_example.py $$prompt $(PDD_DIR)/$${name}.py $(TESTS_DIR)/test_$${name}.py $${name}.log; \
else \
echo "Warning: No verification program found for $$name"; \
fi;
Expand All @@ -401,7 +438,7 @@ else
name="$${rel_path%_python.prompt}"; \
echo "Fixing $$name"; \
if [ -f "$(CONTEXT_DIR)/$${name}_example.py" ]; then \
conda run -n pdd --no-capture-output python -m pdd.cli --strength .9 --temperature 0 --verbose --force fix --loop --auto-submit --max-attempts 5 --output-test output/ --output-code output/ --verification-program $(CONTEXT_DIR)/$${name}_example.py $$prompt $(PDD_DIR)/$${name}.py $(TESTS_DIR)/test_$${name}.py $${name}.log; \
conda run -n $(PDD_CONDA_ENV) --no-capture-output python -m pdd.cli --strength .9 --temperature 0 --verbose --force fix --loop --auto-submit --max-attempts 5 --output-test output/ --output-code output/ --verification-program $(CONTEXT_DIR)/$${name}_example.py $$prompt $(PDD_DIR)/$${name}.py $(TESTS_DIR)/test_$${name}.py $${name}.log; \
else \
echo "Warning: No verification program found for $$name"; \
fi; \
Expand All @@ -425,10 +462,10 @@ ifdef MODULE
fi

@echo "Updating $(PY_PROMPT) based on changes in $(PY_FILE)"
conda run -n pdd --no-capture-output pdd --verbose update --git $(PY_PROMPT) $(PY_FILE)
conda run -n $(PDD_CONDA_ENV) --no-capture-output pdd --verbose update --git $(PY_PROMPT) $(PY_FILE)
else
@echo "Running repository-wide prompt update"
conda run -n pdd --no-capture-output pdd --verbose update --directory pdd --extensions py
conda run -n $(PDD_CONDA_ENV) --no-capture-output pdd --verbose update --directory pdd --extensions py
endif

# Generate requirements.txt
Expand Down Expand Up @@ -520,12 +557,12 @@ test-all-ci: ensure-dev-deps
@mkdir -p test_results
ifdef PR_NUMBER
ifdef PR_URL
@conda run -n pdd --no-capture-output python scripts/run_all_tests_with_results.py --pr-number $(PR_NUMBER) --pr-url $(PR_URL)
@conda run -n $(PDD_CONDA_ENV) --no-capture-output python scripts/run_all_tests_with_results.py --pr-number $(PR_NUMBER) --pr-url $(PR_URL)
else
@conda run -n pdd --no-capture-output python scripts/run_all_tests_with_results.py --pr-number $(PR_NUMBER)
@conda run -n $(PDD_CONDA_ENV) --no-capture-output python scripts/run_all_tests_with_results.py --pr-number $(PR_NUMBER)
endif
else
@conda run -n pdd --no-capture-output python scripts/run_all_tests_with_results.py
@conda run -n $(PDD_CONDA_ENV) --no-capture-output python scripts/run_all_tests_with_results.py
endif

# Run all tests with Infisical (for local development and CI)
Expand All @@ -540,12 +577,12 @@ test-all-with-infisical: ensure-dev-deps
@mkdir -p test_results
ifdef PR_NUMBER
ifdef PR_URL
@infisical run -- conda run -n pdd --no-capture-output python scripts/run_all_tests_with_results.py --pr-number $(PR_NUMBER) --pr-url $(PR_URL)
@infisical run -- conda run -n $(PDD_CONDA_ENV) --no-capture-output python scripts/run_all_tests_with_results.py --pr-number $(PR_NUMBER) --pr-url $(PR_URL)
else
@infisical run -- conda run -n pdd --no-capture-output python scripts/run_all_tests_with_results.py --pr-number $(PR_NUMBER)
@infisical run -- conda run -n $(PDD_CONDA_ENV) --no-capture-output python scripts/run_all_tests_with_results.py --pr-number $(PR_NUMBER)
endif
else
@infisical run -- conda run -n pdd --no-capture-output python scripts/run_all_tests_with_results.py
@infisical run -- conda run -n $(PDD_CONDA_ENV) --no-capture-output python scripts/run_all_tests_with_results.py
endif

# Test a PR from a public or private repo by triggering GitHub Actions
Expand Down Expand Up @@ -583,14 +620,14 @@ install:
build:
@echo "Building pdd"
@rm -rf dist
@conda run -n pdd --no-capture-output python -m build
@conda run -n $(PDD_CONDA_ENV) --no-capture-output python -m build
@rm dist/*.tar.gz #don't upload source distribution

# Post-process the wheel with preprocessed prompts
@echo "Post-processing wheel with preprocessed prompts..."
@conda run -n pdd --no-capture-output python scripts/preprocess_wheel.py 'dist/*.whl'
@conda run -n $(PDD_CONDA_ENV) --no-capture-output python scripts/preprocess_wheel.py 'dist/*.whl'

@conda run -n pdd --no-capture-output twine upload --repository pypi dist/*.whl
@conda run -n $(PDD_CONDA_ENV) --no-capture-output twine upload --repository pypi dist/*.whl

publish:
@echo "Building and uploading package"
Expand Down
Loading