-
Notifications
You must be signed in to change notification settings - Fork 0
Description
Summary
Add an -Scenario All option to Run-IntegrationTests.ps1 that runs every implemented integration test scenario in sequence, performing lightweight resets between scenarios. This enables a full regression test in a single invocation without redundant build or infrastructure startup steps.
Problem
Each integration test scenario must currently be run individually. Running a full regression suite requires manually invoking the script multiple times, each time paying the cost of Docker image builds, Samba AD startup, and service health checks. For 6 implemented scenarios, this means ~15-20 minutes of redundant infrastructure overhead.
Proposed Approach
- Build once — Docker images built at the start and reused
- Start all external systems upfront — All Samba AD instances started once
- Lightweight reset between scenarios — Reset JIM database and clean Samba AD OUs without rebuilding containers
- Aggregated results — Combined pass/fail summary with per-scenario breakdown
- Continue on failure — A failing scenario doesn't abort the remaining tests
Plan Document
See docs/plans/INTEGRATION_TEST_FULL_REGRESSION.md for the full design including infrastructure grouping, reset strategy, results format, and implementation notes.
Scope
- Modify
test/integration/Run-IntegrationTests.ps1to support-Scenario All - Add "All Scenarios" to the interactive scenario menu
- Implement lightweight reset function between scenarios
- Aggregate results into a single JSON output
Acceptance Criteria
-
./Run-IntegrationTests.ps1 -Scenario All -Template Nanoruns all implemented scenarios sequentially - Interactive menu includes "All Scenarios (full regression)" option
- JIM database is fully reset between scenarios (clean state)
- Samba AD test data is cleaned between scenarios
- Docker images are only built once at the start
- External system containers are only started once
- A failing scenario does not prevent subsequent scenarios from running
- Aggregated results JSON saved to
test/integration/results/ - Exit code is non-zero if any scenario failed