Spec Graduation: Absorb completed tier 1 specs into package executable specs

[darko-mijic]

Problem

Completed tier 1 roadmap specs are piling up in delivery-process/specs/ (23 of 38 specs are completed). These served their planning purpose but now require ongoing maintenance without providing proportional value. Meanwhile, package-level executable specs already contain the living behavior proof.

Current state:

• 23 completed tier 1 specs sit alongside 15 active/roadmap specs
• Executable specs in {package}/tests/features/behavior/ already carry @libar-docs-implements backlinks
• Bidirectional traceability exists (@libar-docs-executable-specs ↔ @libar-docs-implements) but tier 1 specs are never "graduated"
• Decision specs (PDRs) are correctly permanent — they document why, not what was planned

The stub analogy: Design stubs follow a clear lifecycle: delivery-process/stubs/ → implementation in src/ → stubs removed. Tier 1 specs should follow the same pattern: delivery-process/specs/ → behavior absorbed into executable specs → tier 1 spec graduated.

Proposal: Spec Graduation Lifecycle

Core Principle

Tier 1 specs are planning artifacts. Like stubs, they get absorbed into the real thing. Once a pattern's executable specs fully cover its behavior, the tier 1 spec has served its purpose.

Tier 1 Spec Lifecycle:
  roadmap → active → completed → graduated
                                     │
                              Behavior lives in executable specs
                              Planning metadata rolls up to epics
                              Tier 1 spec archived or removed

What Migrates Where

Tier 1 Content	Destination	Notes
Behavior scenarios	Package executable specs	Already done via two-tier architecture
Invariant/Rationale prose	Executable spec Feature/Rule descriptions	Enrich package specs with domain documentation
Phase/dependency metadata	Epic specs	One epic per phase cluster (see below)
Deliverable tracking	Git history	Purpose served — no migration needed

What Stays Permanently

Artifact	Why
Decision specs (PDRs)	Architectural events — permanent record of why
Epic specs	Roadmap coordination — group-level planning
Active/roadmap tier 1 specs	Still guiding implementation work

Epic Specs as Graduation Target

Introduce epic-level specs as the surviving roadmap artifacts. One precedent exists: epic-process-enhancements.feature uses @libar-docs-level:epic.

Proposed epic groupings (matching natural phase clusters):

Epic	Phase Range	Graduated Specs	Package Home
Foundation	02-03	EventStoreFoundation, CommandBusFoundation	platform-core, platform-bus
Decider & Type Safety	14-16	DeciderPattern, ProjectionCategories, DynamicConsistencyBoundaries	platform-decider, platform-fsm, platform-core
Durability & Production	18a-18c	DurableFunctionAdapters, WorkpoolPartitioning, EventStoreDurability, EventReplay	platform-core
Service Independence	19-20	BddTestingInfra, EcstFatEvents, ReservationPattern	platform-core
Integration	17, 21	ReactiveProjections, IntegrationPatterns	platform-core
Agent Platform	22-22c	AgentAsBC, AgentComponentIsolation, AgentLLM, AgentCommand	platform-core
Example App	23	ExampleAppModernization	order-management

Epic specs would carry:

• @libar-docs-level:epic
• Phase range and dependency graph (absorbed from child specs)
• List of graduated patterns with links to their executable spec locations
• High-level narrative (Problem/Solution from graduated specs, condensed)

Graduation Process (Per Completed Pattern)

Mirrors the stub resolution workflow:

1. Verify executable spec coverage — all tier 1 scenarios have package-level equivalents
2. Migrate domain documentation — move Invariant/Rationale prose from tier 1 Rules into executable spec Feature/Rule descriptions (enriching the living tests with context)
3. Update epic spec — add pattern to epic's graduated patterns list
4. Archive or remove tier 1 spec — either move to specs/archive/ or delete (git history preserves it)
5. Update generators — codecs should prefer executable specs for content when tier 1 spec is graduated/absent

Generator Changes

The MasterDataset currently merges all sources without lifecycle awareness. Changes needed:

• Content precedence rule: When a pattern has @libar-docs-implements backlinks AND the tier 1 spec is completed/graduated, codecs prefer executable spec content for behavior documentation
• Epic-aware roadmap generation: ROADMAP.md and timeline generators use epic specs for graduated patterns instead of individual tier 1 specs
• Traceability update: Traceability codec should show graduation status and link to executable spec location

Process Guard Updates

• New validation rule: graduation-ready (info/warning) — completed spec with all scenarios covered by executable specs is a candidate for graduation
• Optional: graduated FSM status as terminal state after completed (or use archive/removal as the signal)

Scope & Phasing

Phase 1: Documentation & Decision

• Create PDR for spec graduation (PDR-014 or similar)
• Define graduation criteria formally
• Decide: graduated FSM status vs. archive/removal vs. generator-level preference

Phase 2: Epic Spec Structure

• Define epic spec format (extend @libar-docs-level:epic pattern)
• Create epic specs for the 7 natural phase clusters
• Add epic-level tags: @libar-docs-graduated-patterns, @libar-docs-phase-range

Phase 3: Generator Adaptation

• Implement content precedence in codecs (prefer executable specs for graduated patterns)
• Update roadmap/timeline generators to use epic specs
• Update traceability codec for graduation awareness

Phase 4: Graduation Execution

• Graduate foundation specs (phases 02-03) as pilot
• Migrate domain documentation to executable spec descriptions
• Validate generated docs quality after graduation
• Graduate remaining completed specs in phase-cluster batches

Phase 5: Process API & Tooling

• Add pnpm process:query -- list --graduation-ready query
• Add graduation-ready detection to Process Guard (info level)
• Update planning plugin skills for graduation awareness

Prior Art in This Codebase

• Stub lifecycle (PDR-009): delivery-process/stubs/ → src/ → cleanup
• Epic spec pattern: epic-process-enhancements.feature with @libar-docs-level:epic
• Two-tier spec architecture (PDR-007): already separates planning from implementation proof
• @libar-docs-executable-specs: bidirectional traceability already in place

Questions to Resolve

1. Archive vs. delete? Git preserves history either way. Archive directory adds noise; deletion is cleaner but less discoverable.
2. graduated FSM status vs. implicit? An explicit status is queryable but adds FSM complexity. Removal/archive is simpler but less trackable.
3. Enriching executable specs with documentation — should executable spec Feature descriptions carry the full Invariant/Rationale prose, or just link back to the PDR decisions?
4. Epic spec granularity — one per phase cluster (7 epics) or one per product area (3 epics: Platform, DeliveryProcess, ExampleApp)?

[darko-mijic]

Current State (Quantified)

Metric	Count
Total patterns (all sources)	164
Completed tier 1 specs (gherkin in specs/platform/)	~23
Active/roadmap tier 1 specs	~15
Executable specs (package-level .feature files)	82
Tier 1 specs with @libar-docs-executable-specs backlinks	25
Epic specs (existing precedent)	1 (epic-process-enhancements.feature)
Decision specs (PDRs)	13

★ Insight ─────────────────────────────────────
The 108 "completed" patterns in the process API include both gherkin tier 1 specs and TypeScript-sourced patterns (source annotations in .ts files). The actual tier 1 spec files in specs/platform/ number ~23 completed. This is important — the "pile-up" is in the directory listing, not in the total pattern count where TS patterns dominate.
─────────────────────────────────────────────────

---

The Real Maintenance Burden — Is It Actually Painful?

Before jumping to solutions, let me push back slightly on the problem statement. Completed tier 1 specs are hard-locked (completed protection). You literally cannot modify them without @libar-docs-unlock-reason. So they don't drift from implementation. The actual pain points are:

1. Signal-to-noise in specs/ — 23 completed files alongside 15 active/roadmap ones. When you open the directory to find what you're working on, 60% is done
2. Generated docs bloat — ROADMAP.md lists 23 completed specs individually rather than summarizing them at epic level
3. Conceptual inconsistency — stubs have a lifecycle (created → absorbed → removed), but specs don't
4. Traceability duplication — both the tier 1 spec and the executable spec describe the same behavior; generators merge both without preference

Pain points 1 and 3 are aesthetic/organizational. Pain points 2 and 4 are the real value — they affect the quality of generated artifacts.

---

Downstream Impact Assessment

Here's the critical part — what the proposal would touch in deps-packages/delivery-process/src/:

If Adding graduated FSM State

Module	File	Change	Effort
Status values	taxonomy/status-values.ts	Add 'graduated' to enum	Trivial
Normalized status	taxonomy/normalized-status.ts	Map graduated → display bucket	Trivial
FSM states	validation/fsm/states.ts	Add graduated with protection level	Small
FSM transitions	validation/fsm/transitions.ts	Add completed → [graduated]	Small
Process guard types	lint/process-guard/types.ts	New rule type	Small
Process guard decider	lint/process-guard/decider.ts	New validation rule	Medium
Transform dataset	generators/pipeline/transform-dataset.ts	New byLevel view, graduation awareness	Medium
Roadmap codec	renderable/codecs/timeline.ts	Epic aggregation for graduated specs	Large
Traceability codec	renderable/codecs/reporting.ts	Content precedence, graduation status	Medium
Patterns codec	renderable/codecs/patterns.ts	Hierarchy rendering	Medium
Session codec	renderable/codecs/session.ts	Filter graduated from remaining work	Small
Gherkin extractor	extractors/gherkin-extractor.ts	Graduation metadata extraction	Small
Orchestrator	generators/orchestrator.ts	Content precedence in merge	Medium
MasterDataset schema	validation-schemas/master-dataset.ts	New views/fields	Small

Total: ~14 files, several medium-complexity changes. This is a significant refactor of the delivery-process package.

If Using Archive/Deletion (No FSM Change)

Module	File	Change	Effort
Epic specs (new files)	delivery-process/specs/	Create 7 epic feature files	Medium
Roadmap codec	renderable/codecs/timeline.ts	Render epic-level groupings	Medium
Traceability codec	renderable/codecs/reporting.ts	Handle missing tier 1 specs gracefully	Small
Transform dataset	generators/pipeline/transform-dataset.ts	byLevel grouping for epic awareness	Small
Orchestrator config	Generator config	Exclude specs/archive/ if archiving	Trivial

Total: ~5 files, mostly small changes. Much lighter.

---

Three Realistic Approaches

Approach A: Full Graduation Lifecycle (Your Proposal)

New graduated FSM state, epic specs, generator content precedence, process guard rules, 5-phase rollout.

Pros: Complete, queryable, conceptually clean
Cons: Touches 14+ files, adds FSM complexity that propagates everywhere, every FSM consumer needs updating
Downstream: Heavy

Approach B: Delete + Epic Specs (Minimalist)

Delete completed tier 1 specs once executable specs cover them. Create epic specs to preserve phase/dependency metadata. No FSM changes.

completed tier 1 spec
  → verify executable spec coverage exists
  → migrate unique prose (invariants/rationale) to executable spec Feature description
  → create/update epic spec with pattern entry
  → delete tier 1 spec (git preserves history)

Pros: Simplest, follows stub precedent exactly, minimal downstream changes (~5 files)
Cons: No graduated status to query; deletion is less discoverable than archival
Downstream: Light — mostly new epic specs + minor codec adjustments

Approach C: Tag-Based Soft Graduation (Middle Ground)

Add @libar-docs-graduated as a metadata tag (not an FSM state) on completed specs. Generators recognize the tag and summarize rather than detail. No FSM changes, no file movement.

# On the completed tier 1 spec:
@libar-docs-graduated
@libar-docs-graduated-to:platform-core/tests/features/behavior/dcb

Pros: Queryable without FSM complexity, reversible, no file movement
Cons: Specs still sit in directory (noise), adds a new tag concept without removing files
Downstream: Medium — transform-dataset and 2-3 codecs need tag awareness

---

My Recommendation: Approach B (Delete + Epic Specs)

Here's my reasoning:

1. The stub precedent is the strongest argument. Stubs don't get a graduated FSM state — they get deleted when absorbed. Tier 1 specs should follow the same pattern. Adding a 5th FSM state for something that's essentially "done and absorbed" is over-engineering.

2. FSM state additions are expensive. Every consumer of PROCESS_STATUS_VALUES, every normalizeStatus() call, every codec that switches on status — they all need updating. A new state is the most expensive atom of change in the delivery-process package.

3. Git is the archive. The issue correctly notes this. An archive/ directory is strictly worse than deletion because it adds noise without adding discoverability (who browses archive/ vs. git log?).

4. Epic specs are the right surviving artifact. The epic-process-enhancements.feature precedent already works. The hierarchy system (level, parent, children) is already extracted — codecs just need to render it.

5. The traceability codec already has the data it needs. It checks behaviorFile and behaviorFileVerified. Once tier 1 specs are deleted, the executable specs become the sole source — no precedence logic needed.

---

Answering Your Four Questions

1. Archive vs. delete?

Delete. Archive directories are graveyard patterns — they accumulate without anyone visiting them. Git history is a better archive than a subdirectory. The stub precedent confirms: stubs are deleted, not archived.

2. graduated FSM status vs. implicit?

Implicit (no new state). The cost/benefit doesn't justify it. You'd touch 14+ files for a status that only means "this file is about to be deleted." If you need queryability, a process API command like process:query -- list --graduation-ready can detect the condition dynamically (completed + has executable specs) without an FSM state.

3. Enriching executable specs with documentation?

Minimal enrichment. Move the 1-2 sentence invariant/rationale into the executable spec's Feature: or Rule: description. Don't duplicate full prose — link to the PDR decision spec for the why. The executable spec should explain what behavior is proven, the PDR explains why that design was chosen.

Example:

# In executable spec Feature description:
# Invariant: DCB scope keys are deterministic — same inputs always produce 
# same key. See PDR-010 for cross-component design rationale.

4. Epic granularity?

Phase clusters (7 epics) as proposed. The natural groupings you identified (Foundation, Decider & Type Safety, Durability, etc.) match how the work was actually done. Product-area grouping (3 epics) would be too coarse — Platform alone would have 18+ graduated patterns.

---

A Lighter Phasing

If we go with Approach B, the phasing simplifies:

Phase	Work	Downstream Changes
1. Decision	Create PDR-014 for spec graduation. Define graduation criteria (completed + executable specs cover all scenarios)	None
2. Epic Specs	Create 7 epic feature files. Add @libar-docs-graduated-patterns tag. Populate from completed specs	None — existing extractors handle new tag
3. Codec Awareness	Teach roadmap codec to render epic-level groupings. Teach traceability codec to handle absent tier 1 specs	~3-4 files in delivery-process package
4. Pilot Graduation	Graduate Foundation specs (phases 02-03): verify executable coverage, migrate prose, update epic, delete tier 1	Validate generated docs
5. Batch Graduation	Graduate remaining ~20 completed specs in phase-cluster batches	Mechanical

Phase 3 is the only one that touches the delivery-process package code, and it's ~4 files with small-medium changes instead of 14.

---

What's your take? The key decision point is whether you value the queryability of an explicit graduated FSM state enough to accept the 14-file downstream cost, or whether the deletion-based approach (mirroring stubs) gives you enough.

[darko-mijic]

This is very informative to see side by side. Let me highlight the content gap between tiers, because it directly shapes what "enrichment" means in practice.

Content Gap Analysis: DCB as a Worked Example

Here's what lives only in the tier 1 spec and has no equivalent in the executable specs:

Tier 1 Content	In Executable Spec?	Documentation Value
Problem/Solution narrative (lines 12-23)	No — executable specs have "As a...I want"	High — explains why DCB exists
"Why It Matters" benefits table (lines 25-32)	No	High — quick reference for consumers
Deliverables table with locations (lines 37-47)	No	Low — historical
Conceptual overview table (Rule 1, lines 52-62)	No	High — defines the 4 core concepts
Current-state vs target-state code examples (Rule 2, lines 88-110)	No	High — shows the before/after
Constraints table (Rule 4, lines 175-179)	No	Medium — design rationale
Rule block prose (all 5 rules)	No — executable specs have section comments, not Rule: blocks	High — domain documentation

The executable specs (dcb/execute.feature, dcb/scope-key.feature) are pure test specs: they prove behavior with concrete scenarios but carry zero domain narrative. Their Feature descriptions are user-story format ("As a platform developer, I want...").

★ Insight ─────────────────────────────────────
This gap is significant. The tier 1 spec is doing two jobs: (1) planning artifact (deliverables, status tracking) and (2) domain documentation (problem/solution, rationale, concept tables). Job 1 is done once completed. Job 2 is evergreen — it's what the doc generator extracts.
─────────────────────────────────────────────────

---

What "Enrichment" Actually Means Per Content Type

Here's the practical mapping for DCB:

Moves to executable spec Feature: description

# dcb/execute.feature — BEFORE (current)
Feature: DCB Execution with OCC
  As a platform developer
  I want executeWithDCB to enforce scope-level OCC
  So that cross-entity invariants are protected

# dcb/execute.feature — AFTER (enriched)
Feature: DCB Execution with OCC
  Dynamic Consistency Boundaries enable cross-entity invariants by defining
  a scope that groups related entities and applying OCC on the scope — not
  individual entities.

  | Concept | Description |
  | Scope Key | Unique identifier for consistency boundary |
  | Virtual Stream | Logical composition of events within scope |
  | Scope OCC | expectedVersion checked on commit |

  See: PDR-010 for cross-component design rationale.

Moves to executable spec Rule: blocks (replacing section comments)

# BEFORE: section comment
# ============================================================================
# OCC Pre-Check (Version Validation)
# ============================================================================

# AFTER: proper Rule block (doc generator can extract)
Rule: OCC pre-check validates expectedVersion before decider execution
  Scope version tracks last modification across all entities in the boundary.
  Stale expectedVersion indicates concurrent modification — reject early.

Moves to epic spec

• Phase 16 narrative (condensed Problem/Solution)
• Dependency: depends-on: DeciderPattern
• Graduated pattern entry with link to executable spec location

Stays in graduated tier 1 spec (thin metadata shell)

• All @libar-docs-* tags (pattern, status, phase, effort, release, product-area)
• Minimal Feature line referencing graduated location
• The @libar-docs-executable-specs backlink

---

The Tension: Executable Specs as Documentation Hosts

Here's the honest concern with enrichment. Looking at dcb/scope-key.feature — it's 182 lines of clean, focused test scenarios. Adding domain narrative, concept tables, and code examples would make it 250+ lines and blur its purpose. Developers reading the test file get noise; the doc generator gets content.

Two options for where the domain prose lives:

Option 1: Executable spec carries both (enrichment as proposed)

executable spec = test proof + domain documentation
tier 1 graduated spec = thin metadata shell (in graduated/ folder)
epic spec = phase coordination

• Pro: Single source of truth per pattern for both behavior and documentation
• Con: Test files become heavier; mix of concerns (testing vs documentation)
• Con: Every graduated spec needs executable spec restructuring (add Rule: blocks, rewrite Feature descriptions)

Option 2: Convention-tagged tier 1 spec becomes documentation source

executable spec = test proof only (stays clean)  
tier 1 graduated spec = domain documentation + metadata (in graduated/ folder)
epic spec = phase coordination

Add @libar-docs-convention:dcb to the graduated tier 1 spec. The codec-driven reference generation already consumes conventions. Strip the tier 1 spec of deliverables and duplicate scenarios but keep the domain narrative, Rule blocks, and concept tables.

• Pro: Executable specs stay clean and focused on tests
• Pro: Works immediately with existing convention extraction — no restructuring of 82 executable spec files
• Pro: The content is already there — just tag it and move the file
• Con: Two files per pattern (graduated spec + executable spec) instead of one

---

My Recommendation: Option 2 (Convention-Tagged Graduated Specs)

Here's why I'm gravitating toward this:

1. The codec-driven reference generator already speaks this language. Convention extraction from tagged feature files is a built-in capability. You'd add @libar-docs-convention:dcb (or domain-specific tags) to the graduated tier 1 spec and the pipeline picks it up.

2. Minimal downstream changes. The delivery-process package needs:

   • Convention tags added to graduated specs (~23 files, mechanical)
   • Generator config to scan specs/platform/graduated/ (already covered by specs/**/*.feature glob)
   • Optional: roadmap codec groups graduated specs differently from active ones (small change)
   • That's it. No restructuring of 82 executable spec files.

3. Executable specs stay as tests. Developers reading test files see test scenarios, not documentation prose. The separation of concerns is preserved.

4. The directory move solves the navigation problem. Active/roadmap specs at top level, graduated in subfolder:

specs/platform/
  ├── agent-bc-component-isolation.feature    (active)
  ├── agent-llm-integration.feature           (active)
  ├── confirmed-order-cancellation.feature    (active)
  ├── health-observability.feature            (roadmap)
  ├── integration-patterns-21a.feature        (roadmap)
  ├── ...
  └── graduated/
      ├── decider-pattern.feature             (convention-tagged doc source)
      ├── dynamic-consistency-boundaries.feature
      ├── event-store-foundation.feature
      └── ... (~23 files)

5. The "trim" is targeted. From graduated tier 1 specs, remove:

   • Deliverables table (historical, served its purpose)
   • Duplicate scenarios that exist in executable specs
   • The Background: section (deliverables tracking)

   Keep:

   • All metadata tags + add convention tags
   • Problem/Solution narrative
   • Rule blocks with domain documentation
   • Concept tables and code examples

What About the @libar-docs-convention Tag Semantics?

Currently conventions are on decision records (PDRs). Graduated specs would be a second source. The convention extractor (extractConventions()) filters by @libar-docs-convention tags — it doesn't care whether the source is a PDR or a graduated spec. So this works mechanically.

But semantically: PDRs document why (decisions), graduated specs document what (domain concepts). You might want a different tag to distinguish them:

Tag	Source	Content Type
@libar-docs-convention:dcb	PDR/Decision	Design rationale, architectural constraints
@libar-docs-domain-doc:dcb	Graduated spec	Problem/solution, concept overview, patterns

Or just use conventions for both and let the reference codec compose them. The composition order (AD-5: conventions → shapes → behaviors) would naturally put domain documentation before API shapes before test scenarios.

---

Downstream Impact Summary (Option 2)

Area	Changes	Effort
Graduated specs (~23 files)	Add convention tags, remove deliverables/duplicate scenarios, move to graduated/	Medium (mechanical)
Epic specs (7 new files)	Create with phase metadata, graduated pattern lists	Medium
Generator config	Already scans specs/**/*.feature — no change needed	None
Convention extraction	Already works with any @libar-docs-convention tagged file	None
Roadmap codec	Optional: summarize graduated specs under epic headings	Small
Traceability codec	Handle graduated directory path gracefully	Small
Process Guard	No FSM changes. Optional: graduation-ready info-level rule	Small
Executable specs (82 files)	No changes	None

Compare this to the enrichment-of-executable-specs approach which would touch all 82 executable spec files to add Rule: blocks and domain prose.

---

What's your reaction? The core trade-off is:

• Option 1 (enrich executable specs): One file per pattern but heavier test files + 82 files restructured
• Option 2 (convention-tagged graduated specs): Two files per pattern but clean separation + works with existing infrastructure