Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ end

## Guides and Examples

- User guide track (numbered by complexity): [`docs/user/README.md`](./docs/user/README.md)
- Developer guide track (numbered by architecture depth): [`docs/developer/README.md`](./docs/developer/README.md)
- Quickstart: [`docs/quickstart.md`](./docs/quickstart.md)
- Integration boundaries: [`docs/integration-boundaries.md`](./docs/integration-boundaries.md)
- Troubleshooting: [`docs/troubleshooting.md`](./docs/troubleshooting.md)
Expand All @@ -33,6 +35,8 @@ end
- Minimal sample: `mix run examples/minimal_api_sample.exs`
- Session concurrency sample: `mix run examples/session_concurrency_sample.exs`
- Crash recovery sample: `mix run examples/crash_recovery_sample.exs`
- `jido_skill` format sample: `examples/skill_format_example.md`
- `jido_command` format sample: `examples/command_format_example.md`

## API Contract Snapshot

Expand Down
71 changes: 71 additions & 0 deletions docs/developer/01-architecture-overview.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# 01 - Architecture Overview

`jido_document` is a session-oriented document runtime focused on:

- in-memory document state management
- file-backed persistence with divergence detection
- render preview generation
- operational safety, history, and recovery

This library is intentionally non-UI. Transport and presentation are external
integration concerns.

## Core architectural boundaries

- Stable API boundary:
- `Jido.Document`
- `Jido.Document.Agent`
- `Jido.Document.Document`
- `Jido.Document.Frontmatter`
- `Jido.Document.Renderer`
- `Jido.Document.SchemaMigration`
- `Jido.Document.SessionRegistry`
- `Jido.Document.Signal`
- `Jido.Document.SignalBus`
- Internal implementation boundary:
- `Jido.Document.Actions.*`
- `Jido.Document.Persistence`
- `Jido.Document.History`
- `Jido.Document.Checkpoint`
- `Jido.Document.Revision`
- `Jido.Document.Authorization`
- `Jido.Document.Audit`
- `Jido.Document.Reliability`
- `Jido.Document.Safety`
- `Jido.Document.PathPolicy`

## High-level component map

```mermaid
flowchart LR
Caller["Caller"] --> Entry["Jido.Document / Jido.Document.Agent"]
Entry --> Registry["Jido.Document.SessionRegistry"]
Entry --> Agent["Session Process (Jido.Document.Agent)"]
Registry --> SessionSup["Jido.Document.SessionSupervisor"]
SessionSup --> Agent
Agent --> Actions["Jido.Document.Actions.*"]
Actions --> DocModel["Jido.Document.Document + Frontmatter"]
Actions --> Render["Jido.Document.Renderer"]
Actions --> Persist["Jido.Document.Persistence"]
Agent --> History["History + Revision + Audit"]
Agent --> Recovery["Checkpoint + Recovery"]
Agent --> Bus["Jido.Document.SignalBus"]
```

## Canonical flow

1. Caller resolves/starts a session.
2. Caller sends action commands through `Agent.command/4`.
3. Agent executes one action module with a normalized context.
4. Agent updates session state, history, revision, and audit metadata.
5. Agent emits session-scoped signals for subscribers.
6. Save operations persist using atomic-write and divergence checks.

## Architectural invariants

- Document state is session-owned and process-local (`Agent`).
- Frontmatter and markdown body are modeled explicitly and normalized.
- Save is guarded by baseline snapshots to avoid silent overwrite.
- Signal payloads are versioned and bounded.
- Recovery is explicit (checkpoint discovery, recover, discard).

51 changes: 51 additions & 0 deletions docs/developer/02-supervision-and-runtime-topology.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# 02 - Supervision and Runtime Topology

`Jido.Document.Application` defines a small supervision tree with clear
separation between shared runtime services and dynamic session processes.

## Topology

```mermaid
flowchart TD
App["Jido.Document.Application"] --> RootSup["Jido.Document.Supervisor (one_for_one)"]
RootSup --> Metrics["Jido.Document.Render.Metrics"]
RootSup --> Queue["Jido.Document.Render.JobQueue"]
RootSup --> Bus["Jido.Document.SignalBus"]
RootSup --> SessionSup["Jido.Document.SessionSupervisor"]
RootSup --> Registry["Jido.Document.SessionRegistry"]
Registry --> SessionSup
SessionSup --> AgentA["Jido.Document.Agent (session-a)"]
SessionSup --> AgentN["Jido.Document.Agent (session-n)"]
```

## Responsibilities by process

- `Render.Metrics`:
- tracks render strategy and quality metrics.
- `Render.JobQueue`:
- handles async render jobs.
- `SignalBus`:
- session-scoped pub/sub fanout with queue-aware dropping.
- `SessionSupervisor`:
- dynamic supervisor for session agents.
- `SessionRegistry`:
- session discovery, path-indexing, lock ownership, idle cleanup.

## Startup behavior

Before supervisor start, `Application.start/2` runs render plugin startup
checks (`PluginManager.startup_check/2`) and logs compatibility diagnostics.

## Session lifecycle

1. `SessionRegistry.ensure_session/3` or `ensure_session_by_path/3` is called.
2. Registry starts a new `Agent` under `SessionSupervisor` if missing.
3. Registry monitors agent PID and removes stale entries on `:DOWN`.
4. Idle reclaim can terminate old sessions (`reclaim_idle/2`).

## Failure model

- `one_for_one` supervision isolates process crashes.
- Session failures do not inherently crash sibling sessions.
- Registry and signal bus are shared services and should remain lightweight.

71 changes: 71 additions & 0 deletions docs/developer/03-agent-command-pipeline.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# 03 - Agent Command Pipeline

`Jido.Document.Agent` is the stateful orchestrator. It owns:

- active `Document`
- history model
- checkpoint and recovery state
- revision sequence
- preview/fallback state
- audit trail window

## Command execution flow

```mermaid
sequenceDiagram
participant Caller
participant Agent as Jido.Document.Agent
participant Action as Jido.Document.Action
participant Impl as Jido.Document.Actions.*

Caller->>Agent: command(session, action, params, opts)
Agent->>Agent: normalize params + inject defaults
Agent->>Agent: guard action / lock if needed
Agent->>Action: execute(action_module, params, context)
Action->>Action: authorize + telemetry wrapper
Action->>Impl: run(params, context)
Impl-->>Action: {:ok, value} | {:error, reason}
Action-->>Agent: Result.t()
Agent->>Agent: apply_success/apply_failure
Agent->>Agent: history/revision/audit/signal updates
Agent-->>Caller: Result.t()
```

## Key contracts

- Action behavior (`Jido.Document.Action`):
- `name/0`
- `idempotency/0`
- `run/2`
- Result envelope (`Jido.Document.Action.Result`):
- `status`
- `value` or `error`
- metadata (`action`, `idempotency`, `correlation_id`, `duration_us`)
- Context envelope (`Jido.Document.Action.Context`):
- session identity, actor, document snapshot, options

## Cross-cutting behaviors in pipeline

- Authorization:
- applied in `Action.execute/3`
- role matrix + optional custom hook policy
- Retry:
- controlled by `Jido.Document.Reliability.with_retry/2`
- retries only retryable errors
- Optimistic rollback:
- failed mutation actions can revert to previous state
- Render fallback:
- on render failure, agent emits a fallback preview payload

## Sync vs async

- Sync mode (`mode: :sync`):
- `GenServer.call`, immediate `Result.t()`.
- Async mode (`mode: :async`):
- `GenServer.cast`, returns `:ok`.

## History commands

`:undo` and `:redo` are handled via history transitions and require a loaded
document. They update revision/audit metadata similarly to regular actions.

57 changes: 57 additions & 0 deletions docs/developer/04-document-model-and-frontmatter-engine.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# 04 - Document Model and Frontmatter Engine

`Jido.Document.Document` is the canonical in-memory representation:

- `path`
- `frontmatter` map
- `body` string
- `raw` source content
- schema reference
- dirty/revision tracking

## Parse and serialize pipeline

```mermaid
flowchart LR
Raw["Raw file content"] --> Split["Frontmatter.split/1"]
Split --> ParseFM["Frontmatter.parse/2"]
ParseFM --> NewDoc["Document.new/1"]
NewDoc --> Canon["Document.canonicalize/2"]
Canon --> Serialize["Document.serialize/2"]
Serialize --> Output["Persistable content"]
```

## Frontmatter semantics

- Supported delimiters:
- YAML: `---`
- TOML: `+++`
- Frontmatter is optional:
- no delimiter => `frontmatter: %{}`
- Serialization defaults:
- empty frontmatter omitted unless `emit_empty_frontmatter: true`

## Mutation semantics

- `update_frontmatter/3`:
- `:merge` (default) or `:replace`
- `update_body/3`:
- body normalization policies (line endings, trailing whitespace)
- `apply_body_patch/3`:
- full replace string
- function patch
- `%{search, replace, global}`

All effective mutations increment revision and set dirty state.

## Schema validation

`Jido.Document.Schema` validates frontmatter against `Field` declarations:

- required fields
- defaulting
- type coercion (`string`, `integer`, `float`, `boolean`, arrays, enums)
- unknown key policy (`:warn`, `:ignore`, `:reject`)

This layer validates metadata contracts without coupling to UI or transport.

58 changes: 58 additions & 0 deletions docs/developer/05-rendering-and-preview-subsystem.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# 05 - Rendering and Preview Subsystem

Rendering is split between orchestration and transformation components:

- orchestration:
- `Jido.Document.Actions.Render`
- `Jido.Document.Render.ChangeTracker`
- `Jido.Document.Render.JobQueue`
- `Jido.Document.Render.Metrics`
- transformation:
- `Jido.Document.Renderer`
- `Jido.Document.Render.PluginManager`
- `Jido.Document.Render.ThemeRegistry`

## Render flow

```mermaid
flowchart TD
Cmd["Agent command(:render)"] --> Decision["ChangeTracker.plan/3"]
Decision --> Safety["Safety.scan + Safety.redact"]
Safety --> Mode{"async?"}
Mode -- "yes" --> Queue["JobQueue.enqueue/6"]
Mode -- "no" --> Render["Renderer.render/2"]
Render --> Plugins["PluginManager.apply_plugins/3"]
Render --> Adapter["Adapter selection (:mdex/:simple/:auto)"]
Adapter --> Preview["Preview payload"]
Preview --> Metrics["Render.Metrics"]
```

## Renderer design points

- Frontmatter stripping is default for preview.
- Adapter auto-fallback:
- prefers `:mdex` when available
- falls back to `:simple` with diagnostics
- Fallback preview exists for degraded render conditions.

## Plugin model

- Contract:
- `Jido.Document.Render.Plugin.transform/2`
- `Jido.Document.Render.Plugin.compatible?/1`
- Manager behavior:
- ordered execution by priority
- startup compatibility checks
- failure isolation with diagnostics, not hard crash

## Output contract

Render output includes:

- `html`
- `toc`
- `diagnostics`
- `cache_key`
- `adapter`
- `metadata` (extensions, syntax highlight settings, plugin list, etc.)

65 changes: 65 additions & 0 deletions docs/developer/06-persistence-divergence-and-recovery.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# 06 - Persistence, Divergence, and Recovery

This subsystem combines path policy, atomic writes, baseline snapshots, and
checkpoint-based crash recovery.

## Save pipeline

```mermaid
sequenceDiagram
participant Caller
participant Agent
participant Save as Actions.Save
participant Path as PathPolicy
participant Persist as Persistence

Caller->>Agent: command(:save, params)
Agent->>Save: run(params, context)
Save->>Path: resolve_path(path, workspace_root)
Save->>Persist: detect_divergence(path, baseline)
Save->>Persist: atomic_write(path, serialized)
Save->>Persist: snapshot(path)
Save-->>Agent: {:ok, saved_payload}
Agent-->>Caller: Result.ok
```

## Divergence protection

- Save compares current file state against baseline snapshot.
- On mismatch, conflict error is returned unless explicit strategy allows:
- `:overwrite`
- `:merge_hook`

## Atomic write strategy

`Persistence.atomic_write/3` performs:

1. write fsynced temp file
2. optional metadata preservation
3. atomic rename to target
4. best-effort directory sync

This minimizes partial-write corruption risk.

## Checkpoint and recovery model

`Jido.Document.Checkpoint` stores session payload snapshots to disk:

- schema version
- session id
- document struct
- disk snapshot baseline
- capture timestamp

Recovery APIs on `Agent`:

- `recovery_status/1`
- `recover/2`
- `discard_recovery/1`
- `list_recovery_candidates/1`

## Path security boundary

`Jido.Document.PathPolicy.resolve_path/2` enforces workspace-root containment to
prevent path escapes from session load/save operations.

Loading