diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md new file mode 100644 index 0000000..4c9f13a --- /dev/null +++ b/.github/copilot-instructions.md @@ -0,0 +1,146 @@ +# SoA Workbench - Copilot Instructions + +## Project Overview +Clinical trial Schedule of Activities (SoA) workbench: FastAPI web app + CLI tools for normalizing, expanding, and validating study visit matrices against USDM (Unified Study Definitions Model). + +**Core Architecture:** +- **Web Layer** (`src/soa_builder/web/`): FastAPI app with router-based endpoints, HTMX UI, SQLite persistence +- **Core Logic** (`src/soa_builder/`): Normalization, schedule expansion, validation modules +- **USDM Generators** (`src/usdm/`): Transform database state → USDM JSON artifacts +- **Data Model**: SQLite schema with audit trails, versioning (freezes), and biomedical concept linking + +**USDM Model Entities & Relationships** (critical for understanding the domain): +- **StudyDesign**: Top-level container with arrays of: encounters, activities, arms, epochs, elements, studyCells, scheduleTimelines +- **StudyElement** (`element` table): Structural design components (e.g., treatment periods, cohorts, crossover phases) + - UIDs: `StudyElement_N`; generated by `generate_elements.py` + - Purpose: Define "what study structure exists" (design-time components) + - Grouped via: **StudyCells** (arm + epoch + elementIds array) + - Attributes: transitionStartRule, transitionEndRule, studyInterventionIds +- **ScheduledActivityInstance** (`instances` table): Temporal visit/timepoint occurrences where activities happen + - UIDs: `ScheduledActivityInstance_N`; generated by `generate_scheduled_activity_instances.py` + - Purpose: Define "when/where activities occur" (schedule-specific) + - Relationships: references epochId, encounterId, activityIds[], timelineId + - Contained by: **ScheduleTimeline** (with mainTimeline flag, entryCondition, timings[], instances[]) +- **StudyCell** (`study_cell` table): Junction entity combining armId + epochId + elementIds[] + - Defines which study elements apply to which arm/epoch combinations + - UID pattern: `StudyCell_N` +- **ScheduleTimeline** (`schedule_timelines` table): Container for temporal scheduling + - Contains: instances[] (ScheduledActivityInstance or ScheduledDecisionInstance) + - Contains: timings[] (relative timing definitions), exits[] + - Attributes: mainTimeline (boolean), entryCondition, entryId +- **Encounter** (`visit` table via encounter_uid): Physical/virtual visit where activities occur + - Referenced by: ScheduledActivityInstance.encounterId + - Linked to: Activities via matrix_cells +- **Key Distinction**: Elements = structural design (periods, cohorts) | Instances = temporal schedule (visits, timepoints) + +## Critical Patterns + +### Database & Testing +- **Test isolation**: Tests run against `soa_builder_web_tests.db` (set via `SOA_BUILDER_DB` env). `tests/conftest.py` enforces isolation by removing WAL/SHM files pre-session +- **Connection pattern**: Always use `from .db import _connect` (handles pytest detection, WAL mode, busy timeouts) +- **Schema migrations**: Lifespan event in `app.py` runs migrations in sequence—add new ones to `migrate_database.py` + +### Router Architecture +Endpoints organized by domain in `src/soa_builder/web/routers/`: +- Each router (visits, activities, epochs, arms, elements, etc.) handles JSON API + HTMX UI variants +- Pattern: `@router.post("/soa/{soa_id}/visits")` for API, `@router.post("/ui/soa/{soa_id}/visits/create")` for forms +- Audit trail via `_record_{entity}_audit()` helpers in `audit.py` + +### HTMX UI Conventions +- Templates in `templates/` use `base.html` inheritance +- Form submissions return HTML partials for HTMX swaps +- Matrix edit interface (`edit.html`): drag-drop reordering, cell toggling with status rotation (blank → X → O → blank) +- Modal pattern: target `#modal-host` for freeze/rollback/audit overlays + +### External API Integration +**CDISC Library API** (biomedical concepts): +- Requires `CDISC_SUBSCRIPTION_KEY` or `CDISC_API_KEY` env vars +- Caching: `fetch_biomedical_concepts()` with TTL; force refresh via `POST /ui/soa/{id}/concepts_refresh` +- Override for tests: `CDISC_CONCEPTS_JSON` env (file path or inline JSON) +- Specializations: SDTM codelists via `fetch_sdtm_specializations()` + +### USDM Generation Pipeline +Scripts in `src/usdm/` convert SoA database → USDM JSON: +- `generate_activities.py`, `generate_arms.py`, `generate_study_epochs.py`, etc. +- Each reads from SQLite, constructs USDM objects with UIDs, references, and terminology codes +- Run via CLI: `python -m usdm.generate_activities --soa-id 1 --output-file output/activities.json` +- Relies on junction tables (e.g., `activity_concept`, `code_junction_timings`) for terminology linkage + +## Key Development Workflows + +### Starting the Web Server +```bash +source .venv/bin/activate +soa-builder-web # or uvicorn soa_builder.web.app:app --reload --port 8000 +``` +Access at `http://localhost:8000` + +### Running Tests +```bash +pytest # uses soa_builder_web_tests.db +pytest tests/test_specific.py -v +``` +**Important**: Test DB auto-cleans at session start. Manual cleanup if needed: +```bash +rm -f soa_builder_web_tests.db* +``` + +### Pre-commit Hooks +```bash +pre-commit install +pre-commit run --all-files # runs black + pytest + flake8 +``` + +### CLI Commands +```bash +# Normalize wide CSV → relational tables +soa-builder normalize --input files/SoA.csv --out-dir normalized/ + +# Expand repeating rules → calendar instances +soa-builder expand --normalized-dir normalized/ --start-date 2025-01-01 + +# Validate imaging intervals +soa-builder validate --normalized-dir normalized/ +``` + +## Code Conventions + +### UID Generation +- Auto-generated UIDs follow pattern: `{EntityName}_{incrementing_id}` +- Use `get_next_code_uid()` / `get_next_concept_uid()` from `utils.py` +- Once assigned, UIDs are immutable (e.g., `arm_uid`, `element_uid`) + +### Audit Pattern +All entity mutations log before/after state: +```python +from .audit import _record_element_audit +_record_element_audit(soa_id, "update", element_id, before=old_state, after=new_state) +``` + +### Reorder Operations +- Client sends `order: List[int]` (entity IDs in new sequence) +- Server recomputes `sequence_index` field for all items +- Audit logged with `entity_reorder_audit` table + +### Freeze & Rollback +- **Freeze**: Snapshot visits/activities/cells/epochs/arms to `{entity}_freeze` tables +- **Rollback**: Restore from freeze, track diffs in `rollback_audit` +- UI: Modal shows diff summary, confirms restore + +## Common Gotchas + +1. **Always activate venv first**: `source .venv/bin/activate` before any command +2. **Test DB separation**: Don't run tests against prod DB—conftest enforces `SOA_BUILDER_DB` +3. **HTMX partial responses**: UI endpoints must return HTML fragments, not full pages +4. **SQLite WAL mode**: Production uses WAL; tests use DELETE for simpler cleanup +5. **Concept API 401s in browser**: Direct API URLs fail (no auth headers)—use internal detail pages +6. **Migration order matters**: New migrations go at end of lifespan event sequence +7. **Pydantic schemas**: Use `schemas.py` models for request validation, not raw dicts +8. **Router imports**: Import routers at top of `app.py`, mount with `app.include_router()` + +## Reference Files +- **API endpoints catalog**: `docs/api_endpoints.csv` (165 endpoints: method, path, type, description, response format) +- **Full API docs**: `README_endpoints.md` (curl examples, response schemas) +- **Main README**: Installation, server start, test setup +- **Database schema**: Infer from `initialize_database.py` + migrations in `migrate_database.py` +- **Test patterns**: See `tests/test_bulk_import.py` for matrix operations, `test_element_audit_endpoint.py` for audit trails diff --git a/README.md b/README.md index e7545d9..8e55683 100644 --- a/README.md +++ b/README.md @@ -61,69 +61,45 @@ pytest rm -f soa_builder_web_tests.db soa_builder_web_tests.db-wal soa_builder_web_tests.db-shm ``` -> Full, updated endpoint reference (including Elements, freezes, audits, JSON CRUD and UI helpers) lives in `README_endpoints.md`. Consult that file for detailed request/response examples, curl snippets, and future enhancement notes. +> **Full API Documentation**: See `README_endpoints.md` for complete endpoint reference with curl examples, request/response schemas, and usage patterns. +> +> **Endpoint Catalog**: See `docs/api_endpoints.csv` for sortable/filterable list of all 165+ endpoints. -Endpoints: +## USDM Export +Export USDM-compliant JSON for integration with external systems: +```bash +# Get normalized USDM JSON for a study +curl http://localhost:8000/soa/1/normalized -See **docs/api_endpoints.xlsx** +# Or use the USDM generator scripts directly +python -m usdm.generate_activities --soa-id 1 --output-file activities.json +python -m usdm.generate_encounters --soa-id 1 --output-file encounters.json +python -m usdm.generate_study_epochs --soa-id 1 --output-file epochs.json +# See src/usdm/ for all generator scripts +``` -## Experimental (not yet supported) -After populating data, retrieve normalized artifacts: +## CLI Tools (Legacy) +Command-line tools for CSV normalization and validation: ```bash -curl http://localhost:8000/soa/1/normalized +# Normalize wide CSV → relational tables +soa-builder normalize --input files/SoA.csv --out-dir normalized/ + +# Expand repeating rules → calendar instances +soa-builder expand --normalized-dir normalized/ --start-date 2025-01-01 + +# Validate imaging intervals +soa-builder validate --normalized-dir normalized/ ``` -### Source -Input format: first column `Activity`, subsequent columns are visit/timepoint headers. Cells contain markers `X`, `Optional`, `If indicated`, or repeating patterns (`Every 2 cycles`, `q12w`). - -### Output Artifacts -Running the script produces (in `--out-dir`): -- `visits.csv` — One row per visit/timepoint with parsed window info, inferred category, repeat pattern. -- `activities.csv` — Unique activities (one per original row). -- `visit_activities.csv` — Junction table mapping activities to visits with status and flags. -- `activity_categories.csv` — Heuristic classification of each activity (labs, imaging, dosing, admin, etc.). -- `schedule_rules.csv` — Extracted repeating schedule logic from headers and cells (e.g., `q12w`, `Every 2 cycles`). -- Optional: SQLite database (`--sqlite path`) containing all tables. - -### visits.csv Columns -- `visit_id`: Sequential numeric id. -- `label`: Original header text. -- `visit_name`: Header stripped of parenthetical codes. -- `visit_code`: Code extracted from parentheses (e.g., `C1D1`, `EOT`). -- `sequence_index`: Positional order. -- `window_lower` / `window_upper`: Parsed day offsets if available. -- `repeat_pattern`: Detected repeating pattern (e.g., `every 2 cycles`). -- `category`: Heuristic classification (screening, baseline, treatment, follow_up, eot). - -### activities.csv Columns -- `activity_id`: Sequential id. -- `activity_name`: Name from first column. - -### visit_activities.csv Columns -- `id`: Junction id. -- `visit_id`: FK to visits. -- `activity_id`: FK to activities. -- `status`: Raw cell content. -- `required_flag`: 1 if cell starts with `X`. -- `conditional_flag`: 1 if cell contains `Optional` or `If indicated`. - -### activity_categories.csv Columns -- `activity_id`: FK to activities. -- `category`: Assigned heuristic category label. - -### schedule_rules.csv Columns -- `rule_id`: Unique rule id. -- `pattern`: Normalized repeating pattern token (e.g., `q12w`). -- `description`: Human readable description of pattern source. -- `source_type`: `header` or `cell` origin. -- `activity_id`: Populated if pattern came from a cell (else null). -- `visit_id`: Populated if pattern came from a header. -- `raw_text`: Original text fragment containing the pattern. - - - -# Notes: -- HTMX is loaded via CDN; no build step required. -- For production, configure a persistent DB path via SOA_BUILDER_DB env variable. - -Artifacts stored under `normalized/soa_{id}/`. +See `.github/copilot-instructions.md` for detailed CLI usage patterns. + +--- + +## Architecture Notes +- **Web UI**: HTMX loaded via CDN; no build step required +- **Database**: SQLite with WAL mode (production) or DELETE mode (tests) +- **Test Isolation**: Tests use `soa_builder_web_tests.db` (set via `SOA_BUILDER_DB` env var) +- **Production Config**: Set `SOA_BUILDER_DB` environment variable for persistent DB path +- **USDM Generators**: Python scripts in `src/usdm/` transform database state → USDM JSON artifacts + +For detailed architectural patterns, USDM entity relationships, and development workflows, see `.github/copilot-instructions.md`. diff --git a/README_endpoints.md b/README_endpoints.md index f0a51d5..e914e24 100644 --- a/README_endpoints.md +++ b/README_endpoints.md @@ -1,331 +1,348 @@ # SoA Builder API & UI Endpoints -This document enumerates all public (JSON) API endpoints and key UI (HTML/HTMX) endpoints provided by `soa_builder.web.app`. It groups them by domain with concise purpose, parameters, sample requests, and typical responses. - -> Conventions -> - `{soa_id}` etc. denote path parameters. -> - Unless noted, JSON endpoints return `application/json`. -> - Time values are ISO-8601 UTC. -> - All IDs are integers unless stated otherwise. -> - Errors use FastAPI default error model: `{"detail": "message"}`. +Complete documentation for all 165+ API and UI endpoints in the SoA Workbench application. + +> **Quick Reference**: See `docs/api_endpoints.csv` for a sortable/filterable spreadsheet of all endpoints. +> +> **Conventions** +> - `{soa_id}`, `{visit_id}`, etc. denote path parameters (integers) +> - JSON endpoints return `application/json` unless noted +> - UI endpoints return `text/html` (HTMX partials for partial page updates) +> - Time values are ISO-8601 UTC +> - UIDs follow pattern: `EntityName_N` (e.g., `StudyElement_1`, `ScheduledActivityInstance_5`) +> - Errors use FastAPI default: `{"detail": "message"}`, HTTP status codes: 400, 404, 422 +> +> **Authentication**: Not implemented (all endpoints open). Add auth (API keys / OAuth2) before production use. > -> Authentication: Not implemented (all endpoints open). Add auth (API keys / OAuth2) before production use. +> **Server**: Default runs at `http://localhost:8000` (start via `soa-builder-web` or `uvicorn soa_builder.web.app:app --reload`) --- -## Health / Metadata - -| Method | Path | Purpose | -| ------ | ---- | ------- | -| GET | `/` | Index HTML (lists studies & create form) | -| GET | `/concepts/status` | Diagnostic info about biomedical concepts cache | +## Table of Contents +1. [SoA (Study Container)](#soa-study-container) +2. [Visits](#visits) +3. [Activities](#activities) +4. [Epochs](#epochs) +5. [Arms](#arms) +6. [Elements](#elements) +7. [Instances (ScheduledActivityInstance)](#instances-scheduledactivityinstance) +8. [Schedule Timelines](#schedule-timelines) +9. [Timings](#timings) +10. [Transition Rules](#transition-rules) +11. [Matrix Cells](#matrix-cells) +12. [Study Cells](#study-cells) +13. [Freezes & Rollback](#freezes--rollback) +14. [Audits](#audits) +15. [Biomedical Concepts (CDISC)](#biomedical-concepts-cdisc) +16. [SDTM Specializations](#sdtm-specializations) +17. [Terminology (DDF & Protocol)](#terminology-ddf--protocol) +18. [Curl Examples](#curl-examples) --- ## SoA (Study Container) -| Method | Path | Purpose | -| ------ | ---- | ------- | -| POST | `/soa` | Create new SoA container. Body: `{ "name": str, optional study fields }` | -| GET | `/soa/{soa_id}` | Summary (visits, activities counts, etc.) | -| POST | `/soa/{soa_id}/metadata` | Update study metadata fields (study_id, label, description) | -| GET | `/soa/{soa_id}/normalized` | Normalized SoA JSON (post-processing pipeline) | -| GET | `/soa/{soa_id}/matrix` | Raw matrix: visits, activities, cells | -| POST | `/soa/{soa_id}/matrix/import` | Bulk import matrix (payload structure TBD) | -| GET | `/soa/{soa_id}/export/xlsx` | Download Excel workbook (binary) | - -### Sample: Create SoA +| Method | Path | Type | Description | +| ------ | ---- | ---- | ----------- | +| GET | `/` | UI | Index page - lists all studies with create form | +| POST | `/soa` | API | Create new SoA. Body: `{"name": str, "study_id"?: str, "study_label"?: str, "study_description"?: str}` | +| GET | `/soa/{soa_id}` | API | Get SoA summary (visits, activities, epochs, arms counts) | +| POST | `/soa/{soa_id}/metadata` | API | Update study metadata. Body: `{"study_id"?: str, "study_label"?: str, "study_description"?: str}` | +| GET | `/soa/{soa_id}/normalized` | API | Generate normalized USDM-compatible JSON | +| GET | `/soa/{soa_id}/matrix` | API | Get raw matrix data (visits, activities, cells) | +| POST | `/soa/{soa_id}/matrix/import` | API | Bulk import matrix. Body: `{"instances": [...], "activities": [...], "reset": bool}` | +| GET | `/soa/{soa_id}/export/xlsx` | API | Download Excel workbook | +| GET | `/soa/{soa_id}/export/pdf` | API | Download PDF report | +| POST | `/ui/soa/create` | UI | Create SoA via form | +| POST | `/ui/soa/{soa_id}/update_meta` | UI | Update study metadata via form | +| GET | `/ui/soa/{soa_id}/edit` | UI | Primary editing interface (matrix view) | + +### Example: Create SoA ```bash -curl -X POST http://localhost:8000/soa -H 'Content-Type: application/json' \ - -d '{"name":"Phase I Study","study_id":"STUDY-001"}' +curl -X POST http://localhost:8000/soa \ + -H 'Content-Type: application/json' \ + -d '{"name":"Phase II Trial","study_id":"STUDY-2024-001","study_label":"Phase 2"}' ``` Response: ```json -{ "id": 3, "name": "Phase I Study" } +{"id": 3, "name": "Phase II Trial", "created_at": "2026-01-20T10:30:00.000000+00:00"} ``` --- ## Visits -| Method | Path | Purpose | -| ------ | ---- | ------- | -| POST | `/soa/{soa_id}/visits` | Create visit `{ name, label?, epoch_id? }` | -| PATCH | `/soa/{soa_id}/visits/{visit_id}` | Update visit (partial) returns `updated_fields` | -| DELETE | `/soa/{soa_id}/visits/{visit_id}` | Delete visit (and its cells) | -| GET | `/soa/{soa_id}/visits/{visit_id}` | Fetch visit detail | -| (UI) POST | `/ui/soa/{soa_id}/add_visit` | Form submission create visit | -| (UI) POST | `/ui/soa/{soa_id}/reorder_visits` | Drag reorder (form field `order`) | -| (UI) POST | `/ui/soa/{soa_id}/delete_visit` | Delete via HTMX | -| (UI) POST | `/ui/soa/{soa_id}/set_visit_epoch` | Assign / clear epoch | - -Reorder API (JSON) not implemented for visits yet (only form version). +Visits are **Encounters** in USDM terms - they represent physical or virtual visits where activities occur. + +| Method | Path | Type | Description | +| ------ | ---- | ---- | ----------- | +| GET | `/soa/{soa_id}/visits` | API | List all visits for SoA (ordered by sequence_index) | +| GET | `/ui/soa/{soa_id}/visits` | UI | Visits management page | +| GET | `/soa/visits/{visit_id}` | API | Get visit detail (includes encounter_uid) | +| POST | `/soa/{soa_id}/visits` | API | Create visit. Body: `{"name": str, "label"?: str, "epoch_id"?: int, "encounter_uid"?: str}` | +| PATCH | `/soa/{soa_id}/visits/{visit_id}` | API | Update visit (partial). Returns `{"updated_fields": [...]}` | +| DELETE | `/soa/{soa_id}/visits/{visit_id}` | API | Delete visit (cascades to matrix_cells) | +| POST | `/soa/{soa_id}/visits/reorder` | API | Reorder visits. Body: `[visit_id1, visit_id2, ...]` | +| POST | `/ui/soa/{soa_id}/visits/create` | UI | Create visit via form | +| POST | `/ui/soa/{soa_id}/visits/{visit_id}/update` | UI | Update visit via form | +| POST | `/ui/soa/{soa_id}/visits/{visit_id}/delete` | UI | Delete visit via form | +| POST | `/ui/soa/{soa_id}/reorder_visits` | UI | Reorder visits via drag-drop form | +| POST | `/ui/soa/{soa_id}/set_visit_epoch` | UI | Assign/clear visit epoch | +| POST | `/ui/soa/{soa_id}/set_visit_transition_end_rule` | UI | Set transition end rule | +| POST | `/visits/reorder` | API | Reorder visits (router version) | --- ## Activities -| Method | Path | Purpose | -| ------ | ---- | ------- | -| POST | `/soa/{soa_id}/activities` | Create activity `{ name }` | -| PATCH | `/soa/{soa_id}/activities/{activity_id}` | Update activity (partial) returns `updated_fields` | -| DELETE | `/soa/{soa_id}/activities/{activity_id}` | Delete activity (and its cells & concepts) | -| POST | `/soa/{soa_id}/activities/{activity_id}/concepts` | Set concepts list `{ concept_codes: [...] }` | -| POST | `/soa/{soa_id}/activities/bulk` | Bulk add activities (payload defined in code) | -| GET | `/soa/{soa_id}/activities/{activity_id}` | Fetch activity detail | -| (UI) POST | `/ui/soa/{soa_id}/add_activity` | Form create | -| (UI) POST | `/ui/soa/{soa_id}/reorder_activities` | Drag reorder | -| (UI) POST | `/ui/soa/{soa_id}/delete_activity` | Delete via HTMX | +Activities are **USDM Activity** entities linked to biomedical concepts via `activity_concept` table. + +| Method | Path | Type | Description | +| ------ | ---- | ---- | ----------- | +| GET | `/activities` | API | List all activities | +| GET | `/activities/{activity_id}` | API | Get activity detail (includes activity_uid) | +| POST | `/activities` | API | Create activity. Body: `{"name": str, "activity_uid"?: str}` | +| PATCH | `/activities/{activity_id}` | API | Update activity (partial) | +| DELETE | `/soa/{soa_id}/activities/{activity_id}` | API | Delete activity (cascades to matrix_cells, activity_concept) | +| POST | `/activities/bulk` | API | Bulk add activities. Body: `{"names": [str, ...]}` (deduplicates, skips blanks) | +| POST | `/soa/{soa_id}/activities/{activity_id}/concepts` | API | Set biomedical concepts. Body: `{"concept_codes": [str, ...]}` | +| POST | `/activities/{activity_id}/concepts` | API | Set concepts (router version) | +| POST | `/soa/{soa_id}/activities/reorder` | API | Reorder activities. Body: `[activity_id1, ...]` | +| POST | `/activities/reorder` | API | Reorder activities (router version) | +| POST | `/activities/add` | UI | Add activity via form (router) | +| POST | `/activities/{activity_id}/update` | UI | Update activity via form (router) | +| POST | `/ui/soa/{soa_id}/add_activity` | UI | Add activity via form | +| POST | `/ui/soa/{soa_id}/delete_activity` | UI | Delete activity via form | +| POST | `/ui/soa/{soa_id}/reorder_activities` | UI | Reorder activities via drag-drop | --- ## Epochs -| Method | Path | Purpose | -| ------ | ---- | ------- | -| POST | `/soa/{soa_id}/epochs` | Create epoch `{ name }` (sequence auto-assigned) | -| GET | `/soa/{soa_id}/epochs` | List epochs (ordered) | -| GET | `/soa/{soa_id}/epochs/{epoch_id}` | Fetch epoch detail | -| POST | `/soa/{soa_id}/epochs/{epoch_id}/metadata` | Update name/label/description (returns `updated_fields`) | -| DELETE | `/soa/{soa_id}/epochs/{epoch_id}` | Delete epoch | -| (UI) POST | `/ui/soa/{soa_id}/add_epoch` | Form create | -| (UI) POST | `/ui/soa/{soa_id}/update_epoch` | Update via form | -| (UI) POST | `/ui/soa/{soa_id}/reorder_epochs` | Reorder | -| (UI) POST | `/ui/soa/{soa_id}/delete_epoch` | Delete | +Epochs are **USDM StudyEpoch** entities representing high-level study phases (e.g., Screening, Treatment, Follow-up). + +| Method | Path | Type | Description | +| ------ | ---- | ---- | ----------- | +| GET | `/soa/{soa_id}/epochs` | API | List epochs (ordered by epoch_seq) | +| GET | `/ui/soa/{soa_id}/epochs` | UI | Epochs management page | +| GET | `/soa/{soa_id}/epochs/{epoch_id}` | API | Get epoch detail (includes epoch_uid) | +| POST | `/soa/{soa_id}/epochs/{epoch_id}/metadata` | API | Update epoch metadata. Body: `{"name"?: str, "epoch_label"?: str, "epoch_description"?: str, "type"?: str}` | +| PATCH | `/soa/{soa_id}/epochs/{epoch_id}` | API | Update epoch (partial). Returns `{"updated_fields": [...]}` | +| DELETE | `/soa/{soa_id}/epochs/{epoch_id}` | API | Delete epoch | +| POST | `/soa/{soa_id}/epochs/reorder` | API | Reorder epochs. Body: `[epoch_id1, ...]` | +| POST | `/ui/soa/{soa_id}/epochs/create` | UI | Create epoch via form | +| POST | `/ui/soa/{soa_id}/epochs/{epoch_id}/update` | UI | Update epoch via form | +| POST | `/ui/soa/{soa_id}/epochs/{epoch_id}/delete` | UI | Delete epoch via form | +| POST | `/ui/soa/{soa_id}/reorder_epochs` | UI | Reorder epochs via drag-drop | --- -## Elements (New) -## Arms (New) - -| Method | Path | Purpose | -| ------ | ---- | ------- | -| GET | `/soa/{soa_id}/arms` | List arms (ordered) | -| GET | `/soa/{soa_id}/arms/{arm_id}` | Fetch arm detail (includes immutable `arm_uid`) | -| POST | `/soa/{soa_id}/arms` | Create arm `{ name, label?, description? }` (auto assigns next `arm_uid` = `StudyArm_`) | -| PATCH | `/soa/{soa_id}/arms/{arm_id}` | Update arm (partial) returns `updated_fields` (arm_uid immutable) | -| DELETE | `/soa/{soa_id}/arms/{arm_id}` | Delete arm | -| POST | `/soa/{soa_id}/arms/reorder` | Reorder (body JSON array of IDs) | -| GET | `/soa/{soa_id}/arm_audit` | Arm audit log (create/update/delete/reorder entries) | -| (UI) POST | `/ui/soa/{soa_id}/add_arm` | Form create | -| (UI) POST | `/ui/soa/{soa_id}/update_arm` | Form update | -| (UI) POST | `/ui/soa/{soa_id}/delete_arm` | Form delete | -| (UI) POST | `/ui/soa/{soa_id}/reorder_arms` | Drag reorder (form) | - -Arm rows include immutable `arm_uid` (unique per study). Element linkage has been removed; a migration now physically drops the legacy `element_id` and `etcd` columns from existing databases. Fresh installs never create these columns. - -| Method | Path | Purpose | -| ------ | ---- | ------- | -| GET | `/soa/{soa_id}/elements` | List elements (ordered) | -| GET | `/soa/{soa_id}/elements/{element_id}` | Fetch element detail | -| POST | `/soa/{soa_id}/elements` | Create element `{ name, label?, description?, testrl?, teenrl? }` | -| PATCH | `/soa/{soa_id}/elements/{element_id}` | Update (partial) | -| DELETE | `/soa/{soa_id}/elements/{element_id}` | Delete | -| POST | `/soa/{soa_id}/elements/reorder` | Reorder (body JSON array of IDs) | -| GET | `/soa/{soa_id}/element_audit` | Element audit log (create/update/delete/reorder entries) | -| (UI) POST | `/ui/soa/{soa_id}/add_element` | Form create | -| (UI) POST | `/ui/soa/{soa_id}/update_element` | Form update | -| (UI) POST | `/ui/soa/{soa_id}/delete_element` | Form delete | -| (UI) POST | `/ui/soa/{soa_id}/reorder_elements` | Drag reorder (form) | - -### Element JSON Examples -Create: +## Arms + +Arms are **USDM StudyArm** entities. Each has immutable `arm_uid` (format: `StudyArm_N`). + +| Method | Path | Type | Description | +| ------ | ---- | ---- | ----------- | +| GET | `/soa/{soa_id}/arms` | API | List arms (ordered) | +| GET | `/ui/soa/{soa_id}/arms` | UI | Arms management page | +| POST | `/soa/{soa_id}/arms` | API | Create arm. Body: `{"name": str, "label"?: str, "description"?: str, "type"?: str, "origin"?: str}`. Auto-assigns `arm_uid` | +| PATCH | `/soa/{soa_id}/arms/{arm_id}` | API | Update arm (partial). Returns `{"updated_fields": [...]}`. `arm_uid` immutable | +| POST | `/arms/reorder` | API | Reorder arms. Body: `[arm_id1, ...]` | +| POST | `/ui/soa/{soa_id}/arms/create` | UI | Create arm via form | +| POST | `/ui/soa/{soa_id}/arms/{arm_id}/update` | UI | Update arm via form | +| POST | `/ui/soa/{soa_id}/arms/{arm_id}/delete` | UI | Delete arm via form | +| POST | `/ui/soa/{soa_id}/reorder_arms` | UI | Reorder arms via drag-drop | + +--- +## Elements + +Elements are **USDM StudyElement** entities representing structural design components (e.g., treatment periods, cohorts). Each has immutable `element_id` (format: `StudyElement_N`). + +| Method | Path | Type | Description | +| ------ | ---- | ---- | ----------- | +| GET | `/soa/{soa_id}/elements` | API | List elements (ordered) | +| GET | `/ui/soa/{soa_id}/elements` | UI | Elements management page | +| GET | `/soa/{soa_id}/elements/{element_id}` | API | Get element detail | +| POST | `/elements` | API | Create element. Body: `{"name": str, "label"?: str, "description"?: str, "testrl"?: str, "teenrl"?: str}`. Auto-assigns `element_id` | +| PATCH | `/soa/{soa_id}/elements/{element_id}` | API | Update element (partial) | +| PATCH | `/elements/{element_id}` | API | Update element (router version) | +| DELETE | `/elements/{element_id}` | API | Delete element | +| POST | `/elements/reorder` | API | Reorder elements. Body: `[element_id1, ...]` | +| GET | `/soa/{soa_id}/element_audit` | API | Get element audit log | +| POST | `/ui/soa/{soa_id}/elements/create` | UI | Create element via form | +| POST | `/ui/soa/{soa_id}/elements/{element_id}/update` | UI | Update element via form | +| POST | `/ui/soa/{soa_id}/elements/{element_id}/delete` | UI | Delete element via form | + +### Example: Element Operations ```bash -curl -X POST http://localhost:8000/soa/5/elements \ +# Create element +curl -X POST http://localhost:8000/elements \ -H 'Content-Type: application/json' \ - -d '{"name":"Screening","label":"SCR","description":"Screening element"}' -``` -Reorder: -```bash -curl -X POST http://localhost:8000/soa/5/elements/reorder \ + -d '{"name":"Screening Period","label":"SCR","description":"Initial screening"}' + +# Reorder elements +curl -X POST http://localhost:8000/elements/reorder \ -H 'Content-Type: application/json' \ -d '[3,1,2]' ``` -Audit entry structure (GET `/soa/{soa_id}/element_audit`): -```json -{ - "id": 12, - "element_id": 7, - "action": "update", - "before": {"id":7,"name":"Screening"}, - "after": {"id":7,"name":"Screening Updated"}, - "performed_at": "2025-11-07T12:34:56.123456+00:00" -} -``` - --- -## Cells (Matrix) +## Instances (ScheduledActivityInstance) -| Method | Path | Purpose | -| ------ | ---- | ------- | -| POST | `/soa/{soa_id}/cells` | Upsert a cell `{ visit_id, activity_id, status }` | -| (UI) POST | `/ui/soa/{soa_id}/toggle_cell` | Toggle cell (HTMX) | -| (UI) POST | `/ui/soa/{soa_id}/set_cell` | Explicit set (HTMX) | +Instances are **USDM ScheduledActivityInstance** entities - temporal visit/timepoint occurrences where activities happen. Each has `instance_uid` (format: `ScheduledActivityInstance_N`). -Status typical values: "X" or empty (cleared). +| Method | Path | Type | Description | +| ------ | ---- | ---- | ----------- | +| GET | `/soa/{soa_id}/instances` | API | List instances (ordered) | +| GET | `/ui/soa/{soa_id}/instances` | UI | Instances management page | +| POST | `/ui/soa/{soa_id}/instances/create` | UI | Create instance via form. Fields: name, label, description, epoch_uid, encounter_uid, timeline_id, etc. | +| POST | `/ui/soa/{soa_id}/instances/{instance_id}/update` | UI | Update instance via form | +| POST | `/ui/soa/{soa_id}/instances/{instance_id}/delete` | UI | Delete instance via form | --- -## Biomedical Concepts +## Schedule Timelines -| Method | Path | Purpose | -| ------ | ---- | ------- | -| POST | `/ui/soa/{soa_id}/concepts_refresh` | Force remote re-fetch + cache reset | -| GET | `/concepts/status` | Cache diagnostics (see above) | -| GET | `/ui/concepts` | HTML table listing biomedical concepts (code, title, API href) | -| GET | `/ui/concepts/{code}` | HTML detail page for a single concept (title, API href, parent concept/package links) | +Schedule Timelines are **USDM ScheduleTimeline** containers holding instances, timings, and exits. Each has `schedule_timeline_uid`. -Concept assignment happens via `POST /soa/{soa_id}/activities/{activity_id}/concepts`. - -Payload: -```json -{ "concept_codes": ["C12345", "C67890"] } -``` +| Method | Path | Type | Description | +| ------ | ---- | ---- | ----------- | +| GET | `/ui/soa/{soa_id}/schedule_timelines` | UI | Schedule timelines management page | +| POST | `/ui/soa/{soa_id}/schedule_timelines/create` | UI | Create timeline via form. Fields: name, label, main_timeline (bool), entry_condition, entry_id | +| POST | `/ui/soa/{soa_id}/schedule_timelines/{schedule_timeline_id}/update` | UI | Update timeline via form | +| POST | `/ui/soa/{soa_id}/schedule_timelines/{schedule_timeline_id}/delete` | UI | Delete timeline via form | --- -## Freezes (Versioning) & Rollback +## Timings -| Method | Path | Purpose | -| ------ | ---- | ------- | -| POST | `/ui/soa/{soa_id}/freeze` | Create new version (HTML) | -| GET | `/soa/{soa_id}/freeze/{freeze_id}` | Get freeze snapshot JSON | -| GET | `/ui/soa/{soa_id}/freeze/{freeze_id}/view` | Modal view of snapshot | -| GET | `/ui/soa/{soa_id}/freeze/diff` | HTML diff (query params: `left`, `right`) | -| GET | `/soa/{soa_id}/freeze/diff.json` | JSON diff (`?left=&right=`) | -| POST | `/ui/soa/{soa_id}/freeze/{freeze_id}/rollback` | Restore SoA to snapshot | +Timings are **USDM Timing** definitions for schedule references. Each has `timing_uid` (format: `Timing_N`). -Snapshot includes keys: `epochs`, `elements`, `visits`, `activities`, `cells`, `activity_concepts`, metadata fields. +| Method | Path | Type | Description | +| ------ | ---- | ---- | ----------- | +| GET | `/soa/{soa_id}/timings` | API | List timings (ordered) | +| GET | `/ui/soa/{soa_id}/timings` | UI | Timings management page | +| GET | `/soa/{soa_id}/timing_audit` | API | Get timing audit log | +| POST | `/ui/soa/{soa_id}/timings/create` | UI | Create timing via form. Fields: name, label, type, value, window_upper, window_lower, relative_to_from, etc. | +| POST | `/ui/soa/{soa_id}/timings/{timing_id}/update` | UI | Update timing via form | +| POST | `/ui/soa/{soa_id}/timings/{timing_id}/delete` | UI | Delete timing via form | --- -## Audits +## Transition Rules -| Method | Path | Purpose | -| ------ | ---- | ------- | -| GET | `/soa/{soa_id}/rollback_audit` | JSON rollback audit log | -| GET | `/soa/{soa_id}/reorder_audit` | JSON reorder audit log (visits, activities, epochs, elements) | -| GET | `/ui/soa/{soa_id}/rollback_audit` | HTML modal rollback audit | -| GET | `/ui/soa/{soa_id}/reorder_audit` | HTML modal reorder audit | -| GET | `/soa/{soa_id}/rollback_audit/export/xlsx` | Excel export rollback audit | -| GET | `/soa/{soa_id}/reorder_audit/export/xlsx` | Excel export reorder audit | -| GET | `/soa/{soa_id}/reorder_audit/export/csv` | CSV export reorder audit | -| GET | `/soa/{soa_id}/element_audit` | Element audit (see Elements section) | -| GET | `/soa/{soa_id}/visit_audit` | Visit audit log (create/update/delete) | -| GET | `/soa/{soa_id}/activity_audit` | Activity audit log (create/update/delete) | -| GET | `/soa/{soa_id}/epoch_audit` | Epoch audit log (create/update/delete/reorder) | -| GET | `/soa/{soa_id}/arm_audit` | Arm audit log (create/update/delete/reorder) | +Transition rules define **USDM TransitionRule** entities for element entry/exit conditions. -Rollback audit row fields: `id, soa_id, freeze_id, performed_at, visits_restored, activities_restored, cells_restored, concepts_restored, elements_restored`. +| Method | Path | Type | Description | +| ------ | ---- | ---- | ----------- | +| GET | `/soa/{soa_id}/rules` | API | List transition rules | +| GET | `/ui/soa/{soa_id}/rules` | UI | Transition rules management page | +| PATCH | `/soa/{soa_id}/rules/{rule_id}` | API | Update rule (partial) | +| POST | `/ui/soa/{soa_id}/rules/create` | UI | Create rule via form | +| POST | `/ui/soa/{soa_id}/rules/{rule_id}/update` | UI | Update rule via form | +| POST | `/ui/soa/{soa_id}/rules/{rule_id}/delete` | UI | Delete rule via form | -Reorder audit row fields: `id, soa_id, entity_type, old_order_json, new_order_json, performed_at`. - -### Audit Entry Shapes +--- +## Matrix Cells -Each per-entity audit endpoint (`element_audit`, `visit_audit`, `activity_audit`, `epoch_audit`) returns rows with a common structure: +Matrix cells (`matrix_cells` table) link visits/instances to activities with status markers. -``` -{ - "id": 42, - "_id": 7, - "action": "create" | "update" | "delete" | "reorder", - "before": { ... } | null, - "after": { ... } | null, - "performed_at": "2025-11-07T12:34:56.123456+00:00", - "updated_fields": ["name","label"] // present only for update actions -} -``` +| Method | Path | Type | Description | +| ------ | ---- | ---- | ----------- | +| POST | `/soa/{soa_id}/cells` | API | Create/update matrix cell. Body: `{"visit_id": int, "activity_id": int, "status": str}` | +| POST | `/soa/{soa_id}/cells_instance` | API | Create cell with instance_id. Body: `{"instance_id": int, "activity_id": int, "status": str}` | +| POST | `/ui/soa/{soa_id}/set_cell` | UI | Set cell status via form | +| POST | `/ui/soa/{soa_id}/toggle_cell` | UI | Toggle cell status (blank → X → O → blank) | +| POST | `/ui/soa/{soa_id}/toggle_cell_instance` | UI | Toggle cell instance status | -Notes: -- `before` is null for creates; `after` is null for deletes. -- `updated_fields` lists the keys that changed between `before` and `after` for update actions (omitted otherwise). -- Epoch reorder also creates an entry in `reorder_audit`; if epoch attributes (name/label/description) change, an `update` row appears in `epoch_audit` with `updated_fields`. -- Element reorder emits an `action":"reorder"` row in `element_audit` in addition to the global `reorder_audit` table. +**Status values**: Blank (empty), `"X"` (required), `"O"` (optional) --- -## UI Editing Endpoints (HTMX Helpers) +## Study Cells -| Method | Path | Purpose | -| ------ | ---- | ------- | -| GET | `/ui/soa/{soa_id}/edit` | Primary editing interface (HTML) | -| POST | `/ui/soa/{soa_id}/update_meta` | Update study metadata (form) | -| POST | `/ui/soa/create` | Create new study via form | +Study Cells are **USDM StudyCell** junction entities combining `armId + epochId + elementIds[]`. Each has `study_cell_uid` (format: `StudyCell_N`). -These endpoints render or redirect; they are not intended for API clients. +| Method | Path | Type | Description | +| ------ | ---- | ---- | ----------- | +| POST | `/ui/soa/{soa_id}/add_study_cell` | UI | Add study cell. Form fields: arm_uid, epoch_uid, element_uid | +| POST | `/ui/soa/{soa_id}/update_study_cell` | UI | Update study cell | +| POST | `/ui/soa/{soa_id}/delete_study_cell` | UI | Delete study cell | --- -## Reordering (JSON vs UI) +## Freezes & Rollback -| Domain | JSON Endpoint | UI Endpoint | Body / Form | -| ------ | ------------- | ----------- | ----------- | -| Elements | POST `/soa/{soa_id}/elements/reorder` | POST `/ui/soa/{soa_id}/reorder_elements` | JSON array / form `order` | -| Visits | POST `/soa/{soa_id}/visits/reorder` | POST `/ui/soa/{soa_id}/reorder_visits` | JSON array / form `order` | -| Activities | POST `/soa/{soa_id}/activities/reorder` | POST `/ui/soa/{soa_id}/reorder_activities` | JSON array / form `order` | -| Epochs | POST `/soa/{soa_id}/epochs/reorder` | POST `/ui/soa/{soa_id}/reorder_epochs` | JSON array / form `order` | -| Arms | POST `/soa/{soa_id}/arms/reorder` | POST `/ui/soa/{soa_id}/reorder_arms` | JSON array / form `order` | +Freezes create immutable snapshots of SoA state for versioning. Rollback restores from a freeze. ---- -## Error Handling -Typical errors: -- 400: Validation or duplicate version label. -- 404: Entity not found / SoA not found. -- 409: (Future) uniqueness conflicts (currently 400 for version label). +| Method | Path | Type | Description | +| ------ | ---- | ---- | ----------- | +| POST | `/ui/soa/{soa_id}/freeze` | UI | Create freeze snapshot. Form field: `version_label` (optional) | +| GET | `/soa/{soa_id}/freeze/{freeze_id}` | API | Get freeze snapshot JSON (visits, activities, cells, epochs, arms, elements, concepts) | +| GET | `/ui/soa/{soa_id}/freeze/{freeze_id}/view` | UI | View freeze modal (HTML) | +| GET | `/ui/soa/{soa_id}/freeze/diff` | UI | Compare two freezes. Query params: `?left=freeze_id&right=freeze_id` | +| GET | `/soa/{soa_id}/freeze/diff.json` | API | Get freeze diff JSON. Query params: `?left=&right=` | -Example error: -```json -{"detail":"SOA not found"} -``` +**Freeze includes**: epochs, elements, visits, activities, matrix_cells, activity_concepts, study metadata --- -## Future Enhancements (Suggested) -- Add pagination / filtering for large audit logs. -- Introduce authentication & RBAC. -- Add OpenAPI tags grouping elements vs core vs audits. -- Rate limiting & conditional ETag caching for large snapshots. +## Audits ---- -## Quick Reference (Most Used) -``` -Create SoA POST /soa -Get Visit Detail GET /soa/{id}/visits/{visit_id} -Get Activity Detail GET /soa/{id}/activities/{activity_id} -Get Arm Detail GET /soa/{id}/arms/{arm_id} -List Elements GET /soa/{id}/elements -Create Element POST /soa/{id}/elements -Update Element PATCH /soa/{id}/elements/{element_id} -Reorder Elements POST /soa/{id}/elements/reorder (JSON array) -Reorder Visits POST /soa/{id}/visits/reorder (JSON array) -Reorder Activities POST /soa/{id}/activities/reorder (JSON array) -Reorder Epochs POST /soa/{id}/epochs/reorder (JSON array) -Reorder Arms POST /soa/{id}/arms/reorder (JSON array) -Freeze Version POST /ui/soa/{id}/freeze (form) -Rollback POST /ui/soa/{id}/freeze/{freeze_id}/rollback -Element Audit GET /soa/{id}/element_audit -Rollback Audit GET /soa/{id}/rollback_audit -Reorder Audit GET /soa/{id}/reorder_audit -Export Excel GET /soa/{id}/export/xlsx -Normalized View GET /soa/{id}/normalized -Concepts List GET /ui/concepts +Comprehensive audit trails for all entity mutations and bulk operations. + +| Method | Path | Type | Description | +| ------ | ---- | ---- | ----------- | +| GET | `/soa/{soa_id}/rollback_audit` | API | Rollback audit log (freeze restores) | +| GET | `/ui/soa/{soa_id}/rollback_audit` | UI | View rollback audit modal | +| GET | `/soa/{soa_id}/rollback_audit/export/xlsx` | API | Export rollback audit as Excel | +| GET | `/soa/{soa_id}/reorder_audit` | API | Reorder audit log (visits, activities, epochs, elements, arms) | +| GET | `/ui/soa/{soa_id}/reorder_audit` | UI | View reorder audit modal | +| GET | `/soa/{soa_id}/reorder_audit/export/csv` | API | Export reorder audit as CSV | +| GET | `/soa/{soa_id}/reorder_audit/export/xlsx` | API | Export reorder audit as Excel | +| GET | `/soa/{soa_id}/element_audit` | API | Element-specific audit (create/update/delete/reorder) | +| GET | `/soa/{soa_id}/timing_audit` | API | Timing-specific audit | +| GET | `/ui/soa/{soa_id}/audits` | UI | Combined audits page | + +### Audit Entry Structure +All entity audits follow this pattern: +```json +{ + "id": 42, + "soa_id": 1, + "{entity}_id": 7, + "action": "create|update|delete|reorder", + "before": {"id": 7, "name": "Old Value"}, + "after": {"id": 7, "name": "New Value"}, + "performed_at": "2026-01-20T10:30:00.000000+00:00", + "updated_fields": ["name", "label"] +} ``` +- `before` is null for creates +- `after` is null for deletes +- `updated_fields` present only for updates --- -## Curl Cheat-Sheet -```bash -# Create a study -curl -s -X POST localhost:8000/soa -H 'Content-Type: application/json' -d '{"name":"Demo"}' +## Biomedical Concepts (CDISC) -# Add element -curl -s -X POST localhost:8000/soa/1/elements -H 'Content-Type: application/json' -d '{"name":"Screening"}' +Integration with CDISC Library API for biomedical concept assignment to activities. -# List elements -curl -s localhost:8000/soa/1/elements | jq +| Method | Path | Type | Description | +| ------ | ---- | ---- | ----------- | +| GET | `/concepts/status` | API | Get concepts cache status (TTL, count, last refresh) | +| GET | `/ui/concepts` | UI | List all biomedical concepts (cached) | +| GET | `/ui/concepts/{code}` | UI | View concept detail page | +| POST | `/ui/soa/{soa_id}/concepts_refresh` | UI | Force refresh concepts cache from CDISC API | +| GET | `/ui/concept_categories` | UI | List concept categories | +| GET | `/ui/concept_categories/view` | UI | View concepts by category. Query param: `?name=category_name` | -# Update element -curl -s -X PATCH localhost:8000/soa/1/elements/2 -H 'Content-Type: application/json' -d '{"label":"SCR"}' +**Environment Variables Required**: +- `CDISC_SUBSCRIPTION_KEY` or `CDISC_API_KEY` - for CDISC Library API access +- `CDISC_CONCEPTS_JSON` - (optional) for test overrides -# Reorder elements -curl -s -X POST localhost:8000/soa/1/elements/reorder -H 'Content-Type: application/json' -d '[2,1]' +**Test Override**: Set `CDISC_CONCEPTS_JSON` to file path or inline JSON to bypass remote API -# Freeze -curl -s -X POST localhost:8000/ui/soa/1/freeze -d 'version_label=v1' +--- +## SDTM Specializations -# Diff two freezes -curl -s 'localhost:8000/soa/1/freeze/diff.json?left=5&right=7' | jq -``` +SDTM controlled terminology codelists. + +| Method | Path | Type | Description | +| ------ | ---- | ---- | ----------- | +| GET | `/sdtm/specializations/status` | API | Get SDTM specializations status | +| GET | `/ui/sdtm/specializations/status` | UI | View SDTM status page | +| POST | `/ui/sdtm/specializations/refresh` | UI | Refresh SDTM specializations from API | +| GET | `/ui/sdtm/specializations` | UI | List SDTM specializations | +| GET | `/ui/sdtm/specializations/{idx}` | UI | View SDTM specialization detail | ---- --- ## Terminology (DDF & Protocol) @@ -386,210 +403,144 @@ curl -s --get 'http://localhost:8000/protocol/terminology/audit' | jq '.rows[].d Both accept `.xls` or `.xlsx`. A SHA-256 hash is computed and stored in audit for integrity tracking. --- -Generated on: 2025-11-12 - -## Full Endpoint Inventory (Auto-Generated 2025-11-12) - -Below is a consolidated list of all FastAPI routes currently defined in `src/soa_builder/web/app.py`. "Type" reflects typical response kind (JSON, HTML, Binary, CSV). UI/Form endpoints are primarily for browser interaction (HTMX/HTML forms) and may redirect. - -| Method | Path | Type | Notes | -|--------|------|------|-------| -| GET | `/` | HTML | Index & study creation form | -| GET | `/concepts/status` | JSON | Biomedical concepts cache diagnostics | -| GET | `/sdtm/specializations/status` | JSON | SDTM dataset specializations cache diagnostics | -| GET | `/soa/{soa_id}` | JSON | Study summary (counts, metadata) | -| GET | `/soa/{soa_id}/elements` | JSON | List elements | -| GET | `/soa/{soa_id}/elements/{element_id}` | JSON | Element detail | -| GET | `/soa/{soa_id}/element_audit` | JSON | Element audit log | -| GET | `/soa/{soa_id}/arms` | JSON | List arms | -| GET | `/soa/{soa_id}/arm_audit` | JSON | Arm audit log | -| GET | `/soa/{soa_id}/freeze/{freeze_id}` | JSON | Freeze snapshot JSON | -| GET | `/ui/soa/{soa_id}/freeze/{freeze_id}/view` | HTML | Freeze snapshot modal | -| GET | `/ui/soa/{soa_id}/freeze/diff` | HTML | Diff view (query `left`,`right`) | -| GET | `/soa/{soa_id}/freeze/diff.json` | JSON | Diff JSON (`left`,`right`) | -| GET | `/soa/{soa_id}/rollback_audit` | JSON | Rollback audit log | -| GET | `/soa/{soa_id}/reorder_audit` | JSON | Reorder audit log | -| GET | `/ui/soa/{soa_id}/rollback_audit` | HTML | Rollback audit modal | -| GET | `/ui/soa/{soa_id}/reorder_audit` | HTML | Reorder audit modal | -| GET | `/soa/{soa_id}/rollback_audit/export/xlsx` | Binary | Excel export rollback audit | -| GET | `/soa/{soa_id}/reorder_audit/export/xlsx` | Binary | Excel export reorder audit | -| GET | `/soa/{soa_id}/reorder_audit/export/csv` | CSV | CSV export reorder audit | -| GET | `/soa/{soa_id}/visits/{visit_id}` | JSON | Visit detail | -| GET | `/soa/{soa_id}/activities/{activity_id}` | JSON | Activity detail | -| GET | `/soa/{soa_id}/epochs` | JSON | List epochs (ordered) | -| GET | `/soa/{soa_id}/epochs/{epoch_id}` | JSON | Epoch detail | -| GET | `/soa/{soa_id}/matrix` | JSON | Matrix (visits, activities, cells) | -| GET | `/soa/{soa_id}/export/xlsx` | Binary | Excel workbook export | -| GET | `/soa/{soa_id}/normalized` | JSON | Normalized representation | -| GET | `/ui/soa/{soa_id}/edit` | HTML | Main editing UI | -| GET | `/ui/concepts` | HTML | Concepts listing | -| GET | `/ui/concepts/{code}` | HTML | Concept detail (parent links) | -| GET | `/ui/sdtm/specializations` | HTML | SDTM dataset specializations list | -| GET | `/ui/sdtm/specializations/{idx}` | HTML | SDTM specialization raw JSON detail | -| GET | `/ddf/terminology` | JSON | Query DDF terminology rows | -| GET | `/ui/ddf/terminology` | HTML | DDF terminology UI page | -| GET | `/ddf/terminology/audit` | JSON | DDF audit entries | -| GET | `/ddf/terminology/audit/export.csv` | CSV | DDF audit export CSV | -| GET | `/ddf/terminology/audit/export.json` | JSON | DDF audit export JSON | -| GET | `/ui/ddf/terminology/audit` | HTML | DDF audit UI page | -| GET | `/protocol/terminology` | JSON | Query Protocol terminology rows | -| GET | `/ui/protocol/terminology` | HTML | Protocol terminology UI page | -| GET | `/protocol/terminology/audit` | JSON | Protocol audit entries | -| GET | `/protocol/terminology/audit/export.csv` | CSV | Protocol audit export CSV | -| GET | `/protocol/terminology/audit/export.json` | JSON | Protocol audit export JSON | -| GET | `/ui/protocol/terminology/audit` | HTML | Protocol audit UI page | -| POST | `/soa` | JSON | Create study container | -| POST | `/soa/{soa_id}/metadata` | JSON | Update study metadata fields | -| POST | `/soa/{soa_id}/elements` | JSON | Create element | -| POST | `/soa/{soa_id}/elements/reorder` | JSON | Reorder elements | -| POST | `/soa/{soa_id}/arms` | JSON | Create arm (assigns `arm_uid`) | -| POST | `/soa/{soa_id}/arms/reorder` | JSON | Reorder arms | -| POST | `/soa/{soa_id}/visits` | JSON | Create visit | -| POST | `/soa/{soa_id}/visits/reorder` | JSON | Reorder visits | -| POST | `/soa/{soa_id}/activities` | JSON | Create activity | -| POST | `/soa/{soa_id}/activities/reorder` | JSON | Reorder activities | -| POST | `/soa/{soa_id}/epochs` | JSON | Create epoch | -| POST | `/soa/{soa_id}/epochs/reorder` | JSON | Reorder epochs | -| POST | `/soa/{soa_id}/epochs/{epoch_id}/metadata` | JSON | Update epoch metadata | -| POST | `/soa/{soa_id}/activities/{activity_id}/concepts` | JSON | Set concepts list | -| POST | `/soa/{soa_id}/activities/bulk` | JSON | Bulk create activities | -| POST | `/soa/{soa_id}/cells` | JSON | Upsert cell | -| POST | `/soa/{soa_id}/matrix/import` | JSON | Bulk matrix import | -| POST | `/ui/soa/create` | HTML | Form create study | -| POST | `/ui/soa/{soa_id}/update_meta` | HTML | Update metadata form | -| POST | `/ui/soa/{soa_id}/concepts_refresh` | HTML | Force concepts cache refresh | -| POST | `/ui/soa/{soa_id}/freeze` | HTML | Create freeze version | -| POST | `/ui/soa/{soa_id}/freeze/{freeze_id}/rollback` | HTML | Roll back to freeze | -| POST | `/ui/sdtm/specializations/refresh` | HTML | Force specializations refresh | -| POST | `/ui/soa/{soa_id}/add_visit` | HTML | Add visit form | -| POST | `/ui/soa/{soa_id}/add_arm` | HTML | Add arm form | -| POST | `/ui/soa/{soa_id}/update_arm` | HTML | Update arm form | -| POST | `/ui/soa/{soa_id}/delete_arm` | HTML | Delete arm form | -| POST | `/ui/soa/{soa_id}/reorder_arms` | HTML | Reorder arms form | -| POST | `/ui/soa/{soa_id}/add_element` | HTML | Add element form | -| POST | `/ui/soa/{soa_id}/update_element` | HTML | Update element form | -| POST | `/ui/soa/{soa_id}/delete_element` | HTML | Delete element form | -| POST | `/ui/soa/{soa_id}/reorder_elements` | HTML | Reorder elements form | -| POST | `/ui/soa/{soa_id}/add_activity` | HTML | Add activity form | -| POST | `/ui/soa/{soa_id}/add_epoch` | HTML | Add epoch form | -| POST | `/ui/soa/{soa_id}/update_epoch` | HTML | Update epoch form | -| POST | `/ui/soa/{soa_id}/set_cell` | HTML | Set cell (HTMX) | -| POST | `/ui/soa/{soa_id}/toggle_cell` | HTML | Toggle cell (HTMX) | -| POST | `/ui/soa/{soa_id}/delete_visit` | HTML | Delete visit form | -| POST | `/ui/soa/{soa_id}/set_visit_epoch` | HTML | Assign/clear visit epoch | -| POST | `/ui/soa/{soa_id}/delete_activity` | HTML | Delete activity form | -| POST | `/ui/soa/{soa_id}/delete_epoch` | HTML | Delete epoch form | -| POST | `/ui/soa/{soa_id}/reorder_visits` | HTML | Reorder visits form | -| POST | `/ui/soa/{soa_id}/reorder_activities` | HTML | Reorder activities form | -| POST | `/ui/soa/{soa_id}/reorder_epochs` | HTML | Reorder epochs form | -| POST | `/admin/load_ddf_terminology` | JSON | Reload DDF terminology sheet | -| POST | `/ui/ddf/terminology/upload` | HTML | Upload DDF terminology sheet | -| POST | `/admin/load_protocol_terminology` | JSON | Reload Protocol terminology sheet | -| POST | `/ui/protocol/terminology/upload` | HTML | Upload Protocol terminology sheet | -| PATCH | `/soa/{soa_id}/elements/{element_id}` | JSON | Partial update element | -| PATCH | `/soa/{soa_id}/arms/{arm_id}` | JSON | Partial update arm | -| PATCH | `/soa/{soa_id}/visits/{visit_id}` | JSON | Partial update visit | -| PATCH | `/soa/{soa_id}/activities/{activity_id}` | JSON | Partial update activity | -| DELETE | `/soa/{soa_id}/elements/{element_id}` | JSON | Delete element | -| DELETE | `/soa/{soa_id}/arms/{arm_id}` | JSON | Delete arm | -| DELETE | `/soa/{soa_id}/visits/{visit_id}` | JSON | Delete visit | -| DELETE | `/soa/{soa_id}/activities/{activity_id}` | JSON | Delete activity | -| DELETE | `/soa/{soa_id}/epochs/{epoch_id}` | JSON | Delete epoch | - -If any endpoint returns a non-JSON payload (Excel, CSV, HTML), error responses still use the FastAPI JSON error model. Consider adding explicit OpenAPI tags & descriptions for improved generated docs. -| GET | `/soa/{soa_id}/matrix` | JSON | Raw matrix (visits/activities/cells) | -| POST | `/soa/{soa_id}/matrix/import` | JSON | Bulk import matrix | -| GET | `/soa/{soa_id}/export/xlsx` | Binary | Excel workbook export | -| POST | `/soa/{soa_id}/visits` | JSON | Create visit | -| PATCH | `/soa/{soa_id}/visits/{visit_id}` | JSON | Update visit | -| GET | `/soa/{soa_id}/visits/{visit_id}` | JSON | Visit detail | -| DELETE | `/soa/{soa_id}/visits/{visit_id}` | JSON | Delete visit | -| POST | `/soa/{soa_id}/activities` | JSON | Create activity | -| PATCH | `/soa/{soa_id}/activities/{activity_id}` | JSON | Update activity | -| GET | `/soa/{soa_id}/activities/{activity_id}` | JSON | Activity detail | -| DELETE | `/soa/{soa_id}/activities/{activity_id}` | JSON | Delete activity | -| POST | `/soa/{soa_id}/activities/{activity_id}/concepts` | JSON | Assign concepts to activity | -| POST | `/soa/{soa_id}/activities/bulk` | JSON | Bulk add activities | -| POST | `/soa/{soa_id}/activities/reorder` | JSON | Reorder activities (global audit) | -| POST | `/soa/{soa_id}/epochs` | JSON | Create epoch | -| GET | `/soa/{soa_id}/epochs` | JSON | List epochs | -| GET | `/soa/{soa_id}/epochs/{epoch_id}` | JSON | Epoch detail | -| POST | `/soa/{soa_id}/epochs/{epoch_id}/metadata` | JSON | Update epoch metadata | -| DELETE | `/soa/{soa_id}/epochs/{epoch_id}` | JSON | Delete epoch | -| POST | `/soa/{soa_id}/epochs/reorder` | JSON | Reorder epochs | -| GET | `/soa/{soa_id}/elements` | JSON | List elements | -| GET | `/soa/{soa_id}/elements/{element_id}` | JSON | Element detail | -| POST | `/soa/{soa_id}/elements` | JSON | Create element | -| PATCH | `/soa/{soa_id}/elements/{element_id}` | JSON | Update element | -| DELETE | `/soa/{soa_id}/elements/{element_id}` | JSON | Delete element | -| POST | `/soa/{soa_id}/elements/reorder` | JSON | Reorder elements | -| GET | `/soa/{soa_id}/element_audit` | JSON | Element audit log | -| GET | `/soa/{soa_id}/arms` | JSON | List arms | -| POST | `/soa/{soa_id}/arms` | JSON | Create arm | -| PATCH | `/soa/{soa_id}/arms/{arm_id}` | JSON | Update arm | -| DELETE | `/soa/{soa_id}/arms/{arm_id}` | JSON | Delete arm | -| POST | `/soa/{soa_id}/arms/reorder` | JSON | Reorder arms | -| GET | `/soa/{soa_id}/arm_audit` | JSON | Arm audit log | -| POST | `/soa/{soa_id}/visits/reorder` | JSON | Reorder visits | -| POST | `/soa/{soa_id}/activities/reorder` | JSON | Reorder activities | -| POST | `/soa/{soa_id}/epochs/reorder` | JSON | Reorder epochs | -| GET | `/soa/{soa_id}/rollback_audit` | JSON | Rollback audit log | -| GET | `/soa/{soa_id}/reorder_audit` | JSON | Global reorder audit log | -| GET | `/soa/{soa_id}/rollback_audit/export/xlsx` | Binary | Rollback audit Excel | -| GET | `/soa/{soa_id}/reorder_audit/export/xlsx` | Binary | Reorder audit Excel | -| GET | `/soa/{soa_id}/reorder_audit/export/csv` | CSV | Reorder audit CSV | -| POST | `/soa/{soa_id}/cells` | JSON | Upsert cell | -| GET | `/soa/{soa_id}/matrix` | JSON | (Duplicate listing for completeness) | -| GET | `/soa/{soa_id}/normalized` | JSON | (Duplicate listing for completeness) | -| POST | `/soa/{soa_id}/freeze/{freeze_id}/rollback` | HTML | Rollback via UI | -| GET | `/soa/{soa_id}/freeze/{freeze_id}` | JSON | Freeze snapshot | -| GET | `/ui/soa/{soa_id}/freeze/{freeze_id}/view` | HTML | Modal freeze view | -| GET | `/ui/soa/{soa_id}/freeze/diff` | HTML | Freeze diff view | -| GET | `/soa/{soa_id}/freeze/diff.json` | JSON | Freeze diff JSON | -| POST | `/ui/soa/{soa_id}/freeze` | HTML | Create freeze (form) | -| POST | `/ui/soa/{soa_id}/concepts_refresh` | HTML | Force concepts refresh | -| GET | `/ui/concepts` | HTML | Concepts list | -| GET | `/ui/concepts/{code}` | HTML | Concept detail | -| GET | `/ddf/terminology` | JSON | DDF terminology query | -| POST | `/admin/load_ddf_terminology` | JSON | Load DDF terminology | -| GET | `/ui/ddf/terminology` | HTML | DDF terminology UI | -| POST | `/ui/ddf/terminology/upload` | HTML | Upload DDF terminology | -| GET | `/ddf/terminology/audit` | JSON | DDF audit list | -| GET | `/ddf/terminology/audit/export.csv` | CSV | DDF audit CSV export | -| GET | `/ddf/terminology/audit/export.json` | JSON | DDF audit JSON export | -| GET | `/ui/ddf/terminology/audit` | HTML | DDF audit UI | -| GET | `/protocol/terminology` | JSON | Protocol terminology query | -| POST | `/admin/load_protocol_terminology` | JSON | Load Protocol terminology | -| GET | `/ui/protocol/terminology` | HTML | Protocol terminology UI | -| POST | `/ui/protocol/terminology/upload` | HTML | Upload Protocol terminology | -| GET | `/protocol/terminology/audit` | JSON | Protocol audit list | -| GET | `/protocol/terminology/audit/export.csv` | CSV | Protocol audit CSV export | -| GET | `/protocol/terminology/audit/export.json` | JSON | Protocol audit JSON export | -| GET | `/ui/protocol/terminology/audit` | HTML | Protocol audit UI | -| POST | `/ui/soa/create` | HTML | Create study (form) | -| POST | `/ui/soa/{soa_id}/update_meta` | HTML | Update metadata (form) | -| GET | `/ui/soa/{soa_id}/edit` | HTML | Editing interface | -| POST | `/ui/soa/{soa_id}/add_visit` | HTML | Add visit (form) | -| POST | `/ui/soa/{soa_id}/delete_visit` | HTML | Delete visit (HTMX) | -| POST | `/ui/soa/{soa_id}/reorder_visits` | HTML | Reorder visits (form) | -| POST | `/ui/soa/{soa_id}/set_visit_epoch` | HTML | Assign epoch to visit | -| POST | `/ui/soa/{soa_id}/add_activity` | HTML | Add activity (form) | -| POST | `/ui/soa/{soa_id}/delete_activity` | HTML | Delete activity (HTMX) | -| POST | `/ui/soa/{soa_id}/reorder_activities` | HTML | Reorder activities (form) | -| POST | `/ui/soa/{soa_id}/add_epoch` | HTML | Add epoch (form) | -| POST | `/ui/soa/{soa_id}/update_epoch` | HTML | Update epoch (form) | -| POST | `/ui/soa/{soa_id}/delete_epoch` | HTML | Delete epoch (form) | -| POST | `/ui/soa/{soa_id}/reorder_epochs` | HTML | Reorder epochs (form) | -| POST | `/ui/soa/{soa_id}/add_element` | HTML | Add element (form) | -| POST | `/ui/soa/{soa_id}/update_element` | HTML | Update element (form) | -| POST | `/ui/soa/{soa_id}/delete_element` | HTML | Delete element (form) | -| POST | `/ui/soa/{soa_id}/reorder_elements` | HTML | Reorder elements (form) | -| POST | `/ui/soa/{soa_id}/add_arm` | HTML | Add arm (form) | -| POST | `/ui/soa/{soa_id}/update_arm` | HTML | Update arm (form) | -| POST | `/ui/soa/{soa_id}/delete_arm` | HTML | Delete arm (form) | -| POST | `/ui/soa/{soa_id}/reorder_arms` | HTML | Reorder arms (form) | -| POST | `/ui/soa/{soa_id}/toggle_cell` | HTML | Toggle cell (HTMX) | -| POST | `/ui/soa/{soa_id}/set_cell` | HTML | Set cell status (HTMX) | - -> Not Implemented Endpoints (listed earlier conceptually): per-entity JSON audit endpoints for visits, activities, and epochs (`/soa/{soa_id}/visit_audit`, `/soa/{soa_id}/activity_audit`, `/soa/{soa_id}/epoch_audit`) were described but are not present in code as of this generation. +## Curl Examples + +### Basic Workflow +```bash +# 1. Create a study +RESPONSE=$(curl -s -X POST http://localhost:8000/soa \ + -H 'Content-Type: application/json' \ + -d '{"name":"Phase II Trial","study_id":"TRIAL-2026-001"}') +SOA_ID=$(echo $RESPONSE | jq -r '.id') +echo "Created SoA ID: $SOA_ID" + +# 2. Add visits +curl -s -X POST http://localhost:8000/soa/$SOA_ID/visits \ + -H 'Content-Type: application/json' \ + -d '{"name":"Screening"}' + +curl -s -X POST http://localhost:8000/soa/$SOA_ID/visits \ + -H 'Content-Type: application/json' \ + -d '{"name":"Baseline"}' + +# 3. Add activities +curl -s -X POST http://localhost:8000/activities \ + -H 'Content-Type: application/json' \ + -d '{"name":"Physical Exam"}' + +curl -s -X POST http://localhost:8000/activities \ + -H 'Content-Type: application/json' \ + -d '{"name":"Vital Signs"}' + +# 4. Bulk add activities +curl -s -X POST http://localhost:8000/activities/bulk \ + -H 'Content-Type: application/json' \ + -d '{"names":["ECG","Labs","Imaging"]}' + +# 5. Create epochs +curl -s -X POST http://localhost:8000/soa/$SOA_ID/epochs \ + -H 'Content-Type: application/json' \ + -d '{"name":"Screening","epoch_label":"SCR"}' + +# 6. Create arms +curl -s -X POST http://localhost:8000/soa/$SOA_ID/arms \ + -H 'Content-Type: application/json' \ + -d '{"name":"Treatment A","type":"Experimental"}' + +# 7. Create elements +curl -s -X POST http://localhost:8000/elements \ + -H 'Content-Type: application/json' \ + -d '{"name":"Screening Period","label":"SCR_PERIOD"}' + +# 8. Get matrix +curl -s http://localhost:8000/soa/$SOA_ID/matrix | jq + +# 9. Export to Excel +curl -O http://localhost:8000/soa/$SOA_ID/export/xlsx + +# 10. Create freeze +curl -s -X POST http://localhost:8000/ui/soa/$SOA_ID/freeze \ + -d 'version_label=v1.0' +``` + +### Advanced Operations +```bash +# Reorder elements +curl -X POST http://localhost:8000/elements/reorder \ + -H 'Content-Type: application/json' \ + -d '[3,1,2]' + +# Assign biomedical concepts to activity +curl -X POST http://localhost:8000/soa/1/activities/5/concepts \ + -H 'Content-Type: application/json' \ + -d '{"concept_codes":["C25473","C16960"]}' + +# Update epoch metadata +curl -X POST http://localhost:8000/soa/1/epochs/2/metadata \ + -H 'Content-Type: application/json' \ + -d '{"name":"Treatment Phase","epoch_label":"TRT","type":"TREATMENT"}' + +# Get audit logs +curl -s http://localhost:8000/soa/1/element_audit | jq +curl -s http://localhost:8000/soa/1/reorder_audit | jq + +# Export audits +curl -O http://localhost:8000/soa/1/rollback_audit/export/xlsx +curl -O http://localhost:8000/soa/1/reorder_audit/export/csv + +# Compare freezes +curl -s 'http://localhost:8000/soa/1/freeze/diff.json?left=5&right=7' | jq + +# Query DDF terminology +curl -s --get 'http://localhost:8000/ddf/terminology' \ + --data-urlencode 'codelist_code=C139020' | jq + +# Search protocol terminology +curl -s --get 'http://localhost:8000/protocol/terminology' \ + --data-urlencode 'search=trial' \ + --data-urlencode 'limit=5' | jq +``` + +--- +## Quick Reference Card + +### Most Common Endpoints +| Operation | Method | Endpoint | +|-----------|--------|----------| +| List studies | GET | `/` | +| Create study | POST | `/soa` | +| Edit study | GET | `/ui/soa/{id}/edit` | +| Get matrix | GET | `/soa/{id}/matrix` | +| Export Excel | GET | `/soa/{id}/export/xlsx` | +| Create visit | POST | `/soa/{id}/visits` | +| Create activity | POST | `/activities` | +| Create epoch | POST | `/soa/{id}/epochs` | +| Create arm | POST | `/soa/{id}/arms` | +| Create element | POST | `/elements` | +| Reorder entities | POST | `/elements/reorder`, `/soa/{id}/visits/reorder`, etc. | +| Create freeze | POST | `/ui/soa/{id}/freeze` | +| View audits | GET | `/ui/soa/{id}/audits` | +| Concepts list | GET | `/ui/concepts` | + +### Response Patterns +- **Success**: HTTP 200/201 with JSON body +- **Create**: Returns `{"id": N, ...}` with entity ID +- **Update**: Returns `{"updated_fields": [...]}` for partial updates +- **Reorder**: Returns `{"message": "...", "new_order": [...]}` +- **Delete**: Returns `{"message": "...deleted"}` with cascade info +- **Error**: HTTP 400/404/422 with `{"detail": "..."}` + +--- +## Full Endpoint Inventory + +See **`docs/api_endpoints.csv`** for complete sortable/filterable list of all 165 endpoints with: +- Method (GET/POST/PATCH/DELETE) +- Path with parameters +- Type (API/UI/Admin) +- Description +- Response type (JSON/HTML/Binary) + +--- +*Last Updated: January 20, 2026* +*Version: 4.0* diff --git a/docs/api_endpoints.csv b/docs/api_endpoints.csv new file mode 100644 index 0000000..14fc273 --- /dev/null +++ b/docs/api_endpoints.csv @@ -0,0 +1,157 @@ +Method,Path,Type,Description,Response Type +GET,/,UI,Index page - lists studies & create form,HTML +POST,/soa,API,Create new SoA container,JSON +GET,/soa/{soa_id},API,Get SoA summary (visits activities counts),JSON +POST,/soa/{soa_id}/metadata,API,Update study metadata fields,JSON +GET,/soa/{soa_id}/normalized,API,Get normalized SoA JSON,JSON +GET,/soa/{soa_id}/matrix,API,Get raw matrix (visits activities cells),JSON +POST,/soa/{soa_id}/matrix/import,API,Bulk import matrix data,JSON +GET,/soa/{soa_id}/export/xlsx,API,Download Excel workbook,Binary (XLSX) +GET,/soa/{soa_id}/export/pdf,API,Download PDF report,Binary (PDF) +POST,/ui/soa/create,UI,Create SoA via form,HTML +POST,/ui/soa/{soa_id}/update_meta,UI,Update study metadata via form,HTML +GET,/ui/soa/{soa_id}/edit,UI,Edit SoA page,HTML +GET,/soa/{soa_id}/visits,API,List visits for SoA,JSON +GET,/ui/soa/{soa_id}/visits,UI,List visits page,HTML +GET,/soa/visits/{visit_id},API,Get visit detail,JSON +POST,/soa/{soa_id}/visits,API,Create visit,JSON +PATCH,/soa/{soa_id}/visits/{visit_id},API,Update visit (partial),JSON +DELETE,/soa/{soa_id}/visits/{visit_id},API,Delete visit and cells,JSON +POST,/soa/{soa_id}/visits/reorder,API,Reorder visits,JSON +POST,/ui/soa/{soa_id}/visits/create,UI,Create visit via form,HTML +POST,/ui/soa/{soa_id}/visits/{visit_id}/update,UI,Update visit via form,HTML +POST,/ui/soa/{soa_id}/visits/{visit_id}/delete,UI,Delete visit via form,HTML +POST,/ui/soa/{soa_id}/add_visit,UI,Add visit via form (legacy),HTML +POST,/ui/soa/{soa_id}/update_visit,UI,Update visit (legacy),HTML +POST,/ui/soa/{soa_id}/delete_visit,UI,Delete visit (legacy),HTML +POST,/ui/soa/{soa_id}/reorder_visits,UI,Reorder visits drag-drop,HTML +POST,/ui/soa/{soa_id}/set_visit_epoch,UI,Set visit epoch via form,HTML +POST,/ui/soa/{soa_id}/set_visit_transition_end_rule,UI,Set visit transition end rule,HTML +POST,/visits/reorder,API,Reorder visits (router),JSON +GET,/activities,API,List activities,JSON +GET,/activities/{activity_id},API,Get activity detail,JSON +POST,/activities,API,Create activity,JSON +PATCH,/activities/{activity_id},API,Update activity (partial),JSON +DELETE,/soa/{soa_id}/activities/{activity_id},API,Delete activity and concepts,JSON +POST,/activities/bulk,API,Bulk add activities,JSON +POST,/soa/{soa_id}/activities/{activity_id}/concepts,API,Set activity concepts,JSON +POST,/activities/{activity_id}/concepts,API,Set activity concepts (router),JSON +POST,/soa/{soa_id}/activities/reorder,API,Reorder activities,JSON +POST,/activities/reorder,API,Reorder activities (router),JSON +POST,/activities/add,UI,Add activity via form (router),HTML +POST,/activities/{activity_id}/update,UI,Update activity via form (router),HTML +POST,/ui/soa/{soa_id}/add_activity,UI,Add activity via form,HTML +POST,/ui/soa/{soa_id}/delete_activity,UI,Delete activity via form,HTML +POST,/ui/soa/{soa_id}/reorder_activities,UI,Reorder activities drag-drop,HTML +GET,/soa/{soa_id}/epochs,API,List epochs,JSON +GET,/ui/soa/{soa_id}/epochs,UI,List epochs page,HTML +GET,/soa/{soa_id}/epochs/{epoch_id},API,Get epoch detail,JSON +POST,/soa/{soa_id}/epochs,API,Create epoch,JSON +POST,/soa/{soa_id}/epochs/{epoch_id}/metadata,API,Update epoch metadata,JSON +PATCH,/soa/{soa_id}/epochs/{epoch_id},API,Update epoch (partial),JSON +DELETE,/soa/{soa_id}/epochs/{epoch_id},API,Delete epoch,JSON +POST,/soa/{soa_id}/epochs/reorder,API,Reorder epochs,JSON +POST,/ui/soa/{soa_id}/epochs/create,UI,Create epoch via form,HTML +POST,/ui/soa/{soa_id}/epochs/{epoch_id}/update,UI,Update epoch via form,HTML +POST,/ui/soa/{soa_id}/epochs/{epoch_id}/delete,UI,Delete epoch via form,HTML +POST,/ui/soa/{soa_id}/add_epoch,UI,Add epoch via form (legacy),HTML +POST,/ui/soa/{soa_id}/update_epoch,UI,Update epoch (legacy),HTML +POST,/ui/soa/{soa_id}/delete_epoch,UI,Delete epoch (legacy),HTML +POST,/ui/soa/{soa_id}/reorder_epochs,UI,Reorder epochs drag-drop,HTML +GET,/soa/{soa_id}/arms,API,List arms,JSON +GET,/ui/soa/{soa_id}/arms,UI,List arms page,HTML +POST,/soa/{soa_id}/arms,API,Create arm,JSON +PATCH,/soa/{soa_id}/arms/{arm_id},API,Update arm (partial),JSON +POST,/arms/reorder,API,Reorder arms,JSON +POST,/ui/soa/{soa_id}/arms/create,UI,Create arm via form,HTML +POST,/ui/soa/{soa_id}/arms/{arm_id}/update,UI,Update arm via form,HTML +POST,/ui/soa/{soa_id}/arms/{arm_id}/delete,UI,Delete arm via form,HTML +POST,/ui/soa/{soa_id}/add_arm,UI,Add arm via form (legacy),HTML +POST,/ui/soa/{soa_id}/update_arm,UI,Update arm (legacy),HTML +POST,/ui/soa/{soa_id}/delete_arm,UI,Delete arm (legacy),HTML +POST,/ui/soa/{soa_id}/reorder_arms,UI,Reorder arms drag-drop,HTML +GET,/soa/{soa_id}/elements,API,List elements,JSON +GET,/ui/soa/{soa_id}/elements,UI,List elements page,HTML +GET,/soa/{soa_id}/elements/{element_id},API,Get element detail,HTML +POST,/elements,API,Create element,JSON +PATCH,/soa/{soa_id}/elements/{element_id},API,Update element (partial),JSON +PATCH,/elements/{element_id},API,Update element (router),JSON +DELETE,/elements/{element_id},API,Delete element,JSON +POST,/elements/reorder,API,Reorder elements,JSON +GET,/soa/{soa_id}/element_audit,API,Get element audit log,JSON +POST,/ui/soa/{soa_id}/elements/create,UI,Create element via form,HTML +POST,/ui/soa/{soa_id}/elements/{element_id}/update,UI,Update element via form,HTML +POST,/ui/soa/{soa_id}/elements/{element_id}/delete,UI,Delete element via form,HTML +POST,/ui/soa/{soa_id}/add_element,UI,Add element (legacy),HTML +POST,/ui/soa/{soa_id}/update_element,UI,Update element (legacy),HTML +POST,/ui/soa/{soa_id}/delete_element,UI,Delete element (legacy),HTML +GET,/soa/{soa_id}/instances,API,List instances,JSON +GET,/ui/soa/{soa_id}/instances,UI,List instances page,HTML +POST,/ui/soa/{soa_id}/instances/create,UI,Create instance via form,HTML +POST,/ui/soa/{soa_id}/instances/{instance_id}/update,UI,Update instance via form,HTML +POST,/ui/soa/{soa_id}/instances/{instance_id}/delete,UI,Delete instance via form,HTML +GET,/ui/soa/{soa_id}/schedule_timelines,UI,List schedule timelines page,HTML +POST,/ui/soa/{soa_id}/schedule_timelines/create,UI,Create schedule timeline via form,HTML +POST,/ui/soa/{soa_id}/schedule_timelines/{schedule_timeline_id}/update,UI,Update schedule timeline via form,HTML +POST,/ui/soa/{soa_id}/schedule_timelines/{schedule_timeline_id}/delete,UI,Delete schedule timeline via form,HTML +GET,/soa/{soa_id}/timings,API,List timings,JSON +GET,/ui/soa/{soa_id}/timings,UI,List timings page,HTML +GET,/soa/{soa_id}/timing_audit,API,Get timing audit log,JSON +POST,/ui/soa/{soa_id}/timings/create,UI,Create timing via form,HTML +POST,/ui/soa/{soa_id}/timings/{timing_id}/update,UI,Update timing via form,HTML +POST,/ui/soa/{soa_id}/timings/{timing_id}/delete,UI,Delete timing via form,HTML +POST,/ui/soa/{soa_id}/set_timing,UI,Set timing (legacy),HTML +GET,/soa/{soa_id}/rules,API,List transition rules,JSON +GET,/ui/soa/{soa_id}/rules,UI,List rules page,HTML +PATCH,/soa/{soa_id}/rules/{rule_id},API,Update rule (partial),JSON +POST,/ui/soa/{soa_id}/rules/create,UI,Create rule via form,HTML +POST,/ui/soa/{soa_id}/rules/{rule_id}/update,UI,Update rule via form,HTML +POST,/ui/soa/{soa_id}/rules/{rule_id}/delete,UI,Delete rule via form,HTML +POST,/soa/{soa_id}/cells,API,Create matrix cell,JSON +POST,/soa/{soa_id}/cells_instance,API,Create matrix cell with instance,JSON +POST,/ui/soa/{soa_id}/set_cell,UI,Set cell status,HTML +POST,/ui/soa/{soa_id}/toggle_cell,UI,Toggle cell status,HTML +POST,/ui/soa/{soa_id}/toggle_cell_instance,UI,Toggle cell instance status,HTML +POST,/ui/soa/{soa_id}/add_study_cell,UI,Add study cell,HTML +POST,/ui/soa/{soa_id}/update_study_cell,UI,Update study cell,HTML +POST,/ui/soa/{soa_id}/delete_study_cell,UI,Delete study cell,HTML +POST,/ui/soa/{soa_id}/freeze,UI,Create freeze snapshot,HTML +GET,/soa/{soa_id}/freeze/{freeze_id},API,Get freeze detail,JSON +GET,/ui/soa/{soa_id}/freeze/{freeze_id}/view,UI,View freeze modal,HTML +GET,/ui/soa/{soa_id}/freeze/diff,UI,Compare freezes modal,HTML +GET,/soa/{soa_id}/freeze/diff.json,API,Get freeze diff JSON,JSON +GET,/soa/{soa_id}/rollback_audit,API,Get rollback audit log,JSON +GET,/ui/soa/{soa_id}/rollback_audit,UI,View rollback audit modal,HTML +GET,/soa/{soa_id}/rollback_audit/export/xlsx,API,Export rollback audit XLSX,Binary (XLSX) +GET,/soa/{soa_id}/reorder_audit,API,Get reorder audit log,JSON +GET,/ui/soa/{soa_id}/reorder_audit,UI,View reorder audit modal,HTML +GET,/soa/{soa_id}/reorder_audit/export/csv,API,Export reorder audit CSV,Binary (CSV) +GET,/soa/{soa_id}/reorder_audit/export/xlsx,API,Export reorder audit XLSX,Binary (XLSX) +GET,/ui/soa/{soa_id}/audits,UI,View audits page,HTML +GET,/concepts/status,API,Get concepts cache status,JSON +GET,/ui/concepts,UI,List biomedical concepts,HTML +GET,/ui/concepts/{code},UI,View concept detail,HTML +POST,/ui/soa/{soa_id}/concepts_refresh,UI,Force refresh concepts cache,HTML +GET,/ui/concept_categories,UI,List concept categories,HTML +GET,/ui/concept_categories/view,UI,View concepts by category,HTML +GET,/sdtm/specializations/status,API,Get SDTM specializations status,JSON +GET,/ui/sdtm/specializations/status,UI,View SDTM status page,HTML +POST,/ui/sdtm/specializations/refresh,UI,Refresh SDTM specializations,HTML +GET,/ui/sdtm/specializations,UI,List SDTM specializations,HTML +GET,/ui/sdtm/specializations/{idx},UI,View SDTM specialization detail,HTML +POST,/admin/load_ddf_terminology,Admin,Load DDF terminology from file,JSON +GET,/ddf/terminology,API,Get DDF terminology entries,JSON +GET,/ui/ddf/terminology,UI,View DDF terminology page,HTML +POST,/ui/ddf/terminology/upload,UI,Upload DDF terminology spreadsheet,HTML +GET,/ddf/terminology/audit,API,Get DDF terminology audit,JSON +GET,/ddf/terminology/audit/export.csv,API,Export DDF audit CSV,Binary (CSV) +GET,/ddf/terminology/audit/export.json,API,Export DDF audit JSON,JSON +GET,/ui/ddf/terminology/audit,UI,View DDF terminology audit page,HTML +POST,/admin/load_protocol_terminology,Admin,Load protocol terminology from file,JSON +GET,/protocol/terminology,API,Get protocol terminology entries,JSON +GET,/ui/protocol/terminology,UI,View protocol terminology page,HTML +POST,/ui/protocol/terminology/upload,UI,Upload protocol terminology spreadsheet,HTML +GET,/protocol/terminology/audit,API,Get protocol terminology audit,JSON +GET,/protocol/terminology/audit/export.csv,API,Export protocol audit CSV,Binary (CSV) +GET,/protocol/terminology/audit/export.json,API,Export protocol audit JSON,JSON +GET,/ui/protocol/terminology/audit,UI,View protocol terminology audit page,HTML diff --git a/docs/api_endpoints.xlsx b/docs/api_endpoints.xlsx deleted file mode 100644 index f49a332..0000000 Binary files a/docs/api_endpoints.xlsx and /dev/null differ diff --git a/src/soa_builder/web/app.py b/src/soa_builder/web/app.py index af114a0..667a1ff 100644 --- a/src/soa_builder/web/app.py +++ b/src/soa_builder/web/app.py @@ -2522,31 +2522,43 @@ def export_xlsx(soa_id: int, left: Optional[int] = None, right: Optional[int] = ) concepts_map = {} concepts_uids_map = {} + code_uid_map = {} # Map (activity_id, code) -> uid for aid, code, title, cuid in cur.fetchall(): - concepts_map.setdefault(aid, {})[code] = title + # Use title if available, otherwise use concept_uid as fallback, then code + display_title = title if title else (cuid if cuid else code) + concepts_map.setdefault(aid, {})[code] = display_title if cuid: concepts_uids_map.setdefault(aid, set()).add(cuid) + code_uid_map[(aid, code)] = cuid conn.close() visits, activities, _cells = _fetch_matrix(soa_id) activity_ids_in_order = [a["id"] for a in activities] # Build display strings using EffectiveTitle (override if present) and show code in parentheses concepts_strings = [] - concept_uids_strings = [] + concept_titles_strings = [] # For Concept UIDs column, show titles with UIDs for aid in activity_ids_in_order: cmap = concepts_map.get(aid, {}) cuids = concepts_uids_map.get(aid, set()) if not cmap: concepts_strings.append("") - concept_uids_strings.append("") + concept_titles_strings.append("") continue items = sorted(cmap.items(), key=lambda kv: kv[1].lower()) concepts_strings.append( "; ".join([f"{title} ({code})" for code, title in items]) ) - concept_uids_strings.append(", ".join(sorted(list(cuids))) if cuids else "") + # For Concept UIDs column, show title with UID in parentheses + titles_with_uids = [] + for code, title in items: + uid = code_uid_map.get((aid, code)) + if uid: + titles_with_uids.append(f"{title} ({uid})") + else: + titles_with_uids.append(title) + concept_titles_strings.append("; ".join(titles_with_uids)) if len(concepts_strings) == len(df): df.insert(1, "Concepts", concepts_strings) - df["Concept UIDs"] = concept_uids_strings + df["Concept UIDs"] = concept_titles_strings # Build concept mappings sheet data mapping_rows = [] for a in activities: @@ -3574,6 +3586,7 @@ def ui_edit(request: Request, soa_id: int): i.name, i.instance_uid, i.label, + i.member_of_timeline, (SELECT t.name FROM schedule_timelines t WHERE t.schedule_timeline_uid = i.member_of_timeline @@ -3622,17 +3635,61 @@ def ui_edit(request: Request, soa_id: int): "name": r[1], "instance_uid": r[2], "label": r[3], - "timeline_name": r[4], - "encounter_name": r[5], - "epoch_name": r[6], - "window_label": r[7], - "timing_label": r[8], - "study_day": iso_duration_to_days(r[9]), + "member_of_timeline": r[4], + "timeline_name": r[5], + "encounter_name": r[6], + "epoch_name": r[7], + "window_label": r[8], + "timing_label": r[9], + "study_day": iso_duration_to_days(r[10]), } for r in cur_inst.fetchall() ] cur_inst.close() + # Load Schedule Timelines for timeline selector + conn_tl = _connect() + cur_tl = conn_tl.cursor() + cur_tl.execute( + """ + SELECT schedule_timeline_uid,name,main_timeline + FROM schedule_timelines + WHERE soa_id=? + ORDER BY main_timeline DESC, name + """, + (soa_id,), + ) + timelines = [ + { + "schedule_timeline_uid": r[0], + "name": r[1], + "main_timeline": bool(r[2]), + } + for r in cur_tl.fetchall() + ] + conn_tl.close() + + # Group instances by timeline + instances_by_timeline = {} + for inst in instances: + timeline_key = inst.get("member_of_timeline") or "unassigned" + if timeline_key not in instances_by_timeline: + instances_by_timeline[timeline_key] = [] + instances_by_timeline[timeline_key].append(inst) + + # Determine default timeline (main_timeline or first available) + default_timeline = None + for tl in timelines: + if tl["main_timeline"]: + default_timeline = tl["schedule_timeline_uid"] + break + if not default_timeline and timelines: + default_timeline = timelines[0]["schedule_timeline_uid"] + + # If no default timeline found or no timelines exist, check if there are unassigned instances + if not default_timeline and "unassigned" in instances_by_timeline: + default_timeline = "unassigned" + return templates.TemplateResponse( request, "edit.html", @@ -3662,6 +3719,9 @@ def ui_edit(request: Request, soa_id: int): "study_cells": study_cells, "transition_rules": transition_rules, "timings": timings, + "timelines": timelines, + "instances_by_timeline": instances_by_timeline, + "default_timeline": default_timeline, }, ) diff --git a/src/soa_builder/web/initialize_database.py b/src/soa_builder/web/initialize_database.py index 8386351..aa178f0 100644 --- a/src/soa_builder/web/initialize_database.py +++ b/src/soa_builder/web/initialize_database.py @@ -1,40 +1,41 @@ from .db import _connect +"""Script to check and create all database tables required for application""" + def _init_db(): conn = _connect() cur = conn.cursor() + + # activity cur.execute( - """CREATE TABLE IF NOT EXISTS soa ( - id INTEGER PRIMARY KEY AUTOINCREMENT, - name TEXT, - created_at TEXT - )""" - ) - cur.execute( - """CREATE TABLE IF NOT EXISTS visit ( + """CREATE TABLE IF NOT EXISTS activity ( id INTEGER PRIMARY KEY AUTOINCREMENT, soa_id INTEGER, name TEXT, - label TEXT, order_index INTEGER, - epoch_id INTEGER, - encounter_uid TEXT, + activity_uid TEXT, -- immutable Activity_N identifier unique within an SOA + label TEXT, description TEXT, - UNIQUE(soa_id,encounter_uid) + UNIQUE(soa_id,activity_uid) )""" ) + + # activity_concept + # Mapping table linking activities to biomedical concepts (concept_code + title stored for snapshot purposes) cur.execute( - """CREATE TABLE IF NOT EXISTS activity ( + """CREATE TABLE IF NOT EXISTS activity_concept ( id INTEGER PRIMARY KEY AUTOINCREMENT, - soa_id INTEGER, - name TEXT, - order_index INTEGER, - activity_uid TEXT, -- immutable Activity_N identifier unique within an SOA - label TEXT, - description TEXT + activity_id INTEGER, + concept_code TEXT, + concept_title TEXT, + concept_uid TEXT, -- immutable BiomedicalConcept_N identifier unique within an SOA + activity_uid TEXT, -- joins to the activity table using this uid unique within an SOA + soa_id INT )""" ) + + # arm # Arms: groupings similar to Visits. (Legacy element linkage removed; schema now only stores intrinsic fields.) cur.execute( """CREATE TABLE IF NOT EXISTS arm ( @@ -49,20 +50,204 @@ def _init_db(): arm_uid TEXT -- immutable StudyArm_N identifier unique within an SOA )""" ) + + # cell + # Matrix cells table (renamed from legacy 'cell') + cur.execute( + """CREATE TABLE IF NOT EXISTS matrix_cells ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + soa_id INTEGER, + visit_id INTEGER, + activity_id INTEGER, + status TEXT + )""" + ) + + # code + # create the code table to store unique Code_uid values associated with study objects + cur.execute( + """CREATE TABLE IF NOT EXISTS code ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + soa_id INTEGER NOT NULL, + code_uid TEXT, -- immutable Code_N identifier unique within an SOA + codelist_table TEXT, + codelist_code TEXT NOT NULL, + code TEXT NOT NULL, + UNIQUE(soa_id, code_uid) + )""" + ) + + # ddf_terminology: this table is created dynamically when uploading a new DDF Terminology + # spreadsheet (app.py:5179-5545) + + # element # Elements: finer-grained structural units (optional) that can also be ordered cur.execute( """CREATE TABLE IF NOT EXISTS element ( id INTEGER PRIMARY KEY AUTOINCREMENT, soa_id INTEGER NOT NULL, + element_id TEXT, name TEXT NOT NULL, label TEXT, description TEXT, testrl TEXT, teenrl TEXT, order_index INTEGER, - created_at TEXT + created_at TEXT, + UNIQUE(soa_id,element_id) + )""" + ) + + # epoch + # Epochs: high-level study phase grouping (optional). Behaves like visits/activities list ordering. + cur.execute( + """CREATE TABLE IF NOT EXISTS epoch ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + soa_id INTEGER, + name TEXT, + order_index INTEGER, + epoch_seq INTEGER, + epoch_label TEXT, + epoch_description TEXT, + type TEXT, + epoch_uid TEXT, + UNIQUE (soa_id, epoch_uid) + )""" + ) + + # instance + # create instances table + cur.execute( + """CREATE TABLE IF NOT EXISTS instances ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + soa_id INT NOT NULL, + instance_uid TEXT NOT NULL, -- immutable ScheduledActivityInstance_N identifier unique within SOA + name TEXT NOT NULL, + label TEXT, + description TEXT, + default_condition_uid TEXT, + epoch_uid TEXT, + timeline_id TEXT, + timeline_exit_id TEXT, + order_index INT, + encounter_uid TEXT, + UNIQUE(soa_id, instance_uid) + )""" + ) + + # protocol_terminology: this table is created dynamically when uploading a new Protocol Terminology + # (app.py:5781-6119) + + # schedule_timelines + # create schedule_timelines table + cur.execute( + """CREATE TABLE IF NOT EXISTS schedule_timelines ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + soa_id INT NOT NULL, + schedule_timeline_uid TEXT NOT NULL, -- immutable ScheduleTimeline_N identifier unique within SOA + name TEXT NOT NULL, + label TEXT, + description TEXT, + main_timeline INT, -- 1=True|0=False + entry_condition TEXT, + entry_id, -- dropdown select for ScheduledActivityInstance_ + exit_id TEXT, + order_index INT, + UNIQUE(soa_id, schedule_timeline_uid) + )""" + ) + + # soa + cur.execute( + """CREATE TABLE IF NOT EXISTS soa ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + name TEXT, + created_at TEXT, + study_id TEXT, + study_label TEXT, + study_description TEXT + )""" + ) + + # study_cell + # create the study_cell table to store the relationship between Epoch, Arm and related elements + cur.execute( + """CREATE TABLE IF NOT EXISTS study_cell ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + soa_id INTEGER NOT NULL, + study_cell_uid TEXT NOT NULL, --immutable StudyCell_N identifier unique within SOA + arm_uid TEXT NOT NULL, + epoch_uid TEXT NOT NULL, + element_uid TEXT NOT NULL + )""" + ) + + # timing + # create the timing table + cur.execute( + """CREATE TABLE IF NOT EXISTS timing ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + soa_id INTEGER NOT NULL, + timing_uid TEXT NOT NULL, -- immutable Timing_N identifier unique within SOA + name TEXT NOT NULL, + label TEXT, + description TEXT, + type TEXT, -- value chosen from submissionValue in codelist_code C201264 + value TEXT, + value_label TEXT, + relative_to_from TEXT, -- value chosen from submissionValue in codelist_code C201265 + relative_from_schedule_instance TEXT, + relative_to_schedule_instance TEXT, + window_label TEXT, + window_upper TEXT, + window_lower TEXT, + order_index INTEGER, + member_of_timeline TEXT, + UNIQUE(soa_id, timing_uid) + )""" + ) + + # transition_rule + # create the transition_rule table to store the transition rules for elements, encounters + cur.execute( + """CREATE TABLE IF NOT EXISTS transition_rule ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + soa_id INTEGER NOT NULL, + transition_rule_uid TEXT NOT NULL, --immutable TransitionRule_N identifier unique within SOA + name TEXT NOT NULL, + label TEXT, + description TEXT, + text TEXT, + order_index INTEGER, + created_at TEXT, + UNIQUE(soa_id, transition_rule_uid) )""" ) + + # visit + # Encounters + cur.execute( + """CREATE TABLE IF NOT EXISTS visit ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + soa_id INTEGER, + name TEXT, + label TEXT, + order_index INTEGER, + epoch_id INTEGER, + encounter_uid TEXT, + description TEXT, + type TEXT, + environmentalSettings TEXT, + contactModes TEXT, + transitionStartRule TEXT, + transitionEndRule TEXT, + scheduledAtId TEXT, + UNIQUE(soa_id,encounter_uid) + )""" + ) + + # AUDIT TABLES FOR TRACKING ALL CHANGES TO ENTITIES + # Element audit table capturing create/update/delete operations cur.execute( """CREATE TABLE IF NOT EXISTS element_audit ( @@ -147,51 +332,7 @@ def _init_db(): performed_at TEXT NOT NULL )""" ) - # Epochs: high-level study phase grouping (optional). Behaves like visits/activities list ordering. - cur.execute( - """CREATE TABLE IF NOT EXISTS epoch ( - id INTEGER PRIMARY KEY AUTOINCREMENT, - soa_id INTEGER, - name TEXT, - order_index INTEGER - )""" - ) - # Matrix cells table (renamed from legacy 'cell') - cur.execute( - """CREATE TABLE IF NOT EXISTS matrix_cells ( - id INTEGER PRIMARY KEY AUTOINCREMENT, - soa_id INTEGER, - visit_id INTEGER, - activity_id INTEGER, - status TEXT - )""" - ) - # Mapping table linking activities to biomedical concepts (concept_code + title stored for snapshot purposes) - cur.execute( - """CREATE TABLE IF NOT EXISTS activity_concept ( - id INTEGER PRIMARY KEY AUTOINCREMENT, - activity_id INTEGER, - concept_code TEXT, - concept_title TEXT, - concept_uid TEXT, -- immutable BiomedicalConcept_N identifier unique within an SOA - activity_uid TEXT, -- joins to the activity table using this uid unique within an SOA - soa_id INT - )""" - ) - # Frozen versions (snapshot JSON of current matrix & concepts) - cur.execute( - """CREATE TABLE IF NOT EXISTS soa_freeze ( - id INTEGER PRIMARY KEY AUTOINCREMENT, - soa_id INTEGER, - version_label TEXT, - created_at TEXT, - snapshot_json TEXT - )""" - ) - # Unique index to enforce one label per SoA - cur.execute( - """CREATE UNIQUE INDEX IF NOT EXISTS idx_soafreeze_unique ON soa_freeze(soa_id, version_label)""" - ) + # Rollback audit log cur.execute( """CREATE TABLE IF NOT EXISTS rollback_audit ( @@ -218,70 +359,6 @@ def _init_db(): )""" ) - # create the code table to store unique Code_uid values associated with study objects - cur.execute( - """CREATE TABLE IF NOT EXISTS code ( - id INTEGER PRIMARY KEY AUTOINCREMENT, - soa_id INTEGER NOT NULL, - code_uid TEXT, -- immutable Code_N identifier unique within an SOA - codelist_table TEXT, - codelist_code TEXT NOT NULL, - code TEXT NOT NULL, - UNIQUE(soa_id, code_uid) - )""" - ) - - # create the study_cell table to store the relationship between Epoch, Arm and related elements - cur.execute( - """CREATE TABLE IF NOT EXISTS study_cell ( - id INTEGER PRIMARY KEY AUTOINCREMENT, - soa_id INTEGER NOT NULL, - study_cell_uid TEXT NOT NULL, --immutable StudyCell_N identifier unique within SOA - arm_uid TEXT NOT NULL, - epoch_uid TEXT NOT NULL, - element_uid TEXT NOT NULL - )""" - ) - - # create the transition_rule table to store the transition rules for elements, encounters - cur.execute( - """CREATE TABLE IF NOT EXISTS transition_rule ( - id INTEGER PRIMARY KEY AUTOINCREMENT, - soa_id INTEGER NOT NULL, - transition_rule_uid TEXT NOT NULL, --immutable TransitionRule_N identifier unique within SOA - name TEXT NOT NULL, - label TEXT, - description TEXT, - text TEXT, - order_index INTEGER, - created_at TEXT, - UNIQUE(soa_id, transition_rule_uid) - )""" - ) - - # create the timing table - cur.execute( - """CREATE TABLE IF NOT EXISTS timing ( - id INTEGER PRIMARY KEY AUTOINCREMENT, - soa_id INTEGER NOT NULL, - timing_uid TEXT NOT NULL, -- immutable Timing_N identifier unique within SOA - name TEXT NOT NULL, - label TEXT, - description TEXT, - type TEXT, -- value chosen from submissionValue in codelist_code C201264 - value TEXT, - value_label TEXT, - relative_to_from TEXT, -- value chosen from submissionValue in codelist_code C201265 - relative_from_schedule_instance TEXT, - relative_to_schedule_instance TEXT, - window_label TEXT, - window_upper TEXT, - window_lower TEXT, - order_index INTEGER, - UNIQUE(soa_id, timing_uid) - )""" - ) - # create timing_audit table cur.execute( """CREATE TABLE IF NOT EXISTS timing_audit ( @@ -295,24 +372,6 @@ def _init_db(): )""" ) - # create schedule_timelines table - cur.execute( - """CREATE TABLE IF NOT EXISTS schedule_timelines ( - id INTEGER PRIMARY KEY AUTOINCREMENT, - soa_id INT NOT NULL, - schedule_timeline_uid TEXT NOT NULL, -- immutable ScheduleTimeline_N identifier unique within SOA - name TEXT NOT NULL, - label TEXT, - description TEXT, - main_timeline INT, -- 1=True|0=False - entry_condition TEXT, - entry_id, -- dropdown select for ScheduledActivityInstance_ - exit_id TEXT, - order_index INT, - UNIQUE(soa_id, schedule_timeline_uid) - )""" - ) - # create schedule_timelines_audit table cur.execute( """CREATE TABLE IF NOT EXISTS schedule_timelines_audit ( @@ -339,24 +398,20 @@ def _init_db(): )""" ) - # create instances table + # Frozen versions (snapshot JSON of current matrix & concepts) cur.execute( - """CREATE TABLE IF NOT EXISTS instances ( - id INTEGER PRIMARY KEY AUTOINCREMENT, - soa_id INT NOT NULL, - instance_uid TEXT NOT NULL, -- immutable ScheduledActivityInstance_N identifier unique within SOA - name TEXT NOT NULL, - label TEXT, - description TEXT, - default_condition_uid TEXT, - epoch_uid TEXT, - timeline_id TEXT, - timeline_exit_id TEXT, - order_index INT, - encounter_uid TEXT, - UNIQUE(soa_id, instance_uid) + """CREATE TABLE IF NOT EXISTS soa_freeze ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + soa_id INTEGER, + version_label TEXT, + created_at TEXT, + snapshot_json TEXT )""" ) + # Unique index to enforce one label per SoA + cur.execute( + """CREATE UNIQUE INDEX IF NOT EXISTS idx_soafreeze_unique ON soa_freeze(soa_id, version_label)""" + ) conn.commit() conn.close() diff --git a/src/soa_builder/web/routers/instances.py b/src/soa_builder/web/routers/instances.py index 5c79c11..fcb63af 100644 --- a/src/soa_builder/web/routers/instances.py +++ b/src/soa_builder/web/routers/instances.py @@ -76,6 +76,7 @@ def ui_list_instances(request: Request, soa_id: int): instance_options = get_scheduled_activity_instance(soa_id) return templates.TemplateResponse( + request, "instances.html", { "request": request, diff --git a/src/soa_builder/web/templates/edit.html b/src/soa_builder/web/templates/edit.html index 5ab4b69..7564683 100644 --- a/src/soa_builder/web/templates/edit.html +++ b/src/soa_builder/web/templates/edit.html @@ -186,57 +186,96 @@

Editing SoA {{ soa_id }}

-

Matrix

+ + + {% if timelines and timelines|length > 0 %} +
+ Select Timeline: + {% for tl in timelines %} + + {% endfor %} +
+ {% endif %} + + + +{% for timeline_uid, timeline_instances in instances_by_timeline.items() %} +
+ +

Matrix: {% if timeline_instances and timeline_instances[0].timeline_name %}{{ timeline_instances[0].timeline_name }}{% else %}{{ timeline_uid }}{% endif %}

+ + {% if timeline_instances|length == 0 %} +

No scheduled activity instances for this timeline.

+ {% else %} + - + + - - {% for inst in instances %} - + + {% for inst in timeline_instances %} + {% endfor %} - {% for inst in instances %} + {% for inst in timeline_instances %} {% endfor %} - + - - {% for inst in instances %} + + {% for inst in timeline_instances %} {% endfor %} - {% for inst in instances %} + {% for inst in timeline_instances %} {% endfor %} - - - - {% for inst in instances %} - - {% endfor %} - + - {% for inst in instances %} + {% for inst in timeline_instances %} @@ -245,7 +284,7 @@

Matrix

- {% for inst in instances %} + {% for inst in timeline_instances %} @@ -259,7 +298,7 @@

Matrix

{% set selected_codes = concepts_list | map(attribute='code') | list %} {% set activity_id = a.id %} {% include 'concepts_cell.html' %} - {% for inst in instances %} + {% for inst in timeline_instances %} {% set raw_status = cell_map.get((inst.id, a.id), '') %} {% set display = 'X' if raw_status == 'X' else '' %} {% endfor %}
Timeline Name:-> - {% if inst.timeline_name %}{{ inst.timeline_name }}{% endif %} - Epoch:-> + {% if inst.epoch_name %}{{ inst.epoch_name }}{% endif %} +
Encounter Name:-> {% if inst.encounter_name %}{{ inst.encounter_name }}{% endif %}
Epoch:->Study Day:-> - {% if inst.epoch_name %}{{ inst.epoch_name }}{% endif %} + {% if inst.study_day %}{{ inst.study_day }}{% endif %}
Timing Label:-> {% if inst.timing_label %}{{ inst.timing_label }}{% endif %}
Study Day:-> - {% if inst.study_day %}{{ inst.study_day }}{% endif %} -
Visit Window:-> {% if inst.window_label %}{{ inst.window_label }}{% endif %}
Activity Concepts
{{ inst.name }}
Matrix
+{% endif %} +
+{% endfor %}

Generate Normalized Summary (JSON)

Matrix .activity-actions { display:inline-flex; align-items:center; gap:8px; margin-left:auto; } .transition-rule-actions { display:inline-flex; align-items:center; gap:8px; margin-left:auto; } .hint { font-weight:400; font-size:0.7em; color:#666; } + .timeline-selector {padding: 8px;background: #f9f9f9;border: 1px solid #ddd;border-radius: 4px;} + .timeline-btn:hover {opacity: 0.85;} + .timeline-btn.active {font-weight: 600;}