Skip to content

🔧 Refactor populate_field_type to use type-directed schema walking#1639

Merged
chrisjsewell merged 4 commits intomasterfrom
cjs-improve-reduce-needs
Feb 3, 2026
Merged

🔧 Refactor populate_field_type to use type-directed schema walking#1639
chrisjsewell merged 4 commits intomasterfrom
cjs-improve-reduce-needs

Conversation

@chrisjsewell
Copy link
Member

This PR simplifies the populate_field_type function by replacing runtime keyword detection with type-directed traversal of the SchemasRootType structure.

Changes

Refactored populate_field_type (~190 → ~130 lines)

Before: Generic recursive function operating on Any, detecting JSON schema keywords (properties, items, contains, etc.) at runtime to infer structure, with special-case handling for network context.

After: Type-directed implementation that surgically walks the known typed structure:

  • SchemasRootTypeselect / validate
  • ValidateSchemaTypelocal / network
  • NeedFieldsSchemaTypeproperties / allOf
  • ResolvedLinkSchemaType → recursive items / contains

Key improvements

  1. Clearer structure — 5 focused helper functions with single responsibilities:

    • _process_validate_schema
    • _process_resolved_link
    • _process_need_fields_schema
    • _inject_field_type
    • _inject_type_and_item_type
  2. Eliminated ambiguity — No more is_inside_network path checks; the type structure dictates context

  3. Defensive handling — All field accesses guarded with isinstance() checks; blanket exception handler wraps unexpected errors with context

  4. Unified field lookup — Single pattern for extra/link/core fields with fallback to NeedsCoreFields

Added tests

New test file tests/schema/test_populate_field_type.py with 32 tests covering:

  • Type injection for extra fields, link fields, and core fields
  • Array type injection with items/contains
  • Recursive processing of network and allOf
  • Error handling for unknown fields, type mismatches, and malformed schemas
  • Defensive behavior for missing/invalid structure

Breaking changes

None. All existing tests pass. Error messages for malformed schemas may differ slightly in wording but still provide equivalent information.

This PR simplifies the `populate_field_type` function by replacing runtime keyword detection with type-directed traversal of the `SchemasRootType` structure.

## Changes

### Refactored `populate_field_type` (~190 → ~130 lines)

**Before:** Generic recursive function operating on `Any`, detecting JSON schema keywords (`properties`, `items`, `contains`, etc.) at runtime to infer structure, with special-case handling for network context.

**After:** Type-directed implementation that surgically walks the known typed structure:
- `SchemasRootType` → `select` / `validate`
- `ValidateSchemaType` → `local` / `network`
- `NeedFieldsSchemaType` → `properties` / `allOf`
- `ResolvedLinkSchemaType` → recursive `items` / `contains`

### Key improvements

1. **Clearer structure** — 5 focused helper functions with single responsibilities:
   - `_process_validate_schema`
   - `_process_resolved_link`
   - `_process_need_fields_schema`
   - `_inject_field_type`
   - `_inject_type_and_item_type`

2. **Eliminated ambiguity** — No more `is_inside_network` path checks; the type structure dictates context

3. **Defensive handling** — All field accesses guarded with `isinstance()` checks; blanket exception handler wraps unexpected errors with context

4. **Unified field lookup** — Single pattern for extra/link/core fields with fallback to `NeedsCoreFields`

### Added tests

New test file `tests/schema/test_populate_field_type.py` with 32 tests covering:
- Type injection for extra fields, link fields, and core fields
- Array type injection with `items`/`contains`
- Recursive processing of `network` and `allOf`
- Error handling for unknown fields, type mismatches, and malformed schemas
- Defensive behavior for missing/invalid structure

## Breaking changes

None. All existing tests pass. Error messages for malformed schemas may differ slightly in wording but still provide equivalent information.
@codecov
Copy link

codecov bot commented Feb 2, 2026

Codecov Report

❌ Patch coverage is 97.64706% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.80%. Comparing base (4e10030) to head (fae627e).
⚠️ Report is 234 commits behind head on master.

Files with missing lines Patch % Lines
sphinx_needs/schema/config_utils.py 97.64% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1639      +/-   ##
==========================================
+ Coverage   86.87%   88.80%   +1.92%     
==========================================
  Files          56       70      +14     
  Lines        6532     9972    +3440     
==========================================
+ Hits         5675     8856    +3181     
- Misses        857     1116     +259     
Flag Coverage Δ
pytests 88.80% <97.64%> (+1.92%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Member

@ubmarco ubmarco left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 minor comments, good for merge

@chrisjsewell chrisjsewell merged commit e974e13 into master Feb 3, 2026
20 of 51 checks passed
@chrisjsewell chrisjsewell deleted the cjs-improve-reduce-needs branch February 3, 2026 07:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants