TypeError when generating USDM validation reports with None entity values

[Panikos]

Issue: TypeError when generating USDM validation reports with None entity values

Bug Description

When running USDM validation on certain JSON files, the CORE engine crashes with a TypeError during report generation.

Error Message

TypeError: '<' not supported between instances of 'NoneType' and 'str'

Full Stack Trace

Traceback (most recent call last):
  File "core.py", line 808, in <module>
  File "click\core.py", line 1157, in __call__
  File "click\core.py", line 1078, in main
  File "click\core.py", line 1688, in invoke
  File "click\core.py", line 1434, in invoke
  File "click\core.py", line 783, in invoke
  File "click\decorators.py", line 33, in new_func
  File "core.py", line 355, in validate
  File "scripts\run_validation.py", line 196, in run_validation
  File "cdisc_rules_engine\services\reporting\json_report.py", line 120, in write_report
  File "cdisc_rules_engine\services\reporting\json_report.py", line 98, in get_export
  File "cdisc_rules_engine\services\reporting\base_report.py", line 72, in get_summary_data
TypeError: '<' not supported between instances of 'NoneType' and 'str'

Command Used

core validate -s usdm -v 4-0 -dp protocol_usdm.json -o report -of JSON

Root Cause

In cdisc_rules_engine/services/reporting/usdm_report_data.py, the get_summary_data() and get_detailed_data() methods use sort keys that can return None:

Line 140:

return sorted(
    summary_data,
    key=lambda x: (x.get("entity"), x["core_id"]),
)

Line 151:

return sorted(
    detailed_data,
    key=lambda x: (x["core_id"], x.get("entity")),
)

When entity or core_id is None, Python 3's sorted() cannot compare None with strings, causing the crash.

Suggested Fix

Provide empty string fallbacks for the sort keys:

# Line 140
key=lambda x: (x.get("entity") or "", x["core_id"] or "")

# Line 151
key=lambda x: (x["core_id"] or "", x.get("entity") or "")

This is consistent with the existing pattern in get_entity_detail_data() at line 103:

key=lambda x: x.get("entity", "").lower()

Environment

• CORE Engine version: Latest (packaged executable)
• Standard: USDM 4.0
• OS: Windows 11
• Python: 3.13 (if running from source)

Additional Context

This issue occurs when validating USDM JSON files where some validation results have missing or null entity fields. The validation itself completes successfully, but the report generation fails when trying to sort the results.

[alexfurmenkov]

Hi @Panikos, can you please share the dataset you've used?