JSON Schema Check rules - report SUCCESS not SKIPPED when no errors found

[ASL-rmarshall]

Links to related JIRA Tickets

• https://jira.cdisc.org/projects/CORERULES/issues/CORERULES-9624
• https://jira.cdisc.org/projects/CORERULES/issues/CORERULES-9625
• https://jira.cdisc.org/projects/CORERULES/issues/CORERULES-9668
• https://jira.cdisc.org/projects/CORERULES/issues/CORERULES-9669

Rule Information

• Standard: USDM
• Rule ID: DDF00081
• Rule Description: Class relationships must conform with the USDM schema based on the API specification.

Describe the bug
The JsonSchemaCheckDatasetBuilder validates the input JSON file against the schema extracted from the cache according to standard and standard version. The schema validation is run only once and creates a single dataset containing the list of schema validation errors found - each of which is associated with a dataset/entity.

When a JSON Schema Check is run for a JSON file, the rule is executed for each of the entities found in the JSON file and the JsonSchemaCheckDatasetBuilder.get_dataset method returns a list of errors from the single schema validation error dataset that is filtered according to dataset/entity. These error messages are then processed according to the Check criteria defined in the rule (e.g., so that only schema validation errors of a particular type are reported). If there are no error messages for a particular entity (i.e., the get_dataset method returns an empty dataset), the rule is skipped for the dataset/entity. If schema validation found no issues at all, the rule will be skipped for all datasets/entities, which results in the whole rule being reported as SKIPPED in the reported Summary (which seems to be applied by utils.get_execution_status).

Summary returned from Rule Engine

  "Rules_Report": [
    {
      "core_id": "DDF00081",
      "version": "1",
      "cdisc_rule_id": "DDF00081",
      "message": "The class relationship does not conform with the USDM schema.",
      "status": "SKIPPED"
    }
  ]

Expected behavior
If schema validation found no issues at all (i.e., the single validation error dataset is empty) then the JSON Schema Check rule should report SUCCESS in the summary instead of SKIPPED.

Test data
Negative and positive unit test data is available in the unitTesting SharePoint under USDM > DDF00081.
It's also available for the other 3 rules, but the same unit test data is used for all 4 rules.

[ASL-rmarshall]

@alexfurmenkov As there's a restriction on the budgeted hours available for fixing USDM issues, I think a quick and simple solution for this would be to update get_dataset to return a dataset containing a single row, with an empty string in each column, instead of a completely empty dataset_template. As the returned dataset will not be empty, it will not be skipped. All USDM rules that use this rule type include a restriction of based on the validator column so the row of empty strings will not affect the output.