CORERULES-9645 - is_column_of_iterables method only checks first row #1017
Description
Links to related JIRA Tickets
Rule Information
- Standard: USDM
- Rule ID: DDF00102
- Rule Description: A scheduled activity instance must only reference a timeline exit that is defined within the same schedule timeline as the scheduled activity instance.
Describe the bug
As shown in "Request.txt" attached below, this rule was originally implemented by using the distinct operator to retrieve a list of the distinct schedule timeline exit id values grouped by parent_id, which was merged onto the check dataset. Error records were reported when the value on the timelineExitId variable in the check dataset is_not_contained_by the unique list of schedule timeline exit id values. However, the rule failed with an error:
Cannot perform 'and_' with a dtyped [bool] array and scalar of type [NoneType]
This was happening because the grouped list of unique values was only populated on records with matching group values - it was None for other records. The is_column_of_iterables method is used to check if the comparison column contains iterables to determine the method used to do the "contains" comparison. However, as is_column_of_iterables only checks the first row, the comparison method used when the first row contains an iterable fails for any other row that contains None.
The same issue occurs if the does_not_contain operator is used.
Note that this rule has now been rewritten to use a table join instead, so this is no longer a rule-blocking issue.
Error returned from Rule Engine
"ScheduledActivityInstance": [
{
"executionStatus": "execution_error",
"dataset": "ScheduledActivityInstance.xpt",
"domain": "",
"variables": [],
"message": "rule execution error",
"errors": [
{
"dataset": "ScheduledActivityInstance.xpt",
"error": "An unknown exception has occurred",
"message": "Cannot perform 'and_' with a dtyped [bool] array and scalar of type [NoneType]"
}
]
}
],Expected behavior
The is_not_contained_by and does_not_contain operators should run without error when comparing a variable value against a grouped list of unique values generated by the distinct operator.
Test data
Test data is also available in Excel format in the USDM > DDF00102 > positive > 01 > data folder of the CORE unitTesting SharePoint.