Rule blocked: CORERULES-9609 - codelist_terms operation does not retrieve correct terms #939
Description
- A .txt file with the JSON containing the full request: Request.txt
Links to related JIRA Tickets
Rule Information
- Standard: USDM
- Rule ID: DDF00015
- Rule Description: A study version's study phase must be specified according to the extensible Trial Phase Response (C66737) SDTM codelist (e.g. an entry with a code or decode used from the codelist should be consistent with the full entry in the codelist).
Describe the bug
The codelist_terms operation does not retrieve the correct set of terms for the specified codelist. In this example rule, the specified codelist is "TPHASE". The _execute_operation method tries to retrieve the C-code for the specified codelist by looking up the submission value ("TPHASE") in submission_lookup, which appears to be created from the submission_lookup property of the first controlled terminology package that's found.
However, the submission_lookup property of the CT package appears to have been created based on the assumption that there is a single C-code for each unique submission value. Unfortunately, this is an invalid assumption - there may be as many as 65 different C-codes for any given submission value. Also, the same value might be the submission value for both a codelist and a term, which is the case in this example:
| Code | Codelist Code | Codelist Name | CDISC Submission Value | CDISC Definition |
|---|---|---|---|---|
| C66737 | Trial Phase Response | TPHASE | A terminology codelist relevant to a step in the clinical research and development of a therapy from initial clinical trials to post-approval studies. | |
| C48281 | C66738 | Trial Summary Parameter Test Code | TPHASE | A step in the clinical research and development of a therapy from initial clinical trials to post-approval studies. NOTE: Clinical trials are generally categorized into four (sometimes five) phases. A therapeutic intervention may be evaluated in two or more phases simultaneously in different trials, and some trials may overlap two different phases. [21 CFR section 312.21; After ICH Topic E8 NOTE FOR GUIDANCE ON GENERAL CONSIDERATIONS FOR CLINICAL TRIALS, CPMP/ICH/291/95 March 1998] |
It looks like only the last entry for each submission value is stored in submission_lookup - which might be either a codelist or a term - and codelist_terms then returns all the values from the codelist with the C-code matching this last entry. In this example, the codelist_terms operation returns the terms in the "Trial Summary Parameter Test Code" codelist, which was not what was specified (the codelists parameter of the codelist_terms operation is used to specify the name(s) of the required codelist(s) - not the submission value for any term in the required codelist(s)).
This issue is likely to affect lots of rules, especially those related to Trial Summary codelists (for USDM, SDTM and SEND) where the Trial Summary Parameter Test Code submission values are usually the same as the submission value for the corresponding response codelist.
Error returned from Rule Engine
No error was returned - the rule just did not behave as expected.
Expected behavior
The codelist_terms should return only the terms from the specified codelist - i.e., the terms contained in the codelist whose submission value is specified in the codelists parameter. It should not return terms in any codelist that contains a term whose submission value matches a value specified in the codelists parameter.