Action to run all published rules' tests

[SFJohnson24]

This will involve creating a script to:

• get all published rules
• read through the sharepoint directory of test data
• compile the positive and negative test data resources for all published rules
  • This part may be difficult because the folder and file structure may be inconsistent. Might involve some reorganization of the folder structures to be consistent or the manual creation of a mapping file.
• execute each rule with the test data
• parse the result data
  • for positive test data, there should be at least 1 positive result and no negatiive results. skips can be ignored
  • for negative test data, there should be at least 1 negative result
• summarize the test results

Depending on runtime, we could run this action on push or create a cron job to run periodically using git actions:
https://docs.github.com/en/actions/writing-workflows/choosing-when-your-workflow-runs/events-that-trigger-workflows#schedule

[SFJohnson24]

This would likely involve:

1. standardizing the folder structure and file names for sharepoint (checking to make sure files are in this structure)

2. a private git repo with an action that scrapes sharepoint based on this structure-- taking into account the name/directory structure with potential for multiple pos/negative data. It will also grab Define.xml when applicable. This is sorted into standard/coreID folders with (multiple) positive and negative sub directories. This would likely use the sharepoint API:
   https://learn.microsoft.com/en-us/sharepoint/dev/sp-add-ins/get-to-know-the-sharepoint-rest-service?tabs=csom

3. a second git action on this repo that gets engine, updates cache, uses rules_dictionary.pkl to ID all the published rules. We then iterate through the rule IDs, grab the corresponding folder based on ID since they are unique (will likely need a system for rules that have a/b/c parts as I believe these exist), make the command based on folder contents (if condition for define), then check the resultant json-- expecting no issues for positive data, negative results for negative data.

4. It would be best to create a ledger of the rules run and the number of issues found for positive/negative. It could create a second sheet where problem rules/data are added if there is a misalignment between number of issues and their dataset type

a second copy of this second action could be aimed at the SQL branch for testing