Skip to content

Conversation

@bieryAtFnal
Copy link
Contributor

Description

This PR is correlated with DUNE-DAQ/integrationtest#139.

In a discussion with Pawel and a few other people from the CCM WG, I learned that they are thinking about integration/regression tests that validate the behavior of different types of run control process managers. This led to the request for the ability to specify one or more process manager types in our integtest files.

The core changes to add this functionality were made in the integrationtest repo, and the changes in this repository were the following:

  1. update a couple of tests to use a slightly different way of matching local substrings to the name of the current pytest
  2. provide a sample declaration of the desired process manager types (in the 3ru_3df_multirun_test)
  3. started adding comments to the 3ru_3df_multirun_test to provide more (and updated) information about the variables that need to be defined in each integtest (to pass the necessary information to the integrationtest infrastructure). I haven't finished that comment block because I want to discuss the current, slightly confusing, state of those variables with Eric and others and possibly make some changes to help reduce confusion.
  4. I added a line to the tpreplay_test to check whether the number of raw data files produced by each data-taking run was correct. It looked to me that this line had simply been missing.

Regarding the first change, there was logic in the trigger_bitwords_test and the tpreplay_test that used a regular expression to pull the "confgen" name out of the current pytest name and use that to look up the appropriate test parameters in a locally-defined dictionary. With the addition of the process-manager-type fixture to our tests, the pytest names of our tests will now have three elements: confgen name, process manager type, and index or name of the run control command list. Instead of trying to update the regex to support this change, I simply modified the logic to check which confgen name appears in the pytest current test name. This seems to work fine, and it seems a little more future-proof than the previous logic.

To test these changes, we can use the following steps:

DATE_PREFIX=`date '+%d%b'`
TIME_SUFFIX=`date '+%H%M'`

source /cvmfs/dunedaq.opensciencegrid.org/setup_dunedaq.sh
setup_dbt latest
dbt-create -n NFD_DEV_251229_A9 ${DATE_PREFIX}FDDevTest_${TIME_SUFFIX}
cd ${DATE_PREFIX}FDDevTest_${TIME_SUFFIX}/sourcecode

git clone https://github.com/DUNE-DAQ/daqsystemtest.git -b kbiery/multiple_procmgr_support
cd ..

dbt-workarea-env

git clone https://github.com/DUNE-DAQ/integrationtest.git -b kbiery/multiple_procmgr_support
cd integrationtest; pip install -U . ; cd ..

dbt-build -j 12
dbt-workarea-env

dbt-build --integtest daqsystemtest
echo ""
echo -e "\U1F535 \U2705 Note that all regression tests passed. \U2705 \U1F535"

Type of change

  • New feature or enhancement (non-breaking change which adds functionality)
  • Optimization (non-breaking change that improves code/performance)
  • Bug fix (non-breaking change which fixes an issue)

Testing checklist

  • Unit tests pass (e.g. dbt-build --unittest)
  • Minimal system quicktest passes (pytest -s minimal_system_quick_test.py)
  • Full set of integration tests pass (daqsystemtest_integtest_bundle.sh)

Further checks

  • Code is commented where needed, particularly in hard-to-understand areas

Kurt Biery added 2 commits December 29, 2025 07:52
…k up the datafile_params in the test_data_files() function after the addition of support for multiple run control process managers in integtests.
Copy link
Member

@eflumerf eflumerf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Integration tests still pass

@bieryAtFnal bieryAtFnal merged commit bf124c8 into develop Dec 30, 2025
3 checks passed
@bieryAtFnal bieryAtFnal deleted the kbiery/multiple_procmgr_support branch December 30, 2025 15:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants