Update EE2 Environment Variables: Replace NET-Specific (VARgfs) with Global (VARglobal)#2073
Conversation
There was a problem hiding this comment.
Pull request overview
This PR updates GDASApp scripts, CI harnesses, and Jinja2/YAML configuration templates to follow the EE2 global-workflow naming convention by replacing NET-specific variables (e.g., HOMEgfs, PARMgfs, FIXgfs, EXECgfs) with global equivalents (e.g., HOMEglobal, PARMglobal, FIXglobal, EXECglobal), aligning with global-workflow issue #4410 and companion PR #4540.
Changes:
- Renames
HOMEgfs→HOMEglobal(and related*gfs→*globalvariables) acrossush/,scripts/,test/, andparm/templates. - Updates GW CI test plumbing (
test/gw-ci/) and global-workflow test scripts (test/*/global-workflow/) to export/useHOMEglobal. - Updates build workflow hook(s) to use
HOMEglobalfor workflow-build SOCA linking.
Reviewed changes
Copilot reviewed 82 out of 82 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| ush/ufoeval/setup_phase3.py | Uses HOMEglobal from config and exports it in generated job script. |
| ush/ufoeval/config_jedi.yaml | Renames directory key to HOMEglobal. |
| ush/ufoeval/config_gsi.yaml | Renames directory key to HOMEglobal. |
| ush/ufoeval/README_phase3 | Updates documentation to refer to HOMEglobal. |
| ush/soca/run_jjobs.py | Updates generated j-job environment/script paths to use HOMEglobal. |
| ush/soca/prep_ocean_obs.py | Renames PARMgfs to PARMglobal for dummy CDL path resolution. |
| ush/soca/bkg_utils.py | Switches env lookup to HOMEglobal for GDAS path derivation. |
| ush/gsi_satbias2ioda_all.sh | Uses HOMEglobal for module loading and converter paths. |
| test/snow/apply_jedi_incr.sh | Exports/uses HOMEglobal for test harness setup and job script generation. |
| test/gw-ci/soca/jcb-prototype_3dfgat_3DVarAOWCDA.yaml.j2 | Updates reference/output paths to HOMEglobal. |
| test/gw-ci/create_exp.sh | Uses HOMEglobal to source platform config and create experiments. |
| test/gw-ci/atm/jedi-test_lgetkf_ufs_hybatmDA.yaml.j2 | Updates reference/output paths to HOMEglobal. |
| test/gw-ci/atm/jedi-test_lgetkf-solver_ufs_hybatmDA.yaml.j2 | Updates reference/output paths to HOMEglobal. |
| test/gw-ci/atm/jedi-test_lgetkf-observer_ufs_hybatmDA.yaml.j2 | Updates reference/output paths to HOMEglobal. |
| test/gw-ci/atm/jedi-test_lgetkf-fv3inc_ufs_hybatmDA.yaml.j2 | Updates reference/output paths to HOMEglobal. |
| test/gw-ci/atm/jedi-test_3dvar_ufs_hybatmDA.yaml.j2 | Updates reference/output paths to HOMEglobal. |
| test/gw-ci/atm/jedi-test_3dvar-fv3inc_ufs_hybatmDA.yaml.j2 | Updates reference/output paths to HOMEglobal. |
| test/gw-ci/CMakeLists.txt | Renames CMake plumbing from HOMEgfs to HOMEglobal for cycling test setup. |
| test/atm/global-workflow/setup_workflow_exp.sh | Exports HOMEglobal and uses it for python/module paths. |
| test/atm/global-workflow/jjob_var_run.sh | Exports HOMEglobal and uses it to locate/run variational j-job. |
| test/atm/global-workflow/jjob_var_init.sh | Exports HOMEglobal and uses it to locate/run init j-job. |
| test/atm/global-workflow/jjob_var_inc.sh | Exports HOMEglobal and uses it to locate/run increment j-job. |
| test/atm/global-workflow/jjob_var_final.sh | Exports HOMEglobal and uses it to locate/run finalize j-job. |
| test/atm/global-workflow/jjob_ens_sol.sh | Exports HOMEglobal and uses it to locate/run ensemble solver j-job. |
| test/atm/global-workflow/jjob_ens_obs.sh | Exports HOMEglobal and uses it to locate/run ensemble observer j-job. |
| test/atm/global-workflow/jjob_ens_letkf.sh | Exports HOMEglobal and uses it to locate/run combined LETKF j-job. |
| test/atm/global-workflow/jjob_ens_init_split.sh | Exports HOMEglobal and uses it to locate/run split ensemble init. |
| test/atm/global-workflow/jjob_ens_init.sh | Exports HOMEglobal and uses it to locate/run ensemble init. |
| test/atm/global-workflow/jjob_ens_inc.sh | Exports HOMEglobal and uses it to locate/run ensemble increment. |
| test/atm/global-workflow/jjob_ens_final.sh | Exports HOMEglobal and uses it to locate/run ensemble finalize. |
| test/atm/global-workflow/jedi-test_lgetkf.yaml.j2 | Updates reference/output paths to HOMEglobal. |
| test/atm/global-workflow/jedi-test_lgetkf-solver.yaml.j2 | Updates reference/output paths to HOMEglobal. |
| test/atm/global-workflow/jedi-test_lgetkf-observer.yaml.j2 | Updates reference/output paths to HOMEglobal. |
| test/atm/global-workflow/jedi-test_lgetkf-fv3inc.yaml.j2 | Updates reference/output paths to HOMEglobal. |
| test/atm/global-workflow/jedi-test_3dvar.yaml.j2 | Updates reference/output paths to HOMEglobal. |
| test/atm/global-workflow/jedi-test_3dvar-fv3inc.yaml.j2 | Updates reference/output paths to HOMEglobal. |
| test/atm/global-workflow/config.yaml | Renames base key to HOMEglobal and updates YAML paths accordingly. |
| test/aero/global-workflow/jjob_var_run.sh | Exports/uses HOMEglobal to locate/run aero j-job. |
| test/aero/global-workflow/jjob_var_init.sh | Exports/uses HOMEglobal to locate/run aero init j-job. |
| test/aero/global-workflow/jjob_var_final.sh | Exports/uses HOMEglobal to locate/run aero finalize j-job. |
| test/aero/global-workflow/config.aeroanl | Updates config paths to use HOMEglobal. |
| test/aero/genyaml_3dvar.sh | Exports/uses HOMEglobal for python/module paths when generating YAML. |
| scripts/exgdas_global_marine_analysis_vrfy.py | Uses HOMEglobal to locate EVA plotting templates. |
| parm/snow/snow_stage_jedi_fix.yaml.j2 | Renames FIXgfs → FIXglobal for staged fix files. |
| parm/snow/snow_stage_ims_scf2ioda.yaml.j2 | Renames FIXgfs → FIXglobal mapping source path. |
| parm/snow/snow_stage_bufr2ioda_mapping.yaml.j2 | Renames PARMgfs → PARMglobal mapping sources. |
| parm/snow/snow_stage_berror.yaml.j2 | Renames HOMEgfs → HOMEglobal for berror staging. |
| parm/snow/snow_ens_config.yaml.j2 | Renames EXECgfs/PARMgfs → EXECglobal/PARMglobal. |
| parm/snow/snow_det_config.yaml.j2 | Renames EXECgfs/PARMgfs → EXECglobal/PARMglobal. |
| parm/snow/prep/prep_snocvr_snomad.yaml.j2 | Renames PARMgfs → PARMglobal for mapping files. |
| parm/snow/prep/prep_ims.yaml.j2 | Renames FIXgfs → FIXglobal for IMS mapping files. |
| parm/snow/prep/prep_ghcn.yaml.j2 | Renames FIXgfs → FIXglobal for station list. |
| parm/snow/jcb-base.yaml.j2 | Renames PARMgfs → PARMglobal for JCB search paths. |
| parm/snow/ens_apply_incr_nml.j2 | Renames HOMEgfs → HOMEglobal for orog path. |
| parm/snow/apply_incr_nml.j2 | Renames HOMEgfs → HOMEglobal for orog path. |
| parm/obsolete/snow/hofx/hofx_nomodel.yaml.j2 | Renames FIXgfs → FIXglobal for orography datapath. |
| parm/marine/marine_stage_utilities.yaml.j2 | Renames HOMEgfs → HOMEglobal for utility YAML staging. |
| parm/marine/marine_ens_config.yaml.j2 | Renames EXECgfs/PARMgfs → EXECglobal/PARMglobal. |
| parm/marine/marine_ecen_config.yaml.j2 | Renames EXECgfs/PARMgfs → EXECglobal/PARMglobal. |
| parm/marine/marine_det_config.yaml.j2 | Renames EXECgfs/PARMgfs → EXECglobal/PARMglobal. |
| parm/marine/marine_bmat_config.yaml.j2 | Renames EXECgfs/PARMgfs → EXECglobal/PARMglobal. |
| parm/marine/jcb-base.yaml.j2 | Renames PARMgfs → PARMglobal for JCB search paths. |
| parm/atm/jcb-prototype_lgetkf_solver.yaml.j2 | Renames PARMgfs → PARMglobal for obs paths. |
| parm/atm/jcb-prototype_lgetkf_observer.yaml.j2 | Renames PARMgfs → PARMglobal for obs paths. |
| parm/atm/jcb-prototype_lgetkf.yaml.j2 | Renames PARMgfs → PARMglobal for obs paths. |
| parm/atm/jcb-base.yaml.j2 | Renames PARMgfs → PARMglobal and HOMEgfs → HOMEglobal for script paths. |
| parm/atm/atm_stage_jedi_fix.yaml.j2 | Renames FIXgfs → FIXglobal for staged fix files. |
| parm/atm/atm_ens_config.yaml.j2 | Renames EXECgfs/PARMgfs → EXECglobal/PARMglobal. |
| parm/atm/atm_ecen_config.yaml.j2 | Renames EXECgfs/PARMgfs → EXECglobal/PARMglobal. |
| parm/atm/atm_det_stage_berror_gsibec.yaml.j2 | Renames HOMEgfs → HOMEglobal for GSIBEC staging. |
| parm/atm/atm_det_config.yaml.j2 | Renames EXECgfs/PARMgfs → EXECglobal/PARMglobal. |
| parm/anlstat/snow/jcb-base.yaml.j2 | Renames PARMgfs → PARMglobal for JCB search paths. |
| parm/anlstat/atmos_gsi/jcb-base.yaml.j2 | Renames PARMgfs → PARMglobal for JCB search paths. |
| parm/anlstat/atmos/jcb-base.yaml.j2 | Renames PARMgfs → PARMglobal for JCB search paths. |
| parm/anlstat/anlstat_config.yaml.j2 | Renames EXECgfs/PARMgfs → EXECglobal/PARMglobal and obs list paths. |
| parm/anlstat/aero/jcb-base.yaml.j2 | Renames PARMgfs → PARMglobal for JCB search paths. |
| parm/analcalc/analcalc_config.yaml.j2 | Renames EXECgfs/PARMgfs → EXECglobal/PARMglobal and include path. |
| parm/aero/jcb-base.yaml.j2 | Renames PARMgfs → PARMglobal for JCB search paths. |
| parm/aero/aero_stage_jedi_fix.yaml.j2 | Renames FIXgfs → FIXglobal for staged fix files. |
| parm/aero/aero_det_config.yaml.j2 | Renames EXECgfs/PARMgfs → EXECglobal/PARMglobal. |
| parm/aero/aero_bmat_config.yaml.j2 | Renames EXECgfs/PARMgfs → EXECglobal/PARMglobal. |
| build.sh | Uses HOMEglobal for workflow-build SOCA link targets. |
|
Looks good from what I can tell. Let's work to coordinate the merge as we do not want to merge this into |
|
@AntonMFernando-NOAA are you ready for this to be merged? |
@CoryMartin-NOAA @DavidHuber-NOAA I have updated this branch with the latest changes, and we’ll likely need to review it again. Once you approve, I’ll rerun the CI tests to make sure everything is working properly, especially since the GW PR has been updated as well. After that, we can merge this PR, followed by the GW PR (after updating the hash to point to the latest commit on the GDAS develop branch). |
|
@AntonMFernando-NOAA just let me know when you're done testing and I will merge this in right away |
@CoryMartin-NOAA I’m currently running the tests on WCOSS2 and will update you once they’re finished. Thanks! |
|
@CoryMartin-NOAA I am no longer able to access the WCOSS2 RUNTESTS directory after the development switch. However, when I checked yesterday, most of the tests had already passed. The remaining tests were cycle tests that were running their final cycles, and the first few cycles had already completed successfully. Based on this, I think this PR is ready to be merged. |
|
@AntonMFernando-NOAA ok let's do it and prioritize any hot fixes needed in final testing. |
@CoryMartin-NOAA Thanks! |
Description
Companion PRs
Ref: #4540
Issues
Automated CI tests to run in Global Workflow