Skip to content

Update Hera /scratch1 and /scratch2 paths#1835

Merged
RussTreadon-NOAA merged 9 commits intodevelopfrom
bugfix/hera_scratch
Aug 6, 2025
Merged

Update Hera /scratch1 and /scratch2 paths#1835
RussTreadon-NOAA merged 9 commits intodevelopfrom
bugfix/hera_scratch

Conversation

@RussTreadon-NOAA
Copy link
Contributor

Description

This PR updates /scratch1 and /scratch2 paths in GDASApp in preparation for the 8/5/2025 decommissioning of these filesets.

Companion PRs

GDASApp PR #1830 updates paths in ush/ufoeval/run_ufo_hofx_test.sh

Issues

Resolves #1831

Automated CI tests to run in Global Workflow

  • atm_jjob
  • C96C48_ufs_hybatmDA
  • C96C48_hybatmsnowDA
  • C96_gcafs_cycled
  • C48mx500_3DVarAOWCDA
  • C48mx500_hybAOWCDA
  • C96C48_hybatmDA

@RussTreadon-NOAA RussTreadon-NOAA self-assigned this Aug 1, 2025
@RussTreadon-NOAA RussTreadon-NOAA marked this pull request as draft August 1, 2025 20:57
@RussTreadon-NOAA
Copy link
Contributor Author

Convert to draft until we resolve what to do with the following occurrences of /scratch1:

./ush/ufsda/yamltools.py:                'datapath': '/scratch1/NCEPDEV/global/glopara/fix/orog/20220805/C'+str(case)+'/',
./modulefiles/EVA/hera.lua:local pyenvpath = "/scratch1/NCEPDEV/da/python/envs/"

One option for ./ush/ufsda/yamltools.py is to delete it along with the script that calls it, scripts/old/exgdas_global_marine_analysis_prep.py. If we delete scripts/old/exgdas_global_marine_analysis_prep.py, we should consider removing the entire directory scripts/old.

One option for ./modulefiles/EVA/hera.lua is to see if we can use either of the following modules to replace local pyenvpath = "/scratch1/NCEPDEV/da/python/envs/"

py-virtualenv/20.24.5
python-venv/1.0

The above modules are available when GDASApp modulefiles/GDAS/hera.intel.lua is loaded.

Copy link
Contributor

@guillaumevernieres guillaumevernieres left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@RussTreadon-NOAA
Copy link
Contributor Author

Thank you @guillaumevernieres for the approval. Any thoughts on how to handle the scratch2 reference in ./ush/ufsda/yamltools.py? Below is one option

One option for ./ush/ufsda/yamltools.py is to delete it along with the script that calls it, scripts/old/exgdas_global_marine_analysis_prep.py. If we delete scripts/old/exgdas_global_marine_analysis_prep.py, we should consider removing the entire directory scripts/old.

@RussTreadon-NOAA
Copy link
Contributor Author

Despite two remaining questions, mark this PR as Ready for review.

GDASApp develop at 899848c following the 8/5/2025 Hera maintenance. The scratch1/2 filesets no longer exist.

The GDASApp hera.intel.lua references scratch1/ for CRTM_FIX. The build therefore attempts to download CRTM coefficients and fails. Long term we should be able to load the spack-stack crtm to set CRTM_FIX. This PR changes the CRTM_FIX path to scratch3/.

@RussTreadon-NOAA RussTreadon-NOAA marked this pull request as ready for review August 6, 2025 10:22
@RussTreadon-NOAA
Copy link
Contributor Author

Merge given @guillaumevernieres' approval.

@RussTreadon-NOAA RussTreadon-NOAA merged commit 392cf87 into develop Aug 6, 2025
5 checks passed
@RussTreadon-NOAA RussTreadon-NOAA deleted the bugfix/hera_scratch branch August 6, 2025 10:23
@guillaumevernieres
Copy link
Contributor

Thank you @guillaumevernieres for the approval. Any thoughts on how to handle the scratch2 reference in ./ush/ufsda/yamltools.py? Below is one option

One option for ./ush/ufsda/yamltools.py is to delete it along with the script that calls it, scripts/old/exgdas_global_marine_analysis_prep.py. If we delete scripts/old/exgdas_global_marine_analysis_prep.py, we should consider removing the entire directory scripts/old.

Sorry for the late reply @RussTreadon-NOAA . Deleting sounds good to me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Update references to Hera /scratch1 and /scratch2

2 participants