Skip to content

Wrfcmaq 2020 workflow #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
fisidi merged 9 commits into from
Jun 9, 2021
Merged

Wrfcmaq 2020 workflow #4

Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
a6931f4
Merge pull request #23 from fisidi/WRFCMAQ_2020Workflow
lizadams Jun 8, 2021
9d52abe
Update CMAQ_UG_tutorial_WRF-CMAQ_build_gcc.md
lizadams Jun 8, 2021
b5b2d4e
Update CMAQ_UG_tutorial_WRF-CMAQ_build_gcc.md
lizadams Jun 8, 2021
d1fc461
Update CMAQ_UG_tutorial_WRF-CMAQ_build_gcc.md
lizadams Jun 8, 2021
b8b268c
Update CMAQ_UG_tutorial_WRF-CMAQ_build_gcc.md
lizadams Jun 9, 2021
93e976a
Update CMAQ_UG_tutorial_WRF-CMAQ_build_gcc.md
lizadams Jun 9, 2021
8bacb89
Update CMAQ_UG_tutorial_WRF-CMAQ_build_gcc.md
lizadams Jun 9, 2021
bcf6cdc
Update CMAQ_UG_tutorial_WRF-CMAQ_build_gcc.md
lizadams Jun 9, 2021
e072547
Update CMAQ_UG_tutorial_WRF-CMAQ_build_gcc.md
lizadams Jun 9, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 63 additions & 13 deletions DOCS/Users_Guide/Tutorials/CMAQ_UG_tutorial_WRF-CMAQ_build_gcc.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
## WRF-CMAQ Tutorial ##

### Procedure to build and run the WRF-CMAQ model using gnu compiler: ###
### Procedure to build the WRF-CMAQ model using gnu compiler: ###

### Step 1: choose your compiler, and load it using the module command if it is available on your system

Expand Down Expand Up @@ -128,6 +128,11 @@ BASEDIR = ${INSTALL}/ioapi-3.2-20200828
setenv INSTALL [your_install_path]/LIBRARIES/openmpi_4.0.1_gcc_9.1.0
setenv BIN Linux2_x86_64gfort_openmpi_4.0.1_gcc_9.1.0


### set the CPLMODE environment variable

setenv CPLMODE nocpl


#### Make the installation directory

Expand Down Expand Up @@ -173,12 +178,19 @@ BASEDIR = ${INSTALL}/ioapi-3.2-20200828
#OMPLIBS = -fopenmp
```

### Create the Makefile in the m3tools directory

```
cd ../m3tools
cp Makefile.nocpl Makefile
```


### Build ioapi using the following command
### (Not clear where to run the make command. Do you run it under ioapi-3.2 or ioapi-3.2/ioapi directory?


```
cd ..
make |& tee make.log
```

Expand Down Expand Up @@ -231,6 +243,18 @@ Change directories to the CMAQ_HOME directory
setenv NETCDF netcdf_root_gcc # Note please combine netCDF-C & Fortran Libraries (e.g. /usr/local/netcdf-4.7.0)
setenv IOAPI ioapi_root_gcc (e.g. /usr/local/ioapi-3.2)
setenv WRF_ARCH 34 # [1-75] 64 Bit Linux_x86 Compiler/Architecture options

#> I/O API, netCDF, and MPI include and library locations
setenv IOAPI_INCL_DIR $IOAPI/ioapi/fixed_src #> I/O API include header files
setenv IOAPI_LIB_DIR $IOAPI/Linux2_x86_64gfort_openmpi_4.0.1_gcc_9.1.0 #> I/O API libraries
setenv NETCDF_LIB_DIR $NETCDF/lib #> netCDF C directory path
setenv NETCDF_INCL_DIR $NETCDF/include #> netCDF C directory path
setenv NETCDFF_LIB_DIR $NETCDF/lib #> netCDF Fortran directory path
Copy link
Owner

@fisidi fisidi Jun 8, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lizadams, I changed the config_cmaq.csh to include your changes, but I didn't change the NETCDFF_LIB_DIR and NETCDF_INCL_DIR to have the root $NETCDF, in the case that the user did not combine the NETCDF and NETCDFF directories.

Additionally in the next WRF release it think they are fixing this issue and recognizing that unidata is choosing to separate the C version and dependent Fortran versions of netCDF when built. See wrf-model/WRF#1269.

Copy link
Author

@lizadams lizadams Jun 9, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that is ok. I've added a few more changes to the run portion of the script including adding a new run script for the coupled model with shortwave feedback turned off.

setenv NETCDFF_INCL_DIR $NETCDF/include #> netCDF Fortran directory path
setenv MPI_LIB_DIR mpi_lib_gcc

#> MPI directory path

```

*Note: WRF_ARCH environment variable is based on the following options:*
Expand Down Expand Up @@ -398,21 +422,47 @@ setenv WRF_CMAQ 1

### Step 13: Run the WRF-CMAQ model

- Use the run.twoway_model_411_532_nf_run_script.16pe.csh script and the CMAQv5.3.2 input benchmark dataset to run CMAQ-WRF with no feedback
### Note, in this new coupled model design, the namelist is used to modify settings for WRF.
Environment variables such as WRF_CMAQ_FREQ are no longer used.
The following commonly modified namelist options for WRF are specified in the run script.

1. wrf_cmaq_option (dictates how the coupled model execute)

0 = run WRF only
1 = run WRF only w producing MCIP like GRID and MET files
2 = run WRF-CMAQ coupled model w/o producing MCIP like GRID and MET files
3 = run WRF-CMAQ coupled model w producing MCIP like GRID and MET files

2. wrf_cmaq_freq (indicate how often WRF and CMAQ interact)

3. met_file_tstep (time step size of MCIP like intermediate output files)

4. direct_sw_feedback (indicate to turn on aerosol short wave direct effect)

5. feedback_restart (indicate aerosol SW direct effect information is
available in the WRF restart file or not)

* three sample run scripts are provided; run-p.csh (for pure WRF model), run-cs.csh (for coupled
model with SW feedback on, and run-c-nswf.csh (for coupled model with SW feedback off).


- Start with the run-c-nswf.csh script that specifies direct_sw_feedback = .false.
- and the CMAQv5.3.2 input benchmark dataset to run CMAQ-WRF with no feedback
- It is configured to run on 16 processors and for 2 days of model simulation
- Edit the script to specify the paths, modify the number of processors and batch queue commands
- Verify that the OMIfile definition matches the latest release of CMAQv5.3.2

Modify the following section to specify your local paths

```
set ROOT_PATH = /proj/ie/proj/CMAS/WRF-CMAQ/openmpi_4.0.1_gcc_9.1.0_debug/
set WRF_DIR = $ROOT_PATH/WRF-4.1.1 # WRF source code directory
set INPDIR = /proj/ie/proj/CMAS/WRF-CMAQ/from_EPA/from_gdrive/CMAQv5.3.2_Benchmark_2Day_Input/2016_12SE1
set OMIpath = $WRF_DIR/cmaq # path optics related data files
set OUTPUT_ROOT = $ROOT_PATH/WRF-4.1.1 # output root directory
set NMLpath = $WRF_DIR/cmaq # path with *.nml file mechanism dependent
set NMLpath2 = $WRF_DIR/cmaq # path with Species_Table_TR_0.nml file
set EMISSCTRL = $WRF_DIR/cmaq # path of Emissions Control File
set WORKDIR = /proj/ie/proj/CMAS/WRF-CMAQ/CMAQ_v5.3.2/CCTM/scripts
set WRF_DIR = $WORKDIR/BLD_WRFv4.3_CCTM_v532_gcc # WRF source code directory
set INPDIR = /proj/ie/proj/CMAS/WRF-CMAQ/from_EPA/from_gdrive/CMAQv5.3.2_Benchmark_2Day_Input/2016_12SE1
set OMIpath = $WRF_DIR/cmaq # path optics related data files
set OUTPUT_ROOT = $WORKDIR # output root directory
set NMLpath = $WRF_DIR/cmaq # path with *.nml file mechanism dependent
set NMLpath2 = $WRF_DIR/cmaq # path with Species_Table_TR_0.nml file
set EMISSCTRL = $WRF_DIR/cmaq # path of Emissions Control File
```

- Verify the following settings
Expand All @@ -423,14 +473,14 @@ set EMISSCTRL = $WRF_DIR/cmaq # path of Emissions

- Submit the job using the batch queueing system
```
sbatch run.twoway_model_411_532_nf_run_script.16pe.csh
sbatch run-c-nswf.csh
```

### Step 14: Verify that the run was successful
- look for the output directory

```
cd output_12km_nf_rrtmg_20_5_1_v411532_debug
cd output-nswf
```
If the run was successful you will see the following output

Expand Down