MYNN surface layer - reproducibility notes

[barlage]

It's looking like the surface layer may be responsible for some reproducibility issues. I will use this space to accumulate some notes.

Testing runs with this config:

config_physics_suite = 'hrrrv5'
config_lsm_scheme = "sf_noah"
config_pbl_scheme        = 'bl_mynn'
config_radt_cld_scheme   = 'cld_fraction'
config_sfclayer_scheme   = 'sf_mynn'

this will reproduce the gfs IC/LBC case, commenting just the config_pbl_scheme and config_sfclayer_scheme will not

Added a bunch of prints in mpas_atmphys_driver_sfclayer.F before and after the mynn driver call. It seems the inputs are the same for the timestep after the restart. But after the first call there are differences.

Issue 1: qstar in module_mynnsfclay_driver.F90 and module_mynnsfclay_pre.F90

There is an initflag initialization here.

But this is just a local variable at the driver level here.

I put an assignment to zero before the _pre call and it doesn't affect my restart tests so likely in this case the variable is always being initialized to zero and doesn't matter, but should this be a restart field?

[clark-evans]

#210 is based on the CCPP version, which was known to reproduce. Currently testing in dev2, but not for restarts.