Set notated input parameters not being recognized

[Lvulis]

We've been using set notation but find that the model is not recognizing these. This may be an edge case as this is an unnecessary scenario where only 1 set of alternatives is provided, not multiple.

The input yaml:

out_dir: 'issue_run'
Width: 20000
Length: 15000
h0: 4.0 
S0: .000133
N0_meters: 400
L0_meters: 800
Np_water: 5000
Np_sed: 3000
dx: 200
u0: 1.5
coeff_U_dep_mud: 0.4
save_eta_figs: True
save_eta_grids: True
save_depth_grids: True
save_discharge_grids: True
save_velocity_grids: True
save_sedflux_grids: True
save_sandfrac_grids: True
save_discharge_components: True
set:
  - {f_bedload: 0.24, C0_percent: 0.3}

The resulting log file:

2022-05-18 20:19:08,166 - INFO - Output log file initialized
2022-05-18 20:19:08,166 - INFO - pyDeltaRCM version 2.1.3
2022-05-18 20:19:08,166 - INFO - Python version 3.10.4 | packaged by conda-forge | (main, Mar 24 2022, 17:38:57) [GCC 10.3.0]
2022-05-18 20:19:08,166 - INFO - Platform: Linux-5.13.0-1023-aws-x86_64-with-glibc2.31
2022-05-18 20:19:08,166 - INFO - Setting up model configuration
2022-05-18 20:19:08,166 - INFO - Model type is: DeltaModel
2022-05-18 20:19:08,167 - INFO - Configuration variable `out_dir`: issue_run
2022-05-18 20:19:08,167 - INFO - Configuration variable `verbose`: 0
2022-05-18 20:19:08,167 - INFO - Configuration variable `seed`: None
2022-05-18 20:19:08,167 - INFO - Configuration variable `Length`: 15000
2022-05-18 20:19:08,167 - INFO - Configuration variable `Width`: 20000
2022-05-18 20:19:08,167 - INFO - Configuration variable `dx`: 200
2022-05-18 20:19:08,167 - INFO - Configuration variable `L0_meters`: 800
2022-05-18 20:19:08,167 - INFO - Configuration variable `S0`: 0.000133
2022-05-18 20:19:08,167 - INFO - Configuration variable `itermax`: 3
2022-05-18 20:19:08,167 - INFO - Configuration variable `Np_water`: 5000
2022-05-18 20:19:08,167 - INFO - Configuration variable `u0`: 1.5
2022-05-18 20:19:08,167 - INFO - Configuration variable `N0_meters`: 400
2022-05-18 20:19:08,167 - INFO - Configuration variable `h0`: 4.0
2022-05-18 20:19:08,167 - INFO - Configuration variable `hb`: None
2022-05-18 20:19:08,167 - INFO - Configuration variable `H_SL`: 0.0
2022-05-18 20:19:08,167 - INFO - Configuration variable `SLR`: 0.0
2022-05-18 20:19:08,167 - INFO - Configuration variable `Np_sed`: 3000
2022-05-18 20:19:08,167 - INFO - Configuration variable `f_bedload`: 0.5
2022-05-18 20:19:08,167 - INFO - Configuration variable `active_layer_thickness`: None
2022-05-18 20:19:08,167 - INFO - Configuration variable `C0_percent`: 0.1
2022-05-18 20:19:08,167 - INFO - Configuration variable `Csmooth`: 0.9
2022-05-18 20:19:08,167 - INFO - Configuration variable `toggle_subsidence`: False
2022-05-18 20:19:08,167 - INFO - Configuration variable `subsidence_rate`: 2e-09
2022-05-18 20:19:08,167 - INFO - Configuration variable `start_subsidence`: 216000
2022-05-18 20:19:08,168 - INFO - Configuration variable `save_eta_figs`: True
2022-05-18 20:19:08,168 - INFO - Configuration variable `save_stage_figs`: False
2022-05-18 20:19:08,168 - INFO - Configuration variable `save_depth_figs`: False
2022-05-18 20:19:08,168 - INFO - Configuration variable `save_discharge_figs`: False
2022-05-18 20:19:08,168 - INFO - Configuration variable `save_velocity_figs`: False
2022-05-18 20:19:08,168 - INFO - Configuration variable `save_sedflux_figs`: False
2022-05-18 20:19:08,168 - INFO - Configuration variable `save_sandfrac_figs`: False
2022-05-18 20:19:08,168 - INFO - Configuration variable `save_figs_sequential`: True
2022-05-18 20:19:08,168 - INFO - Configuration variable `save_metadata`: False
2022-05-18 20:19:08,168 - INFO - Configuration variable `save_eta_grids`: True
2022-05-18 20:19:08,168 - INFO - Configuration variable `save_stage_grids`: False
2022-05-18 20:19:08,168 - INFO - Configuration variable `save_depth_grids`: True
2022-05-18 20:19:08,168 - INFO - Configuration variable `save_discharge_grids`: True
2022-05-18 20:19:08,168 - INFO - Configuration variable `save_velocity_grids`: True
2022-05-18 20:19:08,168 - INFO - Configuration variable `save_sedflux_grids`: True
2022-05-18 20:19:08,168 - INFO - Configuration variable `save_sandfrac_grids`: True
2022-05-18 20:19:08,168 - INFO - Configuration variable `save_discharge_components`: True
2022-05-18 20:19:08,168 - INFO - Configuration variable `save_velocity_components`: False
2022-05-18 20:19:08,168 - INFO - Configuration variable `save_dt`: 86400
2022-05-18 20:19:08,168 - INFO - Configuration variable `checkpoint_dt`: None
2022-05-18 20:19:08,168 - INFO - Configuration variable `save_checkpoint`: False
2022-05-18 20:19:08,168 - INFO - Configuration variable `resume_checkpoint`: False
2022-05-18 20:19:08,168 - INFO - Configuration variable `omega_sfc`: 0.1
2022-05-18 20:19:08,168 - INFO - Configuration variable `omega_flow`: 0.9
2022-05-18 20:19:08,168 - INFO - Configuration variable `Nsmooth`: 10
2022-05-18 20:19:08,168 - INFO - Configuration variable `theta_water`: 1.0
2022-05-18 20:19:08,168 - INFO - Configuration variable `coeff_theta_sand`: 2.0
2022-05-18 20:19:08,169 - INFO - Configuration variable `coeff_theta_mud`: 1.0
2022-05-18 20:19:08,169 - INFO - Configuration variable `beta`: 3
2022-05-18 20:19:08,169 - INFO - Configuration variable `sed_lag`: 1.0
2022-05-18 20:19:08,169 - INFO - Configuration variable `coeff_U_dep_mud`: 0.4
2022-05-18 20:19:08,169 - INFO - Configuration variable `coeff_U_ero_mud`: 1.5
2022-05-18 20:19:08,169 - INFO - Configuration variable `coeff_U_ero_sand`: 1.05
2022-05-18 20:19:08,169 - INFO - Configuration variable `alpha`: 0.1
2022-05-18 20:19:08,169 - INFO - Configuration variable `stepmax`: None
2022-05-18 20:19:08,169 - INFO - Configuration variable `sand_frac_bc`: 0
2022-05-18 20:19:08,169 - INFO - Configuration variable `clobber_netcdf`: False
2022-05-18 20:19:08,169 - INFO - Configuration variable `legacy_netcdf`: False
2022-05-18 20:19:08,169 - INFO - Random seed is: 2744010775 
2022-05-18 20:19:08,328 - INFO - Model initialization complete
2022-05-18 20:19:08,328 - INFO - Time: 0.0; timestep: 0
2022-05-18 20:19:10,574 - INFO - Time: 40000.0; timestep: 1

[amoodie]

The config worked fine for me, creating a single job with the specified parameters.

pyDeltaRCM_20220518-143617.log

Can you confirm you are passing the yaml file to the high-level API? Either from the terminal as $ pyDeltaRCM --config issue.yaml or to the Preprocessor as pyDeltaRCM.Preprocessor('issue.yaml')? You can't use the any of the expansion behavior (matrix, ensemble, or set) with the low-level api as pyDeltaRCM.DeltaModel('issue.yaml').

[Lvulis]

We are using the low level processor, so using .DeltaModel. Could be worth updating the docs, skimmed them and didn't see it specifying needing to use the high level API for the expansion behavior.
Example code:

#import necessary packages
import pyDeltaRCM
import time

#set delta model... parameters are set from the configuration .YAML file, which can be adjusted for different case scenarios
default_delta = pyDeltaRCM.DeltaModel(input_file='WLD-configuration.yaml')

#model start time
t1 = time.time()

#model timestep as a forloop
for _ in range(0, 5):
    default_delta.update()
default_delta.finalize()

#model end time
t2 = time.time() 

#total model runtime
tfinal = t2- t1
print ('Total model runtime =', tfinal)

[amoodie]

cool yeah, that is not supported, but I agree I jsut looked at the docs, and I agree it's totally not clear that you can't do that.

I'll leave this open until I can update the docs. Thanks for pointing this out!!

[elbeejay]

Going to re-open this issue as I got the below warning when using the high-level API and specifying timesteps in my YAML:

UserWarning: One or more inputs in yaml file or kwargs were unused by the model during instantiation. The unused keys are: ['timesteps']
  warnings.warn(UserWarning(

Understandably this is from the if-else clause in init_tools.py. An easy solve might be to add an elif statement there to skip the acceptable high-level API parameters (timesteps, time, time_years, If, dryrun, and parallel), or modify the set of acceptable keywords at the start of that block of logic.

[amoodie]

Hm, can you confirm you're on the latest version / the development version? I thought I fixed this by not writing the time variables out to the job yamls here. Can you write a test to reproduce the warning?

[amoodie]

actually #259 that I never finished was supposed to add the parallel kw, so we could add these others you've suggested there, and then need tests to make sure it's working 👀