Skip to content

adding capability to generate ensemble from era5 data on gcp #53

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
StevePny wants to merge 47 commits into
base: main
Choose a base branch
from
Draft

adding capability to generate ensemble from era5 data on gcp #53

StevePny wants to merge 47 commits into from

Conversation

@StevePny
Copy link
Owner

@StevePny StevePny commented Sep 23, 2024

This code pulls era5 data from gcp and either creates an ensemble by lagging through days, months, or years, from a given start date. The ensemble is recentered to the target date and then stored to a specified destination (s3, local, etc.).

@StevePny StevePny self-assigned this Sep 23, 2024
@StevePny StevePny added the enhancement New feature or request label Sep 23, 2024
@StevePny StevePny marked this pull request as draft September 23, 2024 21:21
StevePny and others added 16 commits September 23, 2024 15:27
@StevePny
added era5 var selection to main file
3a1506f
@kysolvik
ETKF can handle irregular obs now, but doesn't have proper indices info
9eddcf1
@kysolvik
Use system_index variable for H
bffcfbc
@kysolvik
Observer adds integer indices for easier H calculation
0cbd940
@kysolvik
Allow datavars to not match observed vars
e023257
@kysolvik
Updated system index and remove sort from the observer
6e9de1a
@kysolvik
Making some dacycler methods part of parent class for ease of mainten…
2f87b45 
…ance
@kysolvik
Initial version of xarray var4dBP, needs testing and cleaning
ccf7e03
@kysolvik
Cleaned var4dbp using xarray
3c09ef3
@kysolvik
All dacyclers working with xarray, but possibly some accuracy issues …
cbc308f 
…with 4dvar and 4dvarBP
@kysolvik
State Vec has delta_t attribute and M is provided as xarray
1690992
@kysolvik
Working RC Model with xarray
8b81d55
@kysolvik
Fixed generator extra step rounding error
e3cd320
@kysolvik
Observer can accept random_time_density now
19572e3
@kysolvik
Add permissible xarray jax to pyproject toml
e5bfe0e
@kysolvik
Remove all vector module imports
789019a
kysolvik and others added 16 commits September 27, 2024 11:48
@kysolvik
Rename i to index for toy data generators
f0f982d
@kysolvik
GCP system dim now specified
e8bd897
@kysolvik
Observer can accept list of error_sds, and now samples with replaceme…
17cab8f 
…nt when there is more than 1 dimension to sample along
@kysolvik
Fixed issue with missing time offset for 4dvar and 4dvarBP
43c9d2b
@kysolvik
Reassign coords to match input state within dacycler
5a916f5
@kysolvik
Apply coord reassing for 4dvar cycler too
6a2e44e
@kysolvik
Remove unnecessary print from 3dvar
2758b82
@kysolvik
Reassign coords for outer loop carry instead of dropping time
a515768
@kysolvik
Updated gcp to properly assign system_dim
96721df
@kysolvik
Assign system_dim as attr, not coord
2c7686d
@kysolvik
Fix typo for xarray_jax git repo
974613c
@kysolvik
XArray accessors for helper methods
3fdfabd
@StevePny
Merge branch 'main' of github.com:StevePny/DataAssimBench into dev/er…
f451be0 
…a5ens
@StevePny
Merge branch 'main' of github.com:StevePny/DataAssimBench into dev/er…
07220fb 
…a5ens
@StevePny
bugfix
93aff5c
@StevePny
add support for setting up era5 initial ensemble
becfa0f
@StevePny StevePny mentioned this pull request Oct 3, 2024
Open
@StevePny StevePny requested a review from kysolvik October 3, 2024 08:52
@StevePny StevePny changed the title adding capbility to generate ensemble from era5 data on gcp adding capability to generate ensemble from era5 data on gcp Oct 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kysolvik kysolvik Awaiting requested review from kysolvik

Assignees

@StevePny StevePny

Labels

enhancement New feature or request

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@StevePny @kysolvik