add new obstype is difficult

[spuynen]

#Createing an obstype

wbgt_obstype = mt.Obstype(obsname='WBGT', std_unit='degree_Celsius',
                  description='The Wet Bulb Globe Temperature as an index for heat stress, calculated from T, BGT and RH')

wbgt_obstype._original_unit = 'degree_Celsius'

mt_dataset.add_new_observationtype(wbgt_obstype)
#%%
from metobs_toolkit import SensorData


for staname in WBGT.index.get_level_values('name').unique():
    stawbgt = WBGT.xs(staname, level='name', drop_level=False)
    
    
    wbgt_sensor = SensorData(stationname=staname,
                             datarecords=stawbgt['value'].to_numpy(),
                             timestamps=stawbgt.index.get_level_values('datetime').to_numpy(),
                             obstype=wbgt_obstype)
    
    mt_dataset.get_station(staname).add_to_sensordata(wbgt_sensor, force_update=True)


mt_dataset.make_plot(obstype='WBGT', colorby='station')

[vergauwenthomas]

I agree that it is difficult to add custom records to the toolkit, after data is imported. However, i don't i can automate this procedure. My proposal is to

• Add an example in the documentation (section special topics)
• setter for original unit
• add some proper error messages that can help the user.

@spuynen can you give me more details (code snipets) on how you compute the WBGT? I can use that as an example.

[spuynen]

Hey Thomas, sounds good, here is the code i use. Note that normally its within a 'calculation module' that i made, but i copy pasted the code from the function so that it should be clear without clutter.

df = mt_dataset.df

# Assigning variables as dataframes

RH_df = pd.DataFrame(df.loc[mt_dataset.df.index.get_level_values('obstype')=='humidity'])
Tair_df = pd.DataFrame(df.loc[mt_dataset.df.index.get_level_values('obstype')=='temp'])
Tg_df = pd.DataFrame(df.loc[mt_dataset.df.index.get_level_values('obstype')=='radiation_temp'])
WS_df = pd.DataFrame(df.loc[mt_dataset.df.index.get_level_values('obstype')=='wind_speed'])

WBGT_df = Tg_df # temp df for WBGT


# prep data for element wise calculations

RH = RH_df.reset_index()['value']
Tair = Tair_df.reset_index()['value']
Tg = Tg_df.reset_index()['value']
WS = WS_df.reset_index()['value']


# prep WBGT df for adding the series after calculation

WBGT_df = WBGT_df.reset_index()
WBGT_df['obstype'] = 'WBGT'


### Create and add new obstypes to later store data in

# WBGT

WBGT_obstype = mt.Obstype(obsname='WBGT', std_unit='degree_Celsius',
                  description='The Wet Bulb Globe Temperature as an index for heat stress, calculated from T, BGT and RH')

WBGT_obstype._original_unit = 'degree_Celsius'

if 'WBGT' in mt_dataset.obstypes == True:
    Warning('WBGT obstype already present in dataset. Data will be overwritten')
    
elif 'WBGT' in mt_dataset.obstypes == False:  
    mt_dataset.add_new_observationtype(WBGT_obstype)
    print('WBGT added as an observation type in the dataset')


# Wet bulb temperature calculation

if method == 'Chen2022':
    Tw = calculate_Tw_Chen2022(Tair, RH)
    
elif method == 'ISO2017':
    Tw = calculate_Tw_ISO2017(Tg, Tair, WS, RH)


# WBGT formula (ISO2017)

WBGT_data = 0.7 * Tw + 0.2 * Tg + 0.1 * Tair
   

# Add WBGT calculated data to the df

WBGT_df.loc[WBGT_data.index, 'value'] = WBGT_data


# Convert to initial Long format

WBGT_df = WBGT_df.reset_index().set_index(["name", "obstype", "datetime"]).drop('index', axis=1)


### Add WBGT data to the dataset

# Add data per station as new sensordata

stations = WBGT_df.index.get_level_values('name').unique()

for station in stations:
    stawbgt = WBGT_df.xs(station, level='name', drop_level=False)

    wbgt_sensor = mt.SensorData(stationname=station,
                                datarecords=stawbgt['value'].to_numpy(),
                                timestamps=stawbgt.index.get_level_values('datetime').to_numpy(),
                                obstype=WBGT_obstype)
    
    mt_dataset.get_station(station).add_to_sensordata(wbgt_sensor, force_update=True)