Error when reading some h5 files

[rui-coelho]

The IDS created with IMAS Python, from h5dump, provenance says:

ids_properties&version_put&access_layer: "5.5.2"
ids_properties&version_put&access_layer_language: "IMAS-Python 2.1.0.post1"
ids_properties&version_put&data_dictionary: "3.42.0"

Another IDS created with AL Python which i can read easily, from h5dump, provenance says:

ids_properties&version_put&access_layer: "5.4.3"
ids_properties&version_put&access_layer_language: "python-5.4.0"
ids_properties&version_put&data_dictionary: "3.42.0"

I attach zip files for the two folders with the master and equilibrium IDSs for both cases.

DD3+AL5_AL-Python.zip
DD3+AL5_IMAS-Python.zip

[rui-coelho]

Forgot to add the environment i am using in AL5+DD3 to read both sets of IDSs:

IMAS-AL-Core/5.4.3-intel-2023b
Data-Dictionary/3.42.0-GCCcore-13.2.0
IMAS-AL-MDSplus-models/5.2.2-GCCcore-13.2.0-DD-3.42.0
IMAS-AL-Cpp/5.4.0-intel-2023b-DD-3.42.0
IMAS-AL-Fortran/5.4.0-intel-2023b-DD-3.42.0
IMAS-AL-Java/5.4.0-intel-2023b-DD-3.42.0
IMAS-AL-Python/5.4.0-intel-2023b-DD-3.42.0
IMAS/3.42.0-2025.01-intel-2023b
HDF5/1.14.4.3-iimpi-2023b(1.14.3-iimpi-2023b)

I get it all from a meta module called imasenv prepared by the ACH team of PSNC which uses the same EasyBuild fabric from ITER.

[olivhoenen]

Hi @rui-coelho, this seems more to be an issue with AL-Python or its install on your system.
Your description does not help understanding what's not working when you read this equilibrium IDS with this legacy library.

Note that with your archive, I can read (get) the IDS with IMAS-AL-Python/5.4.0-foss-2023b-DD-3.42.0 as installed on SDCC:

[rui-coelho]

And with get_slice ?

[olivhoenen]

Same. If you think the error can be reproduced on my end maybe you can share a precise use-case / code snippet, otherwise I'm shooting in the dark here.

>>> eq = db.get_slice("equilibrium",50.0,imas.imasdef.PREVIOUS_SAMPLE)
>>> eq.time
array([49.97502899])

[rui-coelho]

I forwarded the issue to Michal so that he checks on the Gateway what’s wrong since in theory the installation should stem from the EB recipes…I’ll come back with the result.

[bpalak]

@rui-coelho : I will try to reproduce your case, however, I would agree with Olivier that it is not easy to solve this case without knowledge what went wrong :(

[bpalak]

@rui-coelho : I reproduce Olivier's test on the GW, obtaining the same (correct) results.

Please check if any locally installed packages (~/.local etc) does not influence the execution.

[bpalak]

shell> module load imasenv
shell> python test.py

test.py:

import imas

print(imas)

db = imas.DBEntry("imas:hdf5?path=DD3+AL5_IMAS-Python","r")
eq = db.get("equilibrium")
print(eq.ids_properties)

eq = db.get_slice("equilibrium",50.0,imas.imasdef.PREVIOUS_SAMPLE)
print(eq.time)

[rui-coelho]

<g2rcoelh@viz05 ~>module list
Currently Loaded Modulefiles:
 1) profile/base  

Key:
default-version  
<g2rcoelh@viz05 ~>USE_EB_3420 

<g2rcoelh@viz05 ~>idsprint -u "imas:hdf5?path=/afs/eufus.eu/user/g/g2apark/IDS-tests/data"#equilibrium
ERROR:root:b'al_plugin_begin_global_action: [ALBackendException = Unable to open HDF5 group: equilibrium\n]'
Traceback (most recent call last):
  File "/gw/swimas/software/IDStools/2.0.0-iimkl-2023b/bin/idsprint", line 86, in <module>
    ids = connection.get(ids_name, occurrence)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/gw/swimas/software/IMAS-AL-Python/5.4.0-intel-2023b-DD-3.42.0/lib/python3.11/site-packages/imas/db_entry.py", line 387, in get
    ids.get(occurrence, self)
  File "/gw/swimas/software/IMAS-AL-Python/5.4.0-intel-2023b-DD-3.42.0/lib/python3.11/site-packages/imas/equilibrium.py", line 2056, in get
    homogeneousTime = self.readHomogeneous(db_ctx, occurrence)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/gw/swimas/software/IMAS-AL-Python/5.4.0-intel-2023b-DD-3.42.0/lib/python3.11/site-packages/imas/ids_base.py", line 275, in readHomogeneous
    raise ALException(
imas_core.exception.ALException: Error calling al_begin_global_action() in readHomogeneous() operation
Error status=-3


<g2rcoelh@viz05 ~>alias | grep USE_EB_3420
ASCOTenv_EB_3420	USE_EB_3420
USE_EB_3420	module purge ; module load ach-modules ; module load imasenv/3.42.0/intel/2023b/rc; module load h5py/3.11.0-intel-2023b; setenv MPLBACKEND TkAgg; setenv IMAS_AL_DISABLE_VALIDATE 1; setenv XDG_DATA_DIRS /usr/share:/usr/local/share:$XDG_DATA_DIRS
<g2rcoelh@viz05 ~>

[rui-coelho]

that was for the IDS which Adam created using a IMAS Python environment. Now for the IDS i generated using AL Python :

class equilibrium
Attribute ids_properties
	class ids_properties
	Attribute comment: full new default xml input                                                                                          
                Chease run from Kepler                                                                                                         
     Chease run from Kepler                                                                                                              Chease
 run from Kepler                                                                                                              COCOS OUT 11
	Attribute homogeneous_time: 1
	Attribute provider: CHEASE git hash f8c33e9f4e2c3bd471b6100a586567fd805ba5bd_branch_HEAD
	Attribute creation_date: 20260205
	Attribute version_put
		class version_put
		Attribute data_dictionary: 3.42.0
		Attribute access_layer: 5.4.3
		Attribute access_layer_language: python-5.4.0
Attribute vacuum_toroidal_field
	class vacuum_toroidal_field
	Attribute r0: 6.2
	Attribute b0
	[-5.3]
Attribute time_slice
	class time_slicedynamic
	time_slicedynamic[0] = 
		class time_slicedynamicObj
		Attribute boundary
			class boundary
			Attribute type: 0
--More--

[rui-coelho]

i did this in a fresh shell and yes...it's a tcsh shell

[bpalak]

@rui-coelho ... Now I really get lost... :(
It is a new issue? Or something related to the originally reported one?

[rui-coelho]

of course it is the same, i was in a hurry so didn't test your snippet, had at hand ids print instead, I will now test with your snippet

[rui-coelho]

<g2rcoelh@viz05 ~/public/for_CPT>module purge
<g2rcoelh@viz05 ~/public/for_CPT>module load ach-modules
<g2rcoelh@viz05 ~/public/for_CPT>module load imasenv
<g2rcoelh@viz05 ~/public/for_CPT>python3 IMAS-Python.py
<module 'imas' from '/gw/swimas/software/IMAS-AL-Python/5.4.0-intel-2023b-DD-3.42.0/lib/python3.11/site-packages/imas/__init__.py'>
ERROR:root:b'al_plugin_begin_global_action: [ALBackendException = Unable to open HDF5 group: equilibrium\n]'
Traceback (most recent call last):
  File "/afs/eufus.eu/g2itmdev/user/g2rcoelh/public/for_CPT/IMAS-Python.py", line 4, in <module>
    eq = db.get("equilibrium")
         ^^^^^^^^^^^^^^^^^^^^^
  File "/gw/swimas/software/IMAS-AL-Python/5.4.0-intel-2023b-DD-3.42.0/lib/python3.11/site-packages/imas/db_entry.py", line 387, in get
    ids.get(occurrence, self)
  File "/gw/swimas/software/IMAS-AL-Python/5.4.0-intel-2023b-DD-3.42.0/lib/python3.11/site-packages/imas/equilibrium.py", line 2056, in get
    homogeneousTime = self.readHomogeneous(db_ctx, occurrence)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/gw/swimas/software/IMAS-AL-Python/5.4.0-intel-2023b-DD-3.42.0/lib/python3.11/site-packages/imas/ids_base.py", line 275, in readHomogeneous
    raise ALException(
imas_core.exception.ALException: Error calling al_begin_global_action() in readHomogeneous() operation
Error status=-3

[bpalak]

@rui-coelho : Could you kindly make a simple test?
Please COPY /afs/eufus.eu/user/g/g2apark/IDS-tests/data to any of your dir and try idsprint on copied files...