How To Fix "no Numpy Equivalent For Typebitfieldid Exists" Error In Python

I am reading a .h5 file with h5py module. What i am trying to achieve here is print all the groups and all the datasets within a group without knowing the content structure of the

Solution 1:

With this visit function, I can get information on all the datasets that raise this node.dtype error:

deffoo1(name,node):
    #print(name)ifisinstance(node, h5py.Dataset):
        try:
            node.dtype
        except TypeError as err:
            print(name)
            print(node.size, node.shape)
            print(err)

I get a couple of screens worth, with a typical display like:

0 (0,)
No NumPy equivalent for TypeBitfieldID exists
AwakeEventData/GD_BPM.AWAKE.TRIUMF/AcquisitionSPS/posOK
1 (1,)
No NumPy equivalent for TypeBitfieldID exists
AwakeEventData/GD_BPM.AWAKE.TRIUMF/GlobalAcquisition/posOK

So if your goal is just to visit everything, and display the information that you can, add a try/except like this to your visit function.

The h5dump display for one of those datasets is:

2215:~/mypy$ h5dump -d /AwakeEventData/GD_BPM.AWAKE.TRIUMF/AcquisitionSPS/posOK ../Downloads/1541962108935000000_167_838.h5
HDF5 "../Downloads/1541962108935000000_167_838.h5" {
DATASET "/AwakeEventData/GD_BPM.AWAKE.TRIUMF/AcquisitionSPS/posOK" {
   DATATYPE  H5T_STD_B64LE
   DATASPACE  SIMPLE { ( 1 ) / ( H5S_UNLIMITED ) }
   DATA {
   (0): 80:17:00:00:00:00:00:00
   }
   ATTRIBUTE "bitFieldSize" {
      DATATYPE  H5T_STD_I64LE
      DATASPACE  SCALAR
      DATA {
      (0): 14
      }
   }
}
}

Adding print(list(node.attrs.values())) displays that bitFieldSize attribute.

There are other, non-python viewers. I don't know if pytables or pandas could read this file or not.

Solution 2:

Yes, this file is an interesting curiosity. HDFView has no trouble opening or viewing the data (even the troublesome ones). I wrote a little pytables code to walk the group hierarchy and report the leaf names. It issues this warning for several datasets:

DataTypeWarning: Unsupported type for attribute 'exception'in node 'BinningSetting'. Offending HDF5 class: 8

When I look at these datasets in HDFView, they show

Name: exception
Type:8-bit enun (FALSE=0, TRUE=1)

Unfortunately, I don't know enough about HDF5 or pytables to explain what's going on. It's interesting that some of these datasets are different from those mentioned by @hpaulj.

Here's my code (warning: it creates a mountain of output):

import tables as tb

h5f = tb.open_file('1541962108935000000_167_838.h5',mode='r')
for grp in h5f.walk_groups('/') : 
    grp_leaves = grp._v_leaves
    iflen(grp_leaves) > 0 :
        print ('Group: ', grp)
        for grp_leaf in grp_leaves :
            print ('\tLeaf:', grp_leaf)

The first few offending groups are:

Group:  /AwakeEventData/XUCL-SPECTRO/BinningSettingGroup:  /AwakeEventData/XUCL-SPECTRO/CameraSettingsGroup:  /AwakeEventData/XMPP-STREAK/StreakImageGroup:  /AwakeEventData/TT43.BPM.430308/AcquisitionGroup:  /AwakeEventData/TT41.BTV.412426/Image

Does that help?