Low level usage of rhdf5
1
0
Entering edit mode
rubi ▴ 110
@rubi-6462
Last seen 6.3 years ago

I'm trying to create an hdf5 file using R's rhdf5 package.

I want to create this group hierarchy: "top/bottom". In "bottom", I'd like to have two datasets (both H5S_SCALARs):

  1. "score" which is an "H5T_IEEE_F64LE" type
  2. "type" which is an "H5T_STRING" type

I thought this would work:

 

output.path <- "my.h5.dir"
dir.create(output.path,recursive = T)
h5.file <- rhdf5::H5Fcreate(paste0(output.path,"/test.h5"))

top.group <- rhdf5::H5Gcreate(h5.file,"top")
bottom.group <- rhdf5::H5Gcreate(top.group,"_bottom")

h5.space <- rhdf5::H5Screate(type="H5S_SCALAR")
h5.dataset <- rhdf5::H5Dcreate(h5loc=bottom.group,name="score",dtype_id="H5T_IEEE_F64LE",h5space=h5.space)
rhdf5::H5Dwrite(h5.dataset,array(c(1,2)))
rhdf5::H5Dclose(h5.dataset)
rhdf5::H5Sclose(h5.space)

h5.space <- rhdf5::H5Screate(type="H5S_SCALAR")
h5.dataset <- rhdf5::H5Dcreate(h5loc=bottom.group,name="type",dtype_id="H5T_STRING",h5space=h5.space)
rhdf5::H5Dwrite(h5.dataset,"text")
rhdf5::H5Dclose(h5.dataset)
rhdf5::H5Sclose(h5.space)

 

However,

h5.dataset <- rhdf5::H5Dcreate(h5loc=bottom.group,name="type",dtype_id="H5T_STRING",h5space=h5.space)

 

throws this error:

Error in rhdf5::H5Dcreate(h5loc = bottom.group, name = "type", dtype_id = "H5T_STRING",  : 
  HDF5. Invalid arguments to routine. Inappropriate type.

 

Although I do see the H5T_STRING type:

 

> "H5T_STRING" %in% rhdf5::h5const("H5T")
[1] TRUE

 

Any idea?

 

BTW this also happens if I try:

h5.space <- rhdf5::H5Screate(type="H5S_SIMPLE")

h5.dataset <- rhdf5::H5Dcreate(h5loc=bottom.group,name="val",dtype_id="H5T_STD_I64LE",h5space=h5.space)


Error in rhdf5::H5Dcreate(h5loc = bottom.group, name = "val", dtype_id = "H5T_STD_I64LE",  : 
  HDF5. Dataset. Unable to initialize object.

 

 

> sessionInfo()
R version 3.3.3 (2017-03-06)
Platform: x86_64-apple-darwin13.4.0 (64-bit)
Running under: OS X El Capitan 10.11.6

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] rhdf5_2.16.0

loaded via a namespace (and not attached):
[1] zlibbioc_1.18.0 tools_3.3.3    

 

rhdf5 • 1.5k views
ADD COMMENT
0
Entering edit mode

I see the same issue - it might take a while to get the bottom of this. It's interesting that H5T_STRING doesn't appear in the specs at https://support.hdfgroup.org/HDF5/doc1.8/RM/PredefDTypes.html You can try H5T_C_S1 but that only gives you a string of length 1 (which I guess is the null terminator anyway). I'll keep looking, it maybe that I need to add something to rhdf5 to get this to work.

ADD REPLY
0
Entering edit mode

Thanks a lot.

 

It's also failing with these type combinations:

h5.space <- rhdf5::H5Screate(type="H5S_SIMPLE")

h5.dataset <- rhdf5::H5Dcreate(h5loc=bottom.group,name="val",dtype_id="H5T_STD_I32LE",h5space=h5.space)

 

h5.space <- rhdf5::H5Screate(type="H5S_SIMPLE")

h5.dataset <- rhdf5::H5Dcreate(h5loc=bottom.group,name="val",dtype_id="H5T_STRING",h5space=h5.space)

 

h5.space <- rhdf5::H5Screate(type="H5S_SIMPLE")

h5.dataset <- rhdf5::H5Dcreate(h5loc=bottom.group,name="val",dtype_id="H5T_IEEE_F64LE",h5space=h5.space)

 

 

ADD REPLY
0
Entering edit mode
Mike Smith ★ 6.6k
@mike-smith
Last seen 1 minute ago
EMBL Heidelberg

So this example demonstrates how to write a single column dataset containing strings of up to length 10 characters (you can set the limit).

I haven't tried any of your other examples yet, but thank you for listing them. I'll create a Github issue to remind me to look at them later.

I'm not sure the H5T_STRING should be listed in the output from rhdf5::h5const("H5T"), I think this may be an error in the rhdf5 code so I'll check that too.

library(rhdf5)

## file to create
file_name <- '/tmp/test.h5'

## character strings we're going to add to the file
text <- c("text", "text2", 'abcdefghijklmnop')

## create file and 
h5.file <- H5Fcreate(file_name)
top.group <- H5Gcreate(h5.file,"top")

## our string dataset is one-dimensional and
## as long as the number of strings we want to add
h5.space <- H5Screate_simple(dims = length(text), 
                             maxdims = length(text))

## create a string datatype
## this example allows strings upto length 10
## it will truncate any longer like our 3rd entry above
tid <- H5Tcopy(dtype_id="H5T_C_S1")
H5Tset_size(tid, size = 10L)

## if you don't want truncated, so something like
## H5Tset_size(tid, size = max(nchar(text))+1)

## cratee and write the dataset
h5.dataset <- rhdf5::H5Dcreate(h5loc = top.group,
                               name = "type",
                               dtype_id = tid,
                               h5space = h5.space)
rhdf5::H5Dwrite(h5.dataset, buf = text)

## close everything
H5Dclose(h5.dataset)
H5Sclose(h5.space)
H5Gclose(top.group)
H5Fclose(h5.file)
ADD COMMENT

Login before adding your answer.

Traffic: 785 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6