read.idat function misleading error when reading compressed files
2
0
Entering edit mode
@5de73a99
Last seen 3.9 years ago

Contrary to read.ilmn function, it seems that read.idat function is unable to read compressed idat files, returning quite misleading error in that case:


raw_object <- limma::read.idat(idatfiles = paste0(destdir, "/",idat_files), bgxfile = paste0(destdir, "/", bgxfile), ...)

idat_compressed_files

Besides, it would be great, just like read.ilmn function with paramters path and ctrlpath, to add parameters to function read.idat indicating folder where are respectively stored raw idat files and background file.

sessionInfo( )

R version 4.0.2 (2020-06-22)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: CentOS Linux 7 (Core)

Matrix products: default
BLAS:   /softhpc/R/4.0.2/lib64/R/lib/libRblas.so
LAPACK: /softhpc/R/4.0.2/lib64/R/lib/libRlapack.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8   
 [6] LC_MESSAGES=en_US.UTF-8    LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] bmkanalysis_1.0.0
idat illumina limma read.idat • 1.9k views
ADD COMMENT
0
Entering edit mode

This is probably an issue in the illuminaio package rather than limma. I'm pretty sure limma uses that package for reading IDAT files. I haven't looked at the code for a long time, but it could probably be updated to do the unzipping step automatically.

ADD REPLY
0
Entering edit mode

Effectively Illuminaio doesn't read compressed idat files. There may be a way for unzipping them automatically, or like for illumina gzipped files, possibility to read directly compressed files?

ADD REPLY
2
Entering edit mode
@gordon-smyth
Last seen 5 hours ago
WEHI, Melbourne, Australia

read.idat is documented to read IDAT files. There is no suggestion in the documentation that it will read a compressed file.

I suppose we could check for a .gz extension in the file name and throw an error, but otherwise checking for incorrect format binary files is very difficult.

In general, it makes little sense to compress a binary file like an IDAT file, because it is already dense and will not become much smaller. Partly for this reason it is quite rare for read functions in any R package to support compressed binary files. Automatic unzipping is generally only supported when reading text files. read.ilmn supports gzipped files because it is standard for text files to be compressed. read.idat does not support gzipped files because it is not standard for IDAT files to be compressed.

Mike Smith is the author of illuminaio::readIDAT on which read.idat depends. He has shown some interest in his comment above in supporting gzipped IDAT files, but it is up to him.

Five days later

I have now added a check for IDAT files with extension .gz. I have also added arguments path and bgxpath to read.idat, analogous to arguments path and ctrlpath for read.maimages and read.ilmn.

ADD COMMENT
0
Entering edit mode
@5de73a99
Last seen 3.9 years ago

Thanks for your answer, and for updating the code. Actually, this may reveal useful for those using GEO requests, as most of IDAT files are in gzipped format (even though I must admit most of users do not provide their IDAT raw files for Illumina, but such functions you're developping can ease systemic adding of raw data and normalization of conventions used).

ADD COMMENT

Login before adding your answer.

Traffic: 552 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6