Question

A parser for Tiled Data File (TDF) format?

1

Entering edit mode

Henrik Bengtsson ★ 2.4k

@henrik-bengtsson-4333

Last seen 7 months ago

United States

Hi, does anyone know of a parser, preferably in R, for the Tiled Data File (TDF) format that is used by the Broad's IGV Browser?

The TDF format is binary, but appears to be poorly documented. It looks like it is mostly intended as an intermediate file format to speed up IGV, and the Broad's igvtools can generate TDF files from other sources. Unfortunately, there is no method for converting/dumping TDFs into, say, plain text.

The background is a need for reading TDF files published on GEO, e.g. http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE70991

Thxs

file format parser IO TDF IGV • 4.2k views

ADD COMMENT • link 8.8 years ago Henrik Bengtsson ★ 2.4k

score 0 · Answer 1 · 2016-03-01

0

Entering edit mode

Henrik Bengtsson ★ 2.4k

@henrik-bengtsson-4333

Last seen 7 months ago

United States

As a follow up to my own question, I've found that Java class org.broad.igv.tdf.TDFUtils part of IGV to some extent can be used to parse TDF files. For instance, one can convert a TDF to a BED file from within R as:

> library("rJava")
> .jinit(classpath="igvtool.jar")
> TDFUtils <- .jnew("org/broad/igv/tdf/TDFUtils")
> TDFUtils$tdfToBedgraph("GSM1824797_bt142_idm_ctcf_1.tdf", "GSM1824797_bt142_idm_ctcf_1.bed")

Unfortunately, multi-track TDF files cannot be represented as BED files, so this won't work in all cases. There are other methods in the TDFUtils Java class that could be used to parse the TDF file, but that would probably require some additional work in order to coerce Java objects into R objects.

However, for my immediate needs, conversion to BED will probably suffice.

BTW, from this I spun off question Is there an R package providing an R API for IGV / igvtools out there?

ADD COMMENT • link 8.8 years ago Henrik Bengtsson ★ 2.4k

0

Entering edit mode

Do you know if this trick still works, Henrik? I can't get it working on my machine but the combination of rJava, a conda-installed igvtools, and an the general lack of documentation on TDF makes me unsure where the fault lies.

library(rJava)
.jinit(classpath="~/miniconda3/share/igvtools-2.3.93-0/igvtools.jar")
TDFUtils <- .jnew("org/broad/igv/tdf/TDFUtils")
#> Error in .jnew("org/broad/igv/tdf/TDFUtils"): java.lang.ClassNotFoundException

devtools::session_info()
#> ─ Session info ──────────────────────────────────────────────────────────
#>  setting  value                       
#>  version  R version 3.5.1 (2018-07-02)
#>  os       Ubuntu 18.04.1 LTS          
#>  system   x86_64, linux-gnu           
#>  ui       X11                         
#>  language en_AU:en                    
#>  collate  en_AU.UTF-8                 
#>  ctype    en_AU.UTF-8                 
#>  tz       Australia/Melbourne         
#>  date     2018-11-26                  
#> 
#> ─ Packages ──────────────────────────────────────────────────────────────
#>  package     * version date       lib source                         
#>  assertthat    0.2.0   2017-04-11 [3] CRAN (R 3.5.0)                 
#>  backports     1.1.2   2017-12-13 [3] CRAN (R 3.5.0)                 
#>  base64enc     0.1-3   2015-07-28 [3] CRAN (R 3.5.0)                 
#>  callr         3.0.0   2018-08-24 [3] CRAN (R 3.5.1)                 
#>  cli           1.0.1   2018-09-25 [3] CRAN (R 3.5.1)                 
#>  crayon        1.3.4   2017-09-16 [3] CRAN (R 3.5.0)                 
#>  desc          1.2.0   2018-05-01 [3] CRAN (R 3.5.0)                 
#>  devtools      2.0.1   2018-10-26 [1] CRAN (R 3.5.1)                 
#>  digest        0.6.18  2018-10-10 [1] CRAN (R 3.5.1)                 
#>  evaluate      0.12    2018-10-09 [3] CRAN (R 3.5.1)                 
#>  fs            1.2.6   2018-08-23 [1] CRAN (R 3.5.1)                 
#>  glue          1.3.0   2018-07-17 [3] CRAN (R 3.5.1)                 
#>  htmltools     0.3.6   2017-04-28 [3] CRAN (R 3.5.0)                 
#>  knitr         1.20    2018-02-20 [3] CRAN (R 3.5.0)                 
#>  magrittr      1.5     2014-11-22 [3] CRAN (R 3.5.0)                 
#>  memoise       1.1.0   2018-10-25 [1] Github (hadley/memoise@99db7eb)
#>  pkgbuild      1.0.2   2018-10-16 [1] CRAN (R 3.5.1)                 
#>  pkgload       1.0.2   2018-10-29 [3] CRAN (R 3.5.1)                 
#>  prettyunits   1.0.2   2015-07-13 [1] CRAN (R 3.5.1)                 
#>  processx      3.2.0   2018-08-16 [3] CRAN (R 3.5.1)                 
#>  ps            1.2.1   2018-11-06 [1] CRAN (R 3.5.1)                 
#>  R6            2.3.0   2018-10-04 [3] CRAN (R 3.5.1)                 
#>  Rcpp          1.0.0   2018-11-07 [1] CRAN (R 3.5.1)                 
#>  remotes       2.0.2   2018-10-30 [1] CRAN (R 3.5.1)                 
#>  rJava       * 0.9-10  2018-05-29 [3] CRAN (R 3.5.0)                 
#>  rlang         0.3.0.1 2018-10-25 [1] CRAN (R 3.5.1)                 
#>  rmarkdown     1.10    2018-06-11 [3] CRAN (R 3.5.0)                 
#>  rprojroot     1.3-2   2018-01-03 [1] CRAN (R 3.5.1)                 
#>  sessioninfo   1.1.1   2018-11-05 [1] CRAN (R 3.5.1)                 
#>  stringi       1.2.4   2018-07-20 [3] CRAN (R 3.5.1)                 
#>  stringr       1.3.1   2018-05-10 [3] CRAN (R 3.5.0)                 
#>  testthat      2.0.1   2018-10-13 [3] CRAN (R 3.5.1)                 
#>  usethis       1.4.0   2018-08-14 [1] CRAN (R 3.5.1)                 
#>  withr         2.1.2   2018-03-15 [3] CRAN (R 3.5.0)                 
#>  yaml          2.2.0   2018-07-25 [3] CRAN (R 3.5.1)                 
#> 
#> [1] /home/peter/R/x86_64-pc-linux-gnu-library/3.5
#> [2] /usr/local/lib/R/site-library
#> [3] /usr/lib/R/site-library
#> [4] /usr/lib/R/library

ADD REPLY • link 6.1 years ago Peter Hickey ▴ 740

1

Entering edit mode

Don't know, haven't tried for a long time. But, try to use a path without ~, e.g. normalize it with normalizePath(...).

ADD REPLY • link 6.1 years ago Henrik Bengtsson ★ 2.4k

0

Entering edit mode

Using `normalizePath()` worked. Thanks, Henrik!

ADD REPLY • link 6.1 years ago Peter Hickey ▴ 740

0

Entering edit mode

I think this means TDFUtils exists in the .jar file, but I'm pretty outside my knowledge here.

$ jar tf ~/miniconda3/share/igvtools-2.3.93-0/igvtools.jar | grep TDFUtils
org/broad/igv/tdf/TDFUtils.class

ADD REPLY • link 6.1 years ago Peter Hickey ▴ 740