Error with dba.count(dbObj) in DiffBind
1
0
Entering edit mode
Goku • 0
@goku
Last seen 3.7 years ago

Enter the body of text here

Hi

Can anyone help with the following error message. Thanks.

> dbObj <- dba.count(dbObj)
Computing summits... 
Error: Error processing one or more read files. Check warnings().
In addition: There were 50 or more warnings (use warnings() to see the first 50)

> dbObj$class[10:11,]

          XX_ChipH3Ia_DA_Rep1
bamRead    "/XX_ChipH3Ia_DA_Rep1.bam"
bamControl "/XX_input_DA_Rep1.bam"
           XX_ChipH3Ia_DA_Rep2
bamRead    "/XX_ChipH3Ia_DA_Rep2.bam"
bamControl "/XX_input_DA_Rep2.bam"
           XX_ChipH3Ia_DB_Rep1
bamRead    "/XX_ChipH3Ia_DB_Rep1.bam"
bamControl "/XX_input_DB_Rep1.bam"
           XX_ChipH3Ia_DB_Rep2
bamRead    "/XX_ChipH3Ia_DB_Rep2.bam"
bamControl "/XX_input_DB_Rep2.bam"
           XX_ChipH3Ia_DC_Rep1
bamRead    "/XX_ChipH3Ia_DC_Rep1.bam"
bamControl "/XX_input_DC_Rep1.bam"
           XX_ChipH3Ia_DC_Rep2
bamRead    "/XX_ChipH3Ia_DC_Rep2.bam"
bamControl "/XX_input_DC_Rep2.bam"
           XX_ChipH3Ib_DA_Rep1
bamRead    "/XX_ChipH3Ib_DA_Rep1.bam"
bamControl "/XX_input_DA_Rep1.bam"
           XX_ChipH3Ib_DA_Rep2
bamRead    "/XX_ChipH3Ib_DA_Rep2.bam"
bamControl "/XX_input_DA_Rep2.bam"
           XX_ChipH3Ib_DB_Rep1
bamRead    "/XX_ChipH3Ib_DB_Rep1.bam"
bamControl "/XX_input_DB_Rep1.bam"
           XX_ChipH3Ib_DB_Rep2
bamRead    "/XX_ChipH3Ib_DB_Rep2.bam"
bamControl "/XX_input_DB_Rep2.bam"
           XX_ChipH3Ib_DC_Rep1
bamRead    "/XX_ChipH3Ib_DC_Rep1.bam"
bamControl "/XX_input_DC_Rep1.bam"
           XX_ChipH3Ib_DC_Rep2
bamRead    "/XX_ChipH3Ib_DC_Rep2.bam"
bamControl "/XX_input_DC_Rep2.bam"
           XX_ChipH3Ie_DA_Rep1
bamRead    "/XX_ChipH3Ie_DA_Rep1.bam"
bamControl "/XX_input_DA_Rep1.bam"
           XX_ChipH3Ie_DA_Rep2
bamRead    "/XX_ChipH3Ie_DA_Rep2.bam"
bamControl "/XX_input_DA_Rep2.bam"
           XX_ChipH3Ie_DB_Rep1
bamRead    "/XX_ChipH3Ie_DB_Rep1.bam"
bamControl "/XX_input_DB_Rep1.bam"
           XX_ChipH3Ie_DB_Rep2
bamRead    "/XX_ChipH3Ie_DB_Rep2.bam"
bamControl "/XX_input_DB_Rep2.bam"
           XX_ChipH3Ie_DC_Rep1
bamRead    "/XX_ChipH3Ie_DC_Rep1.bam"
bamControl "/XX_input_DC_Rep1.bam"
           XX_ChipH3Ie_DC_Rep2
bamRead    "/XX_ChipH3Ie_DC_Rep2.bam"
bamControl "/XX_input_DC_Rep2.bam"
           XX_ChipH3Ic_DA_Rep1
bamRead    "/XX_ChipH3Ic_DA_Rep1.bam"
bamControl "/XX_input_DA_Rep1.bam"
           XX_ChipH3Ic_DA_Rep2
bamRead    "/XX_ChipH3Ic_DA_Rep2.bam"
bamControl "/XX_input_DA_Rep2.bam"
           XX_ChipH3Ic_DB_Rep1
bamRead    "/XX_ChipH3Ic_DB_Rep1.bam"
bamControl "/XX_input_DB_Rep1.bam"
           XX_ChipH3Ic_DB_Rep2
bamRead    "/XX_ChipH3Ic_DB_Rep2.bam"
bamControl "/XX_input_DB_Rep2.bam"
           XX_ChipH3Ic_DC_Rep1
bamRead    "/XX_ChipH3Ic_DC_Rep1.bam"
bamControl "/XX_input_DC_Rep1.bam"
           XX_ChipH3Ic_DC_Rep2
bamRead    "/XX_ChipH3Ic_DC_Rep2.bam"
bamControl "/XX_input_DC_Rep2.bam"
           XX_ChipH3Id_DA_Rep1
bamRead    "/XX_ChipH3Id_DA_Rep1.bam"
bamControl "/XX_input_DA_Rep1.bam"
           XX_ChipH3Id_DA_Rep2
bamRead    "/XX_ChipH3Id_DA_Rep2.bam"
bamControl "/XX_input_DA_Rep2.bam"
           XX_ChipH3Id_DB_Rep1
bamRead    "/XX_ChipH3Id_DB_Rep1.bam"
bamControl "/XX_input_DB_Rep1.bam"
           XX_ChipH3Id_DB_Rep2
bamRead    "/XX_ChipH3Id_DB_Rep2.bam"
bamControl "/XX_input_DB_Rep2.bam"
           XX_ChipH3Id_DC_Rep1
bamRead    "/XX_ChipH3Id_DC_Rep1.bam"
bamControl "/XX_input_DC_Rep1.bam"
           XX_ChipH3Id_DC_Rep2
bamRead    "/XX_ChipH3Id_DC_Rep2.bam"
bamControl "/XX_input_DC_Rep2.bam"
           YY_ChipH3Ia_DA_Rep1
bamRead    "/YY_ChipH3Ia_DA_Rep1.bam"
bamControl "/YY_input_DA_Rep1.bam"
           YY_ChipH3Ia_DA_Rep2
bamRead    "/YY_ChipH3Ia_DA_Rep2.bam"
bamControl "/YY_input_DA_Rep2.bam"
           YY_ChipH3Ia_DB_Rep1
bamRead    "/YY_ChipH3Ia_DB_Rep1.bam"
bamControl "/YY_input_DB_Rep1.bam"
           YY_ChipH3Ia_DB_Rep2
bamRead    "/YY_ChipH3Ia_DB_Rep2.bam"
bamControl "/YY_input_DB_Rep2.bam"
           YY_ChipH3Ia_DC_Rep1
bamRead    "/YY_ChipH3Ia_DC_Rep1.bam"
bamControl "/YY_input_DC_Rep1.bam"
           YY_ChipH3Ia_DC_Rep2
bamRead    "/YY_ChipH3Ia_DC_Rep2.bam"
bamControl "/YY_input_DC_Rep2.bam"
           YY_ChipH3Ib_DA_Rep1
bamRead    "/YY_ChipH3Ib_DA_Rep1.bam"
bamControl "/YY_input_DA_Rep1.bam"
           YY_ChipH3Ib_DA_Rep2
bamRead    "/YY_ChipH3Ib_DA_Rep2.bam"
bamControl "/YY_input_DA_Rep2.bam"
           YY_ChipH3Ib_DB_Rep1
bamRead    "/YY_ChipH3Ib_DB_Rep1.bam"
bamControl "/YY_input_DB_Rep1.bam"
           YY_ChipH3Ib_DB_Rep2
bamRead    "/YY_ChipH3Ib_DB_Rep2.bam"
bamControl "/YY_input_DB_Rep2.bam"
           YY_ChipH3Ib_DC_Rep1
bamRead    "/YY_ChipH3Ib_DC_Rep1.bam"
bamControl "/YY_input_DC_Rep1.bam"
           YY_ChipH3Ib_DC_Rep2
bamRead    "/YY_ChipH3Ib_DC_Rep2.bam"
bamControl "/YY_input_DC_Rep2.bam"
           YY_ChipH3Ie_DA_Rep1
bamRead    "/YY_ChipH3Ie_DA_Rep1.bam"
bamControl "/YY_input_DA_Rep1.bam"
           YY_ChipH3Ie_DA_Rep2
bamRead    "/YY_ChipH3Ie_DA_Rep2.bam"
bamControl "/YY_input_DA_Rep2.bam"
           YY_ChipH3Ie_DB_Rep1
bamRead    "/YY_ChipH3Ie_DB_Rep1.bam"
bamControl "/YY_input_DB_Rep1.bam"
           YY_ChipH3Ie_DB_Rep2
bamRead    "/YY_ChipH3Ie_DB_Rep2.bam"
bamControl "/YY_input_DB_Rep2.bam"
           YY_ChipH3Ie_DC_Rep1
bamRead    "/YY_ChipH3Ie_DC_Rep1.bam"
bamControl "/YY_input_DC_Rep1.bam"
           YY_ChipH3Ie_DC_Rep2
bamRead    "/YY_ChipH3Ie_DC_Rep2.bam"
bamControl "/YY_input_DC_Rep2.bam"
           YY_ChipH3Ic_DA_Rep1
bamRead    "/YY_ChipH3Ic_DA_Rep1.bam"
bamControl "/YY_input_DA_Rep1.bam"
           YY_ChipH3Ic_DA_Rep2
bamRead    "/YY_ChipH3Ic_DA_Rep2.bam"
bamControl "/YY_input_DA_Rep2.bam"
           YY_ChipH3Ic_DB_Rep1
bamRead    "/YY_ChipH3Ic_DB_Rep1.bam"
bamControl "/YY_input_DB_Rep1.bam"
           YY_ChipH3Ic_DB_Rep2
bamRead    "/YY_ChipH3Ic_DB_Rep2.bam"
bamControl "/YY_input_DB_Rep2.bam"
           YY_ChipH3Ic_DC_Rep1
bamRead    "/YY_ChipH3Ic_DC_Rep1.bam"
bamControl "/YY_input_DC_Rep1.bam"
           YY_ChipH3Ic_DC_Rep2
bamRead    "/YY_ChipH3Ic_DC_Rep2.bam"
bamControl "/YY_input_DC_Rep2.bam"
           YY_ChipH3Id_DA_Rep1
bamRead    "/YY_ChipH3Id_DA_Rep1.bam"
bamControl "/YY_input_DA_Rep1.bam"
           YY_ChipH3Id_DA_Rep2
bamRead    "/YY_ChipH3Id_DA_Rep2.bam"
bamControl "/YY_input_DA_Rep2.bam"
           YY_ChipH3Id_DB_Rep1
bamRead    "/YY_ChipH3Id_DB_Rep1.bam"
bamControl "/YY_input_DB_Rep1.bam"
           YY_ChipH3Id_DB_Rep2
bamRead    "/YY_ChipH3Id_DB_Rep2.bam"
bamControl "/YY_input_DB_Rep2.bam"
           YY_ChipH3Id_DC_Rep1
bamRead    "/YY_ChipH3Id_DC_Rep1.bam"
bamControl "/YY_input_DC_Rep1.bam"
           YY_ChipH3Id_DC_Rep2
bamRead    "/YY_ChipH3Id_DC_Rep2.bam"
bamControl "/YY_input_DC_Rep2.bam"

> file.exists(unique(dbObj$class[10,]))

 [1] TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE
[16] TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE
[31] TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE
[46] TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE

> file.exists(unique(dbObj$class[11,]))

 [1] TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE

> sessionInfo()

R version 4.0.3 (2020-10-10)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: CentOS Linux 7 (Core)

Matrix products: default
BLAS/LAPACK: /spack/apps/linux-centos7-x86_64/gcc-8.3.0/openblas-0.3.8-2no6mfziiclwxb7lstxoos335gnhjpes/lib/libopenblasp-r0.3.8.so

locale:
[1] C

attached base packages:
[1] parallel  stats4    stats     graphics  grDevices utils     datasets
[8] methods   base

other attached packages:
 [1] forcats_0.5.1               stringr_1.4.0
 [3] dplyr_1.0.2                 purrr_0.3.4
 [5] readr_1.4.0                 tidyr_1.1.3
 [7] tibble_3.0.4                ggplot2_3.3.3
 [9] tidyverse_1.3.0             DiffBind_3.0.9
[11] SummarizedExperiment_1.20.0 Biobase_2.50.0
[13] MatrixGenerics_1.2.0        matrixStats_0.57.0
[15] GenomicRanges_1.42.0        GenomeInfoDb_1.26.2
[17] IRanges_2.24.1              S4Vectors_0.28.1
[19] BiocGenerics_0.36.0

loaded via a namespace (and not attached):
  [1] readxl_1.3.1             backports_1.2.1          GOstats_2.56.0
  [4] BiocFileCache_1.14.0     plyr_1.8.6               GSEABase_1.52.1
  [7] splines_4.0.3            BiocParallel_1.24.1      amap_0.8-18
 [10] digest_0.6.27            invgamma_1.1             GO.db_3.12.1
 [13] fansi_0.4.1              SQUAREM_2020.5           magrittr_2.0.1
 [16] checkmate_2.0.0          memoise_1.1.0            BSgenome_1.58.0
 [19] base64url_1.4            limma_3.46.0             Biostrings_2.58.0
 [22] annotate_1.68.0          modelr_0.1.8             systemPipeR_1.24.2
 [25] askpass_1.1              bdsmatrix_1.3-4          prettyunits_1.1.1
 [28] jpeg_0.1-8.1             colorspace_2.0-0         rvest_1.0.0
 [31] blob_1.2.1               rappdirs_0.3.1           apeglm_1.12.0
 [34] ggrepel_0.9.0            haven_2.3.1              crayon_1.3.4
 [37] RCurl_1.98-1.2           jsonlite_1.7.2           graph_1.68.0
 [40] genefilter_1.72.0        brew_1.0-6               survival_3.2-7
 [43] VariantAnnotation_1.36.0 glue_1.4.2               gtable_0.3.0
 [46] zlibbioc_1.36.0          XVector_0.30.0           DelayedArray_0.16.0
 [49] V8_3.2.0                 Rgraphviz_2.34.0         scales_1.1.1
 [52] pheatmap_1.0.12          mvtnorm_1.1-1            DBI_1.1.0
 [55] edgeR_3.32.0             Rcpp_1.0.5               xtable_1.8-4
 [58] progress_1.2.2           emdbook_1.3.12           bit_4.0.4
 [61] rsvg_1.3                 AnnotationForge_1.32.0   truncnorm_1.0-8
 [64] httr_1.4.2               gplots_3.1.1             RColorBrewer_1.1-2
 [67] ellipsis_0.3.1           pkgconfig_2.0.3          XML_3.99-0.5
 [70] dbplyr_2.0.0             locfit_1.5-9.4           tidyselect_1.1.0
 [73] rlang_0.4.10             AnnotationDbi_1.52.0     cellranger_1.1.0
 [76] munsell_0.5.0            tools_4.0.3              cli_2.2.0
 [79] generics_0.1.0           RSQLite_2.2.2            broom_0.7.5
 [82] yaml_2.2.1               fs_1.5.0                 bit64_4.0.5
 [85] caTools_1.18.0           RBGL_1.66.0              xml2_1.3.2
 [88] biomaRt_2.46.0           rstudioapi_0.13          compiler_4.0.3
 [91] curl_4.3                 png_0.1-7                reprex_1.0.0
 [94] stringi_1.5.3            ps_1.5.0                 GenomicFeatures_1.42.1
 [97] lattice_0.20-41          Matrix_1.3-2             vctrs_0.3.6
[100] pillar_1.4.7             lifecycle_1.0.0          data.table_1.13.6
[103] bitops_1.0-6             irlba_2.3.3              rtracklayer_1.50.0
[106] R6_2.5.0                 latticeExtra_0.6-29      hwriter_1.3.2
[109] ShortRead_1.48.0         KernSmooth_2.23-18       MASS_7.3-53
[112] gtools_3.8.2             assertthat_0.2.1         openssl_1.4.3
[115] Category_2.56.0          rjson_0.2.20             withr_2.3.0
[118] GenomicAlignments_1.26.0 batchtools_0.9.14        Rsamtools_2.6.0
[121] GenomeInfoDbData_1.2.4   hms_0.5.3                grid_4.0.3
[124] DOT_0.1                  coda_0.19-4              GreyListChIP_1.22.0
[127] ashr_2.2-47              mixsqp_0.3-43            bbmle_1.0.23.1
[130] lubridate_1.7.10         numDeriv_2016.8-1.1

I checked integrity of bam files using

$ samtools quickcheck -v *.bam
$

but did not get any error.

Can anyone please help. Thanks
DiffBind ChIPSeq • 1.8k views
ADD COMMENT •
0
Entering edit mode

What do you get when you type in :

dbObj

ADD REPLY •
0
Entering edit mode

dbObj

dbObj
4 Samples, 98839 sites in matrix (209518 total):
ID Tissue  Factor Condition Treatment Replicate Intervals
1 1_ChipH3Ia_DA_Rep1 cell_line H3Ia         0        1         1    139874
2 1_ChipH3Ia_DA_Rep2 cell_line H3Ia         0        1         2    150834
3 1_ChipH3Ia_DB_Rep1 cell_line H3Ia         5        1         1     46458
4 1_ChipH3Ia_DB_Rep2 cell_line H3Ia         5        1         2    121136
...

dbObj <- dba.count(dbObj)

ADD REPLY •
0
Entering edit mode

Can you show the warnings?

If you set bParallel=FALSE the warnings will be generated as each file is processed, which makes for a cleaner readout when there is a problem.

Also, if you can try running dba.count() with summits=FALSE, and you do not get an error, that would also be informative.

ADD REPLY •
0
Entering edit mode

dbObj <- dba.count(dbObj)

warnings()

> dbObj <- dba.count(dbObj)
Computing summits...

Error: Error processing one or more read files. Check warnings().
In addition: There were 50 or more warnings (use warnings() to see the first 50)
>
> warnings()
Warning messages:
1: In mclapply(arglist, fn, ..., mc.preschedule = TRUE, mc.allow.recursive = TRUE) :
  scheduled cores 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 19, 24 did not deliver results, all values of the jobs will be affected
2:
3:

....

49:
50:

dbObj <- dba.count(dbObj, bParallel=FALSE)

this did not give any error ....

> dbObj <- dba.count(dbObj, bParallel=FALSE)
Computing summits...
Sample/1_H3Ib_DA_Rep1.bam125
Sample/1_H3Ib_DA_Rep2.bam125
Sample/1_H3Ib_DB_Rep1.bam125
Sample/1_H3Ib_DB_Rep2.bam125
Sample/1_input_DA_Rep1.bam125
Sample/1_input_DA_Rep2.bam125
Sample/1_input_DB_Rep1.bam125
Sample/1_input_DB_Rep2.bam125
Re-centering peaks...
Reads will be counted as Paired-end.
Sample/1_H3Ib_DA_Rep1.bam125
Sample/1_H3Ib_DA_Rep2.bam125
Sample/1_H3Ib_DB_Rep1.bam125
Sample/1_H3Ib_DB_Rep2.bam125
Sample/1_input_DA_Rep1.bam125
Sample/1_input_DA_Rep2.bam125
Sample/1_input_DB_Rep1.bam125
Sample/1_input_DB_Rep2.bam125

but why is parallel not working for me ?

ADD REPLY •
2
Entering edit mode
Rory Stark ★ 5.2k
@rory-stark-5741
Last seen 26 days ago
Cambridge, UK

Interesting that the count works fine in serial mode but fails (with no meaningful error messages) in Parallel mode.

The only other time I saw this, there was an issue with how many sub-processes the user was allowed to spawn, or how much memory the user was allowed to allocate. By default, DiffBind will use up to ALL of the available cores on the machine and can require >1GB per core.. I'm not sure if that is what is happening in this case as in the example you only have 4 samples (8 parallel counting processes with controls) and most machines can handle that.

Did you try running it in parallel on the 4-sample case? If that works, you could try setting the number of cores down for the full experiment, eg:

dbObj$config$cores <- 8

You can try different values to see where it works (or just run in Serial mode).
ADD COMMENT •
0
Entering edit mode

adding

dbObj$config$cores <- 8

solved the problem.

Thanks a lot.

ADD REPLY •

Login before adding your answer.

Similar Posts
Loading Similar Posts
Traffic: 806 users visited in the last hour
Content Search
Users
Tags
Badges
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6