I'd like to first thank the developers for a fine set of tools. I am performing two-group comparison where each group contains 3~4 treated and input pairs. I am running the analysis on AWS EC2 in which I have installed r-base and DiffBind in a conda environment. I am currently trying to resolve the following issue from dba.analyze:
> h3k27ac <- dba.analyze(h3k27ac)
Applying Blacklist/Greylists...
Genome detected: Hsapiens.NCBI.GRCh38
Applying blacklist...
Removed: 5 of 58439 intervals.
Counting control reads for greylist...
Blacklist error: Error in value[[3L]](cond): GreyListChIP error: Error: BiocParallel errors
element index: 1, 2, 3, 4, 5, 6, ...
first error: 'seqlengths' contains NAs or negative values
Applying the blacklist works all right but greylist fails due to BiocParallel errors. From googling, I learned about the following but it did not help at all.
> BiocParallel::register(BiocParallel::SerialParam())
I know dba.analyze(h3k27ac, bGreylist=FALSE)
works but, given my input data, greylisting should have no issue from my perspective. I would appreciate any help or insight. Here's my sessionInfo FYI.
> sessionInfo( )
R version 4.1.0 (2021-05-18)
Platform: x86_64-conda-linux-gnu (64-bit)
Running under: Amazon Linux 2
Matrix products: default
BLAS/LAPACK: /home/ec2-user/miniconda3/envs/R/lib/libopenblasp-r0.3.15.so
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] parallel stats4 stats graphics grDevices utils datasets
[8] methods base
other attached packages:
[1] BiocParallel_1.26.1 DiffBind_3.2.4
[3] SummarizedExperiment_1.22.0 Biobase_2.52.0
[5] MatrixGenerics_1.4.0 matrixStats_0.59.0
[7] GenomicRanges_1.44.0 GenomeInfoDb_1.28.1
[9] IRanges_2.26.0 S4Vectors_0.30.0
[11] BiocGenerics_0.38.0
loaded via a namespace (and not attached):
[1] backports_1.2.1 GOstats_2.58.0 BiocFileCache_2.0.0
[4] plyr_1.8.6 GSEABase_1.54.0 splines_4.1.0
[7] ggplot2_3.3.5 amap_0.8-18 digest_0.6.27
[10] invgamma_1.1 GO.db_3.13.0 SQUAREM_2021.1
[13] fansi_0.5.0 magrittr_2.0.1 checkmate_2.0.0
[16] memoise_2.0.0 BSgenome_1.60.0 base64url_1.4
[19] limma_3.48.1 Biostrings_2.60.1 annotate_1.70.0
[22] systemPipeR_1.26.3 bdsmatrix_1.3-4 prettyunits_1.1.1
[25] jpeg_0.1-8.1 colorspace_2.0-2 blob_1.2.1
[28] rappdirs_0.3.3 apeglm_1.14.0 ggrepel_0.9.1
[31] dplyr_1.0.7 crayon_1.4.1 RCurl_1.98-1.3
[34] jsonlite_1.7.2 graph_1.70.0 genefilter_1.74.0
[37] brew_1.0-6 survival_3.2-11 VariantAnnotation_1.38.0
[40] glue_1.4.2 gtable_0.3.0 zlibbioc_1.38.0
[43] XVector_0.32.0 DelayedArray_0.18.0 V8_3.4.2
[46] Rgraphviz_2.36.0 scales_1.1.1 pheatmap_1.0.12
[49] mvtnorm_1.1-2 DBI_1.1.1 edgeR_3.34.0
[52] Rcpp_1.0.7 xtable_1.8-4 progress_1.2.2
[55] emdbook_1.3.12 bit_4.0.4 rsvg_2.1.2
[58] AnnotationForge_1.34.0 truncnorm_1.0-8 httr_1.4.2
[61] gplots_3.1.1 RColorBrewer_1.1-2 ellipsis_0.3.2
[64] pkgconfig_2.0.3 XML_3.99-0.6 dbplyr_2.1.1
[67] locfit_1.5-9.4 utf8_1.2.1 tidyselect_1.1.1
[70] rlang_0.4.11 AnnotationDbi_1.54.1 munsell_0.5.0
[73] tools_4.1.0 cachem_1.0.5 generics_0.1.0
[76] RSQLite_2.2.5 stringr_1.4.0 fastmap_1.1.0
[79] yaml_2.2.1 bit64_4.0.5 caTools_1.18.2
[82] purrr_0.3.4 KEGGREST_1.32.0 RBGL_1.68.0
[85] xml2_1.3.2 biomaRt_2.48.2 compiler_4.1.0
[88] rstudioapi_0.13 filelock_1.0.2 curl_4.3.2
[91] png_0.1-7 geneplotter_1.70.0 tibble_3.1.2
[94] stringi_1.7.3 GenomicFeatures_1.44.0 lattice_0.20-44
[97] Matrix_1.3-4 vctrs_0.3.8 pillar_1.6.1
[100] lifecycle_1.0.0 irlba_2.3.3 data.table_1.14.0
[103] bitops_1.0-7 rtracklayer_1.52.0 R6_2.5.0
[106] BiocIO_1.2.0 latticeExtra_0.6-29 hwriter_1.3.2
[109] ShortRead_1.50.0 KernSmooth_2.23-20 MASS_7.3-54
[112] gtools_3.9.2 assertthat_0.2.1 DESeq2_1.32.0
[115] Category_2.58.0 rjson_0.2.20 withr_2.4.2
[118] GenomicAlignments_1.28.0 batchtools_0.9.15 Rsamtools_2.8.0
[121] GenomeInfoDbData_1.2.6 hms_1.1.0 grid_4.1.0
[124] DOT_0.1 coda_0.19-4 GreyListChIP_1.24.0
[127] ashr_2.2-47 mixsqp_0.3-43 bbmle_1.0.23.1
[130] numDeriv_2016.8-1.1 restfulr_0.0.13
Hi Rory,
I am having this exact same problem and wonder if you had any other suggestions. I get the same error whether or not I run the blacklist before or after counting. I have my error below and also sessionInfo, but happy to provide more. My BAM files are from an alignment to GrCH37 and so I though this might be the issue - except that the blacklist functionality works fine with hg19..
Hello my friends, I have exactly the same issue. May I ask if you have already solve this problem? I have tried
library(BiocParallel) register(SerialParam())
but it did not work for me.Hi Rory let me know if you think this is a problem with control bam files. I can show you the bam files if you have time. Thank you so much!
Could you let me know the versions? Output of
sessionInfo()
.Hi Rory,
Thank you for your reply! Here is the my
sessionInfo()
:Please let me know what you think. I really appreciate your help!
R version 4.1.2 (2021-11-01) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Ubuntu 20.04.3 LTS
Matrix products: default BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0 LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.9.0
locale: [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8 LC_MONETARY=en_US.UTF-8
[6] LC_MESSAGES=en_US.UTF-8 LC_PAPER=en_US.UTF-8 LC_NAME=C LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages: [1] parallel stats4 stats graphics grDevices utils datasets methods base
other attached packages: [1] BiocParallel_1.26.2 forcats_0.5.1 stringr_1.4.0 dplyr_1.0.9
[5] purrr_0.3.4 readr_2.1.2 tidyr_1.2.0 tibble_3.1.7
[9] ggplot2_3.3.6 tidyverse_1.3.1 DiffBind_3.2.7 SummarizedExperiment_1.22.0 [13] Biobase_2.52.0 MatrixGenerics_1.4.3 matrixStats_0.62.0 GenomicRanges_1.44.0
[17] GenomeInfoDb_1.28.4 IRanges_2.26.0 S4Vectors_0.30.2 BiocGenerics_0.38.0
loaded via a namespace (and not attached): [1] readxl_1.4.0 backports_1.4.1 GOstats_2.58.0 BiocFileCache_2.0.0 plyr_1.8.7
[6] GSEABase_1.54.0 splines_4.1.2 amap_0.8-18 digest_0.6.29 invgamma_1.1
[11] GO.db_3.13.0 SQUAREM_2021.1 fansi_1.0.3 magrittr_2.0.3 checkmate_2.1.0
[16] memoise_2.0.1 BSgenome_1.60.0 base64url_1.4 tzdb_0.3.0 limma_3.48.3
[21] Biostrings_2.60.2 annotate_1.70.0 modelr_0.1.8 systemPipeR_1.26.3 bdsmatrix_1.3-4
[26] prettyunits_1.1.1 jpeg_0.1-9 colorspace_2.0-3 rvest_1.0.2 blob_1.2.3
[31] rappdirs_0.3.3 apeglm_1.14.0 ggrepel_0.9.1 haven_2.5.0 crayon_1.5.1
[36] RCurl_1.98-1.6 jsonlite_1.8.0 graph_1.70.0 genefilter_1.74.1 brew_1.0-7
[41] survival_3.3-1 VariantAnnotation_1.38.0 glue_1.6.2 gtable_0.3.0 zlibbioc_1.38.0
[46] XVector_0.32.0 DelayedArray_0.18.0 V8_4.2.0 Rgraphviz_2.36.0 scales_1.2.0
[51] pheatmap_1.0.12 mvtnorm_1.1-3 DBI_1.1.2 edgeR_3.34.1 Rcpp_1.0.8.3
[56] xtable_1.8-4 progress_1.2.2 emdbook_1.3.12 bit_4.0.4 rsvg_2.3.1
[61] AnnotationForge_1.34.1 truncnorm_1.0-8 httr_1.4.3 gplots_3.1.3 RColorBrewer_1.1-3
[66] ellipsis_0.3.2 pkgconfig_2.0.3 XML_3.99-0.9 dbplyr_2.1.1 locfit_1.5-9.5
[71] utf8_1.2.2 tidyselect_1.1.2 rlang_1.0.2 AnnotationDbi_1.54.1 cellranger_1.1.0
[76] munsell_0.5.0 tools_4.1.2 cachem_1.0.6 cli_3.3.0 generics_0.1.2
[81] RSQLite_2.2.14 broom_0.8.0 fastmap_1.1.0 yaml_2.3.5 fs_1.5.2
[86] bit64_4.0.5 caTools_1.18.2 KEGGREST_1.32.0 RBGL_1.68.0 xml2_1.3.3
[91] biomaRt_2.48.3 debugme_1.1.0 compiler_4.1.2 rstudioapi_0.13 filelock_1.0.2
[96] curl_4.3.2 png_0.1-7 reprex_2.0.1 stringi_1.7.6 GenomicFeatures_1.44.2
[101] lattice_0.20-45 Matrix_1.4-0 vctrs_0.4.1 pillar_1.7.0 lifecycle_1.0.1
[106] irlba_2.3.5 data.table_1.14.2 bitops_1.0-7 rtracklayer_1.52.1 R6_2.5.1
[111] BiocIO_1.2.0 latticeExtra_0.6-29 hwriter_1.3.2.1 ShortRead_1.50.0 KernSmooth_2.23-20
[116] MASS_7.3-55 gtools_3.9.2 assertthat_0.2.1 Category_2.58.0 rjson_0.2.21
[121] withr_2.5.0 GenomicAlignments_1.28.0 batchtools_0.9.15 Rsamtools_2.8.0 GenomeInfoDbData_1.2.6
[126] hms_1.1.1 grid_4.1.2 DOT_0.1 coda_0.19-4 GreyListChIP_1.24.0
[131] ashr_2.2-54 mixsqp_0.3-43 bbmle_1.0.25 lubridate_1.8.0 numDeriv_2016.8-1.1
[136] restfulr_0.0.13
The latest version,
DiffBind_3.6.1
, contains fixes for these issues. You will get these fixes if you upgrade toR_4.2.0
andBioconductor_3.15
.I am still getting an error in
DiffBind_3.6.1
:Seems like it might be a BPPARAM issue though...
Found the problem: link
You can overcome this issue by setting
mc.cores
:didn't work out
Hi all, I am having this same issue now - blacklisting works fine, but greylisting comes up with an error that I could not solve by changing BiocParallel parameters nor cores = 1. Here's my error message. Does anyone have any advice?
dba.blacklist(DBdata_ATOH1, blacklist=TRUE, greylist=TRUE, cores = 10)
Genome detected: Hsapiens.NCBI.GRCh38 Applying blacklist... Removed: 114 of 71504 intervals.
Counting control reads for greylist...
Error in value[3L] : GreyListChIP error: Error: BiocParallel errors element index: 1, 2 first error: 'seqlengths' contains NAs or negative values
I am checking in a fix for this now. It will be available as
DiffBind_3.8.1
.Hi Rory, do you have any updates on this issue? I have just updated to diffbind 3.8.1 but still having the same issue.
Thanks for your help!
Alessia
I can't reproduce this with
DiffBind_3.8.1
on either MacOS or linux.Have you tried running it with
cores=0
to see if you get better error messages?With cores = 0 I get:
Could you make the control .bam file available to me so I can get to the bottom of this issue? You can email me a link of where I can download it. It would also help to have a copy of your
DBA
object.Hi Rory,
We've got to the bottom of the error, it is coming from a difference in the chromosome names between the
ktype
object and those inpv$chrmap
inside the functionpv.countGreylist
. TheDBA
object contains a few peaks for unlocalised chromosomes likeGL000195.1
, which the automatically founddba.ktypes$BSgenome.Hsapiens.NCBI.GRCh38
GRanges object doesn't contain.We've been able to patch it locally with:
But obviously this is a hack! We can't see an easy way to remove the peaks with these non-canonical chromosomes in the
dba
function.This is very helpful! I've logged an issue internally to have a look at this. At minimum,
GreyListChIP
should detect this and give an error, or a warning that the unrecognized chromosomes will be ignored (and won't be greylisted).Thank you! This is the only thing that's worked for me so far.