Entering edit mode
@richardallenfriedmanbrooklyn-24118
Last seen 8 weeks ago
United States
Unable to download hg19.generanges as part of DMRcatedata package
Dear List,
I have been unable to download the entire hg19.generanges for DMRcatedata on a mac at work. I can download it at home, but my mac at work has more memory, so I would like to use the work mac.
Here is my session (I has to eliminate part of the sessionInfo() to stay within character limit. Please raise the character limit):
> results.ranges <- extractRanges(DMRs, genome = "hg19")
snapshotDate(): 2020-04-27
see ?DMRcatedata and browseVignettes('') for documentation
downloading 1 resources
retrieving 1 resource
| | 0%Error: failed to load resource
name: EH3132
title: hg19.generanges
reason: 1 resources failed to download
In addition: Warning messages:
1: download failed
web resource path: 'https://experimenthub.bioconductor.org/fetch/3148'
local file path: '/Users/friedman/Library/Caches/ExperimentHub/17d25be40bc3_3148'
reason: transfer closed with 900409 bytes remaining to read
2: bfcadd() failed; resource removed
rid: BFC15
fpath: 'https://experimenthub.bioconductor.org/fetch/3148'
reason: download failed
3: download failed
hub path: 'https://experimenthub.bioconductor.org/fetch/3148'
cache resource: 'EH3132 : 3148'
reason: bfcadd() failed; see warnings()
> sessionInfo()
R version 4.0.2 (2020-06-22)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Mojave 10.14
Matrix products: default
BLAS: /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRblas.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRlapack.dylib
locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
attached base packages:
[1] stats4 parallel stats graphics grDevices utils datasets methods base
other attached packages:
[1] DMRcatedata_2.6.0 ExperimentHub_1.14.2
[3] AnnotationHub_2.20.2 BiocFileCache_1.12.1
[5] dbplyr_1.4.4 IlluminaHumanMethylation450kanno.ilmn12.hg19_0.6.0
[7] ggplot2_3.3.2 clusterProfiler_3.16.1
[9] xlsx_0.6.3 ChAMP_2.18.2
[11] DT_0.15 IlluminaHumanMethylationEPICmanifest_0.3.0
[13] Illumina450ProbeVariants.db_1.24.0 FEM_3.15.0
[15] graph_1.66.0 org.Hs.eg.db_3.11.4
[17] impute_1.62.0 igraph_1.2.5
[19] corrplot_0.84 marray_1.66.0
[21] limma_3.44.3 Matrix_1.2-18
[23] AnnotationDbi_1.50.3 ChAMPdata_2.20.0
[25] DMRcate_2.2.2 minfi_1.34.0
[27] bumphunter_1.30.0 locfit_1.5-9.4
[29] iterators_1.0.12 foreach_1.5.0
[31] Biostrings_2.56.0 XVector_0.28.0
[33] SummarizedExperiment_1.18.2 DelayedArray_0.14.1
[35] matrixStats_0.56.0 Biobase_2.48.0
[37] GenomicRanges_1.40.0 GenomeInfoDb_1.24.2
[39] IRanges_2.22.2 S4Vectors_0.26.1
[41] BiocGenerics_0.34.0
l
>
Thanks and best wishes, Rich Richard Friedman, Columbia Univeristy
Jim,
Thanks. What you suggest worked on my mac at home, but this is what I got on my work mac
So I have the same problem.
Thanks and best wishes, Rich
Maybe something's weird with your cache. You could try
removeResourcesto just get rid of EH3132, orremoveCacheto just nuke the whole thing and start again.Jim,
I tried these. Are these commands correct?
Thanks an best wishes,
Rich
Richard Friedman
Is
removeResources(ExperimentHub, "EH3132")(quotes around 'EH3123') what you are looking for?Martin,
I wasn;t able to download the hg19.generanges (EH3132) database on one of my macs (please see above). Jim MacDonald suggested that I remove EH3132 and try again. I had neglected to paste in my trying it with quotes,
I also didn;t get a result. My main goal is to get hg19.generanges (EH3132) for use with DMRcate.
Thanks and best wishes,
Rich
Richard Friedman
Oops, again, I guess that should have been
removeResources(ExperimentHub(),"EH3132")with parentheses afterExperimentHub-- I'm interpreting the error message, rather than actually knowing what's going on.Could you also send the results of doing the following:
httr::cache_info(httr::HEAD("https://experimenthub.bioconductor.org/fetch/3148"))andhttr::GET("https://experimenthub.bioconductor.org/fetch/3148")Lori,
I ran the commands that you recommended but still got the same error message when I ran the extractRanges command.
Thanks and best wishes,
Rich
Martin, Here is my session removing the file. When I tried what Lori advised in her post, subsequent to running the command you gave, I still got the same error message as before.
Thanks and best wishes,
Rich
The error message you get should be instructive, so I would recommend paying closer attention to what R is telling you. Any time you get a message like that, where it says it can't find an inherited method, it implies that you are passing in an argument that isn't expected. And it says the signature is 'function', and we used
ExperimentHub()to generate the 'hub' object, so to me at least that's a really big clue?Anyhow, you can always check the help page, which says
Where it clearly states that the first argument is a 'hub object', which again is a pretty good clue. So what I did is
Which is what I was getting at in the first place.
And for completeness, right after removing that resource I got it again:
In which case I had to re-download because it had been removed, as compared to what you get if it's already in your cache:
And if you try to re-add the resource and still have trouble, could you show the full output from each of these commands httr::cache_info(httr::HEAD("https://experimenthub.bioconductor.org/fetch/3148")) and httr::GET("https://experimenthub.bioconductor.org/fetch/3148")
Jim and Lori, Thank you for the advice. I hope that the following is sufficiently complete to show you what is going on. I had to cut off most of the package listing for reasons of space, (and not attached):" for reasons of space
Thanks and best wishes, Rich
This seems really odd. Can you download any other non DMRcatedata resources?
hub[["EH166"]]Lori,
The same thing happened with EH166.
Thanks and best wishes, Rich
Could your institution be blocking downloads from AWS?
Lori,
I have a query into our IT department as to whether they are blocking the download. I am not experiencing this difficulty on my mac at home. I will repeat the query to IT mentioning that the source of the file is AWS.
Thanks and best wishes,
Rich
Lori,
I can download the file from the website via a browser but not via R as you have seen. Is there a kind of block that can block R downloads, but not Netscape downloads?
Thanks and best wishes, Rich
Its very baffling and I have not seen this before. Lets try a little more in depth debugging: Please do the following:
Using
debugoncewill put you inside the function that tries to download the file so you can step line by line through the code. This will show the code and you press the return key to step through.Once you see the following
BEFORE pressing <enter> please investigate -- This should be the section of code that produces an ERROR So BEFORE pressing enter check the value of
Please record these values and then press enter. Please include the output of those values and any ERROR message.
Let me know if you have any troubles trying to use debugonce.
We are thinking right now you may either not have write access or might be out of space in the caching location that is being used.
Lori,
Thank you for your advice. I apologize for the late response. I had several deadlines. I followed your instructions. Here are session excerpts:
```
After some more stepping through
Does this tell you anything?
Best wishes,
Rich
I apologize for the issues and appreciate you debugging with me so we can figure it out. I'll give a few steps:
Does
httr::GET("https://experimenthub.bioconductor.org/fetch/166")work?If not directly after the call please do
traceback().If successful, then try
httr::GET("https://experimenthub.bioconductor.org/fetch/166", write_disk(tempfile()))Similarly if failed,traceback()If successful, the try
httr::GET("https://experimenthub.bioconductor.org/fetch/166", write_disk(paste0("/Users/friedman/Library/Caches/ExperimentHub/", basename(tempfile()))))If fail,traceback()ThanksLori,
Here is what I got,
The write_disk function is not present, although httr is installed.
Thanks and best wishes,
Rich
Its troubling that you are blocked from even doing
httr::GET("https://experimenthub.bioconductor.org/fetch/166")And you are sure your institution computer isn't behind a proxy that should be set at the start?
Is this an 'https'-related issue, e.g., does
Martin,
Thanks for the suggestion. Here is what I get
Does this session tell you anything? Best wishes, Rich
Next up, the http:// and https:// versions of
Martin, Here goes:
Thanks and best wishes, Rich
Just popping out to this level so we have a bit more space.
It looks like the final file is
http://s3.amazonaws.com/experimenthub/alpineData/ERR188297.rda. The command s something likeI get
Martin,
Apologies again. I looked at the support site everyday for a while, and the stopped so I missed your note. Here is my attempt
Does this tell you anything? I suspect that despite Columbia IT's denials they are blocking AWS.
Thanks and best wishes, Rich
Martin,
I found out that curl doesn't work on mac at work (OS 10.14) but does work on my macs at home (OS 10.14.6 and 10.15). I also see that curl works from the cluster at work. So I think that I can solve the problem by updating the OS. Watch this space.
Thanks and best wishes,
Rich
Martin,
I got curl working on my work desktop, but I still get the same error message as before when Iuse experiment hub. I am going to update my mac os in the hope that doing so will help.
Thanks and Happy New Year, Rich