Entering edit mode
I've switched recently from custom cache system to BiocFileCache inside my biodb package.
However, now, in one of the comming extension packages of biodb (biodbHmdb) I need to move more than 100,000 files at once into BiocFileCache.
For this I call bfcadd() on each file inside a loop, and it takes forever to complete.
Is there a faster way of achieving this or would it be possible to improve BiocFileCache for such a purpose?
Do you really need to move all 100,000 files at the same time? It sounds like you're pre-loading the cache. To my mind a cache is useful for speeding up multiple similar operations, but not necessarily the first one. I don't know the context for biodbHmdb, but I'm wondering if it might be more effective to cache the result of a query after it has been executed.
This might also get more traction on the developers mailing list (https://stat.ethz.ch/mailman/listinfo/bioc-devel)