I have used swappedDrops function to remove potential index hopping UMIs from 10x genomics single-cell data. Recently, we sequenced +100K single cells. When I was processing the data, I found that swappedDrops crashed. I am wondering if there is a limitation to the number of cells swappedDrops can handle. Your inputs/insights will be much appreciated. Thank you.
Can you provide some more information about how you ran swappedDrops (e.g., locally or on a compute cluster), and whether there was an error message that was reported?
Loading in very many unfiltered matrices from CellRanger should be quite tough on your RAM. Are you running in an environment with sufficient memory, even before swappedDrops is called?
As Jonny said, more information is necessary to provide meaningful advice. In the absence of such information, I'll just throw out a couple of scenarios for you to ponder.
You ran out of memory and your cluster killed the job. Solution: allocate more memory. Expect swappedDrops to use up about 5 GB of RAM per sample (I don't remember the exact numbers).
You have more than .Machine$integer.max molecules in one of your files. Solution: switch to DropletUtils 1.3.* or higher, which avoids this limitation (up until 2^64 molecules, at least).
You get a segfault due to a stack overflow. Solution: increase the size of your stack, as discussed here.
Something else happened, in which case you should show the commands you used, the error message, sessionInfo(), etc. A reproducible example would be great but I doubt you will be able to provide it.
For future reference, use "add comment" to an existing answer to post updates like this. You should only use "add answer" if you're answering your own question.
Hi Joon,
Can you provide some more information about how you ran swappedDrops (e.g., locally or on a compute cluster), and whether there was an error message that was reported?
Loading in very many unfiltered matrices from CellRanger should be quite tough on your RAM. Are you running in an environment with sufficient memory, even before swappedDrops is called?
Jonny