When doing multiple muscle alignments (in a loop) in R (v. 3.0.3) under OS X 10.9.5 64 bits, the memory usage of R keeps increasing until the system has to stop the App. 

This is muscle 3.8.31-4

I'm doing 100,000+ alignments of pairs of sequences (length 1000 bp or less) in a loop. I don't save the alignments, I just output each alignment in an object that I process.

The command goes like this (simplified)

for (i in 1:100000) {

   aln = muscle(sequences, quiet =T)

   #do stuff with aln

}

The "sequences" object is just a pair of dna sequences. Note that the "aln" object should be purged at every iteration. So I don't see any reason why memory usage goes out of control. I have 32 GB or ram in my Mac. After approx 70000 steps, the system could not compress memory any longer and forced R to stop. R was using 80 gb+ of virtual memory.

And it's not the stuff I do with the aln object that causes the issue.

Thanks for your help.

Jean

Of note : processing speed decreases as memory usage increases to ridiculous levels ( > 20 gb for R).

That also happens when the output of muscle is written to a file instead of an object.

But I see that internally, the muscle() function writes and reads to file using read.table and write.table (even if muscle() returns and object), which are notorious for being extremely slow. The function should use writeChar or readChar, or at least readLines. Or better, there shouldn't be any disk I/O for that, since MUSCLE can read from stdin write to stdout. But I don't think the memory issue has anything to do with disk I/O. 

Anyway, I skipped the muscle package and used system calls from R to do my alignments, which is not only faster, but also much less memory hungry. For one, I can capture the output of MUSCLE from std.out without having to read an alignment from disk. (However, I haven't found a way to align my R sequences without putting them in a file, any advice would be appreciated :-)

For my uses, the muscle package is not suited. :(

Although you mention which version of muscle you are using, it's not clear to me if this is the package version, or the version of muscle that is in the package, or what ...

I'm curious if you've tried using MUSCLE through the bioconductor msa package, or the bioconductor muscle package?

I'm guessing from your R version number that you are not, since both the msa and muscle packages were recent additions to bioconductor, and your R version would not be using the latest bioconductor version, by default.

You could try to upgrade your R and Bioc installation anyway to see if this problem persists ... it would be good to know for posterity, even though it seems you have worked around your immediate problem.

Thanks for the answer. I think I have installed muscle from the bioconductor muscle package. 

I'd like to upgrade R, but last I tried, I got issues. For instance the anova.glm() function was not found (!) even though the default stat package was loaded. I had to revert to a previous version. 

I used the pairwiseAlignment function from biostring once. I thought it was slower than muscle, but I may try it again. 

Jean

Indeed the pairwiseAlignment() function does exactly what I want. I will use it. 

Thanks for the help.   Jean