Question

flowCore --problem with read.FCS()

0

Entering edit mode

mandycm26 • 0

@mandycm26-8247

Last seen 9.5 years ago

United States

I actually have two questions.

1) I tried to use run read.FCS() on some pretty big files, >1GB each. My computer (an old mac) kept freezing. Is there an alternative I can try? What is the general file size limit?

2) I decided to try this on a new and more powerful computer. I installed R and the flowCore package, but I get the error below that I never encountered before. Can someone tell me what the problem might be? Thanks!

> myFile <- read.FCS("sample1.fcs")
Error in if (usedBits < bitwidth) dat <- dat%%(2^usedBits) :
missing value where TRUE/FALSE needed
In addition: Warning message:
In readFCSdata(con, offsets, txt, transformation, which.lines, scale, :
NAs introduced by coercion to integer range

flowcore • 3.0k views

ADD COMMENT • link updated 9.5 years ago by Jiang, Mike ★ 1.3k • written 9.6 years ago by mandycm26 • 0

0

Entering edit mode

Your FCS may be using 64 bit 'integer' which overflows the R's 32 bit integer type. Try to convert it to 'numeric' (float) with the vendor software (if possible).

ADD REPLY • link 9.6 years ago Jiang, Mike ★ 1.3k

0

Entering edit mode

Thanks, Mike. But I don't have problems running read.FCS() on the same file with my old computer --only have the problem with the new computer and new R installation. Do you still think this is the issue? And I don't think I can convert the FCS file.

ADD REPLY • link 9.6 years ago mandycm26 • 0

0

Entering edit mode

Based on what you described, the reading process stopped in the middle on your old computer, which I suspect never got to the point where the error is thrown.

One thing you can inspect your FCS without loading the huge raw data is to read the FCS header. Try to see what these give you:

header <- as.list(read.FCSheader("sample1.fcs")[[1]])

header[["$PAR"]]
header[["$DATATYPE"]]
head[["$TOT"]]
header[["$P1R"]]
header[["$P1B"]]

ADD REPLY • link 9.6 years ago Jiang, Mike ★ 1.3k

0

Entering edit mode

Hi Mike,

Sorry I wasn't clear. I usually had no problem running read.FCS() on my old mac --not until now when I tried to process some bigger files that are > 1GB. I tried the running the read.FCSheader() command as you suggested and it has no problem. Is there anyway way to read FCS files in blocks? Alternatively I want to try this on a new Windows computer but run into error (even with a small FCS file that I have no problem with on my old mac).

ADD REPLY • link 9.5 years ago mandycm26 • 0

0

Entering edit mode

Could you post the result of these commands on your new computer?

ADD REPLY • link 9.5 years ago Jiang, Mike ★ 1.3k

0

Entering edit mode

Hi Mike, the error is as I posted in my original post. This is a small FCS file that I have no problem with on my old mac, but can't get it to work on my new computer.

> myFile <- read.FCS("Plate1.fcs")
Error in if (usedBits < bitwidth) dat <- dat%%(2^usedBits) :
missing value where TRUE/FALSE needed
In addition: Warning message:
In readFCSdata(con, offsets, txt, transformation, which.lines, scale, :
NAs introduced by coercion to integer range

ADD REPLY • link 9.5 years ago mandycm26 • 0

0

Entering edit mode

With regard to file size, I think MATLAB has a 2BG file size limit. I am not familiar with R. Is there a max file size that we can read in?

ADD REPLY • link 9.5 years ago mandycm26 • 0

score 0 · Answer 1 · 2015-07-06

0

Entering edit mode

Jiang, Mike ★ 1.3k

@jiang-mike-4886

Last seen 3.3 years ago

(Private Address)

Here is your example file header info,

> header[["$P1R"]]
[1] "4294967296"
> header[["$P1B"]]
[1] "32"
> header[["$DATATYPE"]]
[1] "I"

It is supposed to be 32-bit 'integer`. But the 'range` is larger than the maximum integer R can represent (i.e. 2^31-1 = 2147483647). So your new R instance gives the accurate error and warning message.

Error in readFCSdata(con, offsets, txt, transformation, which.lines, scale,  : 
  $PnR is larger than the integer limit: 4294967296
In addition: Warning message:
In readFCSdata(con, offsets, txt, transformation, which.lines, scale,  :
  NAs introduced by coercion to integer range

Unless the event intensity values are stored in a format other than what this 'PnR' keyword indicates, your old computer may have parsed it incorrectly.

ADD COMMENT • link 9.5 years ago Jiang, Mike ★ 1.3k

0

Entering edit mode

Thanks so much, Mike. The data my old laptop produce make complete sense so I am surprised. I wonder if this is a Mac/Linux vs Windows difference? My old laptop is a Mac and my new desktop runs Windows. So does this mean there is nothing I can do to run R/FlowCore on my new computer on these files? Is the 32-1 due to signed vs. unsigned? Do you know if there is another library perhaps in Python or Matlab that can handle this? All I am trying to do is to parse the FCS binary data.

Thanks so much again for all the help.

ADD REPLY • link 9.5 years ago mandycm26 • 0

0

Entering edit mode

I haven't used FCS file parsers of other platform. Maybe you can try to use flowJo?

ADD REPLY • link 9.5 years ago Jiang, Mike ★ 1.3k