Re: Memory management in Windows 2000
1
0
Entering edit mode
Eric Blalock ▴ 250
@eric-blalock-78
Last seen 10.2 years ago
I have had similar issues with memory limitations in Win2K (sp2). If you max the command line memory for R, remember to increase the virtual memory/ paging file in Windows (I think the max is around 4GB). Sadly, increasing RAM from 512 meg to 1.5GB did not increase the number of chips I could run in Windows- I'd guess it was some sort of Microsoft shenanigans with memory management. What did help was to run the analysis in stages. First read the data, then save the workspace image and shut R down. Restart R, load the workspace image, normalize the data, and save the image. Then shut down R and restart. Load the new normalized workspace image and see with memory.size() to see the reduction in memory demand compared to running the analysis straight through in a single session. A combination of the paging file/ command line/ analysis staging procedures helped me increase the number of chips on which I could run quantile normalization from 15 to 26- thanks to Benjamin Bolstad for walking me through this. Unfortunately I have far more chips than this, so it looks like the solution for me will probably end up being a dedicated R Linux box, unfortunately what I know about Linux is how to pronounce it, and nothing else, so I'll have to see what kind of a train wreck I create doing this. Cheers, Eric At 04:20 PM 11/26/2002 +0100, you wrote: >Send Bioconductor mailing list submissions to > bioconductor@stat.math.ethz.ch > >To subscribe or unsubscribe via the World Wide Web, visit > http://www.stat.math.ethz.ch/mailman/listinfo/bioconductor >or, via email, send a message with subject or body 'help' to > bioconductor-request@stat.math.ethz.ch > >You can reach the person managing the list at > bioconductor-admin@stat.math.ethz.ch > >When replying, please edit your Subject line so it is more specific >than "Re: Contents of Bioconductor digest..." > > >Today's Topics: > > 1. RE: memory requirement (Stephen Henderson) > 2. RE: memory requirement (Liaw, Andy) > 3. RE: memory requirement (Tom McCarty) > 4. RE: memory requirement (Liaw, Andy) > 5. XML specifs for R GUIs dialog boxes (Philippe Grosjean) > >--__--__-- > >Message: 1 >From: Stephen Henderson <s.henderson@ucl.ac.uk> >To: "'Laurent Gautier'" <laurent@cbs.dtu.dk> >Cc: "'bioconductor@stat.math.ethz.ch'" <bioconductor@stat.math.ethz.ch> >Subject: RE: [BioC] memory requirement >Date: Tue, 26 Nov 2002 11:51:34 -0000 > > >Here is the advice given to me on improving R memory on Win some time ago on >this very list from --Tibor van. This helped a bit. > >If you have a shortcut on your desktop for R (or somewhere else), then if >you right click on the icon and bring up the Properties window, you will see >an entry for >Target with the path to the executable filled in. After that path (and >outside of the quotes) add this command-line argument on my PC, for example >it would look like this: > > > >"C:\Program Files\R\rw1051\bin\Rgui.exe" --max-mem-size=768M > > > >Restart R and you will be in business... > >Stephen Henderson >Wolfson Institute for Biomedical Research >UCL, London > > > > >-----Original Message----- >From: Laurent Gautier [mailto:laurent@cbs.dtu.dk] >Sent: Tuesday, November 26, 2002 12:57 AM >To: Wilkinson, Mark >Cc: 'bioconductor@stat.math.ethz.ch' >Subject: Re: [BioC] memory requirement > >On Mon, Nov 25, 2002 at 03:43:08PM -0600, Wilkinson, Mark wrote: > > Here, here. I'd've asked sooner, but suspected I'd be advised the brute > > force approach of beefing up my machine. > > > > I'm running the affy package, specifically, with U133 chips, but am only > > able to read in 10-15 CEL files at once. Any way around this? I've > > considered using 'merge' to read a few, merge, read a few more, merge, >etc. > > I'm using Windows R, which seems to have poor memory management anyway. >Any > > tips? > > > >How much memory did you have ? > >I do not know much of the behavior with Windows, but with linux I remember >doing >the full monty (raw data -> normalization -> summary expression value) for >about 80 U95A at once on a machine with 2Gb. > >note : the new release of the package spits out something with the size of >the 'AffyBatch' object with the default print method (try: >data(Dilution) >Dilution >) > >Hopin' it helps, > > >L. > > > > > > Mark Wilkinson > > Informatics Analyst > > St. Jude Children's Research Hospital > > Department of Pharmaceutical Sciences > > > > > > -----Original Message----- > > From: Yi-Xiong Zhou [mailto:yzhou@arcturusag.com] > > Sent: Monday, November 25, 2002 3:24 PM > > To: 'bioconductor@stat.math.ethz.ch' > > Subject: [BioC] memory requirement > > > > I am concerning the memory requirement of running bioconductor on Affy > > chips. Could you give me an estimate? > > > > Thanks. > > > > Yi-Xiong > > > > _______________________________________________ > > Bioconductor mailing list > > Bioconductor@stat.math.ethz.ch > > http://www.stat.math.ethz.ch/mailman/listinfo/bioconductor > > > > _______________________________________________ > > Bioconductor mailing list > > Bioconductor@stat.math.ethz.ch > > http://www.stat.math.ethz.ch/mailman/listinfo/bioconductor > >-- >-------------------------------------------------------------- >Laurent Gautier CBS, Building 208, DTU >PhD. Student DK-2800 Lyngby,Denmark >tel: +45 45 25 24 89 http://www.cbs.dtu.dk/laurent > >_______________________________________________ >Bioconductor mailing list >Bioconductor@stat.math.ethz.ch >http://www.stat.math.ethz.ch/mailman/listinfo/bioconductor > > >********************************************************************* * >This email and any files transmitted with it are confidential and >intended solely for the use of the individual or entity to whom they >are addressed. If you have received this email in error please notify >the system manager (wibr.mail@ucl.ac.uk). All files are scanned for viruses. >********************************************************************* * > > > [[alternate HTML version deleted]] > > >--__--__-- > >Message: 2 >Date: Tue, 26 Nov 2002 08:01:09 -0500 >From: "Liaw, Andy" <andy_liaw@merck.com> >Subject: RE: [BioC] memory requirement >To: "'Stephen Henderson'" <s.henderson@ucl.ac.uk> >cc: "'bioconductor@stat.math.ethz.ch'" <bioconductor@stat.math.ethz.ch> > >If you've upgraded to R-1.6.1, note that the limit on Windows has been >raised to the smaller of physical RAM and 1GB (used to be 256MB). So unless >you have more than 1GB of RAM, there's no need to do that. > >Andy > >-----Original Message----- >From: Stephen Henderson [mailto:s.henderson@ucl.ac.uk] >Sent: Tuesday, November 26, 2002 6:52 AM >To: 'Laurent Gautier' >Cc: 'bioconductor@stat.math.ethz.ch' >Subject: RE: [BioC] memory requirement > > > >Here is the advice given to me on improving R memory on Win some time ago on >this very list from --Tibor van. This helped a bit. > >If you have a shortcut on your desktop for R (or somewhere else), then if >you right click on the icon and bring up the Properties window, you will see >an entry for >Target with the path to the executable filled in. After that path (and >outside of the quotes) add this command-line argument on my PC, for example >it would look like this: > > > >"C:\Program Files\R\rw1051\bin\Rgui.exe" --max-mem-size=768M > > > >Restart R and you will be in business... > >Stephen Henderson >Wolfson Institute for Biomedical Research >UCL, London > > > > >-----Original Message----- >From: Laurent Gautier [mailto:laurent@cbs.dtu.dk] >Sent: Tuesday, November 26, 2002 12:57 AM >To: Wilkinson, Mark >Cc: 'bioconductor@stat.math.ethz.ch' >Subject: Re: [BioC] memory requirement > >On Mon, Nov 25, 2002 at 03:43:08PM -0600, Wilkinson, Mark wrote: > > Here, here. I'd've asked sooner, but suspected I'd be advised the brute > > force approach of beefing up my machine. > > > > I'm running the affy package, specifically, with U133 chips, but am only > > able to read in 10-15 CEL files at once. Any way around this? I've > > considered using 'merge' to read a few, merge, read a few more, merge, >etc. > > I'm using Windows R, which seems to have poor memory management anyway. >Any > > tips? > > > >How much memory did you have ? > >I do not know much of the behavior with Windows, but with linux I remember >doing >the full monty (raw data -> normalization -> summary expression value) for >about 80 U95A at once on a machine with 2Gb. > >note : the new release of the package spits out something with the size of >the 'AffyBatch' object with the default print method (try: >data(Dilution) >Dilution >) > >Hopin' it helps, > > >L. > > > > > > Mark Wilkinson > > Informatics Analyst > > St. Jude Children's Research Hospital > > Department of Pharmaceutical Sciences > > > > > > -----Original Message----- > > From: Yi-Xiong Zhou [mailto:yzhou@arcturusag.com] > > Sent: Monday, November 25, 2002 3:24 PM > > To: 'bioconductor@stat.math.ethz.ch' > > Subject: [BioC] memory requirement > > > > I am concerning the memory requirement of running bioconductor on Affy > > chips. Could you give me an estimate? > > > > Thanks. > > > > Yi-Xiong > > > > _______________________________________________ > > Bioconductor mailing list > > Bioconductor@stat.math.ethz.ch > > http://www.stat.math.ethz.ch/mailman/listinfo/bioconductor > > > > _______________________________________________ > > Bioconductor mailing list > > Bioconductor@stat.math.ethz.ch > > http://www.stat.math.ethz.ch/mailman/listinfo/bioconductor > >-- >-------------------------------------------------------------- >Laurent Gautier CBS, Building 208, DTU >PhD. Student DK-2800 Lyngby,Denmark >tel: +45 45 25 24 89 http://www.cbs.dtu.dk/laurent > >_______________________________________________ >Bioconductor mailing list >Bioconductor@stat.math.ethz.ch >http://www.stat.math.ethz.ch/mailman/listinfo/bioconductor > > >********************************************************************* * >This email and any files transmitted with it are confidential and >intended solely for the use of the individual or entity to whom they >are addressed. If you have received this email in error please notify >the system manager (wibr.mail@ucl.ac.uk). All files are scanned for viruses. >********************************************************************* * > > > [[alternate HTML version deleted]] > >_______________________________________________ >Bioconductor mailing list >Bioconductor@stat.math.ethz.ch >http://www.stat.math.ethz.ch/mailman/listinfo/bioconductor > > >--------------------------------------------------------------------- --------- >Notice: This e-mail message, together with any attachments, contains >information of Merck & Co., Inc. (Whitehouse Station, New Jersey, USA) >that may be confidential, proprietary copyrighted and/or legally >privileged, and is intended solely for the use of the individual or entity >named in this message. If you are not the intended recipient, and have >received this message in error, please immediately return this by e-mail >and then delete it. > >===================================================================== ========= > > >--__--__-- > >Message: 3 >From: Tom McCarty <tmccarty@niaid.nih.gov> >To: "'Liaw, Andy'" <andy_liaw@merck.com>, > "'Stephen Henderson'" > <s.henderson@ucl.ac.uk> >Cc: "'bioconductor@stat.math.ethz.ch'" <bioconductor@stat.math.ethz.ch> >Subject: RE: [BioC] memory requirement >Date: Tue, 26 Nov 2002 08:13:28 -0500 > >So if I have more than 1 GB of RAM, following the previous suggestion is the >best way to improve memory allocation to R? > >Tom McCarty > >-----Original Message----- >From: Liaw, Andy [mailto:andy_liaw@merck.com] >Sent: Tuesday, November 26, 2002 8:01 AM >To: 'Stephen Henderson' >Cc: 'bioconductor@stat.math.ethz.ch' >Subject: RE: [BioC] memory requirement > > >If you've upgraded to R-1.6.1, note that the limit on Windows has been >raised to the smaller of physical RAM and 1GB (used to be 256MB). So unless >you have more than 1GB of RAM, there's no need to do that. > >Andy > >-----Original Message----- >From: Stephen Henderson [mailto:s.henderson@ucl.ac.uk] >Sent: Tuesday, November 26, 2002 6:52 AM >To: 'Laurent Gautier' >Cc: 'bioconductor@stat.math.ethz.ch' >Subject: RE: [BioC] memory requirement > > > >Here is the advice given to me on improving R memory on Win some time ago on >this very list from --Tibor van. This helped a bit. > >If you have a shortcut on your desktop for R (or somewhere else), then if >you right click on the icon and bring up the Properties window, you will see >an entry for >Target with the path to the executable filled in. After that path (and >outside of the quotes) add this command-line argument on my PC, for example >it would look like this: > > > >"C:\Program Files\R\rw1051\bin\Rgui.exe" --max-mem-size=768M > > > >Restart R and you will be in business... > >Stephen Henderson >Wolfson Institute for Biomedical Research >UCL, London > > > > >-----Original Message----- >From: Laurent Gautier [mailto:laurent@cbs.dtu.dk] >Sent: Tuesday, November 26, 2002 12:57 AM >To: Wilkinson, Mark >Cc: 'bioconductor@stat.math.ethz.ch' >Subject: Re: [BioC] memory requirement > >On Mon, Nov 25, 2002 at 03:43:08PM -0600, Wilkinson, Mark wrote: > > Here, here. I'd've asked sooner, but suspected I'd be advised the brute > > force approach of beefing up my machine. > > > > I'm running the affy package, specifically, with U133 chips, but am only > > able to read in 10-15 CEL files at once. Any way around this? I've > > considered using 'merge' to read a few, merge, read a few more, merge, >etc. > > I'm using Windows R, which seems to have poor memory management anyway. >Any > > tips? > > > >How much memory did you have ? > >I do not know much of the behavior with Windows, but with linux I remember >doing >the full monty (raw data -> normalization -> summary expression value) for >about 80 U95A at once on a machine with 2Gb. > >note : the new release of the package spits out something with the size of >the 'AffyBatch' object with the default print method (try: >data(Dilution) >Dilution >) > >Hopin' it helps, > > >L. > > > > > > Mark Wilkinson > > Informatics Analyst > > St. Jude Children's Research Hospital > > Department of Pharmaceutical Sciences > > > > > > -----Original Message----- > > From: Yi-Xiong Zhou [mailto:yzhou@arcturusag.com] > > Sent: Monday, November 25, 2002 3:24 PM > > To: 'bioconductor@stat.math.ethz.ch' > > Subject: [BioC] memory requirement > > > > I am concerning the memory requirement of running bioconductor on Affy > > chips. Could you give me an estimate? > > > > Thanks. > > > > Yi-Xiong > > > > _______________________________________________ > > Bioconductor mailing list > > Bioconductor@stat.math.ethz.ch > > http://www.stat.math.ethz.ch/mailman/listinfo/bioconductor > > > > _______________________________________________ > > Bioconductor mailing list > > Bioconductor@stat.math.ethz.ch > > http://www.stat.math.ethz.ch/mailman/listinfo/bioconductor > >-- >-------------------------------------------------------------- >Laurent Gautier CBS, Building 208, DTU >PhD. Student DK-2800 Lyngby,Denmark >tel: +45 45 25 24 89 http://www.cbs.dtu.dk/laurent > >_______________________________________________ >Bioconductor mailing list >Bioconductor@stat.math.ethz.ch >http://www.stat.math.ethz.ch/mailman/listinfo/bioconductor > > >********************************************************************* * >This email and any files transmitted with it are confidential and >intended solely for the use of the individual or entity to whom they >are addressed. If you have received this email in error please notify >the system manager (wibr.mail@ucl.ac.uk). All files are scanned for viruses. >********************************************************************* * > > > [[alternate HTML version deleted]] > >_______________________________________________ >Bioconductor mailing list >Bioconductor@stat.math.ethz.ch >http://www.stat.math.ethz.ch/mailman/listinfo/bioconductor > > >--------------------------------------------------------------------- ------- >-- >Notice: This e-mail message, together with any attachments, contains >information of Merck & Co., Inc. (Whitehouse Station, New Jersey, USA) that >may be confidential, proprietary copyrighted and/or legally privileged, and >is intended solely for the use of the individual or entity named in this >message. If you are not the intended recipient, and have received this >message in error, please immediately return this by e-mail and then delete >it. > >===================================================================== ======= >== > >_______________________________________________ >Bioconductor mailing list >Bioconductor@stat.math.ethz.ch >http://www.stat.math.ethz.ch/mailman/listinfo/bioconductor > > [[alternate HTML version deleted]] > > >--__--__-- > >Message: 4 >Date: Tue, 26 Nov 2002 08:26:02 -0500 >From: "Liaw, Andy" <andy_liaw@merck.com> >Subject: RE: [BioC] memory requirement >To: "'Tom McCarty'" <tmccarty@niaid.nih.gov>, > "Liaw, Andy" <andy_liaw@merck.com>, > "'Stephen Henderson'" <s.henderson@ucl.ac.uk> >cc: "'bioconductor@stat.math.ethz.ch'" <bioconductor@stat.math.ethz.ch> > >Depends on how much memory you want R to be able to use. If you want R to >use more than 1GB (and you have more than 1GB), then yes, you should add the >--max-mem-size argument. > >This is an "improvement" only in the sense that you're giving R more to play >with. > >Andy >-----Original Message----- >From: Tom McCarty [mailto:TMCCARTY@niaid.nih.gov] >Sent: Tuesday, November 26, 2002 8:13 AM >To: 'Liaw, Andy'; 'Stephen Henderson' >Cc: 'bioconductor@stat.math.ethz.ch' >Subject: RE: [BioC] memory requirement > > > >So if I have more than 1 GB of RAM, following the previous suggestion is the >best way to improve memory allocation to R? > >Tom McCarty > >-----Original Message----- >From: Liaw, Andy [ mailto:andy_liaw@merck.com <mailto:andy_liaw@merck.com> ] > >Sent: Tuesday, November 26, 2002 8:01 AM >To: 'Stephen Henderson' >Cc: 'bioconductor@stat.math.ethz.ch' >Subject: RE: [BioC] memory requirement > > >If you've upgraded to R-1.6.1, note that the limit on Windows has been >raised to the smaller of physical RAM and 1GB (used to be 256MB). So unless > >you have more than 1GB of RAM, there's no need to do that. > >Andy > >-----Original Message----- >From: Stephen Henderson [ mailto:s.henderson@ucl.ac.uk ><mailto:s.henderson@ucl.ac.uk> ] >Sent: Tuesday, November 26, 2002 6:52 AM >To: 'Laurent Gautier' >Cc: 'bioconductor@stat.math.ethz.ch' >Subject: RE: [BioC] memory requirement > > > >Here is the advice given to me on improving R memory on Win some time ago on > >this very list from --Tibor van. This helped a bit. > >If you have a shortcut on your desktop for R (or somewhere else), then if >you right click on the icon and bring up the Properties window, you will see > >an entry for >Target with the path to the executable filled in. After that path (and >outside of the quotes) add this command-line argument on my PC, for example >it would look like this: > >"C:\Program Files\R\rw1051\bin\Rgui.exe" --max-mem-size=768M > >Restart R and you will be in business... > >Stephen Henderson >Wolfson Institute for Biomedical Research >UCL, London > > > > >-----Original Message----- >From: Laurent Gautier [ mailto:laurent@cbs.dtu.dk ><mailto:laurent@cbs.dtu.dk> ] >Sent: Tuesday, November 26, 2002 12:57 AM >To: Wilkinson, Mark >Cc: 'bioconductor@stat.math.ethz.ch' >Subject: Re: [BioC] memory requirement > >On Mon, Nov 25, 2002 at 03:43:08PM -0600, Wilkinson, Mark wrote: > > Here, here. I'd've asked sooner, but suspected I'd be advised the brute > > force approach of beefing up my machine. > > > > I'm running the affy package, specifically, with U133 chips, but am only > > able to read in 10-15 CEL files at once. Any way around this? I've > > considered using 'merge' to read a few, merge, read a few more, merge, >etc. > > I'm using Windows R, which seems to have poor memory management anyway. >Any > > tips? > > > >How much memory did you have ? > >I do not know much of the behavior with Windows, but with linux I remember >doing >the full monty (raw data -> normalization -> summary expression value) for >about 80 U95A at once on a machine with 2Gb. > >note : the new release of the package spits out something with the size of >the 'AffyBatch' object with the default print method (try: >data(Dilution) >Dilution >) > >Hopin' it helps, > > >L. > > > > > > Mark Wilkinson > > Informatics Analyst > > St. Jude Children's Research Hospital > > Department of Pharmaceutical Sciences > > > > > > -----Original Message----- > > From: Yi-Xiong Zhou [ mailto:yzhou@arcturusag.com ><mailto:yzhou@arcturusag.com> ] > > Sent: Monday, November 25, 2002 3:24 PM > > To: 'bioconductor@stat.math.ethz.ch' > > Subject: [BioC] memory requirement > > > > I am concerning the memory requirement of running bioconductor on Affy > > chips. Could you give me an estimate? > > > > Thanks. > > > > Yi-Xiong > > > > _______________________________________________ > > Bioconductor mailing list > > Bioconductor@stat.math.ethz.ch > > http://www.stat.math.ethz.ch/mailman/listinfo/bioconductor ><http: www.stat.math.ethz.ch="" mailman="" listinfo="" bioconductor=""> > > > > _______________________________________________ > > Bioconductor mailing list > > Bioconductor@stat.math.ethz.ch > > http://www.stat.math.ethz.ch/mailman/listinfo/bioconductor ><http: www.stat.math.ethz.ch="" mailman="" listinfo="" bioconductor=""> > >-- >-------------------------------------------------------------- >Laurent Gautier CBS, Building 208, DTU >PhD. Student DK-2800 Lyngby,Denmark >tel: +45 45 25 24 89 http://www.cbs.dtu.dk/laurent ><http: www.cbs.dtu.dk="" laurent=""> > >_______________________________________________ >Bioconductor mailing list >Bioconductor@stat.math.ethz.ch >http://www.stat.math.ethz.ch/mailman/listinfo/bioconductor ><http: www.stat.math.ethz.ch="" mailman="" listinfo="" bioconductor=""> > > >********************************************************************* * >This email and any files transmitted with it are confidential and >intended solely for the use of the individual or entity to whom they >are addressed. If you have received this email in error please notify >the system manager (wibr.mail@ucl.ac.uk). All files are scanned for viruses. > >********************************************************************* * > > > [[alternate HTML version deleted]] > >_______________________________________________ >Bioconductor mailing list >Bioconductor@stat.math.ethz.ch >http://www.stat.math.ethz.ch/mailman/listinfo/bioconductor ><http: www.stat.math.ethz.ch="" mailman="" listinfo="" bioconductor=""> > > >--------------------------------------------------------------------- ------- >-- >Notice: This e-mail message, together with any attachments, contains >information of Merck & Co., Inc. (Whitehouse Station, New Jersey, USA) that >may be confidential, proprietary copyrighted and/or legally privileged, and >is intended solely for the use of the individual or entity named in this >message. If you are not the intended recipient, and have received this >message in error, please immediately return this by e-mail and then delete >it. > >===================================================================== ======= >== > >_______________________________________________ >Bioconductor mailing list >Bioconductor@stat.math.ethz.ch >http://www.stat.math.ethz.ch/mailman/listinfo/bioconductor ><http: www.stat.math.ethz.ch="" mailman="" listinfo="" bioconductor=""> > > >--------------------------------------------------------------------- --------- >Notice: This e-mail message, together with any attachments, contains >information of Merck & Co., Inc. (Whitehouse Station, New Jersey, USA) >that may be confidential, proprietary copyrighted and/or legally >privileged, and is intended solely for the use of the individual or entity >named on this message. If you are not the intended recipient, and have >received this message in error, please immediately return this by e-mail >and then delete it. > >===================================================================== ========= > > [[alternate HTML version deleted]] > > >--__--__-- > >Message: 5 >Reply-To: <phgrosjean@sciviews.org> >From: "Philippe Grosjean" <phgrosjean@sciviews.org> >To: "Thomas Friedrichsmeier" <thomas.friedrichsmeier@ruhr-uni- bochum.de="">, > <dmurdoch@pair.com>, <rossini@blindglobe.net>, > <bioconductor@stat.math.ethz.ch> >Cc: <r-sig-gui@stat.math.ethz.ch> >Date: Tue, 26 Nov 2002 11:13:25 +0100 >Subject: [BioC] XML specifs for R GUIs dialog boxes > >Since this becomes a fairly large thread that goes in many different (and >interesting) directions, I propose to start a new one with a more explicit >title. > >I will try to summarize major points discussed so far. > >- Thomas Friedrichsmeier [Thomas.Friedrichsmeier@ruhr-uni-bochum.de] after >"having a short conversation with the obveRsive-people" introduced the idea >of a sort of "plugin" with potentials to contruct dialog boxes at run-time. >He proposes to use a XML-file for specification of the dialog box (ex: >t-test dialog box, http://rkward.sf.net/pics.php). > >- We all agree (I think) that a common semantic for the description of these >dialog boxes accross projects (whatever the language, the graphical toolkit >and the platform used) should be fine. So, we are on the way to define it, >and I'll dedicate a page to it at http://www.r-project.org/GUI, as soon as >we have something consistent (thus, discussed enough in R-SIG-GUI). Also, >this implies it should not be describing features at a too low level, as >pointed out by Thomas, or we take the risk to get something that would be >less portable from one language/graph toolkit to the other. > >- Peter Dalgaard [p.dalgaard@biostat.ku.dk] suggests to contact people in >Bioconductor that may be working on a similar thing (I send them a copy of >this mail). > >- Philippe Grosjean [phgrosjean@sciviews.org] proposes to include >context-sensitive features, allowing for instance to call such dialog boxes >directly from an object explorer, or whatever context menu. > >- While Philippe and Thomas think that a majority (but not all) dialog boxes >could be constructed this way (they probably bear in mind a way to quickly >construct simple to rather complex dialog boxes, but not an exhaustive >approach), Duncan Murdoch [murdoch@stats.uwo.ca] considers that it should be >possible to make it more exhaustive ("I don't see any problem with thinking >about all of them as being made of XML"). > >- Although the specifications could target a run-time dynamic construction >of the dialog box, it could also be used to define dialog boxes at design >time, at least if we follows the Delphi scheme, as pointed out by Duncan >(?). A.J. Rossini [rossini@blindglobe.net] says that it is "just a matter of >translation, for the most part". Probably, this should be more a question of >implementation in the various different languages and should not be >discussed too deeply in the early stages. > >- About layering, Thomas proposes something simple, like a grid with perhaps >tabbed dialog boxes (note: this is roughly the layering potentials of Splus >custom dialog boxes), while Duncan whould like to have something more >powerful. > >- The list of widgets to support is not fixed yet too. Duncan proposes to >focus on low level widgets first (label, text, button, etc...). Thomas >proposes a list that mixes low level and high level widgets (I think we >should make the distinction and propose two separate lists). At this point, >we should also discuss with the Bioconductor team that has already done some >work on creating various interesting high level widgets in tcltk like >objects lists for instance. Here is the widgets list propsed by Thomas: >"+ some sort of widget that lists available objects (...) > + a widget that can hold a (single) object (the "varslot"), and will >probably make some specifications about what types of objects it can hold >(like the attribute type="numeric" in the example) > + maybe a separate widget that can hold one or more objects, but basically >acts the same (might instead simply use a more generalized "varslot") > + maybe a separate widget that can hold a single constant value (number or >string) > + a widget that can be used to select interactions between variables >selected in a "varslot" > + radio-buttons > + checkboxes > + a widget representing a text/label not directly attached to any of the >other widgets > + a widget represent a line or other separation used for layouting >purposes" > >Duncan adds: > + a tree view widget > + a free text widget > >I'll immediately add: > + various classical dialog boxes like file open/close for instance > + a widget that displays a (multi)-selectable list of columns in a >particular data frame, or elements in a list. > + a graph widget (yes a whole R graph device) > >And perhaps: > + a menuing system (with perhaps context menus attached to other widgets) > + sliders to select values in a range > >- There are some open aspects, like the possibility to define new widgets, >or the way results are submitted to R (simple seach/replace mechanism, data >frame or other code describing the whole dialog box and processed by R >code,...). > >- Since someone must initiate a document for the specifs, and Thomas seems >to have something almost ready, I proposed him to make a draft. At this >stage, modifications in a document will be more constructive than loosy >discussions in R-SIG-GUI. > >Best, > >Philippe > > >P.S.: for those of you that receive this mail and are interested, but not >listed yet on R-SIG-GUI, you can register at: > > http://www.stat.math.ethz.ch/mailman/listinfo/r-sig-gui > >Special Interest Group on Development of GUI(s) for R. > >Topic: Discussion of Strategies and approaches; particularly platform >independent ones. See http://www.R-project.org/GUI/ for more. > >...........]<(({?<...............<............................. .. >( ( ( ( ( > ) ) ) ) ) Philippe Grosjean >( ( ( ( ( > ) ) ) ) ) IFREMER Nantes - DEL/AO >( ( ( ( ( rue de l'Ile d'Yeu, BP 21105, 44311 Nantes Cedex 3 > ) ) ) ) ) tel: (33) 02.40.37.42.29, fax: (33) 02.40.37.42.41 >( ( ( ( ( > ) ) ) ) ) SciViews project coordinator (http://www.sciviews.org) >( ( ( ( ( e-mail: phgrosjean@sciviews.org > ) ) ) ) ) >( ( ( ( ( "I'm 100% confident that p is between 0 and 1" > ) ) ) ) ) L. Gonick & W. Smith (1993) >..................................................................... .. > > > >--__--__-- > >_______________________________________________ >Bioconductor mailing list >Bioconductor@stat.math.ethz.ch >http://www.stat.math.ethz.ch/mailman/listinfo/bioconductor > > >End of Bioconductor Digest Eric Blalock, PhD Dept Pharmacology, UKMC 859 323-8033
Normalization affy graph Normalization affy graph • 1.2k views
ADD COMMENT
0
Entering edit mode
Liaw, Andy ▴ 360
@liaw-andy-125
Last seen 10.2 years ago
What follows is what I do for dealing with large data in R, not particular to Bioconductor. It should applies to any platform. Remember that R keeps all objects in the workspace, which sits in the virtual memory. Thus if you have objects you don't need (e.g., from another project) lying around, they take up memory. So the first thing to make sure is that you only have objects that you absolutely need in the workspace. rm() the rest. If you need them, you can use save() to save them and attach() or load() them when needed. I.e., keep your workspace sqeeky (sp?) clean. Another thing that sometimes helps is to run gc() (maybe a couple of times) before/between large computations. HTH, Andy -----Original Message----- From: Eric [mailto:emblal@uky.edu] Sent: Tuesday, November 26, 2002 11:17 AM To: bioconductor@stat.math.ethz.ch Subject: [BioC] Re: Memory management in Windows 2000 I have had similar issues with memory limitations in Win2K (sp2). If you max the command line memory for R, remember to increase the virtual memory/ paging file in Windows (I think the max is around 4GB). Sadly, increasing RAM from 512 meg to 1.5GB did not increase the number of chips I could run in Windows- I'd guess it was some sort of Microsoft shenanigans with memory management. What did help was to run the analysis in stages. First read the data, then save the workspace image and shut R down. Restart R, load the workspace image, normalize the data, and save the image. Then shut down R and restart. Load the new normalized workspace image and see with memory.size() to see the reduction in memory demand compared to running the analysis straight through in a single session. A combination of the paging file/ command line/ analysis staging procedures helped me increase the number of chips on which I could run quantile normalization from 15 to 26- thanks to Benjamin Bolstad for walking me through this. Unfortunately I have far more chips than this, so it looks like the solution for me will probably end up being a dedicated R Linux box, unfortunately what I know about Linux is how to pronounce it, and nothing else, so I'll have to see what kind of a train wreck I create doing this. Cheers, Eric At 04:20 PM 11/26/2002 +0100, you wrote: >Send Bioconductor mailing list submissions to > bioconductor@stat.math.ethz.ch > >To subscribe or unsubscribe via the World Wide Web, visit > http://www.stat.math.ethz.ch/mailman/listinfo/bioconductor >or, via email, send a message with subject or body 'help' to > bioconductor-request@stat.math.ethz.ch > >You can reach the person managing the list at > bioconductor-admin@stat.math.ethz.ch > >When replying, please edit your Subject line so it is more specific >than "Re: Contents of Bioconductor digest..." > > >Today's Topics: > > 1. RE: memory requirement (Stephen Henderson) > 2. RE: memory requirement (Liaw, Andy) > 3. RE: memory requirement (Tom McCarty) > 4. RE: memory requirement (Liaw, Andy) > 5. XML specifs for R GUIs dialog boxes (Philippe Grosjean) > >--__--__-- > >Message: 1 >From: Stephen Henderson <s.henderson@ucl.ac.uk> >To: "'Laurent Gautier'" <laurent@cbs.dtu.dk> >Cc: "'bioconductor@stat.math.ethz.ch'" <bioconductor@stat.math.ethz.ch> >Subject: RE: [BioC] memory requirement >Date: Tue, 26 Nov 2002 11:51:34 -0000 > > >Here is the advice given to me on improving R memory on Win some time ago on >this very list from --Tibor van. This helped a bit. > >If you have a shortcut on your desktop for R (or somewhere else), then if >you right click on the icon and bring up the Properties window, you will see >an entry for >Target with the path to the executable filled in. After that path (and >outside of the quotes) add this command-line argument on my PC, for example >it would look like this: > > > >"C:\Program Files\R\rw1051\bin\Rgui.exe" --max-mem-size=768M > > > >Restart R and you will be in business... > >Stephen Henderson >Wolfson Institute for Biomedical Research >UCL, London > > > > >-----Original Message----- >From: Laurent Gautier [mailto:laurent@cbs.dtu.dk] >Sent: Tuesday, November 26, 2002 12:57 AM >To: Wilkinson, Mark >Cc: 'bioconductor@stat.math.ethz.ch' >Subject: Re: [BioC] memory requirement > >On Mon, Nov 25, 2002 at 03:43:08PM -0600, Wilkinson, Mark wrote: > > Here, here. I'd've asked sooner, but suspected I'd be advised the brute > > force approach of beefing up my machine. > > > > I'm running the affy package, specifically, with U133 chips, but am only > > able to read in 10-15 CEL files at once. Any way around this? I've > > considered using 'merge' to read a few, merge, read a few more, merge, >etc. > > I'm using Windows R, which seems to have poor memory management anyway. >Any > > tips? > > > >How much memory did you have ? > >I do not know much of the behavior with Windows, but with linux I remember >doing >the full monty (raw data -> normalization -> summary expression value) for >about 80 U95A at once on a machine with 2Gb. > >note : the new release of the package spits out something with the size of >the 'AffyBatch' object with the default print method (try: >data(Dilution) >Dilution >) > >Hopin' it helps, > > >L. > > > > > > Mark Wilkinson > > Informatics Analyst > > St. Jude Children's Research Hospital > > Department of Pharmaceutical Sciences > > > > > > -----Original Message----- > > From: Yi-Xiong Zhou [mailto:yzhou@arcturusag.com] > > Sent: Monday, November 25, 2002 3:24 PM > > To: 'bioconductor@stat.math.ethz.ch' > > Subject: [BioC] memory requirement > > > > I am concerning the memory requirement of running bioconductor on Affy > > chips. Could you give me an estimate? > > > > Thanks. > > > > Yi-Xiong > > > > _______________________________________________ > > Bioconductor mailing list > > Bioconductor@stat.math.ethz.ch > > http://www.stat.math.ethz.ch/mailman/listinfo/bioconductor > > > > _______________________________________________ > > Bioconductor mailing list > > Bioconductor@stat.math.ethz.ch > > http://www.stat.math.ethz.ch/mailman/listinfo/bioconductor > >-- >-------------------------------------------------------------- >Laurent Gautier CBS, Building 208, DTU >PhD. Student DK-2800 Lyngby,Denmark >tel: +45 45 25 24 89 http://www.cbs.dtu.dk/laurent > >_______________________________________________ >Bioconductor mailing list >Bioconductor@stat.math.ethz.ch >http://www.stat.math.ethz.ch/mailman/listinfo/bioconductor > > >********************************************************************* * >This email and any files transmitted with it are confidential and >intended solely for the use of the individual or entity to whom they >are addressed. If you have received this email in error please notify >the system manager (wibr.mail@ucl.ac.uk). All files are scanned for viruses. >********************************************************************* * > > > [[alternate HTML version deleted]] > > >--__--__-- > >Message: 2 >Date: Tue, 26 Nov 2002 08:01:09 -0500 >From: "Liaw, Andy" <andy_liaw@merck.com> >Subject: RE: [BioC] memory requirement >To: "'Stephen Henderson'" <s.henderson@ucl.ac.uk> >cc: "'bioconductor@stat.math.ethz.ch'" <bioconductor@stat.math.ethz.ch> > >If you've upgraded to R-1.6.1, note that the limit on Windows has been >raised to the smaller of physical RAM and 1GB (used to be 256MB). So unless >you have more than 1GB of RAM, there's no need to do that. > >Andy > >-----Original Message----- >From: Stephen Henderson [mailto:s.henderson@ucl.ac.uk] >Sent: Tuesday, November 26, 2002 6:52 AM >To: 'Laurent Gautier' >Cc: 'bioconductor@stat.math.ethz.ch' >Subject: RE: [BioC] memory requirement > > > >Here is the advice given to me on improving R memory on Win some time ago on >this very list from --Tibor van. This helped a bit. > >If you have a shortcut on your desktop for R (or somewhere else), then if >you right click on the icon and bring up the Properties window, you will see >an entry for >Target with the path to the executable filled in. After that path (and >outside of the quotes) add this command-line argument on my PC, for example >it would look like this: > > > >"C:\Program Files\R\rw1051\bin\Rgui.exe" --max-mem-size=768M > > > >Restart R and you will be in business... > >Stephen Henderson >Wolfson Institute for Biomedical Research >UCL, London > > > > >-----Original Message----- >From: Laurent Gautier [mailto:laurent@cbs.dtu.dk] >Sent: Tuesday, November 26, 2002 12:57 AM >To: Wilkinson, Mark >Cc: 'bioconductor@stat.math.ethz.ch' >Subject: Re: [BioC] memory requirement > >On Mon, Nov 25, 2002 at 03:43:08PM -0600, Wilkinson, Mark wrote: > > Here, here. I'd've asked sooner, but suspected I'd be advised the brute > > force approach of beefing up my machine. > > > > I'm running the affy package, specifically, with U133 chips, but am only > > able to read in 10-15 CEL files at once. Any way around this? I've > > considered using 'merge' to read a few, merge, read a few more, merge, >etc. > > I'm using Windows R, which seems to have poor memory management anyway. >Any > > tips? > > > >How much memory did you have ? > >I do not know much of the behavior with Windows, but with linux I remember >doing >the full monty (raw data -> normalization -> summary expression value) for >about 80 U95A at once on a machine with 2Gb. > >note : the new release of the package spits out something with the size of >the 'AffyBatch' object with the default print method (try: >data(Dilution) >Dilution >) > >Hopin' it helps, > > >L. > > > > > > Mark Wilkinson > > Informatics Analyst > > St. Jude Children's Research Hospital > > Department of Pharmaceutical Sciences > > > > > > -----Original Message----- > > From: Yi-Xiong Zhou [mailto:yzhou@arcturusag.com] > > Sent: Monday, November 25, 2002 3:24 PM > > To: 'bioconductor@stat.math.ethz.ch' > > Subject: [BioC] memory requirement > > > > I am concerning the memory requirement of running bioconductor on Affy > > chips. Could you give me an estimate? > > > > Thanks. > > > > Yi-Xiong > > > > _______________________________________________ > > Bioconductor mailing list > > Bioconductor@stat.math.ethz.ch > > http://www.stat.math.ethz.ch/mailman/listinfo/bioconductor > > > > _______________________________________________ > > Bioconductor mailing list > > Bioconductor@stat.math.ethz.ch > > http://www.stat.math.ethz.ch/mailman/listinfo/bioconductor > >-- >-------------------------------------------------------------- >Laurent Gautier CBS, Building 208, DTU >PhD. Student DK-2800 Lyngby,Denmark >tel: +45 45 25 24 89 http://www.cbs.dtu.dk/laurent > >_______________________________________________ >Bioconductor mailing list >Bioconductor@stat.math.ethz.ch >http://www.stat.math.ethz.ch/mailman/listinfo/bioconductor > > >********************************************************************* * >This email and any files transmitted with it are confidential and >intended solely for the use of the individual or entity to whom they >are addressed. If you have received this email in error please notify >the system manager (wibr.mail@ucl.ac.uk). All files are scanned for viruses. >********************************************************************* * > > > [[alternate HTML version deleted]] > >_______________________________________________ >Bioconductor mailing list >Bioconductor@stat.math.ethz.ch >http://www.stat.math.ethz.ch/mailman/listinfo/bioconductor > > >--------------------------------------------------------------------- ------ --- >Notice: This e-mail message, together with any attachments, contains >information of Merck & Co., Inc. (Whitehouse Station, New Jersey, USA) >that may be confidential, proprietary copyrighted and/or legally >privileged, and is intended solely for the use of the individual or entity >named in this message. If you are not the intended recipient, and have >received this message in error, please immediately return this by e-mail >and then delete it. > >===================================================================== ====== === > > >--__--__-- > >Message: 3 >From: Tom McCarty <tmccarty@niaid.nih.gov> >To: "'Liaw, Andy'" <andy_liaw@merck.com>, > "'Stephen Henderson'" > <s.henderson@ucl.ac.uk> >Cc: "'bioconductor@stat.math.ethz.ch'" <bioconductor@stat.math.ethz.ch> >Subject: RE: [BioC] memory requirement >Date: Tue, 26 Nov 2002 08:13:28 -0500 > >So if I have more than 1 GB of RAM, following the previous suggestion is the >best way to improve memory allocation to R? > >Tom McCarty > >-----Original Message----- >From: Liaw, Andy [mailto:andy_liaw@merck.com] >Sent: Tuesday, November 26, 2002 8:01 AM >To: 'Stephen Henderson' >Cc: 'bioconductor@stat.math.ethz.ch' >Subject: RE: [BioC] memory requirement > > >If you've upgraded to R-1.6.1, note that the limit on Windows has been >raised to the smaller of physical RAM and 1GB (used to be 256MB). So unless >you have more than 1GB of RAM, there's no need to do that. > >Andy > >-----Original Message----- >From: Stephen Henderson [mailto:s.henderson@ucl.ac.uk] >Sent: Tuesday, November 26, 2002 6:52 AM >To: 'Laurent Gautier' >Cc: 'bioconductor@stat.math.ethz.ch' >Subject: RE: [BioC] memory requirement > > > >Here is the advice given to me on improving R memory on Win some time ago on >this very list from --Tibor van. This helped a bit. > >If you have a shortcut on your desktop for R (or somewhere else), then if >you right click on the icon and bring up the Properties window, you will see >an entry for >Target with the path to the executable filled in. After that path (and >outside of the quotes) add this command-line argument on my PC, for example >it would look like this: > > > >"C:\Program Files\R\rw1051\bin\Rgui.exe" --max-mem-size=768M > > > >Restart R and you will be in business... > >Stephen Henderson >Wolfson Institute for Biomedical Research >UCL, London > > > > >-----Original Message----- >From: Laurent Gautier [mailto:laurent@cbs.dtu.dk] >Sent: Tuesday, November 26, 2002 12:57 AM >To: Wilkinson, Mark >Cc: 'bioconductor@stat.math.ethz.ch' >Subject: Re: [BioC] memory requirement > >On Mon, Nov 25, 2002 at 03:43:08PM -0600, Wilkinson, Mark wrote: > > Here, here. I'd've asked sooner, but suspected I'd be advised the brute > > force approach of beefing up my machine. > > > > I'm running the affy package, specifically, with U133 chips, but am only > > able to read in 10-15 CEL files at once. Any way around this? I've > > considered using 'merge' to read a few, merge, read a few more, merge, >etc. > > I'm using Windows R, which seems to have poor memory management anyway. >Any > > tips? > > > >How much memory did you have ? > >I do not know much of the behavior with Windows, but with linux I remember >doing >the full monty (raw data -> normalization -> summary expression value) for >about 80 U95A at once on a machine with 2Gb. > >note : the new release of the package spits out something with the size of >the 'AffyBatch' object with the default print method (try: >data(Dilution) >Dilution >) > >Hopin' it helps, > > >L. > > > > > > Mark Wilkinson > > Informatics Analyst > > St. Jude Children's Research Hospital > > Department of Pharmaceutical Sciences > > > > > > -----Original Message----- > > From: Yi-Xiong Zhou [mailto:yzhou@arcturusag.com] > > Sent: Monday, November 25, 2002 3:24 PM > > To: 'bioconductor@stat.math.ethz.ch' > > Subject: [BioC] memory requirement > > > > I am concerning the memory requirement of running bioconductor on Affy > > chips. Could you give me an estimate? > > > > Thanks. > > > > Yi-Xiong > > > > _______________________________________________ > > Bioconductor mailing list > > Bioconductor@stat.math.ethz.ch > > http://www.stat.math.ethz.ch/mailman/listinfo/bioconductor > > > > _______________________________________________ > > Bioconductor mailing list > > Bioconductor@stat.math.ethz.ch > > http://www.stat.math.ethz.ch/mailman/listinfo/bioconductor > >-- >-------------------------------------------------------------- >Laurent Gautier CBS, Building 208, DTU >PhD. Student DK-2800 Lyngby,Denmark >tel: +45 45 25 24 89 http://www.cbs.dtu.dk/laurent > >_______________________________________________ >Bioconductor mailing list >Bioconductor@stat.math.ethz.ch >http://www.stat.math.ethz.ch/mailman/listinfo/bioconductor > > >********************************************************************* * >This email and any files transmitted with it are confidential and >intended solely for the use of the individual or entity to whom they >are addressed. If you have received this email in error please notify >the system manager (wibr.mail@ucl.ac.uk). All files are scanned for viruses. >********************************************************************* * > > > [[alternate HTML version deleted]] > >_______________________________________________ >Bioconductor mailing list >Bioconductor@stat.math.ethz.ch >http://www.stat.math.ethz.ch/mailman/listinfo/bioconductor > > >--------------------------------------------------------------------- ------ - >-- >Notice: This e-mail message, together with any attachments, contains >information of Merck & Co., Inc. (Whitehouse Station, New Jersey, USA) that >may be confidential, proprietary copyrighted and/or legally privileged, and >is intended solely for the use of the individual or entity named in this >message. If you are not the intended recipient, and have received this >message in error, please immediately return this by e-mail and then delete >it. > >===================================================================== ====== = >== > >_______________________________________________ >Bioconductor mailing list >Bioconductor@stat.math.ethz.ch >http://www.stat.math.ethz.ch/mailman/listinfo/bioconductor > > [[alternate HTML version deleted]] > > >--__--__-- > >Message: 4 >Date: Tue, 26 Nov 2002 08:26:02 -0500 >From: "Liaw, Andy" <andy_liaw@merck.com> >Subject: RE: [BioC] memory requirement >To: "'Tom McCarty'" <tmccarty@niaid.nih.gov>, > "Liaw, Andy" <andy_liaw@merck.com>, > "'Stephen Henderson'" <s.henderson@ucl.ac.uk> >cc: "'bioconductor@stat.math.ethz.ch'" <bioconductor@stat.math.ethz.ch> > >Depends on how much memory you want R to be able to use. If you want R to >use more than 1GB (and you have more than 1GB), then yes, you should add the >--max-mem-size argument. > >This is an "improvement" only in the sense that you're giving R more to play >with. > >Andy >-----Original Message----- >From: Tom McCarty [mailto:TMCCARTY@niaid.nih.gov] >Sent: Tuesday, November 26, 2002 8:13 AM >To: 'Liaw, Andy'; 'Stephen Henderson' >Cc: 'bioconductor@stat.math.ethz.ch' >Subject: RE: [BioC] memory requirement > > > >So if I have more than 1 GB of RAM, following the previous suggestion is the >best way to improve memory allocation to R? > >Tom McCarty > >-----Original Message----- >From: Liaw, Andy [ mailto:andy_liaw@merck.com <mailto:andy_liaw@merck.com> ] > >Sent: Tuesday, November 26, 2002 8:01 AM >To: 'Stephen Henderson' >Cc: 'bioconductor@stat.math.ethz.ch' >Subject: RE: [BioC] memory requirement > > >If you've upgraded to R-1.6.1, note that the limit on Windows has been >raised to the smaller of physical RAM and 1GB (used to be 256MB). So unless > >you have more than 1GB of RAM, there's no need to do that. > >Andy > >-----Original Message----- >From: Stephen Henderson [ mailto:s.henderson@ucl.ac.uk ><mailto:s.henderson@ucl.ac.uk> ] >Sent: Tuesday, November 26, 2002 6:52 AM >To: 'Laurent Gautier' >Cc: 'bioconductor@stat.math.ethz.ch' >Subject: RE: [BioC] memory requirement > > > >Here is the advice given to me on improving R memory on Win some time ago on > >this very list from --Tibor van. This helped a bit. > >If you have a shortcut on your desktop for R (or somewhere else), then if >you right click on the icon and bring up the Properties window, you will see > >an entry for >Target with the path to the executable filled in. After that path (and >outside of the quotes) add this command-line argument on my PC, for example >it would look like this: > >"C:\Program Files\R\rw1051\bin\Rgui.exe" --max-mem-size=768M > >Restart R and you will be in business... > >Stephen Henderson >Wolfson Institute for Biomedical Research >UCL, London > > > > >-----Original Message----- >From: Laurent Gautier [ mailto:laurent@cbs.dtu.dk ><mailto:laurent@cbs.dtu.dk> ] >Sent: Tuesday, November 26, 2002 12:57 AM >To: Wilkinson, Mark >Cc: 'bioconductor@stat.math.ethz.ch' >Subject: Re: [BioC] memory requirement > >On Mon, Nov 25, 2002 at 03:43:08PM -0600, Wilkinson, Mark wrote: > > Here, here. I'd've asked sooner, but suspected I'd be advised the brute > > force approach of beefing up my machine. > > > > I'm running the affy package, specifically, with U133 chips, but am only > > able to read in 10-15 CEL files at once. Any way around this? I've > > considered using 'merge' to read a few, merge, read a few more, merge, >etc. > > I'm using Windows R, which seems to have poor memory management anyway. >Any > > tips? > > > >How much memory did you have ? > >I do not know much of the behavior with Windows, but with linux I remember >doing >the full monty (raw data -> normalization -> summary expression value) for >about 80 U95A at once on a machine with 2Gb. > >note : the new release of the package spits out something with the size of >the 'AffyBatch' object with the default print method (try: >data(Dilution) >Dilution >) > >Hopin' it helps, > > >L. > > > > > > Mark Wilkinson > > Informatics Analyst > > St. Jude Children's Research Hospital > > Department of Pharmaceutical Sciences > > > > > > -----Original Message----- > > From: Yi-Xiong Zhou [ mailto:yzhou@arcturusag.com ><mailto:yzhou@arcturusag.com> ] > > Sent: Monday, November 25, 2002 3:24 PM > > To: 'bioconductor@stat.math.ethz.ch' > > Subject: [BioC] memory requirement > > > > I am concerning the memory requirement of running bioconductor on Affy > > chips. Could you give me an estimate? > > > > Thanks. > > > > Yi-Xiong > > > > _______________________________________________ > > Bioconductor mailing list > > Bioconductor@stat.math.ethz.ch > > http://www.stat.math.ethz.ch/mailman/listinfo/bioconductor ><http: www.stat.math.ethz.ch="" mailman="" listinfo="" bioconductor=""> > > > > _______________________________________________ > > Bioconductor mailing list > > Bioconductor@stat.math.ethz.ch > > http://www.stat.math.ethz.ch/mailman/listinfo/bioconductor ><http: www.stat.math.ethz.ch="" mailman="" listinfo="" bioconductor=""> > >-- >-------------------------------------------------------------- >Laurent Gautier CBS, Building 208, DTU >PhD. Student DK-2800 Lyngby,Denmark >tel: +45 45 25 24 89 http://www.cbs.dtu.dk/laurent ><http: www.cbs.dtu.dk="" laurent=""> > >_______________________________________________ >Bioconductor mailing list >Bioconductor@stat.math.ethz.ch >http://www.stat.math.ethz.ch/mailman/listinfo/bioconductor ><http: www.stat.math.ethz.ch="" mailman="" listinfo="" bioconductor=""> > > >********************************************************************* * >This email and any files transmitted with it are confidential and >intended solely for the use of the individual or entity to whom they >are addressed. If you have received this email in error please notify >the system manager (wibr.mail@ucl.ac.uk). All files are scanned for viruses. > >********************************************************************* * > > > [[alternate HTML version deleted]] > >_______________________________________________ >Bioconductor mailing list >Bioconductor@stat.math.ethz.ch >http://www.stat.math.ethz.ch/mailman/listinfo/bioconductor ><http: www.stat.math.ethz.ch="" mailman="" listinfo="" bioconductor=""> > > >--------------------------------------------------------------------- ------ - >-- >Notice: This e-mail message, together with any attachments, contains >information of Merck & Co., Inc. (Whitehouse Station, New Jersey, USA) that >may be confidential, proprietary copyrighted and/or legally privileged, and >is intended solely for the use of the individual or entity named in this >message. If you are not the intended recipient, and have received this >message in error, please immediately return this by e-mail and then delete >it. > >===================================================================== ====== = >== > >_______________________________________________ >Bioconductor mailing list >Bioconductor@stat.math.ethz.ch >http://www.stat.math.ethz.ch/mailman/listinfo/bioconductor ><http: www.stat.math.ethz.ch="" mailman="" listinfo="" bioconductor=""> > > >--------------------------------------------------------------------- ------ --- >Notice: This e-mail message, together with any attachments, contains >information of Merck & Co., Inc. (Whitehouse Station, New Jersey, USA) >that may be confidential, proprietary copyrighted and/or legally >privileged, and is intended solely for the use of the individual or entity >named on this message. If you are not the intended recipient, and have >received this message in error, please immediately return this by e-mail >and then delete it. > >===================================================================== ====== === > > [[alternate HTML version deleted]] > > >--__--__-- > >Message: 5 >Reply-To: <phgrosjean@sciviews.org> >From: "Philippe Grosjean" <phgrosjean@sciviews.org> >To: "Thomas Friedrichsmeier" <thomas.friedrichsmeier@ruhr-uni- bochum.de="">, > <dmurdoch@pair.com>, <rossini@blindglobe.net>, > <bioconductor@stat.math.ethz.ch> >Cc: <r-sig-gui@stat.math.ethz.ch> >Date: Tue, 26 Nov 2002 11:13:25 +0100 >Subject: [BioC] XML specifs for R GUIs dialog boxes > >Since this becomes a fairly large thread that goes in many different (and >interesting) directions, I propose to start a new one with a more explicit >title. > >I will try to summarize major points discussed so far. > >- Thomas Friedrichsmeier [Thomas.Friedrichsmeier@ruhr-uni-bochum.de] after >"having a short conversation with the obveRsive-people" introduced the idea >of a sort of "plugin" with potentials to contruct dialog boxes at run-time. >He proposes to use a XML-file for specification of the dialog box (ex: >t-test dialog box, http://rkward.sf.net/pics.php). > >- We all agree (I think) that a common semantic for the description of these >dialog boxes accross projects (whatever the language, the graphical toolkit >and the platform used) should be fine. So, we are on the way to define it, >and I'll dedicate a page to it at http://www.r-project.org/GUI, as soon as >we have something consistent (thus, discussed enough in R-SIG-GUI). Also, >this implies it should not be describing features at a too low level, as >pointed out by Thomas, or we take the risk to get something that would be >less portable from one language/graph toolkit to the other. > >- Peter Dalgaard [p.dalgaard@biostat.ku.dk] suggests to contact people in >Bioconductor that may be working on a similar thing (I send them a copy of >this mail). > >- Philippe Grosjean [phgrosjean@sciviews.org] proposes to include >context-sensitive features, allowing for instance to call such dialog boxes >directly from an object explorer, or whatever context menu. > >- While Philippe and Thomas think that a majority (but not all) dialog boxes >could be constructed this way (they probably bear in mind a way to quickly >construct simple to rather complex dialog boxes, but not an exhaustive >approach), Duncan Murdoch [murdoch@stats.uwo.ca] considers that it should be >possible to make it more exhaustive ("I don't see any problem with thinking >about all of them as being made of XML"). > >- Although the specifications could target a run-time dynamic construction >of the dialog box, it could also be used to define dialog boxes at design >time, at least if we follows the Delphi scheme, as pointed out by Duncan >(?). A.J. Rossini [rossini@blindglobe.net] says that it is "just a matter of >translation, for the most part". Probably, this should be more a question of >implementation in the various different languages and should not be >discussed too deeply in the early stages. > >- About layering, Thomas proposes something simple, like a grid with perhaps >tabbed dialog boxes (note: this is roughly the layering potentials of Splus >custom dialog boxes), while Duncan whould like to have something more >powerful. > >- The list of widgets to support is not fixed yet too. Duncan proposes to >focus on low level widgets first (label, text, button, etc...). Thomas >proposes a list that mixes low level and high level widgets (I think we >should make the distinction and propose two separate lists). At this point, >we should also discuss with the Bioconductor team that has already done some >work on creating various interesting high level widgets in tcltk like >objects lists for instance. Here is the widgets list propsed by Thomas: >"+ some sort of widget that lists available objects (...) > + a widget that can hold a (single) object (the "varslot"), and will >probably make some specifications about what types of objects it can hold >(like the attribute type="numeric" in the example) > + maybe a separate widget that can hold one or more objects, but basically >acts the same (might instead simply use a more generalized "varslot") > + maybe a separate widget that can hold a single constant value (number or >string) > + a widget that can be used to select interactions between variables >selected in a "varslot" > + radio-buttons > + checkboxes > + a widget representing a text/label not directly attached to any of the >other widgets > + a widget represent a line or other separation used for layouting >purposes" > >Duncan adds: > + a tree view widget > + a free text widget > >I'll immediately add: > + various classical dialog boxes like file open/close for instance > + a widget that displays a (multi)-selectable list of columns in a >particular data frame, or elements in a list. > + a graph widget (yes a whole R graph device) > >And perhaps: > + a menuing system (with perhaps context menus attached to other widgets) > + sliders to select values in a range > >- There are some open aspects, like the possibility to define new widgets, >or the way results are submitted to R (simple seach/replace mechanism, data >frame or other code describing the whole dialog box and processed by R >code,...). > >- Since someone must initiate a document for the specifs, and Thomas seems >to have something almost ready, I proposed him to make a draft. At this >stage, modifications in a document will be more constructive than loosy >discussions in R-SIG-GUI. > >Best, > >Philippe > > >P.S.: for those of you that receive this mail and are interested, but not >listed yet on R-SIG-GUI, you can register at: > > http://www.stat.math.ethz.ch/mailman/listinfo/r-sig-gui > >Special Interest Group on Development of GUI(s) for R. > >Topic: Discussion of Strategies and approaches; particularly platform >independent ones. See http://www.R-project.org/GUI/ for more. > >...........]<(({?<...............<............................. .. >( ( ( ( ( > ) ) ) ) ) Philippe Grosjean >( ( ( ( ( > ) ) ) ) ) IFREMER Nantes - DEL/AO >( ( ( ( ( rue de l'Ile d'Yeu, BP 21105, 44311 Nantes Cedex 3 > ) ) ) ) ) tel: (33) 02.40.37.42.29, fax: (33) 02.40.37.42.41 >( ( ( ( ( > ) ) ) ) ) SciViews project coordinator (http://www.sciviews.org) >( ( ( ( ( e-mail: phgrosjean@sciviews.org > ) ) ) ) ) >( ( ( ( ( "I'm 100% confident that p is between 0 and 1" > ) ) ) ) ) L. Gonick & W. Smith (1993) >..................................................................... .. > > > >--__--__-- > >_______________________________________________ >Bioconductor mailing list >Bioconductor@stat.math.ethz.ch >http://www.stat.math.ethz.ch/mailman/listinfo/bioconductor > > >End of Bioconductor Digest Eric Blalock, PhD Dept Pharmacology, UKMC 859 323-8033 _______________________________________________ Bioconductor mailing list Bioconductor@stat.math.ethz.ch http://www.stat.math.ethz.ch/mailman/listinfo/bioconductor ---------------------------------------------------------------------- -------- Notice: This e-mail message, together with any attachments, contains information of Merck & Co., Inc. (Whitehouse Station, New Jersey, USA) that may be confidential, proprietary copyrighted and/or legally privileged, and is intended solely for the use of the individual or entity named on this message. If you are not the intended recipient, and have received this message in error, please immediately return this by e-mail and then delete it. ====================================================================== ========
ADD COMMENT

Login before adding your answer.

Traffic: 633 users visited in the last hour
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6