Re: [MacPerl] Out of memory! error...doing standard file I/O

["Patrick D. F. Ion" <ion@ams.org>]

At 2:58 PM -0500 5/3/00, Brock Gunter-Smith wrote:
>I'm just trying to parse a log file and MacPerl is giving me "Out of
>memory!" errors. I have a 12MB log file that I want to extract entries that
...


>(BTW - the script works fine if I only give it a 1MB log file, but after
>some larger file size limit it just can't seem to handle it properly...)

Don't we have the same problem here as I had not so long ago: a file
probably coming from a Unix environment with non-Mac end-of-lines?
I was advised by Mathhias Neeracher
<http://bumppo.net/lists/macperl/2000/04/msg00151.html>
who said
<<<
There is [no such limit on file sizes}, *provided* you pick the 
correct line terminator. Your problem sounds a lot like your file is 
LF (\012) terminated, while MacPerl expects a CR (\015) terminated 
file by default. It will therefore try to read the entire file into 
RAM, and fail.
<<<
Todd Richmond (2000/04/22 off list), who said
<<<
I've processed 500 MB files using MacPerl. Your problem is probably 
that your file has DOS or Unix line endings. Get rid of those and you 
should be fine. I use a program called DropStrip 
(http://world.std.com/~mattm/dropstrip.html).
<<<
and Jay Bedsole (2000/04/24 off list) who said
<<<
Just wanted to reiterate what Matthias said, your problem
is most likely line/record termination. As a testimonial
to the "no file size limit", I've processed quite a few
very large (more than 100MB) files in MacPerl with no
issue.
<<<
and offered a little script he wrote a long time ago to do the conversion.

This was the advice I needed: learn to use
$/ = "\012" ;
explicitly to set the record separator for files from Unix
to be processed, or adjust it otherwise appropriately for other
sources [Mac \015  (CR = ^M), DOS \015\012 (CR-LF = ^M^J)]
or to something suitable to the records you are treating, which
I had done before.  So now I just always set $/ to remind myself
what I'm doing.  DropStrip too did the trick of fixing the files
I needed to treat, if you like to use a preprocessing filter.

		Patrick

# ===== Want to unsubscribe from this list?
# ===== Send mail with body "unsubscribe" to macperl-request@macperl.org