[Date Prev][Date Next][Thread Prev][Thread Next] [Search] [Date Index] [Thread Index]

[MacPerl] Out Of Memory processing text file?



'lo all,

  I'm running into an odd problem processing a text file.  My script works
fine on smaller files.  But when I try to process the whole 28.8MB text
file at once, it does fine for a while (roughly halfway through, somewhere
in New York) and then I get the message "Out of memory!".

  I know what you're thinking (if you're familiar with MacPerl).  But it's
not line endings.  The text files are in Mac format (confirmed by BBEdit
and DragStrip).  Just for fun I put the line:

$/ = "\012";

at the top of my script.  When I had that line, it ran out of memory
without ANY output (as I would expect).

  I managed to get the script down to a fairly small stub that exhibits
the problem.  I am including it below.

  The text files that I'm working with are avalable at
<http://www.nist.gov/itl/fipspubs/55new/nav-top-fr.htm> in several
formats.  Basically they list every city-like place name in the United
States.

  My Mac is a G4 with 128 MB of RAM.  The text files are 620K for
WIFIPS55.TXT (which works fine) and 28.8MB for FIP55ALL.TXT (which gives
me the out of memory error).

  I suspect that I'm just missing something really obvious here and I'll
smack myself in the head when someone points it out.  But for the life of
me, I can't figure it out....

Thanks!

#!perl -w
#
# This script processes the FIPS 55-DC3 files found at
# http://www.nist.gov/itl/fipspubs/55new/nav-top-fr.htm
# and extracts useful information.


# This is the name of the FIPS 55-DC3 datafile.  Change if needed.
$datafile = "FIP55ALL.TXT";
#$datafile = "WIFIPS55.TXT";

open DATAFILE, $datafile or die "Can't open $datafile: $!\n";

while (<DATAFILE>)  {
  if ( (/Greenfield/) )  {
    print;
  }
}

close DATAFILE;

__END__

# ===== Want to unsubscribe from this list?
# ===== Send mail with body "unsubscribe" to macperl-request@macperl.org