Re: [MacPerl] Searching a VERY large text file

[pudge@pobox.com (Chris Nandor)]

At 12.06 98.02.04, Shyam Hegde wrote:
>I have moved my project on and am finding MacPerl a very powerful tool -
>Thanks Matthias!!
>
>I have two really large text files (6 & 9meg) each containing over 170,000
>lines of text (they contain semantic information about every word/phrase in
>the English language).
>
>Is there a fast way in Perl/MacPerl to search these files for a particular
>string??  The Camel book seemed to suggest using Hashes, but doesn't go
>into detail and using the standard 'Open' and <shove into array> requires
>MacPerl to have 20Mbytes of RAM assigned to it and me to wait a very long
>time.
>
>If anyone has had a similar experience or knows the answer off the top of
>their heads then I would really appreciate hearing about it.

It depends on what you need to do with the data.  If you don't need to keep
all the data around, then you can just look at one line at a time rather
than a whole array of lines at once.

open(F, 'file') || die $!;
while(<F>) {
  print if /match this text/;
}
close(F);

If you need to search it often, and you know the exact string you want, you
can make a tied hash with DB_File, but that depends on what your data looks
like.

--
Chris Nandor          mailto:pudge@pobox.com         http://pudge.net/
%PGPKey=('B76E72AD',[1024,'0824 090B CE73 CA10  1FF7 7F13 8180 B6B6'])
#==       New Book: MacPerl:  Programming for the Rest of Us       ==#
#==    Publishing Date: Early 1998. http://www.ptf.com/macperl/    ==#



***** Want to unsubscribe from this list?
***** Send mail with body "unsubscribe" to mac-perl-request@iis.ee.ethz.ch