Re: [MacPerl] Read Speed

["Jon S. Jaques" <jjaques@grovehillsys.com>]

At 01:21 PM 12/13/97 -0500, Strider wrote:
>the script. Any idea what's up? (the file's 4.7 mb)

IMHO, I think that your variable, $data, is supposed to just be a 'buffer'
space... Say, for about from 4 to 64k, not mb, depending on what works best
for you. I wouldn't imagine that you'd be able to stuff that much into the
buffer space without having problems.

>#!perl
>print "Reading data... ";
>for $i (0 .. $#ARGV) {
>	open (IN, "$ARGV[$i]") || next;
>	read (IN,$data,5120000); # reads up to 5mb
>	close IN;
>}
>print "done.\n";

Try this:

#!perl
$start = time;
open(IN,  "$ARGV[$i]");
open(OUT, ">nodup.tab");
	while(read(IN, $buf, 32768)) {
		print OUT $buf;
	}
close IN;
close OUT;
$finish = time; 
$elapsed = ($finish - $start);
 print "File duplication took $elapsed\n seconds to complete\.\n";

or this:

#!perl
$start = time;
open(IN,  "$ARGV[$i]");
open(OUT, ">nodup.tab");
@LINES = <IN>;
foreach (@LINES) {

>	$line = pop( @data );
>	if ($line eq "") { print "skipped.\n"; next; }
>	$data =~ s/$line//;
>	@data = split/\n/,$data;
>	chomp ($line = $line);
>	print OUT "$line\n";
>	print "$line\n";

}

close OUT;
close IN;

Or some variation thereof... The second version is more likely better if
you need to actually parse each file for it's information, but the first
would probably be faster for a raw copy procedure. (Use binmode() to copy
binaries). 

>open (OUT, ">nodup.tab");
>@data = split/\n/,$data;
>
>until ($#data < 0) {
>	$line = pop( @data );
>	if ($line eq "") { print "skipped.\n"; next; }
>	$data =~ s/$line//;
>	@data = split/\n/,$data;
>	chomp ($line = $line);
>	print OUT "$line\n";
>	print "$line\n";
>}
>
>close OUT;
>close CONSOLE;

I can imagine what you're after here, although I can't quite picture the
data that you're working on! You are close, though!

>My system's memory is at its limit in the dialog. That couldn't cause this,
>could it? (It just grabs more memory when it needs it, right?)

Use that buffer space right, and you should be all set. I'll give that bit
a try on my 540 PowerBook (12mb ram total) after it finishes taking it's
nap! :) (That explains my path delimiters, but either of the above should
work on most any computer!) I created a 5mb text file on my server via
Win32 Perl from the contents of my web-tree ( /\.html/i ) and will try
doing some slicing and dicing for memory tests.

I must profess that I don't know how well parsing the data would work on a
'read'... I think that the buffer sizes would create a problem with line
breaks. In other words, who's to say how many line breaks are in 32k?
Granted, you could split each buffer iteration, but that doesn't sound
feasible. If each line is a record, then reading the files into an array
(@LINES = <IN>) gives you the line breaks as a 'freebee' -- each item in
the array is a line. If you need to do a multi-line search, you can either
do a 
join(' ', @LINES), or enable that other "thing" that I can't remember
because I don't use much, but will "automatically" allow multi-line
searches.--J

Jon S. Jaques
--Jon S. Jaques
--The Grove Hill Pages
--http://www.grovehillsys.com
--jjaques@grovehillsys.com
--jjaques@illuminata.com
--jjaques@almus.com

***** Want to unsubscribe from this list?
***** Send mail with body "unsubscribe" to mac-perl-request@iis.ee.ethz.ch