[MacPerl] Paul: LWP::Simple update

[mark@cheers.jsc.nasa.gov (Mark Manning/Muniz Eng.)]

Well, I got some new information on the LWP::Simple thing I
posted yesterday.  It no longer looks like an LWP::Simple
problem.  This morning, after running the program
overnight, the last three lines on the output screen were
"Out of memory!" lines.  This is quit odd as I UNDEF'd the
downloaded document each time I was through with it.  So
the only thing which continued to accumulate memory was the
MacPerl output window.

821 files were processed last night.  Versus the 879 files
of the night before.

The program consists of two arrays which are being used to
keep track of where we are going or have been:

	use LWP::Simple;

	@curLoc = ();
	@oldLoc = ();

I read in an HTML file and split it on the double quotes.

	$fileCount = -1;
	$curLoc[++$#curLoc] = "http://gulfgreyhound.com";
	while( $#curLoc > -1 ){
		$fileCount++;
		$oldLoc[++$#oldLoc] = $curLoc[$#curLoc];

		$theInfo = get( $curLoc[$#curLoc] );
		$#curLoc--;

		$theInfo =~ s/\012//g;
		open( THEFILE, ">file.$fileCount" ) || die $!;
		print THEFILE $theInfo;
		close( THEFILE );

		@theDoc = split( /"/, $theInfo );
		for( $i=0; $i<=$#theDoc; $i++ ){
			if( $theDoc[$i] =~ /http:/i ){
				$curLoc[++$#curLoc] = $theDoc[$i];
				}
			}

		undef @theDoc;

Then I check to see if there are any duplicates.

		for( $i=0; $i<=$#curLoc; $i++ ){
			if( (($curLoc[$i] !~ /.html/i) &&
				($curLoc[$i] !~ /.txt/i)) ){
				splice( @curLoc, $i, 1 );
				$i--;
				last;
				}
				else {
					for( $j=0; $j<=$#oldLoc; $j++ ){
						if( $curLoc[$i] eq $oldLoc[$j] ){
							splice( @curLoc, $i, 1 );
							$i--;
							last;
							}
						}
					}
				}
			}

	exit( 0 );

And that's it.  (This is hand typed by me.  The original
program has comments embedded in it but this is almost
exactly what the program looks like.)

The @oldLoc only contains one entry for each of the 800
some odd files.  @curLoc never gets above 150 entries
during all of the time it is running.  "About this
computer" always shows MacPerl to only be using 3/4 of the
available memory which has been allocated to MacPerl.

So I'm a bit stumped as to why it says "Out of memory!" or
why it suddenly starts to just dump things to the screen.
Any ideas?  (I know it can't be LWP::Simple any more but
I'm keeping the original title for right now.)

I've upped the memory to 12000K to see if this fixes the
problem.  I started the program before I left for work
today.  So I can let everyone know what's happening
tomorrow.  (ie: if it works or not)

***** Want to unsubscribe from this list?
***** Send mail with body "unsubscribe" to mac-perl-request@iis.ee.ethz.ch