[MacPerl] Memory problems in OO context

[Matthew Langford <langfml [at] eng [dot] auburn [dot] edu>]

I wrote a little program to grab some web pages, parse them, and save the
contents in a big hash, which I save to a file using Storable.

To get the whole deal I had to use more than 55 MB of memory.  (I think
this is what I came back to; for a while I had to crank out some virtual
memory and set it higher.)  The hash, after I finally got it, takes up 4.6
MB. If I read in the file, I don't have too many memory problems.  
Clearly, there are serious memory leaks here in the "getting" stage.

Some say garbage collectors end memory leak worries.  They haven't in this
case.  

How do I help MacPerl reclaim the memory used for each file?

I'm subclassing HTML::Filter, so my functions are in the form of callbacks
for specific tags.  When I know I've hit data I want to save in the HTML
file, I set a switch, and then concatenate all characters into a temp
variable until I see an end tag.  This temp variable is copied into a
hashref off of the $self variable.

At the end of the file, I use that hashref:  $p is an instance of
FileGrabber, which is a subclass of HTML::Filter.  Oh, rats, I'll post the
whole function that grabs one HTML file...


sub GetOneFile
{
    my( $uri ) = @_;
    
    my $p = OneFileGrabber->new();
  
      # Form my URI
    my $theFile = URI->new_abs( $uri, $Base );

      # Create a user agent object
    my $ua = new LWP::UserAgent;
    $ua->agent("$MyName/$VERSION " . $ua->agent);

      # Create a request
    my $req = HTTP::Request->new('GET' => "$theFile");

      # Pass request to the user agent and get a response back
    my $res = $ua->request($req, sub { $p->parse($_[0]) } );

      # Check the outcome of the response
    if ($res->is_success) 
    {
          # try to clear up some memory
        undef $theFile; undef $ua; undef $req; undef $res;
        return \%{$p->{metaData}};
    } 
    else 
    {
        print "Bad luck this time\n";
        die $res->error_as_HTML;
    }
}



If anyone has gotten this far without eyes glazed over and drool
escaping...how can I plug the memory leak?  My guess is that by using the
hashref, I am preventing MacPerl from reclaiming $p, somehow.

Short of that, has anyone else used HTML::Filter-like objects on many
files without memory problems?

In Perl code which returns a hashref, is there something which must
be done to avoid the ref jamming up the garbage collection?

As I say, the metaData itself only totals ~5 MB of memory when read back
in, but the process of getting it (principally this function) leaks more
than 55 MB.

I think this might be a general Perl question, but I have a lingering
suspicion of MacPerl's memory handling.  I don't really know where to
point the finger.


--
MattLangford 


# ===== Want to unsubscribe from this list?
# ===== Send mail with body "unsubscribe" to macperl-request@macperl.org