Re: [MacPerl-WebCGI] Out of Memory! Am I looping it "too far"?Handling readdir with some subdirs.

[Thomas Wegner <toms_email@gmx.de>]

>Hi
>I have this code where I try to read a folder,
>and then exclude some files/folders. So far so good this part works fine.
>But then I want to take the rest of the files and check them if they are
>folders or files
>if they are folders then I need to read them as subfolders an re-read
>the for files and subsubfolders,
>if they are files I need to push them into the array filelist containing
>all the filenames in the directory and subdirectories.
>When I run this script I just get a OUT OF MEMORY.
>Where am I looping it?
>I feel Like I'm spamming all of you with a lot of questions.
>I will return soon with a summery of all of yours replies on my earlier
>question about :
>Putting hash values in an array and checking that with an foreach is not
>working .....
>
>Thanks a lot for all your help with Ideas and pointers.
>I'm still just in the beginning phase of my "perl"ing. (A.K.A Major Newbie)
>
>Yours sincerely
>Jimmy Lantz
>Sweden
>
>
>#!perl
>#########
>#Macperl!
>########
>$infolder = 'Server G4:DATABAS:';
># Exclude the following files/folders.
>@exfiles = ('data');
>#####################
>
>opendir(DIR, "$infolder") or die "cannot open $infolder";
>@files = readdir(DIR);
>closedir(DIR);
>
>&do_exclude;
>
>foreach $file (@files) {
>&do_readdir("$file");
>}
>
>print @filelist;
>
>exit;
>
>###########################
>sub do_readdir
>
>{
>my($specfile) = @_ ;
>
>if ($specfile !~ m/$infolder/) #otherwise add the maindir
>{
>my($adress) = $infolder . $specfile;
>}
>else
>{
>my($adress) = $specfile;
>}
>
>if (-d "$adress")
>{
>&do_readsubdir("$adress");
> }
>else
> {
>push(@filelist, "$adress");
> }
>
>}
>
>######################
>sub do_readsubdir
>
>{
>my($specfolder) = @_ ;
>
>opendir(DIR, "$specfolder") or die "cannot open $specfolder";
>my(@subfiles) = readdir(DIR);
>closedir(DIR);
>
>foreach $subfile (@subfiles) {
>&do_readdir("$subfile");
>}
>
>}
>
>####################
>
>sub do_exclude #This ones works fine from perlfaq4!
>{
>@union = @intersection = @difference = ();
>%count = ();
>foreach $element (@files, @exfiles) { $count{$element}++ }
>foreach $element (keys %count) {
>  push @union, $element;
>  push @{ $count{$element} > 1 ? \@intersection : \@difference }, $element;
>  }
>@files = @difference; # assign the difference to @files
>}
>

Dear Jimmy,

I can not reproduce an 'Out of memory' error. But have you ever tried to
switch on compiler warnings with

#!perl -w

in the first line of your script or in the 'Script' menu? If so, you would
have seen that you have created a wonderful infinite loop ( ;-) ) with the
following warnings:

File 'your_script.pl'; Line xx
# Use of uninitialized value.
File 'your_script.pl'; Line yy
# Use of uninitialized value.

It turned out, that there is a problem in the sub do_readdir. You have
"double-declared" the "my"-variable (lexically local/private variable)
$adress, in the if and the else part of the if-statement. The scope of a
"my"-variable is always its enclosing block, so the variable $adress in the
'if (-d $adress)' test is a complete different variable than the two
my($adress) variables in the if-statement (see 'Private Variables via my()'
in perlsub.pod). Let's look at a small example:

____________
#!perl -w

$nick = 0;
$name = 'Fred'; # global $name No. 1

if ($nick) {            # -|
 my ($name) = 'Freddy'; #  | scope of $name No. 2 from { to } (enclosing block)
}                       # -|
else {                           # -|
 my ($name) = 'Fred Flintstone'; #  | scope of $name No. 3 from { to }
}                                # -|

print "$name\n";  # prints 'Fred' !!!, scope of global $name is whole script
____________

If we had not initialized $name No. 1, we would get the "# Use of
uninitialized value."-warning at runtime, and that is what happened to
$adress. Please note, that the example is syntactically correct (at
compile-time), but it is not what you meant. So what you really meant is:

###########################
sub do_readdir {

  my($specfile) = @_ ;
  my($adress);           # declare it only once !

  if ($specfile !~ m/$infolder/){ # otherwise add the maindir
     $adress = $infolder . $specfile;
  } else {
     $adress = $specfile;
  }
  if (-d "$adress") {
  .........

}#sub


With these changes, your script is running, but it still needs some finetuning.
The paths aren't assembled correct and, in case there is no match with your
exclude file 'data', 'data' will show up in the resulting filelist. I also
noticed, that you are sometimes quoting your "$vars". Please see 'What's
wrong with always quoting "$vars"?' in the perlfaq4.pod.

You are using, what I would call "indirect-recursion", that is, readdir
calls readsubdir, which calls readdir, which calls readsubdir and so on. So
why don't you use "direct" recursion, that is, readdir calls itself. It's
cool and far more elegant :-). What about :

_______________________________________________________________________________

#!perl -w
#########
#Macperl!
#########

#
# main
#

$infolder = 'Server G4:DATABAS'; # remove ':' at the end !
# Exclude the following files/folders.
@exfiles = ('data');

# ---
opendir(DIR, $infolder) or die "cannot open $infolder";
@files = readdir(DIR);
    # only file and folder names show up, no full paths
    # adding the maindir is allways necessary
closedir(DIR);

&do_exclude;

foreach $file (@files) {
   $specfile = $infolder . ':' . $file;
   if (-d $specfile) { # what about Aliases ???
      &do_readdir($specfile);
   } else {
      push(@filelist, $specfile);
   }#if
}#for

# ---

$i = 1;
foreach $the_file (@filelist) {
   print "$i. $the_file\n";
   $i++;
}
print "\n-----------\n";
exit;

###########################
sub do_readdir { # recursive execution
  my($adress) = @_;
  local (*FOLDER); # use local for filehandles

   # itīs allways safer to make the filehandle local
   # see perlfaq5.pod, Files and FileHandles:
   # How can I make a filehandle local to a subroutine?

   my(@subfiles, $file, $specfile);

   opendir(FOLDER, $adress) or die "cannot open $adress";
   @subfiles = readdir(FOLDER);
   closedir(FOLDER);
#
# You may want to include '&do_exclude;' here ???; if so, you have
# to call it with a parameter like '&do_exclude(\@subfiles);' and
# don't use a global for the directory file list.
# Then your main-program can be made much shorter: simply call
# &do_readdir($infolder) after your exclude file declaration
# and remove the rest up to $i = 1;
#
  foreach $file (@subfiles) {
     $specfile = $adress . ':' . $file;
     if (-d $specfile) { # what about Aliases ???
        &do_readdir($specfile); # RECURSION
     } else {
        push(@filelist, $specfile);
     }#if
  }#for
}#sub


####################

sub do_exclude #This ones works fine from perlfaq4!
{
@union = @intersection = @difference = ();
%count = ();
foreach $element (@files, @exfiles) { $count{$element}++ }
foreach $element (keys %count) {
  push @union, $element;
  push @{ $count{$element} > 1 ? \@intersection : \@difference }, $element;
  }
@files = @difference; # assign the difference to @files
}
_______________________________________________________________________________

One problem still remains: If  there is no match with your exclude file
'data', 'data' will show up in the resulting filelist. But this is left as
an exercise to the reader ;-).

Best regards

--Thomas




==== Want to unsubscribe from this list?
==== Send mail with body "unsubscribe" to macperl-webcgi-request@macperl.org