Re: [MacPerl-WebCGI] CGI/WebSTAR problem

[Bruce Van Allen <bva@cruzio.com>]

At 3:13 AM -0800 1/21/00, Thorsten Dittmar wrote:
[snip]

>5 days of hard testing showed to following results:
>
>1) Running the scripts in MacPerl works fine (using "Run Script webshop.cgi"
>and passing parameters from standard input). That's true for both the source
>code version and the "real" cgi-program compiled with MacPerl.
>
>2) Whenever the script aborts output, it always does so at the same point
>
>3) When I don't touch the memory assignment for the cgi file (by default 128
>KB), I get a "document contained to data" message. When I increase the
>memory assigned, I get this incomplete HTML code. However, assigning more
>and more memory has no effect on how much output I get.

[snip]

>I'm using the following code to read the database files:
>
>...
>open(DBS, "+<$databaseFile");
>@dbs = <DBS>;
>close(DBS);
>
>foreach $line (@dbs)
>{
>  ... output html ...
>}
>...

Here are a few ideas:

When you use the approach above of reading in the whole file at once,
you'll stress all the memory limits of the entire execution path, which
would include MacPerl's application memory, the script's (although I've
never actually messed with the memory partition for a cgi/acgi directly..),
total available machine RAM/virtual memory, etc. Since you say your script
works in some configurations and not in others, memory limits are one
likely suspect.

The alternative to your approach above is to read in one line at a time
from the file, process it, output it to a file or to the browser, then get
the next line from the file and repeat.

Above, you have

  open(DBS, "+<$databaseFile");
  @dbs = <DBS>;
  close(DBS);

The key to your mode of input here is the '@' symbol in the statement
  @dbs = <DBS>;

By using an array variable, you're forcing <DBS> to yield all lines of the
file, each line an element of the array @dbs. This is known as 'slurping
the whole file.' I use this approach in many applications, including
flat-file database access (like yours), template handling (Perl now writes
almost all of my HTML), log reading, etc.

But you'll encounter memory constraints if you're slurping large files.
What's large? Well, you seem to have found what's 'large' in your setup...

So, change your code (I hope only slightly), to read in one line at a time:

  open(DBS, "+<$databaseFile");
  while (<DBS>) {
    $dbs = $_;
    [process $dbs]
    [output $dbs -- to file or browser]
  }
  close(DBS);

Using 'while (<DBS>) {...}' is just one way to read in single lines.

There could be other things going on here besides memory issues, but this
jumped out at me.

Two other things about the way you're reading from the file:

1. Is there a reason you're using "+<$databaseFile" in your open file
statement? If you are just getting data from the file, and not writing to
it in this operation, then don't use the '+' sign (and therefore, you don't
need the '<' either). Just say
  open(DBS, "$databaseFile");

Others might know if this in itself adds to your memory requirements.

2. But there's another issue here, unless you've simplified your code for
us: You're not doing any error checking at this critical point in your
script. It's rare that I don't want to catch file-access errors.

Here's my standard way to slurp files, based on the 'best practices'
example from Darin S. Morley (DSMorley@aol.com) as included in the MacPerl
and Perl source examples:

  my $datafile = 'filename';
  my @lines;

  open(IN, $datafile) and
    (@lines = <IN>) and
    close(IN) or
    die(qq/$0: Failed on $datafile: $!\n/);

  chomp @lines;  #optional -- delete line-ends from each line

[There's a variation that disregards line-ends and puts the whole slurp
into one scalar variable, but I think you want to keep your lines (database
records) separate]

The middle statement (starting with open) is a compound of opening the
file, slurping all lines into an array, and closing the file; if any of
those operations fails, then the final clause 'or die...' is invoked, and
it gives useful information about what failed.

If you find that slurping the whole file is what's causing the problem,
then it's a little awkward to do the same elegant file-access statement.
But at least do this:

  open(DBS, "$databaseFile") or die "Can't open $databaseFile: $!";


HTH

1;



- Bruce

__Bruce_Van_Allen___bva@cruzio.com__Santa_Cruz_CA__

==== Want to unsubscribe from this list?
==== Send mail with body "unsubscribe" to macperl-webcgi-request@macperl.org