[MacPerl-Scribes] Draft of CGI Article

[David Steffen <steffen@biomedcomp.com>]

Members:

Please find appended Draft 2 of my article on CGI programming with 
MacPerl.  Some notes:

1) It is very long - nearly twice as long as Arved's XS article.

2) The references are still stubs.  Any additional suggestions for references
    gratefully accepted.

3) Obviously, I'm putting it here because I want comments.

-David-

                        Using MacPerl for CGIs

Although Perl is used for many kinds of tasks, and although Perl
predates the World Wide Web, CGI scripts are sometimes seen as the
"killer app" for Perl.  To a large extent, this is due to the fact that
Perl is particularly well integrated into the Unix family of operating
systems (Unix, Linux, BSD, etc, referred to hereafter as *nix), because
the first Web (HTTP) servers ran under a *nix operating system, and
because *nix operating systems remain the predominant hosts for Web
servers today.  Perl can also be used to write CGIs for Web servers
running under the MacOS, however, and this article tell you how.

In most cases, MacPerl CGIs are identical to their *nix counterparts.
For that reason, and because there is so much information available on
writing CGIs in Perl (e.g. [1]), I will not discuss the writing of a CGI
in Perl at all.  In this column, I discuss what MacPerl CGIs are and are
not good for, the few places where MacPerl CGIs differ from *nix Perl
CGIs, and some of the limitations and pitfalls of using MacPerl to write
CGIs.  In a later column, I will discuss the theory behind synchronous
and asynchronous CGIs (.cgi and .acgi), server push, and taint checking.

Why write a CGI in MacPerl?
--------------------------

The MacOS is primarily a single-user, interactive operating system and
has some significant limitations which impact its use on a public Web
server.  MacPerl has some additional limitations as a CGI language.
Nonetheless, there are a number of reasons you might want to run MacPerl
CGI scripts on your Mac:

1) You want to use MacPerl to prototype cgi scripts before uploading
them to a production server.

2) You want to use a local, private web server on a Mac as a kiosk or to
run a slide show, and you want to add the flexibility and power of a
Perl CGI.

3) You want to add an easy user interface to a MacPerl program running
on a private network or even a single Mac.

4) Despite the difficulties, you want to run a public web server on your
Mac.

In my experience, MacPerl works excellently for prototyping, works well
on a single Mac or small, local network as a front end to a MacPerl
program or for a kiosk or slide show, and with care can be used on a
lightly loaded public server.

Mechanics of Running MacPerl CGIs
---------------------------------

A CGI script for MacPerl is usually identical to a *nix CGI; most *nix
CGIs will run without modification under MacPerl and the ever popular
CGI.pm module is just as useful on the Mac as it is on *nix.

To run a Perl script as a CGI on a Mac, you will need to have both
MacPerl and a Web server installed on your Mac, and have TCP/IP properly
set up, all of which are beyond the scope of this article.  As a Web
server, I have personally used MacHTTP/WebSTAR, Quid Pro Quo, and
Personal Web Sharing and they all work fine.  I suspect that most Mac
HTTP servers will work as well.   There are some differences in
interaction between different web servers and MacPerl, as well as
different versions of the MacOS, so your experience may vary a little
from what's written here.  I will try to point out places where I
suspect such variability to lurk, but be aware that my current
experience is with Quid Pro Quo running under MacOS 8.0.  Since Quid Pro
Quo is free, currently being supported, available via the web [2], and
seems to garner mostly favorable reviews, you might want to give it
serious consideration.

The MacPerl interpreter comes in two forms; one which can be run with
MPW and/or toolserver and one which is a stand-alone application.  The
stand-alone application is typically used for writing MacPerl CGIs and
will be the form of MacPerl discussed here.

If MacPerl is installed properly on your Mac, you will have four options
when you save a Perl script from the MacPerl application: 1) Plain text
2) CGI 3) Droplet 4) Runtime version You must save your script as CGI to
run it on a Mac.  This is a major difference between MacPerl and *nix
Perl, and is perhaps the most common difficulty people have when they
first start trying to use MacPerl for CGIs.  On the other hand, if you
are prototyping a CGI on a Mac which is eventually going to run on a
*nix server, you must save it as Plain Text before transferring it to
the *nix system; the CGI form will be useless to a *nix-based Perl.

Your Web server may have to be configured to use CGIs, may have a
particular location on the disk where it wants the CGIs, and may have
conventions as to how it wants the CGIs named.  The latest version of
Quid Pro Quo (2.1) prompts you for a series of parameters the first time
you run it after an installation and allows you to change these and many
more parameters thereafter.  One of the parameters that it prompts you
for is the root folder for your website; all web pages and CGIs must be
contained within this folder (and its subfolders).  (This folder needs
to exist before the installation.)  By default, you need to have one
subfolder named "cgi-bin" and any CGIs you intend to run need to be
contained therein and named with a .cgi extension (foobar.cgi,
perlscript.cgi, etc.) or a .acgi extension.  For now, you should name
all your MacPerl CGIs with the .cgi extension; the difference between
.cgi and .acgi will be discussed in a later column.

Given the above assumptions, you create your Perl script using any text
editor or the MacPerl application, or you can download a CGI from your
*nix server.  You then open it with the MacPerl application and save it
as a CGI into the cgi-bin folder of the root folder of your Web server.
You should now be able to access it with the URL:

http://your.server.name/cgi-bin/your-script-name.cgi

You do not need run either your CGI or MacPerl running; the Web server
will start them automatically.

What's Happening Here?
---------------------

Normally MacPerl executes a plain text file just like *nix Perl.
However, the CGI interface standard was designed with *nix in mind and
makes use of the Pipes and Environmental Variables of the
application-independent, simple, and powerful IPC model of *nix.
Because the MacOS IPC is completely different than that of *nix, CGIs
had to be implemented in an entirely different way.  Chuck Shotton, when
he wrote MacHTTP (one of the first Macintosh HTTP servers and the
predecessor to WebSTAR), defined a standard for how HTTP servers should
communicate with CGIs on the Mac based on AppleEvents, and as best I can
tell, all Macintosh HTTP servers adhere to this standard.  However, we
don't use this native MacOS model when writing CGIs with MacPerl.
Instead, the author of MacPerl, Matthias Neeracher, provided us with a
mechanism which makes porting CGIs between *nix and MacPerl nearly
seamless.

When you save a MacPerl script as a CGI, MacPerl creates a small, stand
alone application.  The Perl script is included in this application as a
resource.  This application can do nothing useful in the absence of the
MacPerl application; it accepts AppleEvents from a Macintosh Web server,
reformats them to make MacPerl work like *nix Perl, launches MacPerl if
necessary and passes the Perl script and the reformatted data from the
Web server to the MacPerl interpreter.  Finally, the CGI application
redirects the MacPerl output so that it can return it to the Web server.
  This CGI application appears in the application list at the right side
of the Macintosh menu bar just like any other application, and when you
select it, displays its single window with a single function; a "Quit
Now" button.  MacPerl can reopen these CGI applications, allowing you to
edit their scripts.

A feature of the MacPerl CGI mini-application is that it remains running
for 5 minutes after its MacPerl script finishes running.  This is to
allow a second request to that CGI to be serviced more quickly by
avoiding the overhead of starting the CGI.  The 5 minute interval is
actually an editable resource in the application; it can be varied
between 1 minute 127 minutes, or set such that the CGI continues running
indefinitely.  (Instructions for changing this value are given in MPPE,
[3].)  Similarly, a feature of MacPerl is that it continues running
indefinitely after a script finishes running.  (This is different from
*nix Perl.)  This behavior can also be changed from within the Perl
script.  For example, to make MacPerl always quit when the script
finishes running, add the following line to the script:

MacPerl::Quit(2);

Complete instructions for this command can be found in MPPE [4] and in
the MacPerl documentation [5].

Differences between MacPerl and Unix Perl
-----------------------------------------

In my opinion, the two most vexing differences between MacPerl and *nix
Perl both result from differences between MacOS and *nix; the different
interprocess communication (IPC) models used by MacOS and *nix, and the
fact that *nix can run many copies of the same program at the same time,
but the MacOS can only run one copy of a given program.  Because of work
done by Chuck Shotton Matthias Neeracher (described above), the
difference in MacOS and *nix IPC models has little impact on MacPerl
CGIs.  Unfortunately the inability of the MacOS to run multiple copies
of MacPerl has a major adverse impact on MacPerl CGIs.

Differences between Mac and *nix IPC
------------------------------------

The Mac interprocess communication model is based on AppleEvents whereas
that of the *nix operating system is based on Pipes and Environmental
Variables.  In general, interprocess communication is more difficult to
use and much less generally available on a Mac.  Thus, any *nix Perl
program which makes use of interprocess communication is going to be
more difficult to port to MacPerl.  As noted above, this problem has
been largely solved with regards to communication between a Web server
and a MacPerl CGI script.  The place where the difference in IPC models
remains a problem is when a *nix-based CGI uses pipes, the system()
function, or `backquotes` to run other *nix programs via the *nix IPC.
There is no general solution to this problem; in almost all cases, such
commands will not work with MacPerl.  In my opinion, most of these are a
bad idea for CGI's anyway, and thus should be avoided.

Problems resulting from the inability of the MacOS to run two copies of MacPerl
-------------------------------------------------------------------------------

Unlike *nix, the Macintosh OS can only run one copy of any given
program.  Thus, under Unix, you can have 5 copies of the same CGI or 5
different CGIs running at the same time.  Under MacOS, you can only have
one MacPerl CGI running at a time.  What is worse, the current version
of MacPerl CGI "glue" behaves badly when you try to violate this.  The
exact consequence of this bad behavior seems to vary depending on your
version of MacOS and/or your Web server and/or something else; the
bottom line is don't let it happen.

How can you prevent overlapping requests to MacPerl?  The first way is
manually; if you are testing CGIs on a Mac for later use on a *nix
system, just be careful to let one thing finish before starting another.
  The second way is to make it impossible, which with the help of your
Web server, you may be able to do.

The first secret to preventing overlapping calls to MacPerl is to use
the .cgi filename extension (myscript.cgi) rather than the .acgi
filename extension (myscript.acgi).  This should alert your Web server
(and will, if your Web server is the current version of Quid Pro Quo)
that your CGI can only do one thing at a time.  If someone makes a
second request of that CGI while a first request is running, Quid Pro
Quo will hold the second request until the first completes.

The second secret to preventing overlapping calls is to have but a
single MacPerl CGI on your server.  Quid Pro Quo is smart enough to
queue requests to myscript.cgi, but is unaware that otherscript.cgi is
also a MacPerl script, and that trying to run otherscript.cgi while
myscript.cgi is still running is a bad thing.  At first, this may seem
terribly limiting.  But remember, you only need to worry about this when
you cannot otherwise control access to the CGIs (e.g. because they are
on a public website).  Also note that one CGI can do as many different
things as you want, where you can control what it does my an HTML or
environmental variable.  What you might have done as separate CGIs, you
can do as separate Perl modules, "use"ing these modules from one master
cgi.

Different Syntax for Passing Data in the URL
--------------------------------------------

In addition to specifying which CGI to run, the URL can contain
information which is passed to the CGI in environmental variables.  The
most common of these, which works identically under *nix and MacPerl,
includes text after a question mark:

http://foobar.net/cgi-bin/perl.cgi?searchstring

or

http://foobar.net/cgi-bin/perl.cgi?firstname=David&lastname=Steffen

The string "searchstring" or "firstname=David&lastname=Steffen" will be
returned in the QUERY_STRING (and will be parsed into HTML variables by
the CGI.pm module).

However, *nix perl allows another construction:

http://foobar.net/cgi-bin/perl.cgi/more arguments/

This runs the script perl.cgi which resides in /cgi-bin/, passing /more
arguments/ in the PATH_INFO environmental variable.

This will not work with MacPerl.  What will work with MacPerl is:

http://foobar.net/cgi-bin/perl.cgi$/more arguments/

or

http://foobar.net/cgi-bin/perl.cgi$more arguments

...but this MacPerl-specific syntax will not work under *nix Perl.

If you need to go back and forth between the MacOS and *nix, avoid
passing arguments in the PATH_INFO environmental variable.   Passing
arguments via the QUERY_STRING environmental variable using the ? syntax
works fine.

Filenames, case sensitivity, and special characters
---------------------------------------------------

The MacOS keeps track of what case you use when you name a file but then
ignores it.  If you name your CGI MyName.cgi, it will remain MyName.cgi,
but if you request myname.cgi in the URL, your CGI will run.  Not so
under *nix; the URL which works on your Mac server will fail on the *nix
server; you must use the same case in the URL as the filename.

The MacOS is totally indifferent to spaces or other special characters
in a file name (except that the colon character ':' is not allowed) and
we Mac users frequently give files names like 'My First CGI!.cgi'.  *nix
will allow such filenames, but you will have no end of grief trying to
use them.  Just say no to spaces and punctuation in filenames that will
eventually used under *nix.

Putting It All Together: Using MacPerl CGIs
-------------------------------------------

How you plan to use your MacPerl CGI determines how (or even if) you
should use it.  Here I discuss some of the ways I have used MacPerl CGIs
or have seen others use MacPerl CGIs and the implications of those uses.

Testing CGI's that will Eventually Run under *nix
-------------------------------------------------

By and large, this works extremely well and is perhaps the best use for
MacPerl CGIs.  The reason it works well is that such use is highly
controlled, relatively light, and because you are sitting there running
the Mac, a crash or two can be tolerated.

The two main things you need be concerned with when prototyping a Perl
CGI destined for a *nix server using MacPerl are portability issues and
the few *nix Perl features that are not implemented in MacPerl.

If the CGI you are prototyping uses "server push", you are going to have
to read my next column to find out how to make this work under MacPerl.
If the CGI uses pipes or the system call to invoke other programs which
run on the *nix server, you are out of luck; you are going to have to
either change the CGI to avoid these external programs or at least write
stubs or other code to temporarily replace or bypass their use during
testing.  Consider the different file naming requirements of MacOS and
*nix, described above.

Other than that, save the script as a CGI while testing it on the Mac,
then save it as Plain Text for transfer to Unix.  If it works on a Mac,
it will work on Unix, and in most cases, vice versa.  The only other
things to watch out for are: * MacPerl is Perl 5.004.  5.005 and later
features won't work. * Make sure that any modules or other code you
"use" or "require" is available on both the Unix server and the Mac.
Remember that modules which have binary extensions may be difficult (or
even impossible) to port. * Remember to set your ownership and
permissions correctly on the Unix server.

Using a MacPerl CGI in a Slide Show
-----------------------------------

Increasingly, people use their laptop computers to deliver slide show
presentations.  Microsoft Powerpoint is frequently used for this
application, but many of us prefer to use web pages and a browser (e.g.
Netscape).  If you use only static web pages, no web server is required.
  However, if you include a web server, the use of CGIs can provide
additional flexibility, and such CGIs can be written in MacPerl.

Again, this use of MacPerl CGIs works quite well because use is
controlled and light.  However, having your Mac crash during a slide
show is a drag, so you need to be fairly careful about what you do and
do not attempt to minimize the chance of this happening.  Either plan
your presentation so you avoid overlapping calls to MacPerl, or use a
single master CGI for everything, naming it with a .cgi extension.

Using a CGI to Provide a User Interface for a MacPerl Program
-------------------------------------------------------------

Everyone knows how to surf the Web.  Thus, if you are developing a
MacPerl program that is going to be used by a number of people in your
organization, using the Web to provide a user interface has a number of
advantages: 1) You probably already know how to create web pages and
forms.  In any case, developing for the web is easy.  Thus, you can get
a user interface up very quickly. 2) If you have an intranet, this gives
you a client/server architecture almost for free. 3) Because everyone
knows how to surf the web and everyone has a web browser on their
computer, little or no training or software installation will be
required.

There are a couple of disadvantages: 1) The Web is connectionless.
Thus, saving state requires more work on your part.  But if you have
programmed for the web, you already know that. 2) The limitations of
being able to run a single MacPerl process can make your system
unstable.  However, depending on the size of the group, the
sophistication of the users, and physical proximity to the Mac on which
the system is running, a crash may not be the disaster it would be on a
public server.

I suggest using all the suggestions given below for a public server.

Using a MacPerl CGI in a Kiosk
------------------------------

By a kiosk, I mean a Mac which is running an "internal" web server which
provides web pages to the public but only from the one Mac on which it
runs.  In this application, we have gone up a notch in terms of
riskiness.  Use should still be light as only the one Mac can access the
website, but use is uncontrolled and the Mac is (in general) unattended
so a crash is more of a problem.

Again, I suggest using all the suggestions given below for a public
server.  Although only one person at a time will be using your system,
they may accidentally create overlapping calls to MacPerl if the rapidly
move between pages.  Finally, you should set up the server so that upon
restart it is fully functional and you should make it easy to trigger a
restart for whoever is maintaining the kiosk and possibly even for
users.

Using a MacPerl CGI on a Public, Macintosh Web Server
-----------------------------------------------------

I would suggest anyone contemplating running a public webserver on a Mac
think through the alternatives.  Placing your website on someone else's
server means a lot less cost and worry.  The Linux and BSD operating
systems both run on Mac hardware and are considerably more stable than
the MacOS.  Nonetheless, running your own server gives you more
flexibility and a Mac-based web server can be much easier to set up and
administer than a *nix base server, especially for someone from a
Macintosh background.

Given the decision to run a Mac Web server, you might choose a variety
of ways of creating CGIs.  For the ultimate in performance and
flexibility, you can write your CGI in C.  Frontier [6] is a powerful
scripting language which is tailored for web use and which, unlike
MacPerl, supports threading and asynchronous execution.  Applescript
uses very little disk space or RAM, and appears to be somewhat more
stable than MacPerl.  Finally, many applications (e.g. databases) have
built-in CGI support.  However, you may still choose MacPerl because you
already know Perl, because you want to take advantage of the wealth of
Perl CGIs which are available, or because of the power and ease of the
Perl language.

There are a number of things one can do to improve the stability of the
Mac as a server; plenty of RAM, allocating plenty of RAM to all
applications, dedicate a Mac as a server rather than try to use one Mac
both as a workstation and a server, remove as many extensions as
possible, running utilities like "Keep It Up" [7], "Autoboot" [8], and
"Okey Dokey" [9] to restart an application that quits, to restart after
a crash, or to dismiss unexpected dialog boxes which might otherwise
stop everything forever.  Having done as much of that as you can, there
are some MacPerl specific suggestions as well: 1) Give all CGIs the .cgi
extension (which means forgoing the server push feature.) 2) Have a
single, multifunction MacPerl CGI. 3) Only use MacPerl CGIs on lightly
used websites.  If you are trying to develop an amazon.com or a
slashdot, use other tools.

Citations
---------

1) Chapter 9 in "Perl Cookbook" by Tom Christiansen and Nathan
Torkington, ISBN 1-56592-243-3

2) Quid Pro Quo URL

3) MPPE instructions for setting the time in a CGI

4) Ibid, the MacPerl::Quit() function

5) The MacPerl documentation for MacPerl::Quit()

6) Frontier

7) Keep It Up

8) Autoboot

9) Okey Dokey

Biography
---------

David Steffen is founder and current President of Biomedical Computing,
Inc., a provider of custom software to the biomedical research
community.  He has over 20 years experience in biomedical research, in
which software design played a role, and over 5 years experience in full
time software design.  He likes Macs and most forms of *nix, Perl,
Python, and C, and will put up with almost anything that runs on a
computer.
David Steffen, Ph.D.
President, Biomedical Computing, Inc. <http://www.biomedcomp.com/>
Phone: (713) 610-9770 FAX: (713) 610-9769 E-mail: steffen@biomedcomp.com

==== Want to unsubscribe from this list?
==== Send mail with body "unsubscribe" to macperl-scribes-request@macperl.org