Re: [MacPerl] FTP client? [Comments and Questions]

[mark@cheers.jsc.nasa.gov (Mark Manning/Muniz Eng.)]

According to Jon S. Jaques:
> 
> Yes, and no... Doesn't the sort mechanism make temporary duplicates in
> memory? (As mentioned above) Granted, the number of folders would be much
> much smaller, but sorting on the PB is a huge time waister.

*nod*  Yes, it does create temporary arrays to do the
sorting (from what I understand).  The problem was though,
that the PBs were running out of memory trying to handle
everything at once.  So if it was broken up into smaller
chunks it would take a bit longer but the PBs would then be
able to handle it easily.  :-)

Here is an expanded example of what I meant:

    @dirList = ();
    @relFiles = ();
    $dirList[++$#dirList] = "/myDiskDrive";
    while( $#dirList > -1 ){
        $theDir = $dirList[$#dirList];
        $#dirList--;

        opendir( THEDIR, $theDir );
        @theList = grep( !/^\.\.?$/, readdir(THEDIR) );
        closedir( THEDIR );

        for( $i=0; $i<=$#theList; $i++ ){
            if( -d "$theDir/$theList[$i]" ) {
                $dirList[++$#dirList] = "$theDir/$theList[$i]";
                }
                else {
                    $relFiles[++$#relFiles] = "$theList[$i]";
                    }
            }
        }

This creates two lists.  One a directory list and the other
a file list relative to the first directory list.  This
program segment doesn't even need to perform a sort of any
kind.  It simply drops down through the directory listings
and the two lists grow or shrink as needed.  In fact, you
could modify the above like the following in order to
achive what you want without ever sorting a single entry:

#
#	Set up the arrays.
#
    @dirList = ();
    @relFiles = ();
#
#	Put in the top level to begin with.
#
    $dirList[++$#dirList] = "/myDiskDrive";
#
#	Start moving things over.
#
    while( $#dirList > -1 ){
#
#	Pop off the last directory entry.
#
        $theDir = $dirList[$#dirList];
        $#dirList--;

        opendir( THEDIR, $theDir );
        @theList = grep( !/^\.\.?$/, readdir(THEDIR) );
        closedir( THEDIR );

        for( $i=0; $i<=$#theList; $i++ ){
            if( -d "$theDir/$theList[$i]" ) {
#
#	Add the directory to our list to recursively decend into.
#
                $dirList[++$#dirList] = "$theDir/$theList[$i]";
#
#	Create the subdirectory on the Mac.
#
				$newDir = $dirList[$#dirList];
				$newDir =~ s/\//:/g;
				mkdir $dirList[$#dirList];
                }
                else {
#
#	Log the file we are transferring
#
                    $relFiles[++$#relFiles] = "$theList[$i]";
#
#	Transfer it
#
					<put in your commands here>
                    }
            }
        }
> 
> >The time to transfer should stay about the same though.  No
> 
> We do have PCMCIA Ethernet devices on all PBs but one, a 540c, which uses
> an external device... Do you think that the external devices are faster? I
> can test that concept on the next update of the websites...

No.  They should be about the same.  However, with the Gator box I have transferred
hundreds of files in very little time (until Fetch crashed then I just had to reboot
and pick-up where I left off).

> My thought was to use an http server on an NT or Linux server, which can
> serve up the entire list of 9-10,000 files in a minute or two, maybe less,
> due to available clock cycles. Using print statements in Perl, the output
> would begin immediately, and thus would be pushed down the Mac's little
> thro... ahem(!) pipeline at a much greater rate (theoretically) than if
> *it* was handling all of the I/O. Maybe even acheiving the full 10mb
> Ethernet rate? Not likely, I suppose. <g>

I would suggest using Chris' Anarchie program or the FTP
program (converted by Paul to the Mac).  Both of these
programs use the full capabilities of the Mac without the
overhead of having to go through a web server.  You could
use NCSA Telnet or the Anarchie program via AppleScripts.
However, you do not want to overload the PBs by trying to
transfer all 9,000 some odd files at one time.  The PB
would die on such a thing.  So single file transfers are
the best and Chris' or Paul's program would do this for you
really quickly.  :-)

> 
> Off topic, I know, but what "is" the Gator box? A router? Or serial?

It is an Ethernet to Appletalk router box.  It allows you
to have multiple Macs connected to an Ethernet network at
one time.  Up to 256 Macs I believe.  It overrides the
limitations of the Appletalk connectors (which is somewhere
around 9600 baud if I remember correctly) allowing you to
use the full transfer rate (up to something like 192K).  It
does this without messing around Appletalk's slower rate of
transfer.

I hope this helps.  :-)

***** Want to unsubscribe from this list?
***** Send mail with body "unsubscribe" to mac-perl-request@iis.ee.ethz.ch