Re: [MacPerl] Memory requirements for data structures

[Keary Suska <aksuska@insideflyer.com>]

I believe you are looking for the "perlguts" man page (perlguts.pod in the
pod directory under MacPerl's installation directory).

One of the complaints I have heard about Perl in general (and it may not be
a very informed complaint) is that it is memory hungry. That may have as
much to do with programming style and with the internals, but the above info
should show you what Perl is doing.

-K

"Do not meddle in the affairs of dragons, because you are crunchy and taste
good with ketchup."


> From: Henry Penninkilampi <henry@penninkilampi.net>
> Date: Mon, 2 Oct 2000 19:58:20 +0930
> To: <macperl@macperl.org>
> Subject: [MacPerl] Memory requirements for data structures
> 
> Greetings!
> 
> Anyone noticed that data structures seem to tke up an awful lot of memory
> in (Mac)Perl?
> 
> Consider the following:
> 
> If you use a hash to set the value of 1024 8-byte keys to 1, then the
> 'visible payload' is 9 bytes per entry (or 9 KB for the whole thing).
> Real world tests would suggest that it is one (or _two_) orders of
> magnitude higher than that!
> 
> While I appreciate that there is likely to be a 32-bit pointer (for each
> entry) in there somewhere, that would only account for 44% of a
> 1,000..10,000% overhead.  What's taking up the rest?
> 
> In the struggle for memory efficiency, one often turns to the array.  If
> one's keys happen to be integers, then $exists{$id} = 1; can be just as
> easily accomplished using $exists[$id] = 1; and, because the numbers are
> actually _indices_ they shouldn't need to be stored, and hence the
> 'visible payload' should be reduced to a single byte per entry.
> 
> If we continue with the above example, one could hope for a best-case
> scenario of an 89% decrease in the size of the data structure.  But that
> doesn't happen (not even with small keys).
> 
> In extreme cases, one can even use vectors to store data, with
> vec($exists,$id,1); accomplishing much the same thing.  Bit-level
> manipulation of a _single_ string would seem to have 88% less payload
> than even a byte (character) array.  In real-world tests, the difference
> is hardly perceptible.
> 
> So, my question to those of you interested in this sort of thing, is
> "Why?"
> 
> By being 'clever' with how you store certain types of data, it is easily
> possible to acheive a 98.9% decrease in the amount of information needed
> to be manipulated.  The trickle-down benefits of such approaches (lower
> storage requirements, faster processing, etc.) should be obvious, and do
> materialise, but one thing that seems to go against the grain is the
> memory requirement.
> 
> Is this a MacPerl thing or just a Perl thing in general?  Are there any
> references/articles out there which discuss the overheads of Perl data
> structures, and memory optimisation strategies?  Are there any tools out
> there which allow memory usage to be accurately monitored?  Is this the
> price we pay for automated garbage collection?  Is it crippling
> object-oriented approaches to problem-solving?  Will Perl 6 address the
> issue?  Is it even an issue that needs to be addressed?  Do all languages
> suffer from similar 'overheads'?
> 
> Ladies and gentlemen, the chair is yours...
> 
> Henry.
> 
> # ===== Want to unsubscribe from this list?
> # ===== Send mail with body "unsubscribe" to macperl-request@macperl.org
> 


# ===== Want to unsubscribe from this list?
# ===== Send mail with body "unsubscribe" to macperl-request@macperl.org