sigmasternchen/php-doc-en
1
0
Fork 0
mirror of https://github.com/sigmasternchen/php-doc-en synced 2025-03-15 08:28:54 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

- Added a description of the GC algorithm to the documentation

Browse source 
#- I most likely didn't use the most optimal docbook—feel free to fix :)


git-svn-id: https://svn.php.net/repository/phpdoc/en/trunk@291067 c90b9560-bf6c-de11-be94-00142212c4b1
This commit is contained in:
Derick Rethans 2009-11-20 11:32:37 +00:00
parent c9f2372f4c
commit f0f030aaf2
7 changed files with 647 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

BIN
features/figures/gc-algorithm.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 30 KiB

BIN
features/figures/gc-benchmark.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 61 KiB

BIN
features/figures/leak-array.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 5.2 KiB

BIN
features/figures/loop-array.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 5.8 KiB

BIN
features/figures/simple-array.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 9.1 KiB

BIN
features/figures/simple-array2.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 10 KiB

647
features/gc.xml Normal file
View file

@ -0,0 +1,647 @@
<?xml version="1.0" encoding="utf-8"?>
<!-- $Revision: 283795 $ -->
<chapter xml:id="features.gc" xmlns="http://docbook.org/ns/docbook">
<title>Garbage Collection</title>
<para>
This section explains the merits of the new Garbage Collection (also known
as GC) mechanism that is part of PHP 5.3. This was originally written as a
three part column for <link xlink='&url.phparchitect;'>php|architect</link>.
</para>
<sect1 xml:id="features.gc.refcounting-basics">
<title>Reference Counting Basics</title>
<para>
A PHP variable is stored in a container called a "zval". A zval container
contains, besides the variable's type and value, two additional bits of
information. The first is called "is_ref" and is a boolean value
indicating whether or not the variable is part of a "reference set". With
this bit, PHP's engine knows how to differentiate between normal variables
and references. Since PHP allows user-land references, as created by the
&amp; operator, a zval container also has an internal reference counting
mechanism to optimize memory usage. This second piece of additional
information, called "refcount", contains how many variable names—also
called symbols—point to this one zval container. All symbols are stored in
a symbol table, of which there is one per scope. There is a scope for the
main script (i.e., the one requested through the browser), as well as one
for every function or method.
</para>
<para>
A zval container is created when a new variable is created with a constant
value, such as:
<programlisting role="php">
$a = "new string";
</programlisting>
</para>
<para>
In this case, the new symbol name, "a", is created in the current scope,
and a new variable container is created with type "string", value "new
string". The "is_ref" bit is by default set to "false" because no
user-land reference has been created. The "refcount" is set to "1" as
there is only one symbol that makes use of this variable container. Note
that if "refcount" is "1", "is_ref" is always "false". If you have <link
xlink='&url.xdebug;'>Xdebug</link> installed, you can display this
information by calling:
</para>
<para>
<programlisting role="php">
xdebug_debug_zval('a');
</programlisting>
</para>
<para>
which displays:
</para>
<para>
<programlisting role="shell">
a: (refcount=1, is_ref=0)='new string'
</programlisting>
</para>
<para>
Assigning this variable to another variable name increases the refcount:
</para>
<para>
<programlisting role="php">
$a = "new string";
$b = $a;
xdebug_debug_zval( 'a' );
</programlisting>
</para>
<para>
which displays:
</para>
<para>
<programlisting role="shell">
a: (refcount=2, is_ref=0)='new string'
</programlisting>
</para>
<para>
The refcount is "2" here, because the same variable container is linked
with both "a" and "b". PHP is smart enough not to copy the actual variable
container when it is not necessary. Variable containers get destroyed when
the "refcount" reaches zero. The "refcount" gets decreased by one when any
symbol linked to the variable container leaves the scope (e.g. when the
function ends) or when <function>unset</function> is called on a symbol.
The following example shows this:
</para>
<para>
<programlisting role="php">
$a = "new string";
$c = $b = $a;
xdebug_debug_zval( 'a' );
unset( $b, $c );
xdebug_debug_zval( 'a' );
</programlisting>
</para>
<para>
which displays:
</para>
<para>
<programlisting role="shell">
a: (refcount=3, is_ref=0)='new string'
a: (refcount=1, is_ref=0)='new string'
</programlisting>
</para>
<para>
If we now call "unset( $a );", the variable container, including the type
and value, will be removed from memory.
</para>
<sect2 xml:id="features.gc.compound-types">
<title>Compound Types</title>
<para>
Things get a tad more complex with compound types such as arrays and
objects. Instead of a scalar value, arrays and objects store their
properties in a symbol table of their own. This means that the following
example creates three zval containers:
</para>
<para>
<programlisting role="php">
$a = array( 'meaning' => 'life', 'number' => 42 );
xdebug_debug_zval( 'a' );
</programlisting>
</para>
<para>
which displays (after formatting):
</para>
<para>
<programlisting role="shell">
a: (refcount=1, is_ref=0)=array (
'meaning' => (refcount=1, is_ref=0)='life',
'number' => (refcount=1, is_ref=0)=42
)
</programlisting>
</para>
<para>
Or graphically:
<mediaobject>
<imageobject>
<imagedata fileref="en/features/figures/simple-array.png" format="PNG"/>
</imageobject>
</mediaobject>
</para>
<para>
The three zval containers are: "a", "meaning", and "number". Similar rules
apply for increasing and decreasing "refcounts". Below, we add another
element to the array, and set its value to the contents of an already
existing element:
</para>
<para>
<programlisting role="php">
$a = array( 'meaning' => 'life', 'number' => 42 );
$a['life'] = $a['meaning'];
xdebug_debug_zval( 'a' );
</programlisting>
</para>
<para>
which displays (after formatting):
</para>
<para>
<programlisting role="shell">
a: (refcount=1, is_ref=0)=array (
'meaning' => (refcount=2, is_ref=0)='life',
'number' => (refcount=1, is_ref=0)=42,
'life' => (refcount=2, is_ref=0)='life'
)
</programlisting>
</para>
<para>
Or graphically:
<mediaobject>
<imageobject>
<imagedata fileref="en/features/figures/simple-array2.png" format="PNG"/>
</imageobject>
</mediaobject>
</para>
<para>
From the above Xdebug output, we see that both the old and new array
element now point to a zval container whose "refcount" is "2". Of course,
there are now two zval containers, but they are the same one. The
<function>xdebug_debug_zval</function> function does not show this, but
you could see this by also displaying the memory pointer. Removing an
element from the array is like removing a symbol from a scope. By doing
so, the "refcount" of a container that an array element points to is
decreased. Again, when the "refcount" reaches zero, the variable
container is removed from memory. Again, an example to show this:
</para>
<para>
<programlisting role="php">
$a = array( 'meaning' => 'life', 'number' => 42 );
$a['life'] = $a['meaning'];
unset( $a['meaning'], $a['number'] );
xdebug_debug_zval( 'a' );
</programlisting>
</para>
<para>
which displays (after formatting):
</para>
<para>
<programlisting role="shell">
a: (refcount=1, is_ref=0)=array (
'life' => (refcount=1, is_ref=0)='life'
)
</programlisting>
</para>
<para>
Now, things get interesting if we add the array itself as an element of
the array, which we do in the next example, in which I also snuck in a
reference operator, since otherwise PHP would create a copy:
</para>
<para>
<programlisting role="php">
<![CDATA[
$a = array( 'one' );
$a[] =& $a;
xdebug_debug_zval( 'a' );
]]>
</programlisting>
</para>
<para>
which displays (after formatting):
</para>
<para>
<programlisting role="shell">
a: (refcount=2, is_ref=1)=array (
0 => (refcount=1, is_ref=0)='one',
1 => (refcount=2, is_ref=1)=...
)
</programlisting>
</para>
<para>
Or graphically:
<mediaobject>
<imageobject>
<imagedata fileref="en/features/figures/loop-array.png" format="PNG"/>
</imageobject>
</mediaobject>
</para>
<para>
You can see that the array variable ("a") as well as the second element
("1") now point to a variable container that has a "refcount" of "2". The
"..." in the display above shows that there is recursion involved, which,
of course, in this case means that the "..." points back to the original
array.
</para>
<para>
Just like before, unsetting a variable removes the symbol, and the
reference count of the variable container it points to is decreased by
one. So, if we unset variable $a after running the above code, the
reference count of the variable container that $a and element "1" point to
gets decreased by one, from "2" to "1". This can be represented as:
</para>
<para>
<programlisting role="shell">
(refcount=1, is_ref=1)=array (
0 => (refcount=1, is_ref=0)='one',
1 => (refcount=1, is_ref=1)=...
)
</programlisting>
</para>
<para>
Or graphically:
<mediaobject>
<imageobject>
<imagedata fileref="en/features/figures/leak-array.png" format="PNG"/>
</imageobject>
</mediaobject>
</para>
</sect2>
<sect2 xml:id="features.gc.cleanup-problems">
<title>Cleanup Problems</title>
<para>
Although there is no longer a symbol in any scope pointing to this
structure, it cannot be cleaned up because the array element "1" still
points to this same array. Because there is no external symbol pointing to
it, there is no way for a user to clean up this structure; thus you get a
memory leak. Fortunately, PHP will clean up this data structure at the end
of the request, but before then, this is taking up valuable space in
memory. This situation happens often if you're implementing parsing
algorithms or other things where you have a child point back at a "parent"
element. The same situation can also happen with objects of course, where
it actually is more likely to occur, as objects are always implicitly used
by reference.
</para>
<para>
This might not be a problem if this only happens once or twice, but if
there are thousands, or even millions, of these memory losses, this
obviously starts being a problem. This is especially problematic in long
running scripts, such as daemons where the request basically never ends,
or in large sets of unit tests. The latter caused problems while
running the unit tests for the Template component of the eZ Components
library. In some cases, it would require over 2 GB of memory, which the
test server didn't quite have.
</para>
</sect2>
</sect1>
<sect1 xml:id="features.gc.collecting-cycles">
<title>Collecting Cycles</title>
<para>
Traditionally, reference counting memory mechanisms, such as that used by
PHP, fail to address those circular reference memory leaks. Back in 2007,
while looking into this issue, I was pointed to a paper by David F. Bacon
and V.T. Rajan titled "<link xlink='&url.gc-paper;'>Concurrent Cycle
Collection in Reference Counted Systems</link>". Although the paper was
written with Java in mind, I started to play around with it to see if it
was feasible to implement the synchronous algorithm, as outlined in the
paper, in PHP. At that moment, I didn't have a lot of time, but along came
the Google Summer of Code project and we put forward the implementation of
this paper as one of our ideas. Yiduo (David) Wang picked up this idea and
started hacking on the first version as part of the Summer of Code
project.
</para>
<para>
A full explanation of how the algorithm works would be slightly beyond the
scope of this section, but the basics are explained here. First of all,
we have to establish a few ground rules. If a refcount is increased, it's
still in use and therefore, not garbage. If the refcount is decreased and
hits zero, the zval can be freed. This means that garbage cycles can only
be created when a refcount argument is decreased to a non-zero value.
Secondly, in a garbage cycle, it is possible to discover which parts are
garbage by checking whether it is possible to decrease their refcount by
one, and then checking which of the zvals have a refcount of zero.
</para>
<para>
<mediaobject>
<imageobject>
<imagedata fileref="en/features/figures/gc-algorithm.png" format="PNG"/>
</imageobject>
</mediaobject>
</para>
<para>
To avoid having to call the checking of garbage cycles with every possible
decrease of a refcount, the algorithm instead puts all possible roots
(zvals) in the "root buffer" (marking them "purple"). It also makes sure
that each possible garbage root ends up in the buffer only once. Only when
the root buffer is full does the collection mechanism start for all the
different zvals inside. See step A in the figure.
</para>
<para>
In step B, the algorithm runs a depth-first search on all possible roots
to decrease by one the refcounts of each zval it finds, making sure not to
decrease a refcount on the same zval twice (by marking them as "grey"). In
step C, the algorithm again runs a depth-first search from each root node,
to check the refcount of each zval again. If it finds that the refcount is
zero, the zval is marked "white" (blue in the figure). If it's larger than
zero, it reverts the decreasing of the refcount by one with a depth-first
search from that point on, and they are marked "black" again. In the last
step (D), the algorithm walks over the root buffer removing the zval roots
from there, and meanwhile, checks which zvals have been marked "white" in
the previous step. Every zval marked as "white" will be freed.
</para>
<para>
Now that you have a basic understanding of how the algorithm works, we
will look back at how this integrates with PHP. By default, PHP's garbage
collector is turned on. There is, however, a &php.ini;
setting that allows you to change this: <option>zend.enable_gc</option>.
</para>
<para>
When the garbage collector is turned on, the cycle-finding algorithm as
described above is executed whenever the root buffer runs full. The root
buffer has a fixed size of 10,000 possible roots (although you can alter
this by changing the <literal>GC_ROOT_BUFFER_MAX_ENTRIES</literal> constant in
<literal>Zend/zend_gc.c</literal> in the PHP source code, and re-compiling
PHP). When the garbage collector is turned off, the cycle-finding
algorithm will never run. However, possible roots will always be recorded
in the root buffer, no matter whether the garbage collection mechanism has
been activated with this configuration setting.
</para>
<para>
If the root buffer becomes full with possible roots while the garbage
collection mechanism is turned off, further possible roots will simply not
be recorded. Those possible roots that are not recorded will never be
analyzed by the algorithm. If they were part of a circular reference
cycle, they would never be cleaned up and would create a memory leak.
</para>
<para>
The reason why possible roots are recorded even if the mechanism has been
disabled is because it's faster to record possible roots than to have to
check whether the mechanism is turned on every time a possible root could
be found. The garbage collection and analysis mechanism itself, however,
can take a considerable amount of time.
</para>
<para>
Besides changing the <option>zend.enable_gc</option> configuration
setting, it is also possible to turn the garbage collecting mechanism on
and off by calling <function>gc_enable</function> or
<function>gc_disable</function> respectively. Calling those functions has
the same effect as turning on or off the mechanism with the configuration
setting. It is also possible to force the collection of cycles even if the
possible root buffer is not full yet. For this, you can use the
<function>gc_collect_cycles</function> function. This function will return
how many cycles were collected by the algorithm.
</para>
<para>
The rationale behind the ability to turn the mechanism on and off, and to
initiate cycle collection yourself, is that some parts of your application
could be highly time-sensitive. In those cases, you might not want the
garbage collection mechanism to kick in. Of course, by turning off the
garbage collection for certain parts of your application, you do risk
creating memory leaks because some possible roots might not fit into the
limited root buffer. Therefore, it is probably wise to call
<function>gc_collect_cycles</function> just before you call
<function>gc_disable</function> to free up the memory that could be lost
through possible roots that are already recorded in the root buffer. This
then leaves an empty buffer so that there is more space for storing
possible roots while the cycle collecting mechanism is turned off.
</para>
<para>
In this section, we saw how the garbage collection mechanism works and
how it is integrated into PHP. In the third and final section,
we will look at performance considerations and benchmarks.
</para>
</sect1>
<sect1 xml:id="features.gc.performance-considerations">
<title>Performance Considerations</title>
<para>
We have already mentioned in the previous section that simply collecting the
possible roots had a very tiny performance impact, but this is when you
compare PHP 5.2 against PHP 5.3. Although the recording of possible roots
compared to not recording them at all, like in PHP 5.2, is slower, other
changes to the PHP runtime in PHP 5.3 prevented this particular performance
loss from even showing.
</para>
<para>
There are two major areas in which performance is affected. The first
area is reduced memory usage, and the second area is run-time delay when
the garbage collection mechanism performs its memory cleanups. We will
look at both of those issues.
</para>
<sect2 xml:id="features.gc.performance-considerations.reduced-mem">
<title>Reduced Memory Usage</title>
<para>
First of all, the whole reason for implementing the garbage collection
mechanism is to reduce memory usage by cleaning up circular-referenced
variables as soon as the prerequisites are fulfilled. In PHP's
implementation, this happens as soon as the root-buffer is full, or when
the function <function>gc_collect_cycles</function> is called. In
the graph below, we display the memory usage of the script below,
in both PHP 5.2 and PHP 5.3, excluding the base memory that PHP
itself uses when starting up.
</para>
<para>
<mediaobject>
<imageobject>
<imagedata fileref="en/features/figures/gc-benchmark.png" format="PNG"/>
</imageobject>
</mediaobject>
</para>
<para>
<programlisting role="php">
<![CDATA[<?php
class Foo
{
public $var = '3.1415962654';
}
$baseMemory = memory_get_usage();
for ( $i = 0; $i <= 100000; $i++ )
{
$a = new Foo;
$a->self = $a;
if ( $i % 500 === 0 )
{
echo sprintf( '%8d: ', $i ), memory_get_usage() - $baseMemory, "\n";
}
}
?>]]>
</programlisting>
</para>
<para>
In this very academic example, we are creating an object in which a
property is set to point back to the object itself. When the $a variable
in the script is re-assigned in the next iteration of the loop, a memory
leak would typically occur. In this case, two zval-containers are leaked
(the object zval, and the property zval), but only one possible root is
found: the variable that was unset. When the root-buffer is full after
10,000 iterations (with a total of 10,000 possible roots), the garbage
collection mechanism kicks in and frees the memory associated with those
possible roots. This can very clearly be seen in the jagged memory-usage
graph for PHP 5.3. After each 10,000 iterations, the mechanism kicks in
and frees the memory associated with the circular referenced variables.
The mechanism itself does not have to do a whole lot of work in this
example, because the structure that is leaked is extremely simple. From
the diagram, you see that the maximum memory usage in PHP 5.3 is about 9
Mb, whereas in PHP 5.2 the memory usage keeps increasing.
</para>
</sect2>
<sect2 xml:id="features.gc.performance-considerations.slowdowns">
<title>Run-Time Slowdowns</title>
<para>
The second area where the garbage collection mechanism influences
performance is the time taken when the garbage collection mechanism
kicks in to free the "leaked" memory. In order to see how much this is,
we slightly modify the previous script to allow for a larger number of
iterations and the removal of the intermediate memory usage figures. The
second script is here:
</para>
<para>
<programlisting role="php">
<![CDATA[<?php
class Foo
{
public $var = '3.1415962654';
}
for ( $i = 0; $i <= 1000000; $i++ )
{
$a = new Foo;
$a->self = $a;
}
echo memory_get_peak_usage(), "\n";
?>]]>
</programlisting>
</para>
<para>
We will run this script two times, once with the
<option>zend.enable_gc</option> setting turned on, and once with it
turned off:
</para>
<para>
<programlisting role="shell">
time ~/dev/php/php-5.3dev/sapi/cli/php -dzend.enable_gc=0 \
-dmemory_limit=-1 -n Listing1.php
# and
time ~/dev/php/php-5.3dev/sapi/cli/php -dzend.enable_gc=1 \
-dmemory_limit=-1 -n Listing2.php
</programlisting>
</para>
<para>
On my machine, the first command seems to take consistently about 10.7
seconds, whereas the second command takes about 11.4 seconds. This is a
slowdown of about 7%. However, the maximum amount of memory used by the
script is reduced by 98% from 931Mb to 10Mb. This benchmark is not very
scientific, or even representative of real-life applications, but it
does demonstrate the memory usage benefits that this garbage collection
mechanism provides. The good thing is that the slowdown is always the
same 7%, for this particular script, while the memory saving
capabilities save more and more memory as more circular references are
found during script execution.
</para>
<para>
Let's now have a look at non-academic situation. I first started looking
for circular reference collecting algorithms when I found out that while
running the tests of the eZ Components' Template component with PHPUnit,
I ended up swapping a lot, rendering my machine useless in the process.
In order to do some benchmarks for this article, I re-ran those same
tests with an empty php.ini file to disable the overhead and memory
allocation that Xdebug was creating while doing code-coverage analysis.
</para>
<para>
Memory consumption dropped 95% from 1.7Gb to 75Mb, and the runtime as
reported by PHPUnit increased from 2:17 for the non-GC enabled run to
2:33 for the GC enabled run, an increase of about 12%. However, with the
non-GC enabled run, PHP sat there doing "nothing" for almost 15 seconds.
Upon investigation with the Unix debugger, GDB, I noticed that those 15
seconds were all spent on freeing memory allocated for objects inside
the PHP runtime. The actual time that the script ran was about the same
in the end.
</para>
</sect2>
<sect2 xml:id="features.gc.performance-considerations.internal-stats">
<title>PHP's Internal GC Statistics</title>
<para>
It is possible to coax a little bit more information about how the
garbage collection mechanism is run from within PHP. But in order to do
so, you will have to re-compile PHP to enable the benchmark and
data-collecting code. You will have to set the <literal>CFLAGS</literal>
environment variable to <literal>-DGC_BENCH=1</literal> prior to running
<literal>./configure</literal> with your desired options. The following
sequence should do the trick:
</para>
<para>
<programlisting role="shell">
export CFLAGS=GC_BENCH=1
./config.nice
make clean
make
</programlisting>
</para>
<para>
When you run the above example code again with the newly built PHP
binary, you will see the following being shown after PHP has finished
execution:
</para>
<para>
<programlisting role="shell">
GC Statistics
-------------
Runs: 110
Collected: 2072204
Root buffer length: 0
Root buffer peak: 10000
Possible Remove from Marked
Root Buffered buffer grey
-------- -------- ----------- ------
ZVAL 7175487 1491291 1241690 3611871
ZOBJ 28506264 1527980 677581 1025731
</programlisting>
</para>
<para>
The most informative statistics are displayed in the first block. You
can see here that the garbage collection mechanism ran 110 times, and in
total, more than 2 million memory allocations were freed during those
110 runs. As soon as the garbage collection mechanism has run at least
one time, the "Root buffer peak" is always 10000.
</para>
</sect2>
<sect2 xml:id="features.gc.performance-considerations.conclusion">
<title>Conclusion</title>
<para>
In this third and final installment, we had a quick look at the
performance implications of the garbage collection mechanism that is now
part of PHP 5.3. In general, it will only cause a slowdown when the
cycle collecting algorithm actually runs, whereas in normal (smaller)
scripts there should be no performance hit at all.
</para>
<para>
However, in the cases where the cycle collection mechanism does run for
normal scripts, the memory reduction it will provide allows more of
those scripts to run concurrently on your server, since not so much
memory is used in total.
</para>
<para>
The benefits are most apparent for longer-running scripts, such as
lengthy test suites or daemon scripts. Also, for PHP-GTK applications
that generally tend to run longer than scripts for the Web, the new
mechanism should make quite a bit of a difference regarding memory leaks
creeping in over time.
</para>
</sect2>
</sect1>
</chapter>
<!-- Keep this comment at the end of the file
vim600: syn=xml fen fdm=syntax fdl=2 si
vim: et tw=78 syn=sgml
vi: ts=1 sw=1
-->