php-doc-en/reference/mysqlnd/memory.xml

<?xml version="1.0" encoding="utf-8"?>
<!-- $Revision$ -->
<chapter xml:id="mysqlnd.memory" xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink">
 <title>Memory management</title>
 <para>
  <emphasis role="bold">Introduction</emphasis>
 </para>
 <para>
  The MySQL Native Driver manages memory different than the MySQL Client Library.
  The libraries differ in the way memory is allocated and released,
  how memory is allocated in chunks while reading results from MySQL, which debug
  and development options exist, and how results read from MySQL are linked to
  PHP user variables.
 </para>
 <para>
  The following notes are intended as an introduction and summary to users
  interested at understanding the MySQL Native Driver at the C code level.
 </para>
 <para>
  <emphasis role="bold">Memory management functions used</emphasis>
 </para>
 <para>
  All memory allocation and deallocation is done using the PHP memory management
  functions. Therefore, the memory consumption of mysqlnd can be tracked
  using PHP API calls, such as <function>memory_get_usage</function>. Because memory is
  allocated and released using the PHP memory management, the changes may not
  immediately become visible at the operating system level. The PHP
  memory management acts as a proxy which may delay releasing memory towards
  the system. Due to this, comparing the memory usage of the MySQL Native Driver
  and the MySQL Client Library is difficult. The MySQL Client Library
  is using the operating system memory management calls directly, hence the effects
  can be observed immediately at the operating system level.
 </para>
 <para>
  Any memory limit enforced by PHP also affects the MySQL Native Driver. This
  may cause out of memory errors when fetching large result sets that exceed
  the size of the remaining memory made available by PHP. Because the MySQL
  Client Library is not using PHP memory management functions, it does not comply
  to any PHP memory limit set. If using the MySQL Client Library, depending on the
  deployment model, the memory footprint of the PHP process may grow
  beyond the PHP memory limit. But also PHP scripts may be able to process larger
  result sets as parts of the memory allocated to hold the result sets are beyond
  the control of the PHP engine.
 </para>
 <para>
   PHP memory management functions are invoked by the MySQL Native Driver through
   a lightweight wrapper. Among others, the wrapper makes debugging easier.
 </para>
 <para>
   <emphasis role="bold">Handling of result sets</emphasis>
 </para>
 <para>
  The various MySQL Server and the various client APIs differentiate
  between <link linkend="mysqli.quickstart.statements">buffered and unbuffered</link>
  result sets. Unbuffered result sets are transferred row-by-row from MySQL to the client
  as the client iterates over the results. Buffered results are fetched
  in their entirety by the client library before passing them on to the client.
 </para>
 <para>
  The MySQL Native Driver is using PHP Streams for the network communication
  with the MySQL Server. Results sent by MySQL are fetched from the PHP Streams
  network buffers into the result buffer of mysqlnd. The result buffer is
  made of zvals. In a second step the results are made available to the PHP script.
  This final transfer from the result buffer into PHP variables impacts the memory
  consumption and is mostly noticible when using buffered result sets.
 </para>
 <para>
  By default the MySQL Native Driver tries to avoid holding
  buffered results twice in memory. Results are kept only once in the internal
  result buffers and their zvals. When results are fetched into PHP variables
  by the PHP script, the variables will reference the internal result buffers.
  Database query results are not copied and kept in memory only once.
  Should the user modify the contents of a variable holding the database results a
  copy-on-write must be performed to avoid changing the referenced internal result
  buffer. The contents of the buffer must not be modified because the user
  may decide to read the result set a second time. The copy-on-write
  mechanism is implemented using an additional reference
  management list and the use of standard zval reference counters.
  Copy-on-write must also be done if the user reads a result set into PHP variables and
  frees a result set before the variables are unset.
 </para>
 <para>
  Generally speaking, this pattern works well for scripts that read a
  result set once and do not modify variables holding results. Its major drawback
  is the memory overhead caused by the additional reference management which
  comes primarily from the fact that user variables holding results
  cannot be entirely released until the mysqlnd reference management
  stops referencing them. The MySQL Native driver removes the reference to the
  user variables when the result set is freed or a copy-on-write is performed.
  An observer will see the total memory consumption grow until the result set
  is released. Use the <link linkend="mysqlnd.stats">statistics</link> to check whether
  a script does release result sets explicitly or the driver is does implicit
  releases and thus memory is used for a time longer than necessary. Statistics
  also help to see how many copy-on-write operations happened.
 </para>
 <para>
  A PHP script reading many small rows of a buffered result set using a code snippet
  equal or equivalent to <literal>while ($row = $res-&gt;fetch_assoc()) { ... }</literal>
  may optimize memory consumption by requesting copies instead of references.
  Albeit requesting copies means keeping results twice in memory, it allows
  PHP to free the copy contained in <literal>$row</literal> as the result set
  is being iterated and prior to releasing the result set itself. On a loaded server
  optimizing peak memory usage may help improving the overall system performance
  although for an individual script the copy approach may be slower due to
  additional allocations and memory copy operations.
 </para>
 <para>
  The copy mode can be enforced by setting
  <link linkend="ini.mysqlnd.fetch_data_copy">mysqlnd.fetch_data_copy</link>=1.
 </para>
 <para>
   <emphasis role="bold">Monitoring and debugging</emphasis>
 </para>
 <para>
  There are multiple ways of tracking the memory usage of the MySQL Native Driver.
  If the goal is to get a quick high level overview or to verify the memory efficiency
  of PHP scripts, then check the <link linkend="mysqlnd.stats">statistics</link>
  collected by the library. The statistics allow you, for example, to catch
  SQL statements which generate more results than are processed by a PHP script.
 </para>
 <para>
  The <link linkend="ini.mysqlnd.debug">debug</link> trace log can be configured to
  record memory management calls. This helps to see when memory is allocated
  or free'd. However, the size of the requested memory chunks may not be listed.
 </para>
 <para>
  Some, recent versions of the MySQL Native Driver feature the emulation of
  random out of memory situations. This feature is meant to be used by the
  C developers of the library or mysqlnd <link linkend="mysqlnd.plugin">plugin</link>
  authors only. Please, search the source code for corresponding PHP configuration
  settings and further details. The feature is considered private and may be
  modified at any time without prior notice.
 </para>

</chapter>
<!-- Keep this comment at the end of the file
Local variables:
mode: sgml
sgml-omittag:t
sgml-shorttag:t
sgml-minimize-attributes:nil
sgml-always-quote-attributes:t
sgml-indent-step:1
sgml-indent-data:t
indent-tabs-mode:nil
sgml-parent-document:nil
sgml-default-dtd-file:"~/.phpdoc/manual.ced"
sgml-exposed-tags:nil
sgml-local-catalogs:nil
sgml-local-ecat-files:nil
End:
vim600: syn=xml fen fdm=syntax fdl=2 si
vim: et tw=78 syn=sgml
vi: ts=1 sw=1
-->