php-doc-en/reference/mbstring/reference.xml

<?xml version="1.0" encoding="iso-8859-1"?>
<!-- $Revision: 1.22 $ -->
<!-- Purpose: international -->
<!-- Membership: bundled -->

 <reference id="ref.mbstring">
  <title>Multibyte String Functions</title> 
  <titleabbrev>Multibyte String</titleabbrev>
  <partintro>

   <section id="mbstring.intro">
    &reftitle.intro;
    <para>
     While there are many languages in which every necessary character can
     be represented by a one-to-one mapping to a 8-bit value, there are also
     several languages which require so many characters for written
     communication that cannot be contained within the range a mere byte can
     code. Multibyte character encoding schemes were developed to express
     that many (more than 256) characters in the regular bytewise coding
     system.
    </para>
    <para>
     When you manipulate (trim, split, splice, etc.) strings encoded in a
     multibyte encoding, you need to use special functions since two or more
     consecutive bytes may represent a single character in such encoding
     schemes. Otherwise, if you apply a non-multibyte-aware string function
     to the string, it probably fails to detect the beginning or ending of
     the multibyte character and ends up with a corrupted garbage string that
     most likely loses its original meaning.
    </para>
    <para>
     <literal>mbstring</literal> provides these multibyte specific
     string functions that help you deal with multibyte encodings in PHP,
     which is basically supposed to be used with single byte encodings.
     In addition to that, <literal>mbstring</literal> handles character
     encoding conversion between the possible encoding pairs.
    </para>
    <para>
     <literal>mbstring</literal> is also designed to handle Unicode-based
     encodings such as UTF-8 and UCS-2 and many single-byte encodings
     for convenience (listed below), whereas <literal>mbstring</literal> was
     originally developed for use in Japanese web pages.
    </para>

    <section id="mbstring.php4.req">
     <title>PHP Character Encoding Requirements</title>
     <para>
      Encodings of the following types are safely used with PHP.
      <itemizedlist>
       <listitem>
        <para>
         A singlebyte encoding,
         <itemizedlist>
          <listitem>
           <simpara>
            which has ASCII-compatible (ISO646 compatible) mappings for the
            characters in range of <literal>00h</literal> to
            <literal>7fh</literal>.
           </simpara>
          </listitem>
         </itemizedlist>
        </para>
       </listitem>
       <listitem>
        <para>
         A multibyte encoding,
         <itemizedlist>
          <listitem>
           <simpara>
            which has ASCII-compatible mappings for the characters in range of
            <literal>00h</literal> to <literal>7fh</literal>.
           </simpara>
          </listitem>
          <listitem>
           <simpara>
            which don't use ISO2022 escape sequences.
           </simpara>
          </listitem>
          <listitem>
           <simpara>
            which don't use a value from <literal>00h</literal> to
            <literal>7fh</literal> in any of the compounded bytes
            that represents a single character.
           </simpara>
          </listitem>
         </itemizedlist>  
        </para>
       </listitem>
      </itemizedlist>
     </para>
     <para>
      These are examples of character encodings that are unlikely to work
      with PHP.
      <informalexample>
       <programlisting>
<![CDATA[
JIS, SJIS, ISO-2022-JP, BIG-5
]]>
       </programlisting>
      </informalexample>
     </para>
     <para>
      Although PHP scripts written in any of those encodings might not work,
      especially in the case where encoded strings appear as identifiers
      or literals in the script, you can almost avoid using these encodings
      by setting up the <literal>mbstring</literal>'s transparent encoding
      filter function for incoming HTTP queries.
     </para>
     <note>
      <para>
       It's highly discouraged to use SJIS, BIG5, CP936, CP949 and GB18030 for
       the internal encoding unless you are familiar with the parser, the
       scanner and the character encoding.
      </para>
     </note>
     <note>
      <para>
       If you have some database connected with PHP, it is recommended that
       you use the same character encoding for both database and the
       <literal>internal encoding</literal> for ease of use and better
       performance.
      </para>
      <para>
       If you are using PostgreSQL, the character encoding used in the
       database and the one used in the PHP may differ as it supports
       automatic character set conversion between the backend and the frontend.
      </para>
     </note>
    </section>
   </section>

   &reference.mbstring.configure;

   &reference.mbstring.ini;

   <section id="mbstring.resources">
    &reftitle.resources;
    &no.resource;
   </section>

   &reference.mbstring.constants;

   <section id="mbstring.http">
     <title>HTTP Input and Output</title>
     <para>
      HTTP input/output character encoding conversion may convert
      binary data also. Users are supposed to control character
      encoding conversion if binary data is used for HTTP
      input/output.
     </para>
     <note>
      <para>
       In PHP 4.3.2 or earlier versions, there was a limitation in this
       functionality that <literal>mbstring</literal> does not perform
       character encoding conversion in POST data if the
       <literal>enctype</literal> attribute in the <literal>form</literal>
       element is set to <literal>multipart/form-data</literal>.
       So you have to convert the incoming data by yourself in this case
       if necessary.
      </para>
      <para>
       Beginning with PHP 4.3.3, if <literal>enctype</literal> for HTML form is
       set to <literal>multipart/form-data</literal> and
       <literal>mbstring.encoding_translation</literal> is set to On
       in &php.ini; the POST'ed variables and the names of uploaded files
       will be converted to the internal character encoding as well.
       However, the conversion isn't applied to the query keys.
      </para>
     </note>
     <para>
      <itemizedlist>
       <listitem>
        <simpara>
         HTTP Input
        </simpara>
        <para> 
         There is no way to control HTTP input character
         conversion from PHP script. To disable HTTP input character
         conversion, it has to be done in &php.ini;.
         <example>
          <title>
           Disable HTTP input conversion in &php.ini;
          </title>
          <programlisting role="php">
<![CDATA[
;; Disable HTTP Input conversion
mbstring.http_input = pass
;; Disable HTTP Input conversion (PHP 4.3.0 or higher)
mbstring.encoding_translation = Off
]]>
          </programlisting>
         </example>
        </para>
        <para>
         When using PHP as an Apache module, it is possible to
         override those settings in each Virtual Host directive in
         &httpd.conf; or per directory with &htaccess;. Refer to the <link
          linkend="configuration">Configuration</link> section and
         Apache Manual for details.
        </para>
       </listitem>
       <listitem>
        <simpara>
         HTTP Output
        </simpara>
        <para>
         There are several ways to enable output character encoding
         conversion. One is using &php.ini;, another
         is using <function>ob_start</function> with
         <function>mb_output_handler</function> as
         <literal>ob_start</literal> callback function.
        </para>
        <note>
         <para>
          PHP3-i18n users should note that <literal>mbstring</literal>'s output
          conversion differs from PHP3-i18n. Character encoding is
          converted using output buffer.
         </para>
        </note>
       </listitem>
      </itemizedlist>
     </para>
     <para>
      <example>
       <title>&php.ini; setting example</title>
       <programlisting>
<![CDATA[
;; Enable output character encoding conversion for all PHP pages

;; Enable Output Buffering
output_buffering    = On

;; Set mb_output_handler to enable output conversion
output_handler      = mb_output_handler
]]>
       </programlisting>
      </example>
     </para>
     <para>
      <example>
       <title>Script example</title>
       <programlisting role="php">
<![CDATA[
<?php

// Enable output character encoding conversion only for this page

// Set HTTP output character encoding to SJIS
mb_http_output('SJIS');

// Start buffering and specify "mb_output_handler" as
// callback function
ob_start('mb_output_handler');

?>
]]>
       </programlisting>
      </example>
     </para>
   </section>

   <section id="mbstring.supported-encodings">
    <title>Supported Character Encodings</title>
    <simpara>
     Currently the following character encodings are supported by the
     <literal>mbstring</literal> module. Any of those Character encodings
     can be specified in the <literal>encoding</literal> parameter of
     <literal>mbstring</literal> functions.
    </simpara>
    <para>
     The following character encoding is supported in this PHP
     extension: 
    </para>
    <itemizedlist>
     <listitem><simpara>UCS-4</simpara></listitem>
     <listitem><simpara>UCS-4BE</simpara></listitem>
     <listitem><simpara>UCS-4LE</simpara></listitem>
     <listitem><simpara>UCS-2</simpara></listitem>
     <listitem><simpara>UCS-2BE</simpara></listitem>
     <listitem><simpara>UCS-2LE</simpara></listitem>
     <listitem><simpara>UTF-32</simpara></listitem>
     <listitem><simpara>UTF-32BE</simpara></listitem>
     <listitem><simpara>UTF-32LE</simpara></listitem>
     <listitem><simpara>UTF-16</simpara></listitem>
     <listitem><simpara>UTF-16BE</simpara></listitem>
     <listitem><simpara>UTF-16LE</simpara></listitem>
     <listitem><simpara>UTF-7</simpara></listitem>
     <listitem><simpara>UTF7-IMAP</simpara></listitem>
     <listitem><simpara>UTF-8</simpara></listitem>
     <listitem><simpara>ASCII</simpara></listitem>
     <listitem><simpara>EUC-JP</simpara></listitem>
     <listitem><simpara>SJIS</simpara></listitem>
     <listitem><simpara>eucJP-win</simpara></listitem>
     <listitem><simpara>SJIS-win</simpara></listitem>
     <listitem><simpara>ISO-2022-JP</simpara></listitem>
     <listitem><simpara>JIS</simpara></listitem>
     <listitem><simpara>ISO-8859-1</simpara></listitem>
     <listitem><simpara>ISO-8859-2</simpara></listitem>
     <listitem><simpara>ISO-8859-3</simpara></listitem>
     <listitem><simpara>ISO-8859-4</simpara></listitem>
     <listitem><simpara>ISO-8859-5</simpara></listitem>
     <listitem><simpara>ISO-8859-6</simpara></listitem>
     <listitem><simpara>ISO-8859-7</simpara></listitem>
     <listitem><simpara>ISO-8859-8</simpara></listitem>
     <listitem><simpara>ISO-8859-9</simpara></listitem>
     <listitem><simpara>ISO-8859-10</simpara></listitem>
     <listitem><simpara>ISO-8859-13</simpara></listitem>
     <listitem><simpara>ISO-8859-14</simpara></listitem>
     <listitem><simpara>ISO-8859-15</simpara></listitem>
     <listitem><simpara>byte2be</simpara></listitem>
     <listitem><simpara>byte2le</simpara></listitem>
     <listitem><simpara>byte4be</simpara></listitem>
     <listitem><simpara>byte4le</simpara></listitem>
     <listitem><simpara>BASE64</simpara></listitem>
     <listitem><simpara>HTML-ENTITIES</simpara></listitem>
     <listitem><simpara>7bit</simpara></listitem>
     <listitem><simpara>8bit</simpara></listitem>
     <listitem><simpara>EUC-CN</simpara></listitem>
     <listitem><simpara>CP936</simpara></listitem>
     <listitem><simpara>HZ</simpara></listitem>
     <listitem><simpara>EUC-TW</simpara></listitem>
     <listitem><simpara>CP950</simpara></listitem>
     <listitem><simpara>BIG-5</simpara></listitem>
     <listitem><simpara>EUC-KR</simpara></listitem>
     <listitem><simpara>UHC (CP949)</simpara></listitem>
     <listitem><simpara>ISO-2022-KR</simpara></listitem>
     <listitem><simpara>Windows-1251 (CP1251)</simpara></listitem>
     <listitem><simpara>Windows-1252 (CP1252)</simpara></listitem>
     <listitem><simpara>CP866 (IBM866)</simpara></listitem>
     <listitem><simpara>KOI8-R</simpara></listitem>
    </itemizedlist>
    <para>
     &php.ini; entry, which accepts encoding name,
     accepts &quot;<literal>auto</literal>&quot; and
     &quot;<literal>pass</literal>&quot; also.
     <literal>mbstring</literal> functions, which accepts encoding
     name, and accepts &quot;<literal>auto</literal>&quot;.
    </para>
    <para>
     If &quot;<literal>pass</literal>&quot; is set, no character
     encoding conversion is performed.
    </para>
    <para>
     If &quot;<literal>auto</literal>&quot; is set, it is expanded to
     the list of encodings defined per the <link linkend="mbstring.configuration">NLS</link>.
     For instance, if the NLS is set to <literal>Japanese</literal>,
     the value is assumed to be
     &quot;<literal>ASCII,JIS,UTF-8,EUC-JP,SJIS</literal>&quot;.
    </para>
    <para>
     See also <function>mb_detect_order</function>
    </para>
   </section>
    
   <section id="mbstring.overload">
    <title>
     Function Overloading Feature
    </title>
    <para>
     You might often find it difficult to get an existing PHP application
     work in a given multibyte environment. That's mostly because lots of
     PHP applications out there are written with the standard
     string functions such as <function>substr</function>, which are
     known to not properly handle multibyte-encoded strings.
    </para>
    <para>
     mbstring supports 'function overloading' feature which enables
     you to add multibyte awareness to such an application without
     code modification by overloading multibyte counterparts on
     the standard string functions. For example,
     <function>mb_substr</function> is called instead of
     <function>substr</function> if function overloading is enabled.
     This feature makes it easy to port applications that only support
     single-byte encodings to a multibyte environment in many cases.
    </para>
    <para>
     To use the function overloading, set
     <literal>mbstring.func_overload</literal> in &php.ini; to a
     positive value that represents a combination of bitmasks specifying
     the categories of functions to be overloaded. It should be set
     to 1 to overload the <function>mail</function> function. 2 for string
     functions, 4 for regular expression functions. For example,
     if is set for 7, mail, strings and regular expression functions should
     be overloaded. The list of overloaded functions are shown below.
     <table>
      <title>Functions to be overloaded</title>
      <tgroup cols="3">
       <thead>
        <row>
         <entry>value of mbstring.func_overload</entry>
         <entry>original function</entry>
         <entry>overloaded function</entry>
        </row>
      </thead>
      <tbody>
        <row>
         <entry>1</entry>
         <entry><function>mail</function></entry>
         <entry><function>mb_send_mail</function></entry>
        </row>
        <row>
         <entry>2</entry>
         <entry><function>strlen</function></entry>
         <entry><function>mb_strlen</function></entry>
        </row>
        <row>
         <entry>2</entry>
         <entry><function>strpos</function></entry>
         <entry><function>mb_strpos</function></entry>
        </row>
        <row>
         <entry>2</entry>
         <entry><function>strrpos</function></entry>
         <entry><function>mb_strrpos</function></entry>
        </row>
        <row>
         <entry>2</entry>
         <entry><function>substr</function></entry>
         <entry><function>mb_substr</function></entry>
        </row>
        <row>
         <entry>2</entry>
         <entry><function>strtolower</function></entry>
         <entry><function>mb_strtolower</function></entry>
        </row>
        <row>
         <entry>2</entry>
         <entry><function>strtoupper</function></entry>
         <entry><function>mb_strtoupper</function></entry>
        </row>
        <row>
         <entry>2</entry>
         <entry><function>substr_count</function></entry>
         <entry><function>mb_substr_count</function></entry>
        </row>
        <row>
         <entry>4</entry>
         <entry><function>ereg</function></entry>
         <entry><function>mb_ereg</function></entry>
        </row>
        <row>
         <entry>4</entry>
         <entry><function>eregi</function></entry>
         <entry><function>mb_eregi</function></entry>
        </row>
        <row>
         <entry>4</entry>
         <entry><function>ereg_replace</function></entry>
         <entry><function>mb_ereg_replace</function></entry>
        </row>
        <row>
         <entry>4</entry>
         <entry><function>eregi_replace</function></entry>
         <entry><function>mb_eregi_replace</function></entry>
        </row>
        <row>
         <entry>4</entry>
         <entry><function>split</function></entry>
         <entry><function>mb_split</function></entry>
        </row>
       </tbody>
      </tgroup>
     </table>
    </para>
    <note>
     <para>
      It is not recommended to use the function overloading option in
      the per-directory context, because it's not confirmed yet to be
      stable enough in a production environment and may lead to undefined
      behaviour.
     </para>
    </note>
   </section>

   <section id="mbstring.ja-basic">
    <title>Basics of Japanese multi-byte encodings</title>
    <para>
     It is often said quite hard to figure out how Japanese texts are
     handled in the computer. This is not only because Japanese characters
     can only be represented by multibyte encodings, but because different
     encoding standards are adopted for different purposes / platforms.
     Moreover, not a few character set standards are used there, which
     are slightly different from one another. Those facts have often led
     developers to inevitable mess-up.
    </para>
    <para> 
     To create a working web application that would be put in the Japanese
     environment, it is important to use the proper character encoding and
     character set for the task in hand.
    </para>
    <para>
     <itemizedlist>
      <listitem>
       <simpara>Storage for a character can be up to six bytes</simpara>
      </listitem>
      <listitem>
       <simpara>
        Most of multibyte characters often appear twice as wide as 
        a single-byte character on display. Those characters are called
        "zen-kaku" in Japanese which means "full width", and the other
        (narrower) characters are called "han-kaku" - means half width.
        However the graphical properties of the characters depend on
        the glyphs of the type faces used to display them or print them out.
       </simpara>
      </listitem>
      <listitem>
       <simpara>
        Some character encodings use shift(escape) sequences defined
        in ISO2022 to switch the code map of the specific code area
        (<literal>00h</literal> to <literal>7fh</literal>).
       </simpara>
      </listitem>
      <listitem>
       <simpara>
        ISO-2022-JP should be used in SMTP/NNTP, and headers and entities
        should be reencoded as per RFC requirements. Although those are not
        requisites, it's still a good idea because several popular user
        agents cannot recognize any other encoding methods.
       </simpara>
      </listitem>
      <listitem>
       <simpara>
        Webpages created for mobile phone services such as
        <ulink url="&url.imode;">i-mode</ulink>,
        <ulink url="&url.vlife;">Vodafone live!</ulink>, or <ulink url="&url.ezweb;">EZweb</ulink>
        are supposed to use Shift_JIS.
       </simpara>
      </listitem>
     </itemizedlist>
    </para>
   </section>

   <section id="mbstring.ref">
    <title>References</title>
    <para>
     Multibyte character encoding schemes and the related issues are very
     complicated. There should be too few space to cover in sufficient details.
     Please refer to the following URLs and other resources for
     further readings.
     <itemizedlist>
      <listitem>
       <para>
        Unicode materials
       </para>
       <para>
        <ulink url="&url.unicode;">&url.unicode;</ulink>
       </para>
      </listitem>
      <listitem>
       <para>
        Japanese/Korean/Chinese character information
       </para>
       <para>
        <ulink url="&url.oreilly.cjk-inf;">&url.oreilly.cjk-inf;</ulink>
       </para>
      </listitem>
     </itemizedlist>
    </para>
   </section>
&reference.mbstring.encodings;

  </partintro>

&reference.mbstring.functions;

 </reference>
<!-- Keep this comment at the end of the file
Local variables:
mode: sgml
sgml-omittag:t
sgml-shorttag:t
sgml-minimize-attributes:nil
sgml-always-quote-attributes:t
sgml-indent-step:1
sgml-indent-data:t
indent-tabs-mode:nil
sgml-parent-document:nil
sgml-default-dtd-file:"../../../manual.ced"
sgml-exposed-tags:nil
sgml-local-catalogs:nil
sgml-local-ecat-files:nil
End:
vim600: syn=xml fen fdm=syntax fdl=2 si
vim: et tw=78 syn=sgml
vi: ts=1 sw=1
-->