diff --git a/reference/mysqlnd/book.xml b/reference/mysqlnd/book.xml index 91741a62d1..7f6ee0ec8e 100644 --- a/reference/mysqlnd/book.xml +++ b/reference/mysqlnd/book.xml @@ -26,6 +26,7 @@ &reference.mysqlnd.persist; &reference.mysqlnd.stats; &reference.mysqlnd.notes; +&reference.mysqlnd.plugin; + + MySQL Native Driver Plugin API + + The MySQL Native Driver Plugin API is a feature of MySQL Native + Driver, or mysqlnd. Mysqlnd + plugins operate in the layer between PHP applications and the MySQL + server. This is comparable to MySQL Proxy. MySQL Proxy operates on a + layer between any MySQL client application, for example, a PHP + application and, the MySQL server. Mysqlnd plugins + can undertake typical MySQL Proxy tasks such as load balancing, + monitoring and performance optimizations. Due to the different + architecture and location, mysqlnd plugins do not + have some of MySQL Proxy's disadvantages. For example, with plugins, + there is no single point of failure, no dedicated proxy server to + deploy, and no new programming language to learn (Lua). + + + A mysqlnd plugin can be thought of as an extension + to mysqlnd. Plugins can intercept the majority of + mysqlnd functions. The mysqlnd + functions are called by the PHP MySQL extensions such as + ext/mysql, ext/mysqli, and + PDO_MYSQL. As a result, it is possible for a + mysqlnd plugin to intercept all calls made to these + extensions from the client application. + + + Internal mysqlnd function calls can also be + intercepted, or replaced. There are no restrictions on manipulating + mysqlnd internal function tables. It is possible to + set things up so that when certain mysqlnd + functions are called by the extensions that use + mysqlnd, the call is directed to the appropriate + function in the mysqlnd plugin. The ability to + manipulate mysqlnd internal function tables in this + way allows maximum flexibility for plugins. + + + Mysqlnd plugins are in fact PHP Extensions, written + in C, that use the mysqlnd plugin API (which is + built into MySQL Native Driver, mysqlnd). Plugins + can be made 100% transparent to PHP applications. No application + changes are needed because plugins operate on a different layer. The + mysqlnd plugin can be thought of as operating in a + layer below mysqlnd. + + + The following list represents some possible applications of + mysqlnd plugins. + + + + + Load Balancing + + + + + Read/Write Splitting. An example of this is the PECL/mysqlnd_ms + (Master Slave) extension. This extension splits read/write queries + for a replication setup. + + + + + Failover + + + + + Round-Robin, least loaded + + + + + + + Monitoring + + + + + Query Logging + + + + + Query Analysis + + + + + Query Auditing. An example of this is the PECL/mysqlnd_sip (SQL + Injection Protection) extension. This extension inspects queries + and executes only those that are allowed according to a ruleset. + + + + + + + Performance + + + + + Caching. An example of this is the PECL/mysqlnd_qc (Query Cache) + extension. + + + + + Throttling + + + + + Sharding. An example of this is the PECL/mysqlnd_mc (Multi + Connect) extension. This extension will attempt to split a SELECT + statement into n-parts, using SELECT ... LIMIT part_1, SELECT + LIMIT part_n. It sends the queries to distinct MySQL servers and + merges the result at the client. + + + + + + + MySQL Native Driver Plugins Available + + + There are a number of mysqlnd plugins already available. These + include: + + + + + PECL/mysqlnd_mc - Multi Connect + plugin. + + + + + PECL/mysqlnd_ms - Master Slave + plugin. + + + + + PECL/mysqlnd_qc - Query Cache + plugin. + + + + + PECL/mysqlnd_pscache - Prepared + Statement Handle Cache plugin. + + + + + PECL/mysqlnd_sip - SQL Injection + Protection plugin. + + + + + PECL/mysqlnd_uh - User Handler + plugin. + + + +
+ A comparison of mysqlnd plugins with MySQL Proxy + + Mysqlnd plugins and MySQL Proxy are different + technologies using different approaches. Both are valid tools for + solving a variety of common tasks such as load balancing, monitoring, + and performance enhancements. An important difference is that MySQL + Proxy works with all MySQL clients, whereas + mysqlnd plugins are specific to PHP applications. + + + As a PHP Extension, a mysqlnd plugin gets + installed on the PHP application server, along with the rest of PHP. + MySQL Proxy can either be run on the PHP application server or can be + installed on a dedicated machine to handle multiple PHP application + servers. + + + Deploying MySQL Proxy on the application server has two advantages: + + + + + No single point of failure + + + + + Easy to scale out (horizontal scale out, scale by client) + + + + + MySQL Proxy (and mysqlnd plugins) can solve + problems easily which otherwise would have required changes to + existing applications. + + + However, MySQL Proxy does have some disadvantages: + + + + + MySQL Proxy is a new component and technology to master and deploy. + + + + + MySQL Proxy requires knowledge of the Lua scripting language. + + + + + MySQL Proxy can be customized with C and Lua programming. Lua is the + preferred scripting language of MySQL Proxy. For most PHP experts Lua + is a new language to learn. A mysqlnd plugin can + be written in C. It is also possible to write plugins in PHP using + PECL/mysqlnd_uh. + + + MySQL Proxy runs as a daemon - a background process. MySQL Proxy can + recall earlier decisions, as all state can be retained. However, a + mysqlnd plugin is bound to the request-based + lifecycle of PHP. MySQL Proxy can also share one-time computed + results among multiple application servers. A + mysqlnd plugin would need to store data in a + persistent medium to be able to do this. Another daemon would need to + be used for this purpose, such as Memcache. This gives MySQL Proxy an + advantage in this case. + + + MySQL Proxy works on top of the wire protocol. With MySQL Proxy you + have to parse and reverse engineer the MySQL Client Server Protocol. + Actions are limited to those that can be achieved by manipulating the + communication protocol. If the wire protocol changes (which happens + very rarely) MySQL Proxy scripts would need to be changed as well. + + + Mysqlnd plugins work on top of the C API, which + mirrors the libmysql client and Connector/C APIs. + This C API is basically a wrapper around the MySQL Client Server + protocol, or wire protocol, as it is sometimes called. You can + intercept all C API calls. PHP makes use of the C API, therefore you + can hook all PHP calls, without the need to program at the level of + the wire protocol. + + + Mysqlnd implements the wire protocol. Plugins can + therefore parse, reverse engineer, manipulate and even replace the + communication protocol. However, this is usually not required. + + + As plugins allow you to create implementations that use two levels (C + API and wire protocol), they have greater flexibility than MySQL + Proxy. If a mysqlnd plugin is implemented using + the C API, any subsequent changes to the wire protocol do not require + changes to the plugin itself. + +
+
+ Obtaining the mysqlnd plugin API + + The mysqlnd plugin API is simply part of the MySQL + Native Driver PHP extension, ext/mysqlnd. + Development started on the mysqlnd plugin API in + December 2009. It is developed as part of the PHP source repository, + and as such is available to the public either via SVN, or through + source snapshot downloads. + + + The following table shows PHP versions and the corresponding + mysqlnd version contained within. + + + + + + PHP Version + MySQL Native Driver version + + + + + 5.3.0 + 5.0.5 + + + 5.3.1 + 5.0.5 + + + 5.3.2 + 5.0.7 + + + 5.3.3 + 5.0.7 + + + 5.3.4 + 5.0.7 + + + + + + Plugin developers can determine the mysqlnd + version through accessing MYSQLND_VERSION, which + is a string of the format mysqlnd 5.0.7-dev - 091210 - + $Revision: 300535, or through + MYSQLND_VERSION_ID, which is an integer such as + 50007. Developers can calculate the version number as follows: + + + + + + Version (part) + Example + + + + + Major*10000 + 5*10000 = 50000 + + + Minor*100 + 0*100 = 0 + + + Patch + 7 = 7 + + + MYSQLND_VERSION_ID + 50007 + + + + + + During development, developers should refer to the + mysqlnd version number for compatibility and + version tests, as several iterations of mysqlnd + could occur during the lifetime of a PHP development branch with a + single PHP version number. + +
+
+ MySQL Native Driver Plugin Architecture + + This section provides an overview of the mysqlnd + plugin architecture. + + + MySQL Native Driver Overview + + + Before developing mysqlnd plugins, it is useful to + know a little of how mysqlnd itself is organized. + Mysqlnd consists of the following modules: + + + + + + Modules Statistics + mysqlnd_statistics.c + + + Connection + mysqlnd.c + + + Resultset + mysqlnd_result.c + + + Resultset Metadata + mysqlnd_result_meta.c + + + Statement + mysqlnd_ps.c + + + Network + mysqlnd_net.c + + + Wire protocol + mysqlnd_wireprotocol.c + + + + + + C Object Oriented Paradigm + + + At the code level, mysqlnd uses a C pattern for + implementing object orientation. + + + In C you use a struct to represent an object. + Members of the struct represent object properties. Struct members + pointing to functions represent methods. + + + Unlike with other languages such as C++ or Java, there are no fixed + rules on inheritance in the C object oriented paradigm. However, + there are some conventions that need to be followed that will be + discussed later. + + + The PHP Life Cycle + + + When considering the PHP life cycle there are two basic cycles: + + + + + PHP engine startup and shutdown cycle + + + + + Request cycle + + + + + When the PHP engine starts up it will call the module initialization + (MINIT) function of each registered extension. This allows each + module to setup variables and allocate resources that will exist for + the lifetime of the PHP engine process. When the PHP engine shuts + down it will call the module shutdown (MSHUTDOWN) function of each + extension. + + + During the lifetime of the PHP engine it will receive a number of + requests. Each request constitutes another life cycle. On each + request the PHP engine will call the request initialization function + of each extension. The extension can perform any variable setup and + resource allocation required for request processing. As the request + cycle ends the engine calls the request shutdown (RSHUTDOWN) function + of each extension so the extension can perform any cleanup required. + + + How a plugin works + + + A mysqlnd plugin works by intercepting calls made + to mysqlnd by extensions that use + mysqlnd. This is achieved by obtaining the + mysqlnd function table, backing it up, and + replacing it by a custom function table, which calls the functions of + the plugin as required. + + + The following code shows how the mysqlnd function + table is replaced: + + +query = MYSQLND_METHOD(my_conn_class, query); +} +]]> + + + Connection function table manipulations must be done during Module + Initialization (MINIT). The function table is a global shared + resource. In an multi-threaded environment, with a TSRM build, the + manipulation of a global shared resource during the request + processing will almost certainly result in conflicts. + + + + Do not use any fixed-size logic when manipulating the + mysqlnd function table: new methods may be added + at the end of the function table. The function table may change at + any time in the future. + + + + Calling parent methods + + + If the original function table entries are backed up, it is still + possible to call the original function table entries - the parent + methods. + + + In some cases, such as for + Connection::stmt_init(), it is vital to call the + parent method prior to any other activity in the derived method. + + + + + + Extending properties + + + A mysqlnd object is represented by a C struct. It + is not possible to add a member to a C struct at run time. Users of + mysqlnd objects cannot simply add properties to + the objects. + + + Arbitrary data (properties) can be added to a + mysqlnd objects using an appropriate function of + the + mysqlnd_plugin_get_plugin_<object>_data() + family. When allocating an object mysqlnd reserves + space at the end of the object to hold a void * + pointer to arbitrary data. mysqlnd reserves space + for one void * pointer per plugin. + + + The following table shows how to calculate the position of the + pointer for a specific plugin: + + + + + + Memory address + Contents + + + 0 + Beginning of the mysqlnd object C struct + + + n + End of the mysqlnd object C struct + + + n + (m x sizeof(void*)) + void* to object data of the m-th plugin + + + + + + If you plan to subclass any of the mysqlnd object + constructors, which is allowed, you must keep this in mind! + + + The following code shows extending properties: + + +persistent); + (*props)->query_counter = 0; + } + return props; +} +]]> + + + The plugin developer is responsible for the management of plugin data + memory. + + + Use of the mysqlnd memory allocator is recommended + for plugin data. These functions are named using the convention: + mnd_*loc(). The mysqlnd + allocator has some useful features, such as the ability to use a + debug allocator in a non-debug build. + + + When and how to subclass + + + + + + + When to subclass? + Each instance has its own private function table? + How to subclass? + + + Connection (MYSQLND) + MINIT + No + mysqlnd_conn_get_methods() + + + Resultset (MYSQLND_RES) + MINIT or later + Yes + mysqlnd_result_get_methods() or object method function table + manipulation + + + Resultset Meta (MYSQLND_RES_METADATA) + MINIT + No + mysqlnd_result_metadata_get_methods() + + + Statement (MYSQLND_STMT) + MINIT + No + mysqlnd_stmt_get_methods() + + + Network (MYSQLND_NET) + MINIT or later + Yes + mysqlnd_net_get_methods() or object method function table manipulation + + + Wire protocol (MYSQLND_PROTOCOL) + MINIT or later + Yes + mysqlnd_protocol_get_methods() or object method function table + manipulation + + + + + + You must not manipulate function tables at any time later than MINIT + if it is not allowed according to the above table. + + + Some classes contain a pointer to the method function table. All + instances of such a class will share the same function table. To + avoid chaos, in particular in threaded environments, such function + tables must only be manipulated during MINIT. + + + Other classes use copies of a globally shared function table. The + class function table copy is created together with the object. Each + object uses its own function table. This gives you two options: you + can manipulate the default function table of an object at MINIT, and + you can additionally refine methods of an object without impacting + other instances of the same class. + + + The advantage of the shared function table approach is performance. + There is no need to copy a function table for each and every object. + + + Constructors + + + + + + + Allocation, construction, reset + Can be modified? + Caller + + + Connection (MYSQLND) + mysqlnd_init() + No + mysqlnd_connect() + + + Resultset(MYSQLND_RES) + + Allocation: + + + + + Connection::result_init() + + + + + Reset and re-initialized during: + + + + + Result::use_result() + + + + + Result::store_result + + + + Yes, but call parent! + + + + Connection::list_fields() + + + + + Statement::get_result() + + + + + Statement::prepare() (Metadata only) + + + + + Statement::resultMetaData() + + + + + + Resultset Meta (MYSQLND_RES_METADATA) + Connection::result_meta_init() + Yes, but call parent! + Result::read_result_metadata() + + + Statement (MYSQLND_STMT) + Connection::stmt_init() + Yes, but call parent! + Connection::stmt_init() + + + Network (MYSQLND_NET) + mysqlnd_net_init() + No + Connection::init() + + + Wire protocol (MYSQLND_PROTOCOL) + mysqlnd_protocol_init() + No + Connection::init() + + + + + + It is strongly recommended that you do not entirely replace a + constructor. The constructors perform memory allocations. The memory + allocations are vital for the mysqlnd plugin API + and the object logic of mysqlnd. If you do not + care about warnings and insist on hooking the constructors, you + should at least call the parent constructor before doing anything in + your constructor. + + + Regardless of all warnings, it can be useful to subclass + constructors. Constructors are the perfect place for modifying the + function tables of objects with non-shared object tables, such as + Resultset, Network, Wire Protocol. + + + Destruction + + + + + + + Derived method must call parent? + Destructor + + + Connection + yes, after method execution + free_contents(), end_psession() + + + Resultset + yes, after method execution + free_result() + + + Resultset Meta + yes, after method execution + free() + + + Statement + yes, after method execution + dtor(), free_stmt_content() + + + Network + yes, after method execution + free() + + + Wire protocol + yes, after method execution + free() + + + + + + The destructors are the appropriate place to free properties, + mysqlnd_plugin_get_plugin_<object>_data(). + + + The listed destructors may not be equivalent to the actual + mysqlnd method freeing the object itself. However, + they are the best possible place for you to hook in and free your + plugin data. As with constructors you may replace the methods + entirely but this is not recommended. If multiple methods are listed + in the above table you will need to hook all of the listed methods + and free your plugin data in whichever method is called first by + mysqlnd. + + + The recommended method for plugins is to simply hook the methods, + free your memory and call the parent implementation immediately + following this. + + + + Due to a bug in PHP versions 5.3.0 to 5.3.3, plugins do not + associate plugin data with a persistent connection. This is because + ext/mysql and ext/mysqli do + not trigger all the necessary mysqlnd + end_psession() method calls and the plugin may + therefore leak memory. This has been fixed in PHP 5.3.4. + + +
+
+ The mysqlnd plugin API + + The following is a list of functions provided in the + mysqlnd plugin API: + + + + + mysqlnd_plugin_register() + + + + + mysqlnd_plugin_count() + + + + + mysqlnd_plugin_get_plugin_connection_data() + + + + + mysqlnd_plugin_get_plugin_result_data() + + + + + mysqlnd_plugin_get_plugin_stmt_data() + + + + + mysqlnd_plugin_get_plugin_net_data() + + + + + mysqlnd_plugin_get_plugin_protocol_data() + + + + + mysqlnd_conn_get_methods() + + + + + mysqlnd_result_get_methods() + + + + + mysqlnd_result_meta_get_methods() + + + + + mysqlnd_stmt_get_methods() + + + + + mysqlnd_net_get_methods() + + + + + mysqlnd_protocol_get_methods() + + + + + There is no formal definition of what a plugin is and how a plugin + mechanism works. + + + Components often found in plugins mechanisms are: + + + + + A plugin manager + + + + + A plugin API + + + + + Application services (or modules) + + + + + Application service APIs (or module APIs) + + + + + The mysqlnd plugin concept employs these features, + and additionally enjoys an open architecture. + + + No Restrictions + + + A plugin has full access to the inner workings of + mysqlnd. There are no security limits or + restrictions. Everything can be overwritten to implement friendly or + hostile algorithms. It is recommended you only deploy plugins from a + trusted source. + + + As discussed previously, plugins can use pointers freely. These + pointers are not restricted in any way, and can point into another + plugin's data. Simple offset arithmetic can be used to read another + plugin's data. + + + It is recommended that you write cooperative plugins, and that you + always call the parent method. The plugins should always cooperate + with mysqlnd itself. + + + Issues: an example of chaining and + cooperation + + + + + + Extension + mysqlnd.query() pointer + call stack if calling parent + + + ext/mysqlnd + mysqlnd.query() + mysqlnd.query + + + ext/mysqlnd_cache + mysqlnd_cache.query() + + + + mysqlnd_cache.query() + + + + + mysqlnd.query + + + + + + ext/mysqlnd_monitor + mysqlnd_monitor.query() + + + + mysqlnd_monitor.query() + + + + + mysqlnd_cache.query() + + + + + mysqlnd.query + + + + + + + + + In this scenario, a cache (ext/mysqlnd_cache) and + a monitor (ext/mysqlnd_monitor) plugin are loaded. + Both subclass Connection::query(). Plugin + registration happens at MINIT using the logic + shown previously. PHP calls extensions in alphabetical order by + default. Plugins are not aware of each other and do not set extension + dependencies. + + + By default the plugins call the parent implementation of the query + method in their derived version of the method. + + + PHP Extension Recap + + + This is a recap of what happens when using an example plugin, + ext/mysqlnd_plugin, which exposes the + mysqlnd C plugin API to PHP: + + + + + Any PHP MySQL application tries to establish a connection to + 192.168.2.29 + + + + + The PHP application will either use ext/mysql, + ext/mysqli or PDO_MYSQL. All + three PHP MySQL extensions use mysqlnd to + establish the connection to 192.168.2.29. + + + + + Mysqlnd calls its connect method, which has been + subclassed by ext/mysqlnd_plugin. + + + + + ext/mysqlnd_plugin calls the userspace hook + proxy::connect() registered by the user. + + + + + The userspace hook changes the connection host IP from 192.168.2.29 + to 127.0.0.1 and returns the connection established by + parent::connect(). + + + + + ext/mysqlnd_plugin performs the equivalent of + parent::connect(127.0.0.1) by calling the + original mysqlnd method for establishing a + connection. + + + + + ext/mysqlnd establishes a connection and returns + to ext/mysqlnd_plugin. + ext/mysqlnd_plugin returns as well. + + + + + Whatever PHP MySQL extension had been used by the application, it + receives a connection to 127.0.0.1. The PHP MySQL extension itself + returns to the PHP application. The circle is closed. + + + +
+
+ Getting started building a mysqlnd plugin + + It is important to remember that a mysqlnd plugin + is itself a PHP extension. + + + The following code shows the basic structure of the MINIT function + that will be used in the typical mysqlnd plugin: + + +query = MYSQLND_METHOD(mysqlnd_plugin_conn, query); + conn_m->connect = MYSQLND_METHOD(mysqlnd_plugin_conn, connect); +} +]]> + + + + + + Task analysis: from C to userspace + + + + + + Process: + + + + + PHP: user registers plugin callback + + + + + PHP: user calls any PHP MySQL API to connect to MySQL + + + + + C: ext/*mysql* calls mysqlnd method + + + + + C: mysqlnd ends up in ext/mysqlnd_plugin + + + + + C: ext/mysqlnd_plugin + + + + Calls userspace callback + + + + + Or orginal mysqlnd method, if userspace + callback not set + + + + + + + + You need to carry out the following: + + + + + Write a class "mysqlnd_plugin_connection" in C + + + + + Accept and register proxy object through + "mysqlnd_plugin_set_conn_proxy()" + + + + + Call userspace proxy methods from C (optimization - + zend_interfaces.h) + + + + + Userspace object methods can either be called using + call_user_function() or you can operate at a level + closer to the Zend Engine and use + zend_call_method(). + + + Optimization: calling methods from C using + zend_call_method + + + The following code snippet shows the prototype for the + zend_call_method function, taken from + zend_interfaces.h. + + + + + + Zend API supports only two arguments. You may need more, for example: + + + + + + To get around this problem you will need to make a copy of + zend_call_method() and add a facility for + additional parameters. You can do this by creating a set of + MY_ZEND_CALL_METHOD_WRAPPER macros. + + + Calling PHP userspace + + + This code snippet shows the optimized method for calling a userspace + function from C: + + + + + + Calling userspace: simple arguments + + + + + + + Calling userspace: structs as arguments + + + + + + + The first argument of many mysqlnd methods is a C + "object". For example, the first argument of the connect() method is + a pointer to MYSQLND. The struct MYSQLND + represents a mysqlnd connection object. + + + The mysqlnd connection object pointer can be + compared to a standard I/O file handle. Like a standard I/O file + handle a mysqlnd connection object shall be linked + to the userspace using the PHP resource variable type. + + + From C to userspace and back + + + + + + PHP users must be able to call the parent implementation of an + overwritten method. + + + As a result of subclassing it is possible to refine only selected + methods and you can choose to have "pre" or "post" hooks. + + + Buildin class: + mysqlnd_plugin_connection::connect() + + + + +
+ +