From 5e6265f846011016807c9714cdad1c36418d1791 Mon Sep 17 00:00:00 2001 From: Ulf Wendel Date: Tue, 1 Jul 2014 17:17:37 +0000 Subject: [PATCH] Quickstart addition, outline of XA related setup and one of the XA transaction related functions. git-svn-id: https://svn.php.net/repository/phpdoc/en/trunk@334062 c90b9560-bf6c-de11-be94-00142212c4b1 --- reference/mysqlnd_ms/changes.xml | 32 ++ .../functions/mysqlnd-ms-xa-begin.xml | 156 +++++++ reference/mysqlnd_ms/quickstart.xml | 248 ++++++++++- reference/mysqlnd_ms/setup.xml | 421 +++++++++++++++++- reference/mysqlnd_ms/versions.xml | 1 + 5 files changed, 836 insertions(+), 22 deletions(-) create mode 100644 reference/mysqlnd_ms/functions/mysqlnd-ms-xa-begin.xml diff --git a/reference/mysqlnd_ms/changes.xml b/reference/mysqlnd_ms/changes.xml index 240b27f3aa..5b86a4b8b1 100644 --- a/reference/mysqlnd_ms/changes.xml +++ b/reference/mysqlnd_ms/changes.xml @@ -87,6 +87,38 @@ Feature changes + + + Added a basic distributed transaction abstraction. + XA transactions can are supported ever since using standard SQL calls. + This is inconvenient as XA participants must be managed manually. + PECL/mysqlnd_ms introduces API calls to control XA transaction among MySQL servers. + When using the new functions, PECL/mysqlnd_ms acts as a transaction coordinator. + After starting a distributed transaction, the plugin tracks all servers + involved until the transaction is ended and issues appropriate SQL statements + on the XA participants. + + + This is useful, for example, when using Fabric and sharding. When using + Fabric the actual shard servers involved in a business transaction may not be known + in advance. Thus, manually controlling a transaction that spawns multiple shards becomes + difficult. + + + + + Introduced automatic retry loop for + transient errors and + corresponding statistic + to count the number of implicit retries. Some distributed + database clusters use transient errors to hint a client to retry + its operation in a bit. Most often, the client is then supposed + to halt execution (sleep) for a short moment before retrying the + desired operation. Immediately failing over to another node + is not necessary in response to the error. Instead, a retry loop + can be performed. Common situation when using MySQL Cluster. + + Introduced automatic retry loop for diff --git a/reference/mysqlnd_ms/functions/mysqlnd-ms-xa-begin.xml b/reference/mysqlnd_ms/functions/mysqlnd-ms-xa-begin.xml new file mode 100644 index 0000000000..a6be59db77 --- /dev/null +++ b/reference/mysqlnd_ms/functions/mysqlnd-ms-xa-begin.xml @@ -0,0 +1,156 @@ + + + + + + mysqlnd_ms_xa_begin + Starts a distributed/XA transaction among MySQL servers + + + + &reftitle.description; + + int + mysqlnd_ms_xa_begin + mixedconnection + stringgtrid + + + Starts a XA transaction among MySQL servers. PECL/mysqlnd_ms acts as a transaction + coordinator the distributed transaction. + + + Once a global transaction has been started, the plugin injects appropriate + XA BEGIN SQL statements on all MySQL servers used in the following. + The global transaction is either ended by calling mysqlnd_ms_xa_commit, + mysqlnd_ms_xa_rollback or by an implicit rollback in case + of an error. + + + During a global transaction, the plugin tracks all server switches, + for example, when switching from one MySQL shard to another MySQL shard. + Immediately before a query is run on a server that has not been participating + in the global transaction yet, XA BEGIN is executed on + the server. From a users perspective the injection happens during a call to a + query execution function such as mysqli_query. Should + the injection fail an error is reported to the caller of the query execution + function. The failing server does not become a participant in the global + transaction. The user may retry executing a query on the server and hereby retry + injecting XA BEGIN, abort the global transaction because + not all required servers can participat or, ignore continue the global without + the failed server. + + + Reasons to fail executing XA BEGIN include but are not limited to + a server being unreachable or the server having an open, concurrent + XA transaction using the same xid. + + + Please note, global and local transactions + are mutually exclusive. You cannot start a XA transaction when you have a local + transaction open. The local transaction must be ended first. The plugin tries + to detect this conflict as early as possible. It monitors API calls for controlling + local transactions to learn about the current state. However, if using + SQL statements for local transactions such as BEGIN, the + plugin may not know the current state and the conflict is not detected + before XA BEGIN is injected and executed. + + + The use of other XA resources but MySQL servers is not supported by + the function. To carry out a global transaction among, for example, + a MySQL server and another vendors database system, you should issue + the systems SQL commands yourself. + + + Experimental + + The feature is currently under development. There may be issues and/or + feature limitations. Do not use in production environments. + + + + + + &reftitle.parameters; + + + connection + + + A MySQL connection handle obtained from any of the + connect functions of the mysqli, + mysql or + PDO_MYSQL extensions. + + + + + gtrid + + + Global transaction identifier (gtrid). The gtrid is a binary string + upto 64 bytes long. Please note, depending on your character set settings, + 64 characters may require more than 64 bytes to store. + + + In accordance with the MySQL SQL syntax, + XA transactions use identifiers made of three parts. An xid consists of a + global transaction identifier (gtrid), a branch qualifier (bqual) and a + format identifier (formatID). Only the global transaction identifier can + and needs to be set. + + + The branch qualifier and format identifier are set automatically. The details + should be considered implementation dependent, which may change without + prior notice. In version 1.6 the branch qualifier is consecutive number + which is incremented whenever a participant joins the global transaction. + + + + + + + + &reftitle.returnvalues; + + Returns &true; if there is no open local or global transaction and a new global + transaction can be started. Otherwise, returns &false; + + + + + &reftitle.seealso; + + + + Quickstart XA/Distributed transactions + + + Runtime configuration + + + + + + + + diff --git a/reference/mysqlnd_ms/quickstart.xml b/reference/mysqlnd_ms/quickstart.xml index 973061d3b9..c57ff3caf9 100755 --- a/reference/mysqlnd_ms/quickstart.xml +++ b/reference/mysqlnd_ms/quickstart.xml @@ -215,7 +215,7 @@ mysqlnd_ms.config_file=/path/to/mysqlnd_ms_plugin.ini mysql, and PDO_MYSQL) that is compiled to use the mysqlnd library. - PECL/mysqlnd_ms plugs into the mysqlnd library. + PECL/mysqlnd_ms plugs into the mysqlnd library. It does not change the API or behavior of those extensions. @@ -285,7 +285,7 @@ $mysql = mysql_connect("myapp", "username", "password"); The username, password and schema name are taken from the connect API calls and used for all servers. In other words: you must use the same username and password for every MySQL server listed in a plugin configuration - file section. The is not a general limitation. As of PECL/mysqlnd_ms 1.1.0, + file section. The is not a general limitation. As of PECL/mysqlnd_ms 1.1.0, it is possible to set the username and password for any server in the @@ -622,7 +622,7 @@ $mysqli->close();
- Transactions + Local transactions The current version of the plugin is not transaction safe by default, because it is not aware of running transactions in all cases. SQL transactions are @@ -731,7 +731,7 @@ $mysqli->close(); API based transaction boundary detection has been improved with PHP 5.5.0 and - PECL/mysqlnd_ms 1.5.0 to cover not only calls to mysqli_autocommit + PECL/mysqlnd_ms 1.5.0 to cover not only calls to mysqli_autocommit but also mysqli_begin, mysqli_commit and mysqli_rollback. @@ -819,6 +819,236 @@ $mysqli->close(); section.
+ +
+ XA/Distributed Transactions + + Version requirement + + XA related functions have been introduced in PECL mysqlnd_ms version 1.6.0-alpha. + + + + Experimental + + The feature is currently under development. There may be issues and/or + feature limitations. Do not use in production environments. + + + + XA transactions are a standardized method for executing transactions across + multiple resources. Those resources can be databases or other transactional + systems. The MySQL server supports XA SQL statements which allows users + to carry out a distributed SQL transaction that spawns multiple database servers + or any kind as long as they support the SQL statements too. In such a scenario + it is in the responsibility of the user to coordinate the participating + servers. + + + PECL/mysqlnd_ms can act as a transaction coordinator for a global (distributed, XA) + transaction carried out on MySQL servers only. As a transaction coordinator, the plugin + tracks all servers involved in a global transaction and transparently issues + appropriate SQL statements on the participants. The global transactions are controlled with + mysqlnd_ms_xa_begin, mysqlnd_ms_xa_commit + and mysqlnd_ms_xa_rollback. SQL details are mostly hidden from + the application as is the need to track and coordinate participants. + + + + General pattern for XA transactions + +errno, $mysqli->error)); +} + +/* run queries as usual: XA BEGIN will be injected upon running a query */ +if (!$mysqli->query("INSERT INTO orders(order_id, item) VALUES (1, 'christmas tree, 1.8m')")) { + /* Either INSERT failed or the injected XA BEGIN failed */ + if ('XA' == substr($mysqli->sqlstate, 0, 2)) { + printf("Global transaction/XA related failure, [%d] %s\n", $mysqli->errno, $mysqli->error); + } else { + printf("INSERT failed, [%d] %s\n", $mysqli->errno, $mysqli->error); + } + /* rollback global transaction */ + mysqlnd_ms_xa_rollback($mysqli, $xid); + die("Stopping."); +} + +/* continue carrying out queries on other servers, e.g. other shards */ + +/* commit the global transaction */ +if (!mysqlnd_ms_xa_commit($mysqli, $xa_id)) { + printf("[%d] %s\n", $mysqli->errno, $mysqli->error); +} +?> +]]> + + + + + Unlike with local transactions, which are carried out on a single server, + XA transactions have an identifier (xid) associated with them. + The XA transaction identifier is composed of a global transaction + identifier (gtrid), a branch qualifier (bqual) + a format identifier (formatID). Only the global transaction identifier can + and must be given when calling any of the plugins XA functions. + + + Once a global transaction has been started, the plugin begins tracking servers + until the global transaction ends. When a server is picked for query execution, + the plugin injects the SQL statement XA BEGIN prior to + executing the actual SQL statement on the server. XA BEGIN + makes the server participate in the global transaction. If the injected + SQL statement fails, the plugin will report the issue in reply to the query + execution function that was used. In the above example, + $mysqli->query("INSERT INTO orders(order_id, item) VALUES (1, 'christmas tree, 1.8m')") + would indicate such an error. You may want to check the errors SQL state code to + determine whether the actual query (here: INSERT) has failed + or the error is related to the global transaction. It is up to you to ignore the + failure to start the global transaction on a server and continue execution + without having the server participate in the global transaction. + + + + Local and global transactions are mutually exclusive + +begin_transaction()) + die(sprintf("[%d/%s] %s\n", $mysqli->errno, $mysqli->sqlstate, $mysqli->error)); + +/* cannot start global transaction now - must end local transaction first */ +$gtrid_id = "12345"; +if (!mysqlnd_ms_xa_begin($mysqli, $gtrid_id)) { + die(sprintf("[%d/%s] %s\n", $mysqli->errno, $mysqli->sqlstate, $mysqli->error)); +} +?> +]]> + + &example.outputs; + + + + + + + A global transaction cannot be started when a local transaction is active. + The plugin tries to detect this situation as early as possible, that is when + mysqlnd_ms_xa_begin is called. If using API calls only to + control transactions, the plugin will know that a local transaction is open and + return an error for mysqlnd_ms_xa_begin. However, note the + plugins limitations on detecting + transaction boundaries.. In the worst case, if using direct SQL + for local transactions (BEGIN, + COMMIT, ...), it may happen that an error is delayed + until some SQL is executed on a server. + + + To end a global transaction invoke mysqlnd_ms_xa_commit or + mysqlnd_ms_xa_rollback. When a global transaction is ended + all participants must be informed of the end. Therefore, PECL/mysqlnd_ms + transparently issues appropriate XA related SQL statements + on some or all of them. Any failure during this phase will cause an implicit + rollback. The XA related API is intentionally kept simple here. A more + complex API that gave more control would bare few, if any, advantages over + a user implementation that issues all lower level XA SQL statements itself. + + + XA transactions use the two-phase commit protocol. The two-phase commit + protocol is a blocking protocol. There are cases when no progress can + be made, not even when using timeouts. Transaction coordinators + should survive their own failure, be able to detect blockades and break ties. + PECL/mysqlnd_ms takes the role of a transaction coordinator and can be + configured to survive its own crash to avoid issues with blocked MySQL servers. + Therefore, the plugin can and should be configured to use a persistent and crash-safe state + to allow garbage collection of unfinished, aborted global transactions. + A global transaction can be aborted in an open state if either the plugin fails (crashes) + or a connection from the plugin to a global transaction participant fails. + + + + Transaction coordinator state store + + + + + + + Currently, PECL/mysqlnd_ms supports only using MySQL database tables + as a state store. The SQL definitions of the tables are given in the + plugin configuration section. + Please, make sure to use a transactional and crash-safe + storage engine for the tables, such as InnoDB. InnoDB is the default + table engine in recent versions of the MySQL server. Make also sure + the database server itself is higly available. + + + If a state store has been configured, the plugin can perform a garbage collection. + During garbage collection it may be necessary to connect to a participant + of a failed global transaction. Thus, the state store holds a list of participants + and, among others, their host names. If the garbage collection is run + on another host but the one that has written a participant entry with the + host name localhost, then localhost + resolves to different machines. There are two solutions to the problem. + Either you do not configure any servers with the host name localhost but + configure an IP address (and port) or, you hint the garbage collection. + In the above example, localhost is used for + master_0, hence it may not resolve to the correct + host during garbage collection. However, participant_localhost_ip + is also set to hint the garbage collection that localhost + stands for the IP 192.168.2.12. + +
+
Service level and consistency @@ -1116,14 +1346,14 @@ PHP Warning: mysqli::query(): (mysqlnd_ms) No connection selected by the last f As of MySQL 5.6.5-m8 release candidate the MySQL server features built-in global transaction identifiers. - The MySQL built-in global transaction ID feature is supported by PECL/mysqlnd_ms 1.3.0-alpha or + The MySQL built-in global transaction ID feature is supported by PECL/mysqlnd_ms 1.3.0-alpha or later. However, the final feature set found in MySQL 5.6 production releases to date is not sufficient to support the ideas discussed below in all cases. Please, see also the concepts section. - PECL/mysqlnd_ms can either use its own global transaction ID emulation or the + PECL/mysqlnd_ms can either use its own global transaction ID emulation or the global transaction ID feature built-in to MySQL 5.6.5-m8 or later. From a developer perspective the client-side and server-side approach offer the same features with regards to service levels provided by PECL/mysqlnd_ms. Their differences @@ -1131,7 +1361,7 @@ PHP Warning: mysqli::query(): (mysqlnd_ms) No connection selected by the last f The quickstart first demonstrates the use of the client-side global transaction ID emulation - built-in to PECL/mysqlnd_ms before its show how to use the server-side counterpart. + built-in to PECL/mysqlnd_ms before its show how to use the server-side counterpart. The order ensures that the underlying idea is discussed first. @@ -1148,7 +1378,7 @@ PHP Warning: mysqli::query(): (mysqlnd_ms) No connection selected by the last f already. - PECL/mysqlnd_ms can inject SQL for every committed transaction to increment a GTID counter. + PECL/mysqlnd_ms can inject SQL for every committed transaction to increment a GTID counter. The so created GTID is accessible by the application to identify an applications write operation. This enables the plugin to deliver session consistency (read your writes) service level by not only querying masters but also slaves which have replicated @@ -1545,7 +1775,7 @@ var_dump($res->fetch_assoc()); Databases clusters can deliver different levels of consistency. As of - PECL/mysqlnd_ms 1.2.0 it is possible to advice the plugin to consider only + PECL/mysqlnd_ms 1.2.0 it is possible to advice the plugin to consider only cluster nodes that can deliver the consistency level requested. For example, if using asynchronous MySQL Replication with its cluster-wide eventual consistency, it is possible to request session consistency (read your writes) diff --git a/reference/mysqlnd_ms/setup.xml b/reference/mysqlnd_ms/setup.xml index 03b660f9bd..08d9ada72f 100755 --- a/reference/mysqlnd_ms/setup.xml +++ b/reference/mysqlnd_ms/setup.xml @@ -421,7 +421,7 @@ Fatal error: Call to a member function fetch_assoc() on a non-object in Command to set the error type used to display buffered errors. By default an error of type E_WARNING will be emitted. - + Improved configuration file validation since 1.5.0 @@ -731,7 +731,7 @@ Warning: mysqli::mysqli(): (mysqlnd_ms) Exclusive usage of configuration enforce Since 1.2.0. - + wait_for_gtid_timeout @@ -912,7 +912,7 @@ Warning: mysqli::mysqli(): (mysqlnd_ms) Exclusive usage of configuration enforce mysqlnd_ms_fabric_select_global, set the boolean flag trx_warn_server_list_changes. - + Warnings about the violation of transaction boundaries @@ -1560,8 +1560,8 @@ PHP Warning: mysqli::query(): Couldn't fetch mysqli in filter_warning.php on li - - + + Using a callback - Filter: node_groups @@ -1783,7 +1782,7 @@ function pick_server($connected, $query, $masters, $slaves, $last_used_connectio must be followed by other filters to reduce the number of candidates down to one for statement execution. - + @@ -1797,10 +1796,10 @@ function pick_server($connected, $query, $masters, $slaves, $last_used_connectio - - user defined node group name - - + + user defined node group name + + One or more node groups must be defined. A node group can have an arbitrary user defined name. The name is used in combination with @@ -2046,6 +2045,7 @@ function pick_server($connected, $query, $masters, $slaves, $last_used_connectio + failover @@ -2233,6 +2233,7 @@ function pick_server($connected, $query, $masters, $slaves, $last_used_connectio + lazy_connections @@ -2294,6 +2295,7 @@ function pick_server($connected, $query, $masters, $slaves, $last_used_connectio + server_charset @@ -2396,6 +2398,7 @@ $mysqli->set_charset("latin1"); + master_on_write @@ -2453,6 +2456,7 @@ $mysqli->set_charset("latin1"); + trx_stickiness @@ -2724,10 +2728,401 @@ $mysqli->set_charset("latin1"); + + + xa + object + + + + The setting has been introduced in 1.6.0. + + + + Experimental + + The feature is currently under development. There may be issues and/or + feature limitations. Do not use in production environments. + + + + + + + + + + Keyword + Description + + + + + + state_store + + + + + + + + + Keyword + Description + + + + + + record_participant_credentials + + + + Whether to store the username and password of a global transaction + participant in the participants table. If disabled, the garbage + collection will use the default username and password when connecting + to the participants. Unless you are using a different username and + password for each of your MySQL servers, you can use the default + and avoid storing the sensible information in state store. + + + Please note, username and password are stored in clear text when using + the MySQL state store, which is the only one available. It is in your + responsibility to protect this sensible information. + + + Default: false + + + + + + participant_localhost_ip + + + + During XA garbage collection the plugin may find a participant server + for which the host localhost + has been recorded. If the garbage collection takes place on another host + but the host that has written the participant record to the state store, + the host name localhost now resolves to a different + host. Therefore, when recording a participant servers host name + in the state store, a value of localhost must + be replaced with the actual IP address of localhost. + + + Setting participant_localhost_ip should be considered + only if using localhost cannot be avoided. + From a garbage collection point of view only, it is preferrable not to + configure any socket connection but to provide an IP address and port + for a node. + + + + + + mysql + + + + + The MySQL state store is the only state store available. + + + + + + + + Keyword + Description + + + + + + global_trx_table + + + + Name of the MySQL table used to store the state of an ongoing or aborted + global transaction. Use the below SQL statement to create the table. + Make sure to edit the table name to match your configuration. + + + + SQL definition for the MySQL state store transaction table + + + + + + + Default: mysqlnd_ms_xa_trx + + + + + + participant_table + + + + Name of the MySQL table used to store participants of an ongoing or aborted + global transaction. Use the below SQL statement to create the table. + Make sure to edit the table name to match your configuration. + + + Storing credentials can be enabled and disabled using + record_participant_credentials + + + + SQL definition for the MySQL state store transaction table + + + + + + + Default: mysqlnd_ms_xa_participants + + + + + + garbage_collection_table + + + + Name of the MySQL table used to track and synchronize garbage collection runs. + Use the below SQL statement to create the table. + Make sure to edit the table name to match your configuration. + + + + SQL definition for the MySQL state store garbage collection table + + + + + + + Default: mysqlnd_ms_xa_gc + + + + + + host + + + + Host name of the MySQL server. + + + + + + user + + + + Name of the user used to connect to the MySQL server. + + + + + + password + + + + Password for the MySQL server user. + + + + + + db + + + + Database that holds the garbage collection tables. + Please note, you have to create the garbage collection + tables prior to using the plugin. The tables will not be + created implicitly during runtime but garbage collection + will fail if the tables to not exist. + + + + + + port + + + + Port of the MySQL server. + + + + + + socket + + + + Unix domain socket of the MySQL server. Please note, + if you have multiple PHP servers each of them will + try to carry out garbage collection and need to be able + to connect to the state store. In this case, you may + prefer configuring an IP address and a port for + the MySQL state store server to ensure all PHP servers + can reach it. + + + + + + + + + + + + + + + + garbage_collection + + + + + + + + + + Keyword + Description + Version + + + + + + max_retries + + + + Maximum number of garbage collection runs before giving up. + Allowed values are from 0 to 100. + A setting of 0 means no limit, unless + the state store enforces a limit. Should the state store enforce a limit, + it can be supposed to be significantly higher than 100. + + + Default: 5 + + + Since 1.6.0. + + + + probability + + + + Garbage collection probability. Setting the value has not effect. + The feature is under development. + + + Default: 5 + + + Since 1.6.0. + + + + + + + + + rollback_on_close + + + + Whether to automatically rollback an open global transaction is + a connection is closed. + + + Default: true + + + + + + + + + - -
+
Plugin configuration file (<= 1.0.x) diff --git a/reference/mysqlnd_ms/versions.xml b/reference/mysqlnd_ms/versions.xml index fae5e6f081..11963f86e5 100755 --- a/reference/mysqlnd_ms/versions.xml +++ b/reference/mysqlnd_ms/versions.xml @@ -14,6 +14,7 @@ +