tidb-server v2.0.8

Improvements

• Slow down the AUTO_INCREMENT ID increasing speed when the Update statement does not modify the corresponding AUTO_INCREMENT column #7846

Bug fixes

• Quickly create a new etcd session to recover the service when the PD leader goes down #7810
• Fix the issue that the time zone is not considered when the default value of the DateTime type is calculated #7672
• Fix the issue that duplicate key update inserts values incorrectly in some conditions #7685
• Fix the issue that the predicate conditions of UnionScan are not pushed down #7726
• Fix the issue that the time zone is not correctly handled when you add the TIMESTAMP index #7812
• Fix the memory leak issue caused by the statistics module in some conditions #7864
• Fix the issue that the results of ANALYZE cannot be obtained in some abnormal conditions #7871
• Do not fold the function SYSDATE, to ensure the returned results are correct #7894
• Fix the substring_index panic issue in some conditions #7896
• Fix the issue that OUTER JOIN is mistakenly converted to INNER JOIN in some conditions #7899

tidb-server v2.1.0-rc.3

SQL Optimizer

• Fix the incorrect result issue when a statement contains embedded LEFT OUTER JOIN #7689
• Enhance the optimization rule of predicate pushdown on the JOIN statement #7645
• Fix the optimization rule of predicate pushdown for the UnionScan operator #7695
• Fix the issue that the unique key property of the Union operator is not correctly set #7680
• Enhance the optimization rule of constant folding #7696
• Optimize the data source in which the filter is null after propagation to table dual #7756

SQL Execution Engine

• Optimize the performance of read requests in a transaction #7717
• Optimize the cost of allocating Chunk memory in some executors #7540
• Fix the "index out of range" panic caused by the columns where point queries get all NULL values #7790

Server

• Fix the issue that the memory quota in the configuration file does not take effect #7729
• Add the tidb_force_priority system variable to set the execution priority for each statement #7694
• Support using the admin show slow statement to obtain the slow query log #7785

Compatibility

• Fix the issue that the result of charset/collation is incorrect in information_schema.schemata #7751
• Fix the issue that the value of the hostname system variable is empty #7750

Expressions

• Support the init_vecter argument in the AES_ENCRYPT/AES_DECRYPT built-in function #7425
• Fix the issue that the result of Format is incorrect in some expressions #7770
• Support the JSON_LENGTH built-in function #7739
• Fix the incorrect result issue when casting the unsigned integer type to the decimal type #7792

DML

• Fix the issue that the result of the INSERT … ON DUPLICATE KEY UPDATE statement is incorrect while updating the unique key #7675

DDL

• Fix the issue that the index value is not converted between time zones when you create a new index on a new column of the timestamp type #7724
• Support appending new values for the enum type #7767
• Support creating an etcd session quickly, which improves the cluster availability after network isolation #7774

tidb-server v2.1.0-rc.2

SQL Optimizer

• Put forward a proposal of the next generation Planner #7543
• Improve the optimization rules of constant propagation #7276
• Enhance the computing logic of Range to enable it to handle multiple IN or EQUAL conditions simultaneously #7577
• Fix the issue that the estimation result of TableScan is incorrect when Range is empty #7583
• Support the PointGet operator for the UPDATE statement #7586
• Fix the panic issue during the process of executing the FirstRow aggregate function in some conditions #7624

SQL Execution Engine

• Fix the potential DataRace issue when the HashJoin operator encounters an error #7554
• Make the HashJoin operator read the inner table and build the hash table simultaneously #7544
• Optimize the performance of Hash aggregate operators #7541
• Optimize the performance of Join operators #7493, #7433
• Fix the issue that the result of UPDATE JOIN is incorrect when the Join order is changed #7571
• Improve the performance of Chunk’s iterator #7585

Statistics

• Fix the issue that the auto Analyze work repeatedly analyzes the statistics #7550
• Fix the statistics update error that occurs when there is no statistics change #7530
• Use the RC isolation level and low priority when building Analyze requests #7496
• Support enabling statistics auto-analyze on certain period of a day #7570
• Fix the panic issue when logging the statistics information #7588
• Support configuring the number of buckets in the histogram using the ANALYZE TABLE WITH BUCKETS statement #7619
• Fix the panic issue when updating an empty histogram #7640
• Update information_schema.tables.data_length using the statistics information #7657

Server

• Add Trace related dependencies #7532
• Enable the mutex profile feature of Golang #7512
• The Admin statement requires the Super_priv privilege #7486
• Forbid users to Drop crucial system tables #7471
• Switch from juju/errors to pkg/errors #7151
• Complete the functional prototype of SQL Tracing #7016
• Remove the goroutine pool #7564
• Support viewing the goroutine information using the USER1 signal #7587
• Set the internal SQL to high priority while TiDB is started #7616
• Use different labels to filter internal SQL and user SQL in monitoring metrics #7631
• Store the top 30 slow queries in the last week to the TiDB server #7646
• Put forward a proposal of setting the global system time zone for the TiDB cluster #7656
• Enrich the error message of “GC life time is shorter than transaction duration” #7658
• Set the global system time zone when starting the TiDB cluster #7638

Compatibility

• Add the unsigned flag for the Year type #7542
• Fix the issue of configuring the result length of the Year type in the Prepare/Execute mode #7525
• Fix the issue of inserting zero timestamp in the Prepare/Execute mode #7506
• Fix the error handling issue of the integer division #7492
• Fix the compatibility issue when processing ComStmtSendLongData #7485
• Fix the error handling issue during the process of converting string to integer #7483
• Optimize the accuracy of values in the information_schema.columns_in_table table #7463
• Fix the compatibility issue when writing or updating the string type of data using the MariaDB client #7573
• Fix the compatibility issue of aliases of the returned value #7600
• Fix the issue that the NUMERIC_SCALE value of the float type is incorrect in the information_schema.COLUMNS table #7602
• Fix the issue that Parser reports an error when the single line comment is empty #7612

Expressions

• Check the value of max_allowed_packet in the insert function #7528
• Support the built-in function json_contains #7443
• Support the built-in function json_contains_path #7596
• Support the built-in function encode/decode #7622
• Fix the issue that some time related functions are not compatible with the MySQL behaviors in some cases #7636
• Fix the compatibility issue of parsing the time type of data in string #7654
• Fix the issue that the time zone is not considered when computing the default value of the DateTime data #7655

DML

• Set correct last_insert_id in the InsertOnDuplicateUpdate statement #7534
• Reduce the cases of updating the auto_increment_id counter #7515
• Optimize the error message of Duplicate Key #7495
• Fix the insert...select...on duplicate key update issue #7406
• Support the LOAD DATA IGNORE LINES statement #7576

DDL

• Add the DDL job type and the current schema version information in the monitor #7472
• Complete the design of the Admin Restore Table feature #7383
• Fix the issue that the default value of the Bit type exceeds 128 #7249
• Fix the issue that the default value of the Bit type cannot be NULL #7604
• Reduce the interval of checking CREATE TABLE/DATABASE in the DDL queue #7608
• Use the ddl/owner/resign HTTP interface ro release the DDL owner and start electing a new owner #7649

TiKV Go Client

• Support the issue that the Seek operation only obtains Key #7419

Table Partition (Experimental)

• Fix the issue that the Bigint type cannot be used as the partition key #7520
• Support the rollback operation when an issue occurs during adding an index in the partitioned table #7437

tidb-server v2.0.7

New Feature

• Add the PROCESSLIST table in information_schema #7286

Improvements

• Collect more details about SQL statement execution and output the information in the SLOW QUERY log #7364
• Drop the partition information in SHOW CREATE TABLE #7388
• Improve the execution efficiency of the ANALYZE statement by setting it to the RC isolation level and low priority #7500
• Speed up adding a unique index #7562
• Add an option of controlling the DDL concurrency #7563

Bug Fixes

• Fix the issue that USE INDEX(PRIMARY) cannot be used in a table whose primary key is an integer #7298
• Fix the issue that Merge Join and Index Join output incorrect results when the inner row is NULL #7301
• Fix the issue that Join outputs an incorrect result when the chunk size is set too small #7315
• Fix the panic issue caused by a statement of creating a table involving range column #7379
• Fix the issue that admin check table mistakenly reports an error of a time-type column #7457
• Fix the issue that the data with a default value current_timestamp cannot be queried using the = condition #7467
• Fix the issue that the zero-length parameter inserted by using the ComStmtSendLongData command is mistakenly parsed to NULL #7508
• Fix the issue that auto analyze is repeatedly executed in specific scenarios #7556
• Fix the issue that the parser cannot parse a single line comment ended with a newline character #7635

tidb-server v2.1.0-rc.1

• SQL Optimizer

  • Fix the issue that a wrong result is returned after the correlated subquery is decorrelated in some cases #6972
  • Optimize the output result of Explain #7011#7041
  • Optimize the choosing strategy of the outer table for IndexJoin #7019
  • Remove the Plan Cache of the non-PREPARE statement #7040
  • Fix the issue that the INSERT statement is not parsed and executed correctly in some cases #7068
  • Fix the issue that the IndexJoin result is not correct in some cases #7150
  • Fix the issue that the NULL value cannot be found using the unique index in some cases #7163
  • Fix the range computing issue of the prefix index in UTF-8 #7194
  • Fix the issue that result is not correct caused by eliminating the Project operator in some cases #7257
  • Fix the issue that USE INDEX(PRIMARY) cannot be used when the primary key is an integer #7316
  • Fix the issue that the index range cannot be computed using the correlated column in some cases #7357

• SQL Execution Engine

  • Fix the issue that the daylight saving time is not computed correctly in some cases #6823
  • Refactor the aggregation function framework to improve the execution efficiency of the Stream and Hash aggregation operators #6852
  • Fix the issue that the Hash aggregation operator cannot exit normally in some cases #6982
  • Fix the issue that BIT_AND/BIT_OR/BIT_XOR does not handle the non-integer data correctly #6994
  • Optimize the execution speed of the REPLACE INTO statement and increase the performance nearly 10 times #7027
  • Optimize the memory usage of time type data and decrease the memory usage of the time type data by fifty percent #7043
  • Fix the issue that the returned result is mixed with signed and unsigned integers in the UNION statement is not compatible with MySQL #7112
  • Fix the panic issue caused by the too much memory applied by LPAD/RPAD/TO_BASE64/FROM_BASE64/REPEAT #7171 #7266 #7409 #7431
  • Fix the incorrect result when MergeJoin/IndexJoin handles the NULL value #7255
  • Fix the incorrect result of Outer Join in some cases #7288
  • Improve the error message of Data Truncated to facilitate locating the wrong data and the corresponding field in the table #7401
  • Fix the incorrect result for decimal in some cases #7001 #7113 #7202 #7208
  • Optimize the point select performance #6937
  • Prohibit the isolation level of Read Commited to avoid the underlying problem #7211
  • Fix the incorrect result of LTRIM/RTRIM/TRIM in some cases #7291
  • Fix the issue that the MaxOneRow operator cannot guarantee that the returned result does not exceed one row #7375
  • Divide the Coprocessor requests with too many ranges #7454

• Statistics

  • Optimize the mechanism of statistics dynamic collection #6796
  • Fix the issue that Auto Analyze does not work when data is updated frequently #7022
  • Decrease the Write conflicts during the statistics dynamic update process #7124
  • Optimize the cost estimation when the statistics is incorrect #7175
  • Optimize the row count estimation #7233

• Server

  • Fix the bug in loading privilege information #6976
  • Fix the issue that the Kill command is too strict with privilege check #6954
  • Fix the issue of removing some binary numeric types #6922
  • Shorten the output log #7029
  • Handle the mismatchClusterID issue #7053
  • Add the advertise-address configuration item #7078
  • Add the GrpcKeepAlive option #7100
  • Add the connection or Token time monitor #7110
  • Optimize the data decoding performance #7149
  • Add the PROCESSLIST table in INFORMMATION_SCHEMA #7236
  • Fix the order issue when multiple rules are hit in verifying the privilege #7211
  • Change some default values of encoding related system variables to UTF-8 #7198
  • Make the slow query log show more detailed information #7302
  • Support registering tidb-serverrelated information in PD and obtaining this information by HTTP API #7082

• Compatibility

  • Support Session variables warning_count and error_count #6945
  • Add Scope check when reading the system variables #6958
  • Support the MAX_EXECUTION_TIME syntax #7012
  • Support more statements of the SET syntax #7020
  • Add validity check when setting system variables #7117
  • Add the verification of the number of PlaceHolders in the Prepare statement #7162
  • Support set character_set_results = null #7353
  • Support the flush status syntax #7369
  • Fix the column size of SET and ENUM types in information_schema #7347
  • Support the NATIONAL CHARACTER syntax of statements for creating a table #7378
  • Support the CHARACTER SET syntax in the LOAD DATA statement #7391
  • Fix the column information of the SET and ENUM types #7417
  • Support the IDENTIFIED WITH syntax in the CREATE USER statement #7402
  • Fix the precision losing issue during TIMESTAMP computing process #7418
  • Support the validity verification of more SYSTEM variables #7196
  • Fix the incorrect result when the CHAR_LENGTH function computes the binary string #7410
  • Fix the incorrect CONCAT result in a statement involving GROUP BY #7448
  • Fix the imprecise type length issue when casting the DECIMAL type to the STRING type #7451

• DML

  • Fix the stability issue of the Load Data statement #6927
  • Fix the memory usage issue when performing some Batch operations #7086
  • Improve the performance of the Replace Into statement #7027
  • Fix the inconsistent precision issue when writing CURRENT_TIMESTAMP #7355

• DDL

  • Improve the method of DDL judging whether Schema is synchronized to avoid misjudgement in some cases #7319
  • Fix the SHOW CREATE TABLE result in adding index process #6993
  • Allow the default value of text/blob/json to be NULL in non-restrict sql-mode #7230
  • Fix the ADD INDEX issue in some cases #7142
  • Increase the speed of adding UNIQUE-KEY index operation largely #7132
  • Fix the truncating issue of the prefix index in UTF-8 character set #7109
  • Add the environment variable tidb_ddl_reorg_priority to control the priority of the add-index operation #7116
  • Fix the display issue of `AUTO-INCREME...

tidb-server v2.0.6

Improvements

• Make “set system variable” log shorter to save disk space #7031
• Record slow operations during the execution of ADD INDEX in the log, to make troubleshooting easier #7083
• Reduce transaction conflicts when updating statistics #7138
• Improve the accuracy of row count estimation when the values pending to be estimated exceeds the statistics range #7185
• Choose the table with a smaller estimated row count as the outer table for Index Join to improve its execution efficiency #7277
• Add the recover mechanism for panics occurred during the execution of ANALYZE TABLE, to avoid that the tidb-server is unavailable caused by abnormal behavior in the process of collecting statistics #7228
• Return NULL and the corresponding warning when the results of RPAD/LPAD exceed the value of the max_allowed_packet system variable, compatible with MySQL #7244
• Set the upper limit of placeholders count in the PREPARE statement to 65535, compatible with MySQL #7250

Bug Fixes

• Fix the issue that the DROP USER statement is incompatible with MySQL behavior in some cases #7014
• Fix the issue that statements like INSERT/LOAD DATA meet OOM aftering opening tidb_batch_insert #7092
• Fix the issue that the statistics fail to automatically update when the data of a table keeps updating #7093
• Fix the issue that the firewall breaks inactive gPRC connections #7099
• Fix the issue that prefix index returns a wrong result in some scenarios #7126
• Fix the panic issue caused by outdated statistics in some scenarios #7155
• Fix the issue that one piece of index data is missed after the ADD INDEX operation in some scenarios #7156
• Fix the wrong result issue when querying NULL values using the unique index in some scenarios #7172
• Fix the messy code issue of the DECIMAL multiplication result in some scenarios #7212
• Fix the wrong result issue of DECIMAL modulo operation in some scenarios #7245
• Fix the issue that the UPDATE/DELETE statement in a transaction returns a wrong result under some special sequence of statements #7219
• Fix the panic issue of the UNION ALL/UPDATE statement during the process of building the execution plan in some scenarios #7225
• Fix the issue that the range of prefix index is calculated incorrectly in some scenarios #7231
• Fix the issue that the LOAD DATA statement fails to write the binlog in some scenarios #7242
• Fix the wrong result issue of SHOW CREATE TABLE during the execution process of ADD INDEX in some scenarios #7243
• Fix the issue that panic occurs when Index Join does not initialize timestamps in some scenarios #7246
• Fix the false alarm issue when ADMIN CHECK TABLE mistakenly uses the timezone in the session #7258
• Fix the issue that ADMIN CLEANUP INDEX does not clean up the index in some scenarios #7265
• Disable the Read Committed isolation level #7282

tidb-server v2.0.5

New Features

• Add the tidb_disable_txn_auto_retry system variable which is used to disable the automatic retry of transactions #6877

Improvements

• Optimize the cost calculation of Selection to make the result more accurate #6989
• Select the query condition that completely matches the unique index or the primary key as the query path directly #6966
• Execute necessary cleanup when failing to start the service #6964
• Handle \N as NULL in the Load Data statement #6962
• Optimize the code structure of CBO #6953
• Report the monitoring metrics earlier when starting the service #6931
• Optimize the format of slow queries by removing the line breaks in SQL statements and adding user information #6931
• Support multiple asterisks in comments #6931

Bug Fixes

• Fix the issue that KILL QUERY always requires SUPER privilege #6931
• Fix the issue that users might fail to login when the number of users exceeds 1024 #6986
• Fix an issue about inserting unsigned float/double data #6940
• Fix the compatibility of the COM_FIELD_LIST command to resolve the panic issue in some MariaDB clients #6929
• Fix the CREATE TABLE IF NOT EXISTS LIKE behavior #6928
• Fix an issue in the process of TopN pushdown #6923
• Fix the ID record issue of the currently processing row when an error occurs in executing Add Index #6903

tidb-server v2.1.0-beta

• SQL Optimizer
  • Optimize the selection range of Index Join to improve the execution performance
  • Optimize correlated subquery, push down Filter, and extend the index range, to improve the efficiency of some queries by orders of magnitude
  • Support Index Hint and Join Hint in the UPDATE and DELETE statements
  • Validate Hint TIDM_SMJ when no available index exists
  • Support pushdown of the ABS, CEIL, FLOOR, IS TRUE, and IS FALSE functions
  • Handle the IF and IFNULL functions especially in the constant folding process
• SQL Execution Engine
  • Implement parallel Hash Aggregate operators and improve the computing performance of Hash Aggregate by 350% in some scenarios
  • Implement parallel Project operators and improve the performance by 74% in some scenarios
  • Read the data of the Inner table and Outer table of Hash Join concurrently to improve the execution performance
  • Fix incorrect results of INSERT … ON DUPLICATE KEY UPDATE … in some scenarios
  • Fix incorrect results of the CONCAT_WS, FLOOR, CEIL, and DIV built-in functions
• Server
  • Add the HTTP API to scatter the distribution of table Regions in the TiKV cluster
  • Add the auto_analyze_ratio system variable to control the threshold value of automatic Analyze
  • Add the HTTP API to control whether to open the general log
  • Add the HTTP API to modify the log level online
  • Add the user information in the general log and the slow query log
  • Support the server side cursor
• Compatibility
  • Support more MySQL syntax
  • Make the bit aggregate function support the ALL parameter
  • Support the SHOW PRIVILEGES statement
• DML
  • Decrease the memory usage of the INSERT INTO SELECT statement
  • Fix the performance issue of PlanCache
  • Add the tidb_retry_limit system variable to control the automatic retry times of transactions
  • Add the tidb_disable_txn_auto_retry system variable to control whether the transaction tries automatically
  • Fix the accuracy issue of the written data of the time type
  • Support the queue of locally conflicted transactions to optimize the conflicted transaction performance
  • Fix Affected Rows of the UPDATE statement
  • Optimize the statement performance of insert ignore on duplicate key update
• DDL
  • Optimize the execution speed of the CreateTable statement
  • Optimize the execution speed of ADD INDEX and improve it greatly in some scenarios
  • Fix the issue that the number of added columns by Alter table add column exceeds the limit of the number of table columns
  • Fix the issue that DDL job retries lead to an increasing pressure on TiKV in abnormal conditions
  • Fix the issue that TiDB continuously reloads the schema information in abnormal conditions
  • Do not output the FOREIGN KEY related information in the result of SHOW CREATE TABLE
  • Support the select tidb_is_ddl_owner() statement to facilitate judging whether TiDB is DDL Owner
  • Fix the issue that the index is deleted in the Year type in some scenarios
  • Fix the renaming table issue in the concurrent execution scenario
  • Support the AlterTableForce syntax
  • Support the AlterTableRenameIndex syntax with FromKey and ToKey
  • Add the table name and database name in the output information of admin show ddl jobs

tidb-server v2.0.4

New Features

• Support the ALTER TABLE t DROP COLUMN a CASCADE syntax
• Support configuring the value of tidb_snapshot to TSO

Improvements

• Refine the display of statement types in monitoring items
• Optimize the accuracy of query cost estimation
• Configure the backoff max delay parameter of gRPC
• Support configuring the memory threshold of a single statement in the configuration file
• Refactor the error of Optimizer

Bug Fixes

• Fix the side effects of the Cast Decimal data
• Fix the wrong result issue of the Merge Join operator in specific scenarios
• Fix the issue of converting the Null object to String
• Fix the issue of casting the JSON type of data to the JSON type
• Fix the issue that the result order is not consistent with MySQL in the condition of Union + OrderBy
• Fix the compliance rules issue when the Union statement checks the Limit/OrderBy clause
• Fix the compatibility issue of the Union All result
• Fix a bug in predicate pushdown
• Fix the compatibility issue of the Union statement with the For Update clause
• Fix the issue that the concat_ws function mistakenly truncates the result

tidb-server v2.1.0-alpha

add pull request template (#6718)