docs icon indicating copy to clipboard operation
docs copied to clipboard

tidb-configuration-file: add more notes to txn-entry-size-limit

Open dveeden opened this issue 3 years ago • 2 comments

What is changed, added or deleted? (Required)

Growing the txn-entry-size-limit to the maximum size needs adjustments in other settings as well.

When txn-entry-size-limit is too small:

sql> INSERT INTO t1(t) VALUES(REPEAT('x',6*1024*1024));
ERROR: 8025 (HY000): entry too large, the max entry size is 6291456, the size of data is 6291489

When raft-entry-max-size is too small:

sql> INSERT INTO t1(t) VALUES(REPEAT('x',15*1024*1024));
ERROR: 1105 (HY000): message:"raft entry is too large, region 2, entry size 15728790" raft_entry_too_large:<region_id:2 entry_size:15728790 > 

When max_allowed_packet is too small:

sql> INSERT INTO t1(t) VALUES(REPEAT('x',100*1024*1024));
ERROR: 1301 (HY000): Result of repeat() was larger than max_allowed_packet (67108864) - truncated

When txn-total-size-limit is too small:

sql> INSERT INTO t1(t) VALUES(REPEAT('x',100*1024*1024));
ERROR: 8004 (HY000): Transaction is too large, size: 104857633

Tested with:

tiup playground --tiflash 0 --without-monitor --db.config /tmp/tidb.toml --kv.config /tmp/tikv.toml v6.3.0

tikv.toml:

[raftstore]
raft-entry-max-size = "150MB"

tidb.toml:

[performance]
txn-entry-size-limit = 125829120
txn-total-size-limit = 1073741824

And SET GLOBAL max_allowed_packet=130*1024*1024 (and then reconnect)

Which TiDB version(s) do your changes apply to? (Required)

Tips for choosing the affected version(s):

By default, CHOOSE MASTER ONLY so your changes will be applied to the next TiDB major or minor releases. If your PR involves a product feature behavior change or a compatibility change, CHOOSE THE AFFECTED RELEASE BRANCH(ES) AND MASTER.

For details, see tips for choosing the affected versions.

  • [x] master (the latest development version)
  • [x] v6.4 (TiDB 6.4 versions)
  • [x] v6.3 (TiDB 6.3 versions)
  • [x] v6.1 (TiDB 6.1 versions)
  • [ ] v5.4 (TiDB 5.4 versions)
  • [ ] v5.3 (TiDB 5.3 versions)
  • [ ] v5.2 (TiDB 5.2 versions)
  • [ ] v5.1 (TiDB 5.1 versions)
  • [ ] v5.0 (TiDB 5.0 versions)

What is the related PR or file link(s)?

  • This PR is translated from:
  • Other reference link(s):

Do your changes match any of the following descriptions?

  • [ ] Delete files
  • [ ] Change aliases
  • [ ] Need modification after applied to another branch
  • [ ] Might cause conflicts after applied to another branch

dveeden avatar Oct 07 '22 10:10 dveeden

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • TomShawn
  • qiancai

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment. After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review. Reviewer can cancel approval by submitting a request changes review.

ti-chi-bot avatar Oct 07 '22 10:10 ti-chi-bot

Note that the actual limit might be slightly less. Instead of 6MiB (default config) exactly it is 33 bytes less for the defaults. I don't think this is a problem.

sql> CREATE TABLE t1(id bigint primary key auto_random, t longtext);
Query OK, 0 rows affected, 1 warning (0.1575 sec)
Note (code 1105): Available implicit allocation times: 288230376151711743

sql> INSERT INTO t1(t) VALUES(REPEAT('x',5*1024*1024));
Query OK, 1 row affected (0.0811 sec)

sql> INSERT INTO t1(t) VALUES(REPEAT('x',6*1024*1024));
ERROR: 8025 (HY000): entry too large, the max entry size is 6291456, the size of data is 6291489

sql> SELECT 6291489-6291456;
+-----------------+
| 6291489-6291456 |
+-----------------+
|              33 |
+-----------------+
1 row in set (0.0003 sec)

sql> INSERT INTO t1(t) VALUES(REPEAT('x',(6*1024*1024)-33));
Query OK, 1 row affected (0.0803 sec)

sql> SELECT LENGTH(t), FORMAT_BYTES(LENGTH(t)) FROM t1;
+-----------+-------------------------+
| LENGTH(t) | FORMAT_BYTES(LENGTH(t)) |
+-----------+-------------------------+
|   6291423 | 6.00 MiB                |
|   5242880 | 5.00 MiB                |
+-----------+-------------------------+
2 rows in set (0.0921 sec)

dveeden avatar Oct 07 '22 12:10 dveeden

@TomShawn PTAL

dveeden avatar Oct 10 '22 14:10 dveeden

@TomShawn PTAL

dveeden avatar Oct 18 '22 19:10 dveeden

The ci / tidb-check found this:

 tidb-configuration-file.md
  365:24-365:87  warning  Dead anchor: /system-variables.md#max_allowed_packet  pingcap-docs-anchors  remark-lint

However the link seems fine to me.

dveeden avatar Oct 25 '22 13:10 dveeden

/rebase

TomShawn avatar Oct 25 '22 14:10 TomShawn

The ci / tidb-check found this:

 tidb-configuration-file.md
  365:24-365:87  warning  Dead anchor: /system-variables.md#max_allowed_packet  pingcap-docs-anchors  remark-lint

However the link seems fine to me.

Anchor fixed. If @qiancai has no further comment, let's merge this PR.

TomShawn avatar Oct 25 '22 14:10 TomShawn

/merge

qiancai avatar Oct 26 '22 05:10 qiancai

This pull request has been accepted and is ready to merge.

Commit hash: 64542efb05e0be661cd0b6b20b5ad082cf6619c8

ti-chi-bot avatar Oct 26 '22 05:10 ti-chi-bot

In response to a cherrypick label: new pull request created: #11014.

ti-chi-bot avatar Oct 26 '22 05:10 ti-chi-bot

In response to a cherrypick label: new pull request created: #11015.

ti-chi-bot avatar Oct 26 '22 05:10 ti-chi-bot