server
server copied to clipboard
MariaDB
MDEV-34062 Implement innodb_log_file_mmap on 64-bit systems
- [x] The Jira issue number for this PR is: MDEV-34062
Description
In MariaDB Server 10.11 (but not 10.6), copying the InnoDB write-ahead log (innodb_logfile0) is taking an extremely long time, copying at most 3 MiB/s from a RAM disk.
The problem appears to be connected to the rather small default value innodb_log_buffer_size=2m (log_sys.buf_size). If I specify --innodb-log-buffer-size=1g, then xtrabackup_copy_logfile() will process much more data per a system call, the backup would finish in reasonable time during my Sysbench workload, and the top of perf report would look more reasonable as well.
We can do better read the log file via memory-mapped I/O when available. A fallback for file based I/O will remain.
Release Notes
On 64-bit systems, we will introduce a read-only Boolean parameter innodb_log_file_mmap that that will be OFF by default on most platforms. On Linux and FreeBSD the default is innodb_log_file_mmap=ON. When enabled, the InnoDB crash recovery as well as mariadb-backup will use memory-mapped I/O for reading the ib_logfile0, instead of regular file system I/O.
How can this PR be tested?
The added parameter is lightly covered by the test innodb.log_file_size_online.
The code is covered by existing tests in ./mtr --suite=mariabackup. With this fix, the backup during the following is expected to complete in a few seconds:
sysbench /usr/share/sysbench/oltp_update_index.lua --mysql-socket="$MYSQL_SOCK" --mysql-user="$MYSQL_USER" --mysql-db=test --tables=64 --table_size=100000 &
mariadb-backup -u"$MYSQL_USER" --socket="$MYSQL_SOCK" --backup --target-dir=/dev/shm/backup
In MDEV-34062 you can find results for testing with all 8 combinations of innodb_log_file_mmap (backup and server) and innodb_log_file_buffering (server) when the log resides on a SATA HDD, as well as results for testing the server with each setting.
Basing the PR against the correct MariaDB version
- [ ] This is a new feature and the PR is based against the latest MariaDB development branch.
- [x] This is a bug fix and the PR is based against the earliest maintained branch in which the bug can be reproduced.
PR quality check
- [x] I checked the CODING_STANDARDS.md file and my PR conforms to this where appropriate.
- [ ] For any trivial modifications to the PR, I am ok with the reviewer making the changes themselves.
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.
The top commit db576beedf2a197d16a43cff44959fc0a7ad6a00 is only for better testing this on our CI. It disables the PMEM interface, so that the new mmap logic will be exercised on Linux, and it enables the mmap logic by default on all platforms. (Except in mariadb-backup on Windows.)
As requested by @vaintroub, I am removing the logic to allow the regular ib_logfile0 to be written via a memory mapping. For future reference, I created 0ae45445a065b0e635ab83f27f12bea1512da494 in case someone might want to enable that logic later.