Dynamically modify the location of the slowlog file

[wxiaoke]

Feature Request

Is your feature request related to a problem? Please describe:

reference：https://github.com/pingcap/tiup/issues/1440

Describe the feature you'd like:

 We can modify the location of the slow log through the following steps
 1. tiup cluster edit-config <cluster-name>
     add or modify related parameter about slow log file
 2. tiup cluster <cluster-name> reload  -R tidb
 3. the configurations take effect

Why the featue is needed:

  customers maybe want to  store slow log in a separate log directory, and store generally log files in another disk.

Describe alternatives you've considered:

Teachability, Documentation, Adoption, Migration Strategy:

[AstroProfundis]

Currently the path of slow log file is under LogDir of the instance, and is set in the startup script. However, TiDB also supports setting location of slow log file in configuration file, but if both command line argument and config file is set, only the value in command line argument is used.

We do not have plan to support custom slow log file location at present (and keep it inside LogDir), but if there's any situation that separated locations of slow log and common log files are necessary, we're open to discuss.

[ptecza]

Hello @AstroProfundis

What do you mean by LogDir? I would like to keep all TiDB related logs in one place, but unfortunately tidb_slow_query.log seems to stored in a different location then the rest logs:

root@my-tidb-node-1:/var/lib/tidb# ls -1 log/tidb/*log
log/tidb/tidb.log
log/tidb/tidb_stderr.log
root@my-tidb-node-1:/var/lib/tidb#
root@my-tidb-node-1:/var/lib/tidb# ls -1 deploy/tidb/*log
tidb_slow_query.log
root@my-tidb-node-1:/var/lib/tidb#

Below you can see my directories design:

root@my-tidb-node-1:/var/lib/tidb# ls -1
data
deploy
log
lost+found
root@my-tidb-node-1:/var/lib/tidb#
root@my-tidb-node-1:/var/lib/tidb# ls -1 deploy/
alertmanager
grafana
monitor
pd
prometheus
tidb
tikv
root@my-tidb-node-1:/var/lib/tidb#
root@my-tidb-node-1:/var/lib/tidb# ls -1 log/
alertmanager
monitor
pd
prometheus
tidb
tikv
root@my-tidb-node-1:/var/lib/tidb#

and a piece of TiDB cluster topology file:

global:
  user: "tidb"
  ssh_port: 22
  deploy_dir: "/var/lib/tidb/deploy"
  data_dir: "/var/lib/tidb/data"
[...]
server_configs:
  tidb:
    enable-telemetry: false
    log.slow-threshold: 300
    log.slow-query-file: /var/lib/tidb/log/tidb/tidb_slow_query.log
    performance.max-procs: 2
    new_collations_enabled_on_first_bootstrap: true
[...]
tidb_servers:
  - host: my-tidb-node-1.dev-platform.cds
    port: 4000
    status_port: 10080
    deploy_dir: "/var/lib/tidb/deploy/tidb"
    log_dir: "/var/lib/tidb/log/tidb"

tiup command generates the configuration properly:

root@my-tidb-node-1:/var/lib/tidb# cat deploy/tidb/conf/tidb.toml
# WARNING: This file is auto-generated. Do not edit! All your modification will be overwritten!
# You can use 'tiup cluster edit-config' and 'tiup cluster reload' to update the configuration
# All configuration items you want to change can be added to:
# server_configs:
#   tidb:
#     aa.b1.c3: value
#     aa.b2.c4: value
enable-telemetry = false
new_collations_enabled_on_first_bootstrap = true

[log]
slow-query-file = "/var/lib/tidb/log/tidb/tidb_slow_query.log"
slow-threshold = 300

[performance]
max-procs = 2
root@my-tidb-node-1:/var/lib/tidb#

but log.slow-query-file option is being ignored by run_tidb.sh script:

root@my-tidb-node-1:/var/lib/tidb# cat deploy/tidb/scripts/run_tidb.sh
#!/bin/bash
set -e

# WARNING: This file was auto-generated. Do not edit!
#          All your edit might be overwritten!
DEPLOY_DIR=/var/lib/tidb/deploy/tidb

cd "${DEPLOY_DIR}" || exit 1
exec env GODEBUG=madvdontneed=1 bin/tidb-server \
    -P 4000 \
    --status="10080" \
    --host="0.0.0.0" \
    --advertise-address="my-tidb-node-1.example.com" \
    --store="tikv" \
    --path="my-tidb-node-1.example.com:2379,my-tidb-node-2.example.com:2379,my-tidb-node-3.example.com:2379" \
    --log-slow-query="log/tidb_slow_query.log" \
    --config=conf/tidb.toml \
    --log-file="/var/lib/tidb/log/tidb/tidb.log" 2>> "/var/lib/tidb/log/tidb/tidb_stderr.log"
root@my-tidb-node-1:/var/lib/tidb#

[AstroProfundis]

Sorry I might missed the updates.

global:
  user: "tidb"
  ssh_port: 22
  deploy_dir: "/var/lib/tidb/deploy"
  data_dir: "/var/lib/tidb/data"
[...]
server_configs:
  tidb:
    log.slow-query-file: /var/lib/tidb/log/tidb/tidb_slow_query.log
tidb_servers:
  - host: my-tidb-node-1.dev-platform.cds
    log_dir: "/var/lib/tidb/log/tidb"

In this case, the slow log file should be under log_dir: "/var/lib/tidb/log/tidb", and the config in log.slow-query-file: /var/lib/tidb/log/tidb/tidb_slow_query.log should be ignored by tidb-server. As "if both command line argument and config file is set, only the value in command line argument is used."

However, the generated start script is using:

--log-slow-query="log/tidb_slow_query.log"

without the full log_dir as prefix, that's indeed a bug.

I believe it could be fixed by simply update the script template for tidb.