kennytm

Seems we can use https://pkg.go.dev/github.com/cockroachdb/pebble#DB.EstimateDiskUsage to fetch the disk usage. # Abstract Periodically, before a `WriteRows`, check every engine's total estimated disk usage. If the total disk usage exceeds the...

> By this way, we can avoid overwriting accidentally. could you elaborate how this works?

While AWS S3 still supports both path and virtual-host style addressing (for now), some other S3-compatible cloud provider only supported virtual-host styling ([in particular](https://rclone.org/s3/#s3-force-path-style) Aliyun OSS, Netease COS, Tencent COS),...

Thanks for the report. The error is that scattering failed. It is not fatal but the cluster would not be balanced initially. PTAL @disksing The "not fully replicated" error for...

@glorv that code is used not just by sst_importer, but the entire TiKV code base when interacting with PD. I think you'd better ask the PD team (sig-scheduling) for this.

> Can lightning discover a tidb-host and tikv-importer host from PD? That might not work perfectly, since it won't know which one is closest. I think it could absolutely discover...

@morgo > .. but it could be? :-) So start tikv-importer with the address of a pd server. This means we also need to supply the PD address to `tikv-importer`...

For Lightning via SQL we should have already required the SUPER privilege. For TiDB backend, there is no security flaw as the RBAC system should be able to reject those...

We likely won't do anything for Importer backend. In Local backend, when `on-duplicate` is "error", we can perform a Get() on the (indexed) Batch before calling Set(). If the result...

By default Lightning runs in a non-strict `SQL_Mode`, which would reduce those errors to only warnings. You could reproduce as: ```sql mysql> set sql_mode = ''; Query OK, 0 rows...