br icon indicating copy to clipboard operation
br copied to clipboard

Add log to make error more clear when restore error using s3 sink

Open yuqi1129 opened this issue 4 years ago • 0 comments

Please answer these questions before submitting your issue. Thanks!

  1. What did you do? If possible, provide a recipe for reproducing the error.

./br restore cdclog --pd=http://10.142.99.8:2379 --start-ts=422756483571646465 --end-ts=422756525015040000 -s "s3://tidb/tx31?endpoint=http://endpoint"

  1. What did you expect to see?
Error: NoSuchKey: The specified key does not exist.
	status code: 404, request id: e12fcec0-1ca2-0e8c-0000-01777fbe7ce4, host id:

and log is

[2021/02/08 11:45:27.717 +08:00] [ERROR] [main.go:59] ["br failed"] [error="NoSuchKey: The specified key does not exist.\n\tstatus code: 404, request id: e12fcec0-1ca2-0e8c-0000-01777fbe7ce4, host id: "] [errorVerbose="NoSuchKey: The specified key does not exist.\n\tstatus code: 404, request id: e12fcec0-1ca2-0e8c-0000-01777fbe7ce4, host id: \ngithub.com/pingcap/errors.AddStack\n\tgithub.com/pingcap/[email protected]/errors.go:174\ngithub.com/pingcap/errors.Trace\n\tgithub.com/pingcap/[email protected]/juju_adaptor.go:15\ngithub.com/pingcap/br/pkg/storage.(*S3Storage).Read\n\tgithub.com/pingcap/br@/pkg/storage/s3.go:340\ngithub.com/pingcap/br/pkg/restore.(*LogClient).RestoreLogData\n\tgithub.com/pingcap/br@/pkg/restore/log_client.go:930\ngithub.com/pingcap/br/pkg/task.RunLogRestore\n\tgithub.com/pingcap/br@/pkg/task/restore_log.go:136\ngithub.com/pingcap/br/cmd.runLogRestoreCommand\n\tgithub.com/pingcap/br@/cmd/restore.go:36\ngithub.com/pingcap/br/cmd.newLogRestoreCommand.func1\n\tgithub.com/pingcap/br@/cmd/restore.go:132\ngithub.com/spf13/cobra.(*Command).execute\n\tgithub.com/spf13/[email protected]/command.go:842\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\tgithub.com/spf13/[email protected]/command.go:950\ngithub.com/spf13/cobra.(*Command).Execute\n\tgithub.com/spf13/[email protected]/command.go:887\nmain.main\n\tgithub.com/pingcap/br@/main.go:58\nruntime.main\n\truntime/proc.go:203\nruntime.goexit\n\truntime/asm_amd64.s:1357"] [stack="main.main\n\tgithub.com/pingcap/br@/main.go:59\nruntime.main\n\truntime/proc.go:203"]

Which is really too hard to find the root cause, in fact the reasons is s3://tidb/tx31 does not exist

We need to add some log about the s3 configuration and value to help us get the error quickly

  1. What did you see instead?

Log the the possible(i say possible) sections of the error and thus it's convenient for us to figure out the real cause

  1. What version of BR and TiDB/TiKV/PD are you using?

Master branch

yuqi1129 avatar Feb 08 '21 03:02 yuqi1129