Neil Shen

tests: stop all fake-gcs-server and oauth during stop services

12

### What problem does this PR solve? This PR tries to prevent fake gcs server port conflicting by randomlize port number. Close #795 ### Check List Tests - Integration test...

Test pauseSuite.TestPause fails

13

1. What did you do? If possible, provide a recipe for reproducing the error. ``` [2021-03-02T11:32:19.313Z] FAIL: pause_test.go:62: pauseSuite.TestPause [2021-03-02T11:32:19.313Z] [2021-03-02T11:32:19.313Z] pause_test.go:99: [2021-03-02T11:32:19.313Z] c.Assert(&wg, unblocksBetween, 500*time.Millisecond, 520*time.Millisecond) [2021-03-02T11:32:19.313Z] ... waitGroupPtr...

1. What did you do? If possible, provide a recipe for reproducing the error. https://github.com/pingcap/br/blob/f628f4ea420aee5dcc743806fb1d890d18bec22d/pkg/backup/client.go#L362 We should use the `DumpStatsToJSONBySnapshot` API instead, otherwise, we may backup wrong stats.

Add metrics to BR to monitor internal state

1

## Feature Request ### Describe your feature request related problem: When we debug some abnormal backup/restore jobs, there is no way to know internal state of BR. Add metrics to...

Test testChecksumSuite.TestChecksum fails

1

1. What did you do? If possible, provide a recipe for reproducing the error. ``` [2021-03-03T07:06:36.488Z] FAIL: executor_test.go:52: testChecksumSuite.TestChecksum [2021-03-03T07:06:36.488Z] [2021-03-03T07:06:36.488Z] executor_test.go:53: [2021-03-03T07:06:36.488Z] c.Assert(s.mock.Start(), IsNil) [2021-03-03T07:06:36.488Z] ... value *errors.withStack =...

Unstable test TestAutoSend fails

1

Please answer these questions before submitting your issue. Thanks! 1. What did you do? If possible, provide a recipe for reproducing the error. Run CI on macOS. https://github.com/pingcap/br/pull/730/checks?check_run_id=1806130705#step:5:619 ``` FAIL:...

From BR's heap profile, we find there a lot of memory is consumed by `json.Marshal` and `json.Unmarshal`, we need to find out why and reduce memory. https://gist.github.com/overvenus/4491e987dca672fd62705d29bb254d03

After #665, BR repo contains both BR code and TiDB-Lightning code. There are some packages are very similar, they should be unified to reduce duplicate code and improve maintenance efficiency....

br_tiflash fails with ``` [2020-11-04T04:52:30.624Z] failed to restore, before: 1000; after: 960 ``` https://internal.pingcap.net/idc-jenkins/blue/organizations/jenkins/br_ghpr_unit_and_integration_test/detail/br_ghpr_unit_and_integration_test/3611/pipeline

br_log_restore fails with `start ts:421355834788020227 is greater than resolved ts:0`

1

br_log_restore fails https://internal.pingcap.net/idc-jenkins/blue/organizations/jenkins/br_ghpr_unit_and_integration_test/detail/br_ghpr_unit_and_integration_test/4116/pipeline#step-1318-log-326 ``` [2020-12-07T12:43:29.384Z] [2020/12/07 20:43:26.743 +08:00] [ERROR] [restore.go:37] ["failed to restore"] [error="start ts:421355834788020227 is greater than resolved ts:0: [BR:Restore:ErrRestoreResolvedTsConstrain]resolved ts constrain violation"] [errorVerbose="[BR:Restore:ErrRestoreResolvedTsConstrain]resolved ts constrain violation\nstart ts:421355834788020227 is...