Remove Cluster label in TiMonitor's prometheus scrape job.

[just1900]

Backgroud

In my multi-kubernetes monitoring setup, we are using cluster label as external_labels to indicate an unique kubernetes cluster(I guess there are some more people doing the same🤔), when deploying timonitor and send its metrics to our central monitoring storage, I find some confusing results, cluster label are setted as tidb cluster.

Information I found

After digging into the impl, I find there are two kinds of cluster label providing almost the same information:

1. cluster label: https://github.com/pingcap/tidb-operator/blob/d4c238c3d36377487f2225b627fc38a429e5c219/pkg/monitor/monitor/template.go#L375-L380

2. tidb_cluster label https://github.com/pingcap/tidb-operator/blob/d4c238c3d36377487f2225b627fc38a429e5c219/pkg/monitor/monitor/template.go#L396-L402

Thus, I think it is okay to leave only one of them, namely tidb_cluster. This way, also avoiding the conflict I mentioned above.

Suggestions

I think we can first provide a config to disable this relabel setting for the compatiblity, and fianlly remove it after few versions later when we community reach concensus.

[DanielZhangQD]

Please also take this issue https://github.com/pingcap/tidb-operator/issues/3118 into consideration, not sure if they are related or not.