Add TiKV support for cache persistence

[tstromberg]

It seems to be a popular Kubernetes option.

[tstromberg]

If someone wants to take this on, it's pretty simple:

• Copy an existing persistence driver like https://github.com/google/triage-party/blob/master/pkg/persist/postgres.go
• Replace all the m.db calls with the appropriate TiKV calls. For example: https://github.com/tikv/tikv/blob/master/docs/reference/clients/go-client-api.md
• Add the "tikv" driver registration here: https://github.com/google/triage-party/blob/aed733794a73c1f2ae003de2361bf8716541856f/pkg/persist/persist.go#L55

[masihtehrani]

If someone wants to take this on, it's pretty simple:

• Copy an existing persistence driver like https://github.com/google/triage-party/blob/master/pkg/persist/postgres.go
• Replace all the m.db calls with the appropriate TiKV calls. For example: https://github.com/tikv/tikv/blob/master/docs/reference/clients/go-client-api.md
• Add the "tikv" driver registration here: https://github.com/google/triage-party/blob/aed733794a73c1f2ae003de2361bf8716541856f/pkg/persist/persist.go#L55

I started to implement Tikv same Postgres but it has different concept because it just has GET/PUT/DELETE/SCAN operation and got a problem for implement for methods loadItems() & cleanup() because I and find operation for select all and delete all keys values, I think it must be implement the same mem

[tstromberg]

Thanks!

• cleanup() is more of a bad hack than an important feature. With TiKV, you can simply set an expiration time for keys to keep them from filling up the database. For instance, a 30-day expiration should be plenty of time.

• For loadItems(), here's an idea: have Save() store a key (named something like saved-keys) with a list of keys that it has saved. Then your loadItems() implementation can call fetch the value for saved-keys, iterate over the values, and call setMem() on each item.

What do you think?

[masihtehrani]

Thanks!

• cleanup() is more of a bad hack than an important feature. With TiKV, you can simply set an expiration time for keys to keep them from filling up the database. For instance, a 30-day expiration should be plenty of time.
• For loadItems(), here's an idea: have Save() store a key (named something like saved-keys) with a list of keys that it has saved. Then your loadItems() implementation can call fetch the value for saved-keys, iterate over the values, and call setMem() on each item.

What do you think?

Tikv is the tiny project doesn't have TTL and select all it just has Scan(startKey, endKey []byte, limit int) maybe we can do something with scan method. I have some suggestions.

1. I think about we can get all persist data in one key-value with storage with JSON data for example key: persis, value: {"key1":"value1"} and handle in Golang but I don't have an idea about implement TTL and this function m.cleanup(start.Add(-1 * time.Hour)) now.
2. We can use TIDB it has SQL and we can implement same Postgres
3. Chose Another KV store same Redis does have TTL, select all and...

[tstromberg]

OK. Let's skip cleanup(), as it's a private function and not necessary. TTL's aren't necessary either, as they are not part of the API.

For an initial iteration, I say let's just ignore TTL's altogether. If you can figure out how to make Scan work, then great. If not, you can fall back on saving a list of keys to load at a specific key.

If you go the Scan route, in a second iteration, we can implement TTL's in a round-about way by refusing to load values into memory if the creation time is older than a set time (30-days?).

[masihtehrani]

OK. Let's skip cleanup(), as it's a private function and not necessary. TTL's aren't necessary either, as they are not part of the API.

For an initial iteration, I say let's just ignore TTL's altogether. If you can figure out how to make Scan work, then great. If not, you can fall back on saving a list of keys to load at a specific key.

If you go the Scan route, in a second iteration, we can implement TTL's in a round-about way by refusing to load values into memory if the creation time is older than a set time (30-days?).

I try to do this with a simple key, Please review my PR

[tstromberg]

I'm temporarily going to back the PR out because I can't seem to get it to compile:

 ~/s/triage-party   tikv-cleanup $…  go run cmd/server/main.go                                                                                                                                             Mon May 11 21:57:38 2020
go: finding module for package google.golang.org/grpc/codes
go: finding module for package google.golang.org/grpc/keepalive
go: finding module for package github.com/pingcap/pd/pd-client
go: finding module for package google.golang.org/grpc/grpclog
go: finding module for package google.golang.org/grpc/encoding
go: finding module for package google.golang.org/grpc/internal
go: finding module for package google.golang.org/grpc/encoding/proto
go: finding module for package google.golang.org/grpc/status
go: finding module for package google.golang.org/grpc/peer
go: finding module for package google.golang.org/grpc/balancer
go: finding module for package google.golang.org/grpc/attributes
go: finding module for package google.golang.org/grpc/serviceconfig
go: finding module for package google.golang.org/grpc/internal/resolver/passthrough
go: finding module for package google.golang.org/grpc/connectivity
go: finding module for package google.golang.org/grpc/internal/resolver/dns
go: found google.golang.org/grpc/codes in google.golang.org/grpc v1.29.1
go: finding module for package github.com/pingcap/pd/pd-client
../../go/pkg/mod/github.com/pingcap/[email protected]+incompatible/store/tikv/kv.go:28:2: module github.com/pingcap/pd@latest found (v2.1.19+incompatible), but does not contain package github.com/pingcap/pd/pd-client

I spent an hour trying to debug the go.mod madness, and came to the conclusion that:

• pingcap did bad things
• grpc did bad things
• tikv and cloudsql-proxy want different versions of grpc, ultimately.

[masihtehrani]

I'm temporarily going to back the PR out because I can't seem to get it to compile:

 ~/s/triage-party   tikv-cleanup $…  go run cmd/server/main.go                                                                                                                                             Mon May 11 21:57:38 2020
go: finding module for package google.golang.org/grpc/codes
go: finding module for package google.golang.org/grpc/keepalive
go: finding module for package github.com/pingcap/pd/pd-client
go: finding module for package google.golang.org/grpc/grpclog
go: finding module for package google.golang.org/grpc/encoding
go: finding module for package google.golang.org/grpc/internal
go: finding module for package google.golang.org/grpc/encoding/proto
go: finding module for package google.golang.org/grpc/status
go: finding module for package google.golang.org/grpc/peer
go: finding module for package google.golang.org/grpc/balancer
go: finding module for package google.golang.org/grpc/attributes
go: finding module for package google.golang.org/grpc/serviceconfig
go: finding module for package google.golang.org/grpc/internal/resolver/passthrough
go: finding module for package google.golang.org/grpc/connectivity
go: finding module for package google.golang.org/grpc/internal/resolver/dns
go: found google.golang.org/grpc/codes in google.golang.org/grpc v1.29.1
go: finding module for package github.com/pingcap/pd/pd-client
../../go/pkg/mod/github.com/pingcap/[email protected]+incompatible/store/tikv/kv.go:28:2: module github.com/pingcap/pd@latest found (v2.1.19+incompatible), but does not contain package github.com/pingcap/pd/pd-client

I spent an hour trying to debug the go.mod madness, and came to the conclusion that:

• pingcap did bad things
• grpc did bad things
• tikv and cloudsql-proxy want different versions of grpc, ultimately.

I try to fix it

[tstromberg]

Once https://github.com/etcd-io/etcd/pull/11881 is merged, this may just work.

Alternatively, selecting a version of cloudsqlproxy and tikv of the same age will likely work as well, but I couldn't get it on my first attempt.