[WIP] (proof-of-concept) sqlite port of etcd

[logicalhan]

[DO NOT MERGE] This is just a demonstrative PR, demonstrating the proof-of-concept of:

• extracting out bolt and putting it behind an interface
• implementing the interface using bolt
• implementing the interface using sqlite

If you want to run etcd using sqlite, please apply this patch and run:

• make
• rm -rf default.etcd
• ./bin/etcd

You can benchmark the performance of etcd using etcdctl by running ./bin/etcdctl check perf.

Note that this implementation of etcd-on-sqlite uses a pure golang version of sqlite, which is roughly half as fast as the full blown C implementation. Even so, the etcdctl performance tests suggest that the full golang version of sqlite seems roughly on par with bolt (if not a tad bit faster). For speed, we may want to consider using github.com/mattn/go-sqlite3, which compiles with CGO. However, this does break all of our build scripts and github actions since we presume that etcd can be compiled with CGO disabled.

All github actions (unmodified) are passing.

[xiang90]

Normal ops do not matter that much. They are bounded by gRPC and disk performance more likely as etcd batches requests aggressively and the read path would probably not even hit disk io...

Compaction and snapshot performance and isolation probably matters more.

[siyuanfoundation]

I did some memory profiling of the sqlite backend. This is what I did:

1. start etcd with sqlite bin/etcd --experimental-backend-type=sqlite --data-dir=sqlite.etcd --quota-backend-bytes=10737418240 --enable-pprof or bbolt backend bin/etcd --data-dir=bolt.etcd --quota-backend-bytes=10737418240 --enable-pprof.
2. populate the db with go run tools/benchmark/main.go put --total=10000000 --key-size=64 --val-size=512.
3. kill and restart etcd with sqlite or bbolt backend.
4. profile with go tool pprof -http=":8000" pprofbin http://127.0.0.1:2379/debug/pprof/heap?debug=1

They both consume similar amount of memory. Even though sqlite should have much lower memory footprint in theory, there is not clear advantage of using sqlite over bbolt.

• in-use memory profile with sqlite:

• in-use memory profile with bbolt:

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 21 days if no further activity occurs. Thank you for your contributions.

[williamcaban]

What are the next steps for this? Is there any follow-up discussion or plans for possible adoption? Specifically, are there noticeable benefits when handling defragmentation?

[k8s-ci-robot]

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.