badger
badger copied to clipboard
GC doesn't seem to run
What version of Badger are you using?
Latest v4
What version of Go are you using?
1.20.3
Have you tried reproducing the issue with the latest release?
Yes
What is the hardware spec (RAM, CPU, OS)?
16gb ram, i5 intel, mac os
What steps will reproduce the bug?
package main
import (
"fmt"
"log"
"math/rand"
"strconv"
"time"
"github.com/dgraph-io/badger/v4"
)
var db *badger.DB
const charset = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"
func generateRandomString(length int) string {
b := make([]byte, length)
for i := range b {
b[i] = charset[rand.Intn(len(charset))]
}
return string(b)
}
func main() {
opts := badger.DefaultOptions("badger")
opts.ValueLogFileSize = 1 << 20
var err error
db, err = badger.Open(opts)
if err != nil {
log.Fatal(err)
}
defer db.Close()
fmt.Println("Start inserting 5 million items.")
for i := 0; i < 5_000_000; i++ {
err := db.Update(func(txn *badger.Txn) error {
key := strconv.Itoa(i)
value := []byte(generateRandomString(300))
e := badger.NewEntry([]byte(key), value).WithTTL(30 * time.Second)
err := txn.SetEntry(e)
if err != nil {
return err
}
return nil
})
if err != nil {
log.Fatal("Error while inserting:", err)
}
}
fmt.Println("Successfully inserted 5 million items.")
startGarbageCollection()
}
func startGarbageCollection() {
ticker := time.NewTicker(10 * time.Second)
defer ticker.Stop()
for range ticker.C {
again:
err := db.RunValueLogGC(0.01)
fmt.Println(err)
if err == nil {
goto again
}
}
}
Expected behavior and actual result.
Since the keys are expiring in 30 seconds, I thought that the files in badger/ would get removed.
However, heres what I see:
Additional information
No response
may be related to #1995. Thanks for filling an issue with code to reproduce. I will try to find some time to look into it.
Maybe the compaction is not triggered. Similar to https://discuss.dgraph.io/t/gc-may-not-work-in-some-cases/17197
Hi Friends. Any updates on this? Ty
@mangalaman93 Would it possible for you to take a look when you have some time ? Thank you !
This issue has been stale for 60 days and will be closed automatically in 7 days. Comment to keep it open.
Any updates ? Thanks !