fastcache icon indicating copy to clipboard operation
fastcache copied to clipboard

Add VisitAllEntries method to iterate all entries in cache.

Open andreiavrammsd opened this issue 5 years ago • 9 comments

Iterate all entries in cache with a given callback with signature f func(k, v []byte) error. Iteration stops if error is returned from callback.

Outdated:

Retrieves cached keys by regex pattern.

type User struct {
	ID       int    `json:"id"`
	Username string `json:"username"`
}

const userCacheKey = "user:%d"

cache := fastcache.New(1000)

// Insert data into cache
u := &User{ID: 1, Username: "John"}
b, _ := json.Marshal(u)
cache.Set([]byte(fmt.Sprintf(userCacheKey, u.ID)), b)

u = &User{ID: 2, Username: "Doe"}
b, _ = json.Marshal(u)
cache.Set([]byte(fmt.Sprintf(userCacheKey, u.ID)), b)

// Retrieve all users
users, err := cache.Keys(`user:\d+`)
if err != nil {
	panic(err)
}

fmt.Printf("Found %d user(s)\n", len(users))

for _, u := range users {
	user := &User{}
	if err := json.Unmarshal(cache.Get(nil, u), user); err != nil {
		fmt.Printf("Error retrieving key \"%s\"", u)
		continue
	}
	fmt.Printf("%#v\n", user)
}

andreiavrammsd avatar Dec 03 '18 19:12 andreiavrammsd

Codecov Report

Merging #3 into master will increase coverage by 1.14%. The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff            @@
##           master      #3      +/-   ##
=========================================
+ Coverage   76.05%   77.2%   +1.14%     
=========================================
  Files           4       4              
  Lines         497     522      +25     
=========================================
+ Hits          378     403      +25     
  Misses         65      65              
  Partials       54      54
Impacted Files Coverage Δ
fastcache.go 89.79% <100%> (+1.49%) :arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 78aabb6...39cf0ef. Read the comment docs.

codecov-io avatar Dec 03 '18 19:12 codecov-io

Will it be better to filter keys outside fastcache? Just get all the keys, filter them, fetch matched key-value pairs.

cristaloleg avatar Dec 04 '18 19:12 cristaloleg

Will it be better to filter keys outside fastcache? Just get all the keys, filter them, fetch matched key-value pairs.

This won't be better, since returned keys for huge number of cached items (typical workload for fastcache) would require a lot of additional memory and pointers.

valyala avatar Dec 04 '18 20:12 valyala

Please don't put everything inside doc only. You can write the limitations, caveats and few basic complete examples on the readme.md

Things to look out for should be most visible.

There are a lot of nice features and TTL is definitely needed here.

Please do make TTL ready at least as soon as possible.

I think the format of method can be implemented and set, the function / method / strategy / optimization can be improved on later.

e.g. SetWithTTL(k,v,t) etc or just SetT(k,v,t) etc can be implemented already. As for how the actual method should be, we can improve on overall later.

No one will read the doc complete but at least they will definitely read the readme.md AND a wiki section will be nice with FAQs section

hiqinternational avatar Jun 05 '19 06:06 hiqinternational

some case TTL is must and cool, but some others cache had. but , TTL is no need for high performance cache, there another ( easy ) way to update cache item or remove it.

fastcache is perfect in my trial project and running in product now.

tsingson avatar Aug 20 '19 13:08 tsingson

@andreiavrammsd do you need any help to finish this PR? fastcache is perfect for my use case, it's missing just the feature to iterate over all the entries.

diegobernardes avatar Nov 04 '19 17:11 diegobernardes

@diegobernardes, the PR is finished but not accepted because of performance concerns. Please see the discussions.

andreiavrammsd avatar Nov 04 '19 18:11 andreiavrammsd

Is there a feature for VisitAllEntries()? How can I code this / add to personal repo?

hiqsociety avatar Mar 12 '21 21:03 hiqsociety

Why not develop a function of simple Keys without regex pattern,I think there will be fewer performance problems 😄

miaoyc666 avatar Nov 17 '22 10:11 miaoyc666