v3.0.2 bytesqueue peak panic

[tcsgogogo]

found a panic:

runtime error: slice bounds out of range [:31874] with capacity 24992
goroutine 1708235262 [running]:
panic({0x1618300, 0xc030365740})
        /data/golang/1.18/src/runtime/panic.go:838 +0x207
github.com/allegro/bigcache/v3/queue.(*BytesQueue).peek(0x140e960?, 0xc001730630?)
        /root/go/pkg/mod/github.com/allegro/bigcache/[email protected]/queue/bytes_queue.go:242 +0x177
github.com/allegro/bigcache/v3/queue.(*BytesQueue).Get(...)
        /root/go/pkg/mod/github.com/allegro/bigcache/[email protected]/queue/bytes_queue.go:193
github.com/allegro/bigcache/v3.(*cacheShard).getWrappedEntry(0xc0004f5200, 0xc06ab11d60?)
        /root/go/pkg/mod/github.com/allegro/bigcache/[email protected]/shard.go:92 +0x46
github.com/allegro/bigcache/v3.(*cacheShard).get(0xc0004f5200, {0x18a8321, 0x1}, 0x1000000000001?)
        /root/go/pkg/mod/github.com/allegro/bigcache/[email protected]/shard.go:64 +0x7e
github.com/allegro/bigcache/v3.(*BigCache).Get(0xc002c324e0, {0x18a8321, 0x1})
        /root/go/pkg/mod/github.com/allegro/bigcache/[email protected]/bigcache.go:123 +0x69

Environment:

• Version (git sha or release): v3.0.2
• OS (e.g. from /etc/os-release or winver.exe): centos
• go version: 1.18

[janisz]

Do you have any way to reproduce it?

[tcsgogogo]

Do you have any way to reproduce it?

The Panic occurred in the K8S deployment of the production environment. I tried to redeploy it, but there was a persistent Panic. After a brief look at the code location of the Panic, I guess that there is a problem in the key generation. I removed the customized hasher in the NewBigCache configuration, and then recompiled and packaged, then the Panic never appeared.

My custom hasher key is uint64, and the definition is as follows:

//NewUint64Hasher uses uint64 as key
func NewUint64Hasher() Hasher {
	return uint64Hasher{}
}

type uint64Hasher struct{}

// Sum64 hash calc func
func (f uint64Hasher) Sum64(key string) uint64 {
	hash, err := strconv.ParseUint(key, 10, 64)
	if err != nil {
		log.Error("key:%s err!", key)
	}

	return hash
}

// FormatUint64CacheKey uint64 map as key
func FormatUint64CacheKey(key uint64) string {
	return strconv.FormatUint(key, 10)
}

[janisz]

Interesting, what entries and keys looks like? Were the keys randomly distributed?

[tcsgogogo]

Interesting, what entries and keys looks like? Were the keys randomly distributed?

1. In fact, both key and value are uint64 numbers.
2. The key comes from the uint64 number entered by the user, so it may not be randomly distributed.

Will data skew in a shard cause this problem? Shouldn't it cause a set error rather than a get error, let alone a panic.

[janisz]

Will data skew in a shard cause this problem? Shouldn't it cause a set error rather than a get error, let alone a panic.

You are right. But maybe this approach will help us debug #283

[tcsgogogo]

Will data skew in a shard cause this problem? Shouldn't it cause a set error rather than a get error, let alone a panic.

You are right. But maybe this approach will help us debug #283
PS:
I only use the set and get methods to access data. It seems that this is the easiest way to use.
func (c *BigCache) Set(key string, entry []byte) error
func (c *BigCache) Get(key string) ([]byte, error)