Version 4.0.0-alpha.0

[folkvir]

Hello my friend! :tada:

This is a new life for this package, here are my modifications:

• Use @node-rs/xxhash in place of the xxhashjs package
  • The deps contains the @node-rs/xxhash-wasm32-wasi, this is for the browser build and is not included by default wit @node-rs/xxhash because of the cpu option which by default targets your plateform. We may pay attention
• Replace the usage of the Buffer built-in package (and of course the polyfill for browser users) by built-in UInt8array(s)
• Replace the usage of the Long.js package by built-in bigints in the XorFilter
• Use only 64-bits and 128-bits hash functions everywhere
• Add XOR 32-bits and 64-bits, straightforward addition with our modular implementation
• Update all dependencies to latest
• Replace mocha by jest
• Enforce eslint with plugin:@typescript-eslint/strict-type-checked
• Fix missing Hyperloglog large estimation correction
• Remove dead code, no more support for getDistinctIndexes, its last usage was for the Invertible Bloom Filter which do not use it anymore.
• Invertible Bloom Filters now only accept strings as input
• The project use yarn v4
  • Hence the new .yarn folder which needs to be shared
• It outputs both commonjs and module js. The package.json reflects the exports of every files of the project
• Examples on how to bundle using rspack and webpack have been added
• Update workflows version to latest
• Remove all .equals functions; there is no point to check for equality because items that are added using indexes could have identical indexes for 2 different elements with a same seed.

Fix the issues:

• https://github.com/Callidon/bloom-filters/issues/68
  • Changing the seed to a random one effectively trigger false negative, so there is a problem somewhere
    • Fix fingerprints computation
  • ~~Investigating other problems... 👀~~
  • Fix add algorithm and refacto the tests
• https://github.com/Callidon/bloom-filters/issues/69
  • Not reproductible anymore; seems to be fixed but I updated the example in the readme to reflect what the hyperloglog does and should report
• fix typos of https://github.com/Callidon/bloom-filters/pull/59

Versioning/Publishing:

• version was bumped to 4.0.0-alpha.0 with yarn version premajor --preid alpha from the 3.0.1
• incrementing is: yarn version prerelease
• publishing is: yarn publish --tag alpha

Feedback is welcome! @Callidon

TODO:

• update the documentation
• check issues:
  • https://github.com/Callidon/bloom-filters/issues/70

[github-advanced-security[bot]]

This pull request sets up GitHub code scanning for this repository. Once the scans have completed and the checks have passed, the analysis results for this pull request branch will appear on this overview. Once you merge this pull request, the 'Security' tab will show more code scanning analysis results (for example, for the default branch). Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results. For more information about GitHub code scanning, check out the documentation.

[Lwdthe1]

Thanks for working on this. Is there a timeline for releasing this version?

I'm looking forward to the browser updates 🙏

[folkvir]

✋ No timeline and we both have small free time to work on the package. And this PR is too big to be merged like this. The first thing I'll work on is the replacement of the buffer and long.js package:

Replace the usage of the Buffer built-in package (and of course the polyfill for browser users) by built-in UInt8array(s) Replace the usage of the Long.js package by built-in bigints in the XorFilter

And there is a bug within the Hashing.serialize function that I found when migrating everything to uint8arrays and bigints so I rather prefer to work on this before adding new features

What you can take into account is that we will soon merge the v4.0.0 (https://github.com/Callidon/bloom-filters/pull/76) which contains a major dependency upgrade. And I will work from this version. The v4 will support versions for node 20+ only.