webknossos icon indicating copy to clipboard operation
webknossos copied to clipboard

[Draft/deferred] Cuckoo hashing for data rendering

Open philippotto opened this issue 3 years ago • 1 comments

This branch was created in March 2021 and deprioritzed shortly after. I opened the PR for it to avoid it being completely "gone".

URL of deployed dev instance (used for testing):

  • https://___.webknossos.xyz

Steps to test:

  • abc

Issues:

  • fixes #3915

(Please delete unneeded items, merge only when none are left open)

  • [ ] Updated (unreleased) changelog
  • [ ] Updated (unreleased) migration guide if applicable
  • [ ] Updated documentation if applicable
  • [ ] Adapted wk-connect if datastore API changes
  • [ ] Adapted wk-libs python client if relevant API parts change
  • [ ] Needs datastore update after deployment
  • [ ] Ready for review

philippotto avatar Feb 08 '22 12:02 philippotto

Short update:

  • cuckoo hashing is implemented (in JS) and should work
  • the integration of the new hashing is blocked because the murmur hash is not compatible with webgl 1
  • switching to webgl 2 is prepared, but does not work for some reason with the look up buffer (which is a 2-channel float texture)
  • I updated threejs a bit, hoping that this would solve some problems (or at least reduce technical debt). however, I'm afraid this causes more work than it solves problems for now.
  • Details for the float texture problem:
    • the console is spammed with GL_INVALID_OPERATION: Level of detail outside of range. when the look up texture is updated
    • I tried to switch to signed integer textures, but the shader still only finds zeros in the texture apparently
    • I also tried to use another format than LuminanceAlpha for the two-channel format (I tried RGFormat which seems to be new to webgl2). not much luck, either, though.
    • I suspect that the implementation of UpdatableTexture could be at fault

philippotto avatar Jul 01 '22 13:07 philippotto

Closing in favor of #6748 which implements the basic idea of this PR, too.

philippotto avatar Feb 17 '23 13:02 philippotto