John Readey

It seems like it's more or less drop in. I was a bit confused with the dispatching logic at first, but my naive attempt seems to work: https://github.com/HDFGroup/anndata/commit/1a4833fed8d9cdeec1d297e224fbeffc030dc304, at least...

No problem! Thanks for getting back on this. The goal of HSDS is to support the use of HDF in a cloud-native context. This means having a REST-based API, ability...

As you can imagine, HSDS doesn't support pluggable filters (I imagine security people wouldn't be happy with clients injecting code onto the server for one thing...). So any supported filter...

Hey, @ajelenak has updated the code to use the numcodecs package for unshuflling (vs. the original code that was using python with numba): https://github.com/HDFGroup/hsds/commit/f44f0718648be4559faeb85e8d4e0c07d348d858. If nothing else, this has reduced...

Ah, sorry, I was thinking regular shuffle, not bitshuffle. We'll look into adding bitshuffle support. Do you have a sample file we can use for testing?

If someone would like to create a package that could do bitshuffle without the HDF library dependency, that would be much appreciated!

@bilalshaikh42, The https://github.com/HDFGroup/hsds/security/code-scanning/1 issue comes from using eval to do sql-like queries over datasets. The user input runs through the checkQuery function (https://github.com/HDFGroup/hsds/blob/master/hsds/util/chunkUtil.py#L1032) before it gets executed. Do you know...

In this commit: https://github.com/HDFGroup/hsds/commit/76519798b2b6beda87e010367de52629a0ab1e01, the BooleanParser ( https://github.com/HDFGroup/hsds/blob/master/hsds/util/boolparser.py) is used to evaluate the SQL-like expression. So rather than taking any user input and directly running eval on it, it's fed...

The https://github.com/HDFGroup/hsds/security/code-scanning/6 issue should be resolved in this commit: https://github.com/HDFGroup/hsds/commit/c2c3e7e6ef8cbf03339f7bb474599af1da9cdb0d. Rather than using regualr expression, there's a custom globparser matcher here: https://github.com/HDFGroup/hsds/blob/master/hsds/util/globparser.py. This will match domain names using a subset...

These should all be resolved now.