finalfusion-rust
finalfusion-rust copied to clipboard
finalfusion
finalfusion embeddings in Rust
While reimplementing finalfusion-inspector in Rust, I bumped into a small annoyance. The `analogy` method takes an array of `&str`: `query: [&str; 3]` However, oftentimes you have a [String; 3]. We...
Add support for pruning embeddings, where N embeddings are retained. Words for which embeddings are removed are mapped to their nearest neighbor. This should provide more or less the same...
I think it would be nice to have a small utility data structure to fetch pretrained embeddings. I don't think this needs to be part of the `finalfusion` crate, since...
Trying to read the GoogleNews-vectors-negative300.bin word2vec model triggers this assert: https://github.com/finalfusion/finalfusion-rust/blob/main/src/chunks/vocab/simple.rs#L28 ``` thread 'main' panicked at 'assertion failed: `(left == right)` left: `3000000`, right: `2999997`: words contained duplicate entries.' ```...
I tried to use this library with a pretrained models from https://github.com/sdadas/polish-nlp-resources?tab=readme-ov-file#word2vec and I found out that these are in [keyed vector format](https://radimrehurek.com/gensim/models/keyedvectors.html#) which is currently not supported.
Just wanted to let you know all links [here](https://finalfusion.github.io/pretrained.html) return 404.
