Kenneth Heafield comments

Results 290 comments of


                                            Kenneth Heafield

Building intermediate models with a predefined vocabulary leads to "poison" error

Confirmed I can reproduce with: ``` build/bin/lmplz -o 2 --text README.md --intermediate delme --limit_vocab_file LICENSE ``` (abusing text files as corpora). Looking into it.

Building intermediate models with a predefined vocabulary leads to "poison" error

I think @nmatthews-asapp has a different problem. Regarding this bug, apparently I hadn't thought about limit_vocab when permuting the vocabulary ids to put them in a consistent order. Fun. Planning...

Building intermediate models with a predefined vocabulary leads to "poison" error

@R1ckShi This is not relevant to the issue at hand. You need more disk space.

Building intermediate models with a predefined vocabulary leads to "poison" error

@ritwikmishra Please don't spam unrelated issues with duplicate posts.

I don't know what this kenlmd.lib is... How to fix that error?

I don't know what that is either. But it looks like you're using windows, which is only supported by contributors like you.

std::string std::bad_alloc

Smells like a different compiler version or different options. Try building with `make VERBOSE=1` and copy the compiler flags, especially any `-std=` flags exactly to your project. Also, please include...

Add prebuilt for windows

I don't maintain the windows build; random volunteers do.

Building diffs for patching lm binaries?

In theory this is possible but you'd be digging into smoothing algorithms because the discount parameters impact probability globally. And the quantizer is free to move centers. Possible but annoying.

How to uninstall?

https://stackoverflow.com/questions/41471620/cmake-support-make-uninstall https://gitlab.kitware.com/cmake/community/-/wikis/FAQ#can-i-do-make-uninstall-with-cmake ``` xargs rm < install_manifest.txt ```

How to uninstall?

Did you run `cmake install` or something? The default build instructions just make stuff in the `build` directory. I guess you could install to another directory, list the files, then...