sentencepiece icon indicating copy to clipboard operation
sentencepiece copied to clipboard

pipx install sentencepiece fails to install

Open gonsolo opened this issue 7 months ago • 3 comments

  1. Fix with gcc 15 (include cstdint)
  2. Release a new version and upload to Pypi, so that CMake 4 works

gonsolo avatar May 07 '25 08:05 gonsolo

diff --git a/src/sentencepiece_processor.h b/src/sentencepiece_processor.h
index dd3f092..010d500 100644
--- a/src/sentencepiece_processor.h
+++ b/src/sentencepiece_processor.h
@@ -15,6 +15,7 @@
 #ifndef SENTENCEPIECE_PROCESSOR_H_
 #define SENTENCEPIECE_PROCESSOR_H_
 
+#include <cstdint>
 #include <cstring>
 #include <memory>
 #include <string>

gonsolo avatar May 07 '25 08:05 gonsolo

Thanks for reporting this here too. This definitely is going to be a pain point going forward as more systems update CMake and GCC versions.

Ahajha avatar May 08 '25 20:05 Ahajha

Seems there's been a PR with that fix for a few weeks now (April 22): https://github.com/google/sentencepiece/pull/1109

The author hasn't signed the CLA apparently which might be why no review has taken place 🤷‍♂

No new releases since Feb 2024 though? Ideally a new release is tagged after that's resolved.

polarathene avatar May 12 '25 08:05 polarathene

This issue is fixed on the head master. Thank you.

taku910 avatar Aug 02 '25 11:08 taku910

This issue is fixed on the head master. Thank you.

No it's not. You fixed 1. (GCC issue), but 2. (Release and upload to Pypi) is not done. 👎 Until that is done, all packages relying on sentencepiece (Modular, mojo) are broken. Unfortunately I'm not allowed to reopen my own issue. 👎

gonsolo avatar Aug 02 '25 15:08 gonsolo

For modular/mojo/max, the dependency was removed, at least in the last few weeks of nightlies, and the next stable will also have the dependency removed.

Ahajha avatar Aug 02 '25 16:08 Ahajha

For modular/mojo/max, the dependency was removed, at least in the last few weeks of nightlies, and the next stable will also have the dependency removed.

Ok. But that doesn't change the fact that the fixed cstdint is worthless without a release.

gonsolo avatar Aug 02 '25 16:08 gonsolo

Yes, agreed.

Ahajha avatar Aug 02 '25 16:08 Ahajha