fasttext-langdetect icon indicating copy to clipboard operation
fasttext-langdetect copied to clipboard

low_memory parameter breaks the detect function

Open muzgash opened this issue 1 year ago • 1 comments

running python 3.10.12 and fasttext_langdetect 1.0.5

The function "detect! runs fine as explained in the docs:

detect(text="Bugün hava çok güzel")

but when adding the parameter low_memory like this:

detect(text="Bugün hava çok güzel",low_memory=True)

It shows the following traceback:

ValueError Traceback (most recent call last) in ----> 1 result = fd.detect(text="Bugün hava çok güzel",low_memory=True)

~/.local/lib/python3.10/site-packages/ftlangdetect/detect.py in detect(text, low_memory) 42 43 def detect(text: str, low_memory=False) -> Dict[str, Union[str, float]]: ---> 44 model = get_or_load_model(low_memory) 45 labels, scores = model.predict(text) 46 label = labels[0].replace("label", '')

~/.local/lib/python3.10/site-packages/ftlangdetect/detect.py in get_or_load_model(low_memory) 29 if not model: 30 model_path = download_model("lid.176.ftz") ---> 31 model = fasttext.load_model(model_path) 32 models["low_mem"] = model 33 return model

/usr/local/lib/python3.10/dist-packages/fasttext/FastText.py in load_model(path) 439 """Load a model given a filepath and return a model object.""" 440 eprint("Warning : load_model does not return WordVectorModel or SupervisedModel any more, but a FastText object which is very similar.") --> 441 return _FastText(model_path=path) 442 443

/usr/local/lib/python3.10/dist-packages/fasttext/FastText.py in init(self, model_path, args) 96 self.f = fasttext.fasttext() 97 if model_path is not None: ---> 98 self.f.loadModel(model_path) 99 self._words = None 100 self._labels = None

ValueError: /tmp/fasttext-langdetect/lid.176.ftz has wrong file format!

muzgash avatar Oct 11 '23 21:10 muzgash

this is because u dont have /tmp/fasttext-langdetect/lid.176.ftz file

amazingTest avatar Jan 04 '24 10:01 amazingTest