ERROR: Failed building wheel for kenlm (Python 3.9.13 & 3.10.9)

[TrinhNhatTuyen]

Hi, I used !pip install https://github.com/kpu/kenlm/archive/master.zip to install kenlm, and the error is:

Installing build dependencies ... done
 Getting requirements to build wheel ... done
 Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: kenlm
Building wheel for kenlm (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for kenlm (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [74 lines of output]
      'bash' is not recognized as an internal or external command,
      operable program or batch file.
      'bash' is not recognized as an internal or external command,
      operable program or batch file.
      'bash' is not recognized as an internal or external command,
      operable program or batch file.
      running bdist_wheel
      running build
      running build_ext
      -- Building for: NMake Makefiles
      CMake Error at CMakeLists.txt:14 (project):
        Generator
     
          NMake Makefiles
     
        does not support platform specification, but platform
     
          x64
     
        was specified.
     
     
      CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage
      CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage
      -- Configuring incomplete, errors occurred!

. . .

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for kenlm
Failed to build kenlm
ERROR: Could not build wheels for kenlm, which is required to install pyproject.toml-based projects

I tried with Python 3.9.13 & 3.10.9 but i didn't work. is there any way to fix it?

[kpu]

What platform is this that doesn't have bash installed?

[TheWorldIsNot]

I get same problem,but have no solution

[kpu]

Install bash...

[simozhu]

I got the same trouble

[michael-newsrx]

Building wheels for collected packages: sentence_transformers, kenlm, audioread, pylatexenc
  Building wheel for sentence_transformers (setup.py) ... done
  Created wheel for sentence_transformers: filename=sentence_transformers-2.2.2-py3-none-any.whl size=125925 sha256=2a549fa103869746a8ecc40d4fa0c9b3e96a765dbc7a75f559fd7ca00741d490
  Stored in directory: /home/michael/.cache/pip/wheels/ff/27/bf/ffba8b318b02d7f691a57084ee154e26ed24d012b0c7805881
  Building wheel for kenlm (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [89 lines of output]
      running bdist_wheel
      running build
      running build_ext
      building 'kenlm' extension
      creating build
      creating build/temp.linux-x86_64-cpython-311
      creating build/temp.linux-x86_64-cpython-311/lm
      creating build/temp.linux-x86_64-cpython-311/python
      creating build/temp.linux-x86_64-cpython-311/util
      creating build/temp.linux-x86_64-cpython-311/util/double-conversion
      gcc -pthread -B /home/michael/miniconda3/envs/scholarly-editions/compiler_compat -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/michael/miniconda3/envs/scholarly-editions/include -fPIC -O2 -isystem /home/michael/miniconda3/envs/scholarly-editions/include -fPIC -I. -I/home/michael/miniconda3/envs/scholarly-editions/include/python3.11 -c lm/bhiksha.cc -o build/temp.linux-x86_64-cpython-311/lm/bhiksha.o -O3 -DNDEBUG -DKENLM_MAX_ORDER=6 -std=c++11 -DHAVE_CLOCKGETTIME
      gcc -pthread -B /home/michael/miniconda3/envs/scholarly-editions/compiler_compat -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/michael/miniconda3/envs/scholarly-editions/include -fPIC -O2 -isystem /home/michael/miniconda3/envs/scholarly-editions/include -fPIC -I. -I/home/michael/miniconda3/envs/scholarly-editions/include/python3.11 -c lm/binary_format.cc -o build/temp.linux-x86_64-cpython-311/lm/binary_format.o -O3 -DNDEBUG -DKENLM_MAX_ORDER=6 -std=c++11 -DHAVE_CLOCKGETTIME
      lm/binary_format.cc: In member function ‘void lm::ngram::BinaryFormat::FinishFile(const lm::ngram::Config&, lm::ngram::ModelType, unsigned int, const std::vector<long unsigned int>&)’:
      lm/binary_format.cc:261:9: warning: ‘void* memset(void*, int, size_t)’ clearing an object of type ‘struct lm::ngram::Parameters’ with no trivial copy-assignment; use assignment or value-initialization instead [-Wclass-memaccess]
        261 |   memset(&params, 0, sizeof(Parameters));
            |   ~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      In file included from lm/binary_format.cc:1:
      lm/binary_format.hh:42:8: note: ‘struct lm::ngram::Parameters’ declared here
         42 | struct Parameters {
            |        ^~~~~~~~~~
      gcc -pthread -B /home/michael/miniconda3/envs/scholarly-editions/compiler_compat -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/michael/miniconda3/envs/scholarly-editions/include -fPIC -O2 -isystem /home/michael/miniconda3/envs/scholarly-editions/include -fPIC -I. -I/home/michael/miniconda3/envs/scholarly-editions/include/python3.11 -c lm/config.cc -o build/temp.linux-x86_64-cpython-311/lm/config.o -O3 -DNDEBUG -DKENLM_MAX_ORDER=6 -std=c++11 -DHAVE_CLOCKGETTIME
      gcc -pthread -B /home/michael/miniconda3/envs/scholarly-editions/compiler_compat -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/michael/miniconda3/envs/scholarly-editions/include -fPIC -O2 -isystem /home/michael/miniconda3/envs/scholarly-editions/include -fPIC -I. -I/home/michael/miniconda3/envs/scholarly-editions/include/python3.11 -c lm/lm_exception.cc -o build/temp.linux-x86_64-cpython-311/lm/lm_exception.o -O3 -DNDEBUG -DKENLM_MAX_ORDER=6 -std=c++11 -DHAVE_CLOCKGETTIME
      gcc -pthread -B /home/michael/miniconda3/envs/scholarly-editions/compiler_compat -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/michael/miniconda3/envs/scholarly-editions/include -fPIC -O2 -isystem /home/michael/miniconda3/envs/scholarly-editions/include -fPIC -I. -I/home/michael/miniconda3/envs/scholarly-editions/include/python3.11 -c lm/model.cc -o build/temp.linux-x86_64-cpython-311/lm/model.o -O3 -DNDEBUG -DKENLM_MAX_ORDER=6 -std=c++11 -DHAVE_CLOCKGETTIME
      gcc -pthread -B /home/michael/miniconda3/envs/scholarly-editions/compiler_compat -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/michael/miniconda3/envs/scholarly-editions/include -fPIC -O2 -isystem /home/michael/miniconda3/envs/scholarly-editions/include -fPIC -I. -I/home/michael/miniconda3/envs/scholarly-editions/include/python3.11 -c lm/quantize.cc -o build/temp.linux-x86_64-cpython-311/lm/quantize.o -O3 -DNDEBUG -DKENLM_MAX_ORDER=6 -std=c++11 -DHAVE_CLOCKGETTIME
      gcc -pthread -B /home/michael/miniconda3/envs/scholarly-editions/compiler_compat -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/michael/miniconda3/envs/scholarly-editions/include -fPIC -O2 -isystem /home/michael/miniconda3/envs/scholarly-editions/include -fPIC -I. -I/home/michael/miniconda3/envs/scholarly-editions/include/python3.11 -c lm/read_arpa.cc -o build/temp.linux-x86_64-cpython-311/lm/read_arpa.o -O3 -DNDEBUG -DKENLM_MAX_ORDER=6 -std=c++11 -DHAVE_CLOCKGETTIME
      gcc -pthread -B /home/michael/miniconda3/envs/scholarly-editions/compiler_compat -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/michael/miniconda3/envs/scholarly-editions/include -fPIC -O2 -isystem /home/michael/miniconda3/envs/scholarly-editions/include -fPIC -I. -I/home/michael/miniconda3/envs/scholarly-editions/include/python3.11 -c lm/search_hashed.cc -o build/temp.linux-x86_64-cpython-311/lm/search_hashed.o -O3 -DNDEBUG -DKENLM_MAX_ORDER=6 -std=c++11 -DHAVE_CLOCKGETTIME
      gcc -pthread -B /home/michael/miniconda3/envs/scholarly-editions/compiler_compat -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/michael/miniconda3/envs/scholarly-editions/include -fPIC -O2 -isystem /home/michael/miniconda3/envs/scholarly-editions/include -fPIC -I. -I/home/michael/miniconda3/envs/scholarly-editions/include/python3.11 -c lm/search_trie.cc -o build/temp.linux-x86_64-cpython-311/lm/search_trie.o -O3 -DNDEBUG -DKENLM_MAX_ORDER=6 -std=c++11 -DHAVE_CLOCKGETTIME
      gcc -pthread -B /home/michael/miniconda3/envs/scholarly-editions/compiler_compat -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/michael/miniconda3/envs/scholarly-editions/include -fPIC -O2 -isystem /home/michael/miniconda3/envs/scholarly-editions/include -fPIC -I. -I/home/michael/miniconda3/envs/scholarly-editions/include/python3.11 -c lm/sizes.cc -o build/temp.linux-x86_64-cpython-311/lm/sizes.o -O3 -DNDEBUG -DKENLM_MAX_ORDER=6 -std=c++11 -DHAVE_CLOCKGETTIME
      gcc -pthread -B /home/michael/miniconda3/envs/scholarly-editions/compiler_compat -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/michael/miniconda3/envs/scholarly-editions/include -fPIC -O2 -isystem /home/michael/miniconda3/envs/scholarly-editions/include -fPIC -I. -I/home/michael/miniconda3/envs/scholarly-editions/include/python3.11 -c lm/trie.cc -o build/temp.linux-x86_64-cpython-311/lm/trie.o -O3 -DNDEBUG -DKENLM_MAX_ORDER=6 -std=c++11 -DHAVE_CLOCKGETTIME
      gcc -pthread -B /home/michael/miniconda3/envs/scholarly-editions/compiler_compat -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/michael/miniconda3/envs/scholarly-editions/include -fPIC -O2 -isystem /home/michael/miniconda3/envs/scholarly-editions/include -fPIC -I. -I/home/michael/miniconda3/envs/scholarly-editions/include/python3.11 -c lm/trie_sort.cc -o build/temp.linux-x86_64-cpython-311/lm/trie_sort.o -O3 -DNDEBUG -DKENLM_MAX_ORDER=6 -std=c++11 -DHAVE_CLOCKGETTIME
      gcc -pthread -B /home/michael/miniconda3/envs/scholarly-editions/compiler_compat -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/michael/miniconda3/envs/scholarly-editions/include -fPIC -O2 -isystem /home/michael/miniconda3/envs/scholarly-editions/include -fPIC -I. -I/home/michael/miniconda3/envs/scholarly-editions/include/python3.11 -c lm/value_build.cc -o build/temp.linux-x86_64-cpython-311/lm/value_build.o -O3 -DNDEBUG -DKENLM_MAX_ORDER=6 -std=c++11 -DHAVE_CLOCKGETTIME
      gcc -pthread -B /home/michael/miniconda3/envs/scholarly-editions/compiler_compat -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/michael/miniconda3/envs/scholarly-editions/include -fPIC -O2 -isystem /home/michael/miniconda3/envs/scholarly-editions/include -fPIC -I. -I/home/michael/miniconda3/envs/scholarly-editions/include/python3.11 -c lm/virtual_interface.cc -o build/temp.linux-x86_64-cpython-311/lm/virtual_interface.o -O3 -DNDEBUG -DKENLM_MAX_ORDER=6 -std=c++11 -DHAVE_CLOCKGETTIME
      gcc -pthread -B /home/michael/miniconda3/envs/scholarly-editions/compiler_compat -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/michael/miniconda3/envs/scholarly-editions/include -fPIC -O2 -isystem /home/michael/miniconda3/envs/scholarly-editions/include -fPIC -I. -I/home/michael/miniconda3/envs/scholarly-editions/include/python3.11 -c lm/vocab.cc -o build/temp.linux-x86_64-cpython-311/lm/vocab.o -O3 -DNDEBUG -DKENLM_MAX_ORDER=6 -std=c++11 -DHAVE_CLOCKGETTIME
      gcc -pthread -B /home/michael/miniconda3/envs/scholarly-editions/compiler_compat -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/michael/miniconda3/envs/scholarly-editions/include -fPIC -O2 -isystem /home/michael/miniconda3/envs/scholarly-editions/include -fPIC -I. -I/home/michael/miniconda3/envs/scholarly-editions/include/python3.11 -c python/kenlm.cpp -o build/temp.linux-x86_64-cpython-311/python/kenlm.o -O3 -DNDEBUG -DKENLM_MAX_ORDER=6 -std=c++11 -DHAVE_CLOCKGETTIME
      python/kenlm.cpp: In function ‘void __Pyx_AddTraceback(const char*, int, int, const char*)’:
      python/kenlm.cpp:452:62: error: invalid use of incomplete type ‘PyFrameObject’ {aka ‘struct _frame’}
        452 |   #define __Pyx_PyFrame_SetLineNumber(frame, lineno)  (frame)->f_lineno = (lineno)
            |                                                              ^~
      python/kenlm.cpp:8778:5: note: in expansion of macro ‘__Pyx_PyFrame_SetLineNumber’
       8778 |     __Pyx_PyFrame_SetLineNumber(py_frame, py_line);
            |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~
      In file included from /home/michael/miniconda3/envs/scholarly-editions/include/python3.11/Python.h:42,
                       from python/kenlm.cpp:6:
      /home/michael/miniconda3/envs/scholarly-editions/include/python3.11/pytypedefs.h:22:16: note: forward declaration of ‘PyFrameObject’ {aka ‘struct _frame’}
         22 | typedef struct _frame PyFrameObject;
            |                ^~~~~~
      python/kenlm.cpp: In function ‘PyObject* __Pyx_Coroutine_SendEx(__pyx_CoroutineObject*, PyObject*, int)’:
      python/kenlm.cpp:10248:14: error: invalid use of incomplete type ‘PyFrameObject’ {aka ‘struct _frame’}
      10248 |             f->f_back = PyThreadState_GetFrame(tstate);
            |              ^~
      In file included from /home/michael/miniconda3/envs/scholarly-editions/include/python3.11/Python.h:42,
                       from python/kenlm.cpp:6:
      /home/michael/miniconda3/envs/scholarly-editions/include/python3.11/pytypedefs.h:22:16: note: forward declaration of ‘PyFrameObject’ {aka ‘struct _frame’}
         22 | typedef struct _frame PyFrameObject;
            |                ^~~~~~
      In file included from /home/michael/miniconda3/envs/scholarly-editions/include/python3.11/Python.h:38,
                       from python/kenlm.cpp:6:
      python/kenlm.cpp: In function ‘void __Pyx_Coroutine_ResetFrameBackpointer(__Pyx_ExcInfoStruct*)’:
      python/kenlm.cpp:10285:19: error: invalid use of incomplete type ‘PyFrameObject’ {aka ‘struct _frame’}
      10285 |         Py_CLEAR(f->f_back);
            |                   ^~
      /home/michael/miniconda3/envs/scholarly-editions/include/python3.11/pyport.h:24:38: note: in definition of macro ‘_Py_CAST’
         24 | #define _Py_CAST(type, expr) ((type)(expr))
            |                                      ^~~~
      /home/michael/miniconda3/envs/scholarly-editions/include/python3.11/object.h:581:29: note: in expansion of macro ‘_PyObject_CAST’
        581 |         PyObject *_py_tmp = _PyObject_CAST(op); \
            |                             ^~~~~~~~~~~~~~
      python/kenlm.cpp:10285:9: note: in expansion of macro ‘Py_CLEAR’
      10285 |         Py_CLEAR(f->f_back);
            |         ^~~~~~~~
      In file included from /home/michael/miniconda3/envs/scholarly-editions/include/python3.11/Python.h:42,
                       from python/kenlm.cpp:6:
      /home/michael/miniconda3/envs/scholarly-editions/include/python3.11/pytypedefs.h:22:16: note: forward declaration of ‘PyFrameObject’ {aka ‘struct _frame’}
         22 | typedef struct _frame PyFrameObject;
            |                ^~~~~~
      In file included from /home/michael/miniconda3/envs/scholarly-editions/include/python3.11/Python.h:44,
                       from python/kenlm.cpp:6:
      python/kenlm.cpp:10285:19: error: invalid use of incomplete type ‘PyFrameObject’ {aka ‘struct _frame’}
      10285 |         Py_CLEAR(f->f_back);
            |                   ^~
      /home/michael/miniconda3/envs/scholarly-editions/include/python3.11/object.h:583:14: note: in definition of macro ‘Py_CLEAR’
        583 |             (op) = NULL;                        \
            |              ^~
      In file included from /home/michael/miniconda3/envs/scholarly-editions/include/python3.11/Python.h:42,
                       from python/kenlm.cpp:6:
      /home/michael/miniconda3/envs/scholarly-editions/include/python3.11/pytypedefs.h:22:16: note: forward declaration of ‘PyFrameObject’ {aka ‘struct _frame’}
         22 | typedef struct _frame PyFrameObject;
            |                ^~~~~~
      error: command '/usr/bin/gcc' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for kenlm
  Running setup.py clean for kenlm
  Building wheel for audioread (setup.py) ... done
  Created wheel for audioread: filename=audioread-3.0.0-py3-none-any.whl size=23704 sha256=d76c32a34c53300e5c282bfafc5b9a72439f82c12201c01ea2e0d8899356ba87
  Stored in directory: /home/michael/.cache/pip/wheels/34/6d/02/40eff8045c8287806e7d83333301c1c7bab6109045424e8558
  Building wheel for pylatexenc (setup.py) ... done
  Created wheel for pylatexenc: filename=pylatexenc-2.10-py3-none-any.whl size=136819 sha256=a2e16ff1dfaa941c29808a22f8ac8a9b1c432d867062e5217d22128f60e1e3cf
  Stored in directory: /home/michael/.cache/pip/wheels/b1/7a/33/9fdd892f784ed4afda62b685ae3703adf4c91aa0f524c28f03
Successfully built sentence_transformers audioread pylatexenc
Failed to build kenlm
ERROR: Could not build wheels for kenlm, which is required to install pyproject.toml-based projects

[michael-newsrx]

echo $SHELL
/bin/bash

[kpu]

@michael-newsrx different issue, looks like cython is broken. Try now that I've updated cython. Or cd kenlm/python && cython -3 --cplus kenlm.pyx yourself.

[chrisma2021]

I got the same trouble.

Python version information: Python 3.11 (64-bit)

OS version information: OS Name Microsoft Windows 10 Home Version 10.0.19045 Build 19045

PIP version information: Name: pip Version: 23.2.1 Summary: The PyPA recommended tool for installing Python packages. Home-page: https://pip.pypa.io/ Author: The pip developers Author-email: [email protected] License: MIT

[boots-coder]

What platform is this that doesn't have bash installed?

Windows10

[GNishanth7]

I am also getting the same error

[kpu]

Windows is a community-supported platform. I don't support it but you're welcome to fix it.

[boots-coder]

you can install the *whl which is compiled in some webs;i fixxed that on Fasttext by this way

[razi-tm]

https://stackoverflow.com/questions/29846087/error-microsoft-visual-c-14-0-is-required-unable-to-find-vcvarsall-bat

check out the answers provided here. it's for a different question but can solve the problem of kenlm installation on windows.

@TrinhNhatTuyen @TheWorldIsNot @simozhu @chrisma2021 @GNishanth7

[quyet12308]

https://stackoverflow.com/questions/29846087/error-microsoft-visual-c-14-0-is-required-unable-to-find-vcvarsall-bat

kiểm tra các câu trả lời được cung cấp ở đây. đó là một câu hỏi khác nhưng có thể giải quyết vấn đề cài đặt kenlm trên windows.

@TrinhNhatTuyen @TheWorldIsNot @simozhu @chrisma2021 @GNishanth7

Can you give more details? I tried the methods but I still can't install kenlm for windows

[razi-tm]

download and run visual studio installer and install desktop development with C++, it worked for me.

https://visualstudio.microsoft.com/thank-you-downloading-visual-studio/?sku=Community&channel=Release&version=VS2022&source=VSLandingPage&cid=2030&passive=false

@quyet12308

[Lizi-12]

download and run visual studio installer and install desktop development with C++, it worked for me.

https://visualstudio.microsoft.com/thank-you-downloading-visual-studio/?sku=Community&channel=Release&version=VS2022&source=VSLandingPage&cid=2030&passive=false

@quyet12308

It works for me! Thanks!

[shibing624]

use python3.8, and it work for me.

python3.8 -m pip install kenlm

[GoodCabbage]

install about Microsoft Visual C++ 14.0，it works