Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ERROR: Could not build wheels for kenlm, which is required to install pyproject.toml-based projects #438

Open
solee0022 opened this issue Aug 9, 2023 · 2 comments
Open

ERROR: Could not build wheels for kenlm, which is required to install pyproject.toml-based projects #438

solee0022 opened this issue Aug 9, 2023 · 2 comments

Comments

@solee0022
Copy link

solee0022 commented Aug 9, 2023 •
edited
Loading

I want to install kenlm, but I got this error. And I've already installed 'pyproject.toml'. How can I fix this issue?

(asr2) $ pip install https://github.com/kpu/kenlm/archive/master.zip
Collecting https://github.com/kpu/kenlm/archive/master.zip
  Using cached https://github.com/kpu/kenlm/archive/master.zip
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: kenlm
  Building wheel for kenlm (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for kenlm (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [218 lines of output]
      Will build with KenLM max_order set to 6
      running bdist_wheel
      running build
      running build_ext
      CMake Deprecation Warning at CMakeLists.txt:1 (cmake_minimum_required):
        Compatibility with CMake < 3.5 will be removed from a future version of
        CMake.
      
        Update the VERSION argument <min> value or use a ...<max> suffix to tell
        CMake that the project does not need compatibility with older versions.
      
      
      -- The C compiler identification is NVHPC 22.2.0
      -- The CXX compiler identification is NVHPC 22.2.0
      -- Detecting C compiler ABI info
      -- Detecting C compiler ABI info - done
      -- Check for working C compiler: /opt/ohpc/pub/apps/nvidia/hpc_sdk/Linux_x86_64/22.2/compilers/bin/nvc - skipped
      -- Detecting C compile features
      -- Detecting C compile features - done
      -- Detecting CXX compiler ABI info
      -- Detecting CXX compiler ABI info - done
      -- Check for working CXX compiler: /opt/ohpc/pub/apps/nvidia/hpc_sdk/Linux_x86_64/22.2/compilers/bin/nvc++ - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      -- Could NOT find Eigen3 (missing: Eigen3_DIR)
      CMake Deprecation Warning at python/BuildStandalone.cmake:1 (cmake_minimum_required):
        Compatibility with CMake < 3.5 will be removed from a future version of
        CMake.
      
        Update the VERSION argument <min> value or use a ...<max> suffix to tell
        CMake that the project does not need compatibility with older versions.
      Call Stack (most recent call first):
        CMakeLists.txt:31 (include)
      
      
      -- Found ZLIB: /usr/lib64/libz.so (found version "1.2.11")
      -- Found BZip2: /usr/lib64/libbz2.so (found version "1.0.6")
      -- Looking for BZ2_bzCompressInit
      -- Looking for BZ2_bzCompressInit - found
      -- Looking for lzma_auto_decoder in /usr/lib64/liblzma.so
      -- Looking for lzma_auto_decoder in /usr/lib64/liblzma.so - found
      -- Looking for lzma_easy_encoder in /usr/lib64/liblzma.so
      -- Looking for lzma_easy_encoder in /usr/lib64/liblzma.so - found
      -- Looking for lzma_lzma_preset in /usr/lib64/liblzma.so
      -- Looking for lzma_lzma_preset in /usr/lib64/liblzma.so - found
      -- Found LibLZMA: /usr/lib64/liblzma.so (found version "5.2.4")
      -- Configuring done (1.7s)
      -- Generating done (0.0s)
      -- Build files have been written to: /tmp/pip-req-build-8tk25yr5/build/temp.linux-x86_64-cpython-39
      [  2%] Building CXX object CMakeFiles/kenlm.dir/lm/bhiksha.cc.o
      [  4%] Building CXX object CMakeFiles/kenlm.dir/lm/binary_format.cc.o
      [  9%] Building CXX object CMakeFiles/kenlm.dir/lm/lm_exception.cc.o
      [  9%] Building CXX object CMakeFiles/kenlm.dir/lm/config.cc.o
      [ 12%] Building CXX object CMakeFiles/kenlm.dir/lm/model.cc.o
      "/tmp/pip-req-build-8tk25yr5/lm/state.hh", line 77: warning: pointer points outside of underlying object
            for (uint64_t * i = pointers + length; i < pointers + KENLM_MAX_ORDER - 1; ++i)
                                                                ^
      
      "/tmp/pip-req-build-8tk25yr5/lm/vocab.hh", line 67: warning: overloaded virtual function "lm::base::Vocabulary::Index" is only partially overridden in class "lm::ngram::SortedVocabulary"
        class SortedVocabulary : public base::Vocabulary {
              ^
      
      "/tmp/pip-req-build-8tk25yr5/lm/vocab.hh", line 156: warning: overloaded virtual function "lm::base::Vocabulary::Index" is only partially overridden in class "lm::ngram::ProbingVocabulary"
        class ProbingVocabulary : public base::Vocabulary {
              ^
      
      [ 14%] Building CXX object CMakeFiles/kenlm.dir/lm/quantize.cc.o
      [ 17%] Building CXX object CMakeFiles/kenlm.dir/lm/read_arpa.cc.o
      [ 19%] Building CXX object CMakeFiles/kenlm.dir/lm/search_hashed.cc.o
      "/tmp/pip-req-build-8tk25yr5/lm/state.hh", line 77: warning: pointer points outside of underlying object
            for (uint64_t * i = pointers + length; i < pointers + KENLM_MAX_ORDER - 1; ++i)
                                                                ^
      
      "/tmp/pip-req-build-8tk25yr5/lm/vocab.hh", line 67: warning: overloaded virtual function "lm::base::Vocabulary::Index" is only partially overridden in class "lm::ngram::SortedVocabulary"
        class SortedVocabulary : public base::Vocabulary {
              ^
      
      "/tmp/pip-req-build-8tk25yr5/lm/vocab.hh", line 156: warning: overloaded virtual function "lm::base::Vocabulary::Index" is only partially overridden in class "lm::ngram::ProbingVocabulary"
        class ProbingVocabulary : public base::Vocabulary {
              ^
      
      [ 21%] Building CXX object CMakeFiles/kenlm.dir/lm/search_trie.cc.o
      "/tmp/pip-req-build-8tk25yr5/lm/vocab.hh", line 67: warning: overloaded virtual function "lm::base::Vocabulary::Index" is only partially overridden in class "lm::ngram::SortedVocabulary"
        class SortedVocabulary : public base::Vocabulary {
              ^
      
      "/tmp/pip-req-build-8tk25yr5/lm/vocab.hh", line 156: warning: overloaded virtual function "lm::base::Vocabulary::Index" is only partially overridden in class "lm::ngram::ProbingVocabulary"
        class ProbingVocabulary : public base::Vocabulary {
              ^
      
      "/tmp/pip-req-build-8tk25yr5/lm/search_trie.cc", line 180: warning: pointer points outside of underlying object
              for (BackoffMessages *i = messages_; i != messages_ + KENLM_MAX_ORDER - 1; ++i)
                                                                  ^
      
      "/tmp/pip-req-build-8tk25yr5/lm/search_trie.cc", line 430: warning: variable "backoffs" was declared but never referenced
          std::vector<float> probs, backoffs;
                                    ^
                detected during:
                  instantiation of "void lm::ngram::trie::BuildTrie(lm::ngram::trie::SortedFiles &, std::vector<uint64_t, std::allocator<uint64_t>> &, const lm::ngram::Config &, lm::ngram::trie::TrieSearch<Quant, Bhiksha> &, Quant &, lm::ngram::SortedVocabulary &, lm::ngram::BinaryFormat &) [with Quant=lm::ngram::DontQuantize, Bhiksha=lm::ngram::trie::DontBhiksha]" at line 587
                  instantiation of "void lm::ngram::trie::TrieSearch<Quant, Bhiksha>::InitializeFromARPA(const char *, util::FilePiece &, std::vector<uint64_t, std::allocator<uint64_t>> &, const lm::ngram::Config &, lm::ngram::SortedVocabulary &, lm::ngram::BinaryFormat &) [with Quant=lm::ngram::DontQuantize, Bhiksha=lm::ngram::trie::DontBhiksha]" at line 590
      
      [ 24%] Building CXX object CMakeFiles/kenlm.dir/lm/sizes.cc.o
      "/tmp/pip-req-build-8tk25yr5/lm/state.hh", line 77: warning: pointer points outside of underlying object
            for (uint64_t * i = pointers + length; i < pointers + KENLM_MAX_ORDER - 1; ++i)
                                                                ^
      
      "/tmp/pip-req-build-8tk25yr5/lm/vocab.hh", line 67: warning: overloaded virtual function "lm::base::Vocabulary::Index" is only partially overridden in class "lm::ngram::SortedVocabulary"
        class SortedVocabulary : public base::Vocabulary {
              ^
      
      "/tmp/pip-req-build-8tk25yr5/lm/vocab.hh", line 156: warning: overloaded virtual function "lm::base::Vocabulary::Index" is only partially overridden in class "lm::ngram::ProbingVocabulary"
        class ProbingVocabulary : public base::Vocabulary {
              ^
      
      [ 26%] Building CXX object CMakeFiles/kenlm.dir/lm/trie.cc.o
      [ 29%] Building CXX object CMakeFiles/kenlm.dir/lm/trie_sort.cc.o
      "/tmp/pip-req-build-8tk25yr5/lm/vocab.hh", line 67: warning: overloaded virtual function "lm::base::Vocabulary::Index" is only partially overridden in class "lm::ngram::SortedVocabulary"
        class SortedVocabulary : public base::Vocabulary {
              ^
      
      "/tmp/pip-req-build-8tk25yr5/lm/vocab.hh", line 156: warning: overloaded virtual function "lm::base::Vocabulary::Index" is only partially overridden in class "lm::ngram::ProbingVocabulary"
        class ProbingVocabulary : public base::Vocabulary {
              ^
      
      "/tmp/pip-req-build-8tk25yr5/lm/trie_sort.cc", line 124: warning: parameter "entry_size" was declared but never referenced
          void operator()(std::size_t entry_size, unsigned char order, const void *first, const void *second, FILE * /*out*/) const {
                                      ^
      
      "/tmp/pip-req-build-8tk25yr5/lm/trie_sort.cc", line 124: warning: parameter "second" was declared but never referenced
          void operator()(std::size_t entry_size, unsigned char order, const void *first, const void *second, FILE * /*out*/) const {
                                                                                                      ^
      
      "/tmp/pip-req-build-8tk25yr5/lm/trie_sort.cc", line 241: warning: variable "deleter" was declared but never referenced
                util::scoped_FILE deleter(*i);
                                  ^
      
      "/tmp/pip-req-build-8tk25yr5/lm/trie_sort.cc", line 246: warning: variable "deleter" was declared but never referenced
              util::scoped_FILE deleter(files_.front());
                                ^
      
      [ 31%] Building CXX object CMakeFiles/kenlm.dir/lm/value_build.cc.o
      "/tmp/pip-req-build-8tk25yr5/lm/state.hh", line 77: warning: pointer points outside of underlying object
            for (uint64_t * i = pointers + length; i < pointers + KENLM_MAX_ORDER - 1; ++i)
                                                                ^
      
      "/tmp/pip-req-build-8tk25yr5/lm/vocab.hh", line 67: warning: overloaded virtual function "lm::base::Vocabulary::Index" is only partially overridden in class "lm::ngram::SortedVocabulary"
        class SortedVocabulary : public base::Vocabulary {
              ^
      
      "/tmp/pip-req-build-8tk25yr5/lm/vocab.hh", line 156: warning: overloaded virtual function "lm::base::Vocabulary::Index" is only partially overridden in class "lm::ngram::ProbingVocabulary"
        class ProbingVocabulary : public base::Vocabulary {
              ^
      
      [ 34%] Building CXX object CMakeFiles/kenlm.dir/lm/virtual_interface.cc.o
      [ 36%] Building CXX object CMakeFiles/kenlm.dir/lm/vocab.cc.o
      "/tmp/pip-req-build-8tk25yr5/lm/vocab.hh", line 67: warning: overloaded virtual function "lm::base::Vocabulary::Index" is only partially overridden in class "lm::ngram::SortedVocabulary"
        class SortedVocabulary : public base::Vocabulary {
              ^
      
      "/tmp/pip-req-build-8tk25yr5/lm/vocab.hh", line 156: warning: overloaded virtual function "lm::base::Vocabulary::Index" is only partially overridden in class "lm::ngram::ProbingVocabulary"
        class ProbingVocabulary : public base::Vocabulary {
              ^
      
      [ 39%] Building CXX object CMakeFiles/kenlm.dir/python/score_sentence.cc.o
      "/tmp/pip-req-build-8tk25yr5/lm/state.hh", line 77: warning: pointer points outside of underlying object
            for (uint64_t * i = pointers + length; i < pointers + KENLM_MAX_ORDER - 1; ++i)
                                                                ^
      
      [ 41%] Building CXX object CMakeFiles/kenlm.dir/util/bit_packing.cc.o
      [ 43%] Building CXX object CMakeFiles/kenlm.dir/util/double-conversion/bignum-dtoa.cc.o
      [ 46%] Building CXX object CMakeFiles/kenlm.dir/util/double-conversion/bignum.cc.o
      [ 48%] Building CXX object CMakeFiles/kenlm.dir/util/double-conversion/cached-powers.cc.o
      [ 51%] Building CXX object CMakeFiles/kenlm.dir/util/double-conversion/double-to-string.cc.o
      [ 53%] Building CXX object CMakeFiles/kenlm.dir/util/double-conversion/fast-dtoa.cc.o
      [ 56%] Building CXX object CMakeFiles/kenlm.dir/util/double-conversion/fixed-dtoa.cc.o
      [ 58%] Building CXX object CMakeFiles/kenlm.dir/util/double-conversion/string-to-double.cc.o
      [ 60%] Building CXX object CMakeFiles/kenlm.dir/util/double-conversion/strtod.cc.o
      "/tmp/pip-req-build-8tk25yr5/util/double-conversion/strtod.cc", line 562: warning: variable "f2" was set but never used
          float f2 = float_guess;
                ^
      
      [ 63%] Building CXX object CMakeFiles/kenlm.dir/util/ersatz_progress.cc.o
      "/tmp/pip-req-build-8tk25yr5/util/double-conversion/strtod.cc", line 457: warning: function "double_conversion::AssertTrimmedDigits" was declared but never referenced
        static bool AssertTrimmedDigits(const Vector<const char>& buffer) {
                    ^
      
      [ 65%] Building CXX object CMakeFiles/kenlm.dir/util/exception.cc.o
      [ 68%] Building CXX object CMakeFiles/kenlm.dir/util/file.cc.o
      [ 70%] Building CXX object CMakeFiles/kenlm.dir/util/file_piece.cc.o
      [ 73%] Building CXX object CMakeFiles/kenlm.dir/util/float_to_string.cc.o
      [ 75%] Building CXX object CMakeFiles/kenlm.dir/util/integer_to_string.cc.o
      [ 78%] Building CXX object CMakeFiles/kenlm.dir/util/mmap.cc.o
      [ 80%] Building CXX object CMakeFiles/kenlm.dir/util/murmur_hash.cc.o
      "/tmp/pip-req-build-8tk25yr5/util/murmur_hash.cc", line 163: warning: function "util::<unnamed>::MurmurHashNativeBackend<L>(const void *, std::size_t, uint64_t) [with L=4U]" was declared but never referenced
        template <> inline uint64_t MurmurHashNativeBackend<4>(const void * key, std::size_t len, uint64_t seed) {
                                    ^
      
      [ 82%] Building CXX object CMakeFiles/kenlm.dir/util/parallel_read.cc.o
      [ 85%] Building CXX object CMakeFiles/kenlm.dir/util/pool.cc.o
      [ 87%] Building CXX object CMakeFiles/kenlm.dir/util/read_compressed.cc.o
      "/tmp/pip-req-build-8tk25yr5/util/mmap.cc", line 323: warning: statement is unreachable
              break;
              ^
      
      [ 90%] Building CXX object CMakeFiles/kenlm.dir/util/scoped.cc.o
      [ 92%] Building CXX object CMakeFiles/kenlm.dir/util/spaces.cc.o
      [ 95%] Building CXX object CMakeFiles/kenlm.dir/util/string_piece.cc.o
      [ 97%] Building CXX object CMakeFiles/kenlm.dir/util/usage.cc.o
      [100%] Linking CXX shared library /tmp/pip-req-build-8tk25yr5/build/lib.linux-x86_64-cpython-39/libkenlm.so
      [100%] Built target kenlm
      building 'kenlm' extension
      creating build/temp.linux-x86_64-cpython-39/lm
      creating build/temp.linux-x86_64-cpython-39/python
      creating build/temp.linux-x86_64-cpython-39/util
      creating build/temp.linux-x86_64-cpython-39/util/double-conversion
      /opt/ohpc/pub/apps/nvidia/hpc_sdk/Linux_x86_64/22.2/compilers/bin/nvc -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /home/solee0022/anaconda3/envs/asr2/include -I/home/solee0022/anaconda3/envs/asr2/include -fPIC -O2 -isystem /home/solee0022/anaconda3/envs/asr2/include -fPIC -I. -I/home/solee0022/anaconda3/envs/asr2/include/python3.9 -c lm/bhiksha.cc -o build/temp.linux-x86_64-cpython-39/lm/bhiksha.o -O3 -DNDEBUG -DKENLM_MAX_ORDER=6 -std=c++11 -DHAVE_CLOCKGETTIME -DHAVE_ZLIB -DHAVE_BZLIB -DHAVE_XZLIB
      nvc-Error-Unknown switch: -Wno-unused-result
      error: command '/opt/ohpc/pub/apps/nvidia/hpc_sdk/Linux_x86_64/22.2/compilers/bin/nvc' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for kenlm
Failed to build kenlm
ERROR: Could not build wheels for kenlm, which is required to install pyproject.toml-based projects
@lapolonio
Copy link

looks related https://kheafield.com/code/kenlm/dependencies/

@tnq177
Copy link

tnq177 commented Apr 1, 2024

Installing boost & eigen fixed this for me. Inside a conda env:

conda install conda-forge::boost
conda install conda-forge::eigen
pip install https://github.com/kpu/kenlm/archive/master.zip

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@lapolonio @tnq177 @solee0022