blazingsql icon indicating copy to clipboard operation
blazingsql copied to clipboard

[BUG] Cannot import BlazingContext when processor type unknown

Open callofdutyops opened this issue 2 years ago • 2 comments

Describe the bug Cannot import BlazingContext when processor type unknown.

Steps/Code to reproduce bug Code and output from ipython (personal info hidden).

In [1]: from blazingsql import BlazingContext
---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
<ipython-input-7-0b19b5b41f48> in <module>
----> 1 from blazingsql import BlazingContext

~/miniconda3/envs/blazingsql/lib/python3.7/site-packages/blazingsql/__init__.py in <module>
      1 from pyblazing.apiv2 import S3EncryptionType
      2 from pyblazing.apiv2 import DataType
----> 3 from pyblazing.apiv2.context import BlazingContext
      4
      5 from cio import getProductDetailsCaller

~/miniconda3/envs/blazingsql/lib/python3.7/site-packages/pyblazing/apiv2/context.py in <module>
    105         )
    106
--> 107 jpype.startJVM("-ea", convertStrings=False, jvmpath=jvm_path)
    108 # jpype.startJVM()
    109

~/miniconda3/envs/blazingsql/lib/python3.7/site-packages/jpype/_core.py in startJVM(*args, **kwargs)
    225     try:
    226         _jpype.startup(jvmpath, tuple(args),
--> 227                        ignoreUnrecognized, convertStrings, interrupt)
    228         initializeResources()
    229     except RuntimeError as ex:

FileNotFoundError: [Errno 2] JVM DLL not found: /home/{my_username}/miniconda3/envs/blazingsql/lib/server/libjvm.so


In [2]: !uname -p
unknown

Expected behavior Should be imported without any errors.

Environment overview (please complete the following information)

  • Environment location: Bare-metal
  • Method of BlazingSQL install: conda
  • BlazingSQL Version
BlazingSQL version (git hash): 13618d177a37bd34bb20ac832fb8a14f8243ff5c
BlazingSQL branch name: HEAD
BlazingSQL branch tag: v21.08.02
BlazingSQL build id: 0
BlazingSQL compiler version: GNU /usr/local/gcc9/bin/g++ 9.4.0
BlazingSQL cuda flags: -Xcompiler -Wno-parentheses -gencode=arch=compute_60,code=sm_60 -gencode=arch=compute_70,code=sm_70 -gencode=arch=compute_75,code=sm_75 -gencode=arch=compute_75,code=compute_75 --expt-extended-lambda --expt-relaxed-constexpr -Werror=cross-execution-space-call -Xcompiler -Wall,-Wno-error=deprecated-declarations --default-stream=per-thread -DHT_DEFAULT_ALLOCATOR
BlazingSQL Operating system kernel: Linux-5.4.0-1054-aws
BlazingSQL Operating system architecture: x86_64
BlazingSQL Linux Operating system release: NAME=CentOS Linux|VERSION=7 (Core)|ID=centos|ID_LIKE=rhel fedora|VERSION_ID=7|PRETTY_NAME=CentOS Linux 7 (Core)|ANSI_COLOR=031|CPE_NAME=cpe:/o:centos:centos:7|HOME_URL=[https://www.centos.org/|BUG_REPORT_URL=https://bugs.centos.org/||CENTOS_MANTISBT_PROJECT=CentOS-7|CENTOS_MANTISBT_PROJECT_VERSION=7|REDHAT_SUPPORT_PRODUCT=centos|REDHAT_SUPPORT_PRODUCT_VERSION=7|](https://www.centos.org/%7CBUG_REPORT_URL=https://bugs.centos.org/%7C%7CCENTOS_MANTISBT_PROJECT=CentOS-7%7CCENTOS_MANTISBT_PROJECT_VERSION=7%7CREDHAT_SUPPORT_PRODUCT=centos%7CREDHAT_SUPPORT_PRODUCT_VERSION=7%7C)

----For BlazingSQL Developers---- Suspected source of the issue https://github.com/BlazingDB/blazingsql/blob/branch-21.08/pyblazing/pyblazing/apiv2/context.py#L70

machine_processor = platform.processor()

if machine_processor in ("x86_64", "x64"):
    machine_processor = "amd64"

when the uname -p is unknown, platform.processor() equals to '', thus machine_processor is empty, which leads to wrong jvm lib path.

callofdutyops avatar May 28 '22 10:05 callofdutyops

I think change machine_processor = platform.processor() to machine_processor = platform.processor() or platform.machine() can fix this.

callofdutyops avatar May 28 '22 10:05 callofdutyops

For those who suffer this, tmp solution is monkey patch:

import platform
def x64():
    return "x86_64"
platform.processor = x64

callofdutyops avatar May 28 '22 10:05 callofdutyops