Build fails on ubuntu 20.04

[irthomasthomas]

Hi, I can't get 0.9.0 to build on ubuntu 20.04 $ make setup produces error E: Unable to locate package python-gevent command failed: { apt-get -qq install -y python-psutil python-gevent; } >/tmp/tmp9QhBZF 2>&1 make: *** [Makefile:240: setup] Error 1

$ make all gives error: Building cpython... Creating ../../bin/linux-x64-release/cpython make[1]: *** No rule to make target '../../deps/cpython/configure', needed by '../../bin/linux-x64-release/cpython/Makefile'. Stop. make: *** [Makefile:258: bin/linux-x64-release/cpython/libpython3.7m-fixed.a] Error 2

[gkorland]

@irthomasthomas did you run make fetch? I got the same error on make setup and still worked for me after.

make fetch
make clean 
make all

[irthomasthomas]

@gkorland I hadn't noticed in the logs the make fetch was failing because I was using the zip file instead of the repo. But after fixing that the make all step is failing for another reason...

Creating a virtualenv for this project… Using /usr/bin/python3.7m (3.7.3) to create virtualenv… ⠋RuntimeError: failed to query /usr/bin/python3.7m with code 1 err: 'Traceback (most recent call last):\n File "/usr/lib/python3/dist-packages/virtualenv/discovery/py_info.py", line 16, in \n from distutils.command.install import SCHEME_KEYS\nModuleNotFoundError: No module named 'distutils.command'\n'

Virtualenv location: /root/RedisGears/bin/linux-x64-release/redislabs/lib/modules/python3/.venv Warning: Your Pipfile requires python_version 3.7, but you are using None (/root/R/bin/l/r/lib/m/p/.venv/bin/python). $ pipenv check will surely fail. Creating a virtualenv for this project… Using /var/opt/redislabs/lib/modules/python3/bin/python3 (3.7.2) to create virtualenv… ⠙created virtual environment CPython3.7.2.final.0-64 in 427ms creator CPython3Posix(dest=/root/RedisGears/bin/linux-x64-release/redislabs/lib/modules/python3/.venv, clear=False, global=False)
seeder FromAppData(download=False, pep517=latest, webencodings=latest, html5lib=latest, pkg_resources=latest, lockfile=latest, colorama=latest, progress=latest, distlib=latest, ipaddr=latest, chardet=latest, retrying=latest, appdirs=latest, six=latest, pyparsing=latest, setuptools=latest, wheel=latest, msgpack=latest, distro=latest, requests=latest, pytoml=latest, certifi=latest, idna=latest, packaging=latest, urllib3=latest, contextlib2=latest, pip=latest, CacheControl=latest, via=copy, app_data_dir=/root/.local/share/virtualenv/seed-app-data/v1.0.1.debian)
activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator

Virtualenv location: /root/RedisGears/bin/linux-x64-release/redislabs/lib/modules/python3/.venv Pipfile.lock not found, creating… Locking [dev-packages] dependencies… Locking [packages] dependencies… Updated Pipfile.lock (69552c)! Installing dependencies from Pipfile.lock (69552c)… An error occurred while installing git+https://github.com/RedisGears/redisgears-py.git#egg=redisgears-py! Will try again. 🐍 ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 8/8 — 00:00:06 Installing initially–failed dependencies… Collecting redisgears-py▉▉▉▉▉▉▉▉▉▉▉▉▉ 0/1 — 00:00:00 Cloning https://github.com/RedisGears/redisgears-py.git to /tmp/pip-install-q0i01o71/redisgears-py
Requirement already satisfied: cloudpickle in ./.venv/lib/python3.7/site-packages (from gearsclient) (1.4.0)
Requirement already satisfied: redis in ./.venv/lib/python3.7/site-packages (from gearsclient) (3.4.1)
Building wheels for collected packages: gearsclient, gearsclient
Building wheel for gearsclient (setup.py): started
Building wheel for gearsclient (setup.py): finished with status 'error'
Running setup.py clean for gearsclient
Building wheel for gearsclient (setup.py): started
Building wheel for gearsclient (setup.py): finished with status 'error'
Running setup.py clean for gearsclient
Failed to build gearsclient gearsclient
Installing collected packages: gearsclient
Running setup.py install for gearsclient: started
Running setup.py install for gearsclient: finished with status 'error'

Running command git clone -q https://github.com/RedisGears/redisgears-py.git /tmp/pip-install-q0i01o71/redisgears-py WARNING: Generating metadata for package redisgears-py produced metadata for project name gearsclient. Fix your #egg=redisgears-py fragments. ERROR: Command errored out with exit status 1:
command: /root/RedisGears/bin/linux-x64-release/redislabs/lib/modules/python3/.venv/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-q0i01o71/redisgears-py/setup.py'"'"'; file='"'"'/tmp/pip-install-q0i01o71/redisgears-py/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-c6uawxr8
cwd: /tmp/pip-install-q0i01o71/redisgears-py/
Complete output (48 lines):
running bdist_wheel
running build
running build_py
creating build
creating build/lib
creating build/lib/gearsclient
copying gearsclient/init.py -> build/lib/gearsclient
copying gearsclient/redisgears_builder.py -> build/lib/gearsclient
warning: build_py: byte-compiling is disabled, skipping.

installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
Traceback (most recent call last):
File "", line 1, in
File "/tmp/pip-install-q0i01o71/redisgears-py/setup.py", line 28, in
author_email='[email protected]'
File "/root/RedisGears/bin/linux-x64-release/redislabs/lib/modules/python3/.venv/lib/python3.7/site-packages/setuptools/init.py", line 145, in setup
return distutils.core.setup(**attrs)
File "/var/opt/redislabs/lib/modules/python3/lib/python3.7/distutils/core.py", line 148, in setup
dist.run_commands()
File "/var/opt/redislabs/lib/modules/python3/lib/python3.7/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/var/opt/redislabs/lib/modules/python3/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/root/RedisGears/bin/linux-x64-release/redislabs/lib/modules/python3/.venv/lib/python3.7/site-packages/wheel/bdist_wheel.py", line 259, in run
self.run_command('install')
File "/var/opt/redislabs/lib/modules/python3/lib/python3.7/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/var/opt/redislabs/lib/modules/python3/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/root/RedisGears/bin/linux-x64-release/redislabs/lib/modules/python3/.venv/lib/python3.7/site-packages/setuptools/command/install.py", line 61, in run
return orig.install.run(self)
File "/var/opt/redislabs/lib/modules/python3/lib/python3.7/distutils/command/install.py", line 557, in run
self.run_command(cmd_name)
File "/var/opt/redislabs/lib/modules/python3/lib/python3.7/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/var/opt/redislabs/lib/modules/python3/lib/python3.7/distutils/dist.py", line 984, in run_command
cmd_obj.ensure_finalized()
File "/var/opt/redislabs/lib/modules/python3/lib/python3.7/distutils/cmd.py", line 107, in ensure_finalized
self.finalize_options()
File "/root/RedisGears/bin/linux-x64-release/redislabs/lib/modules/python3/.venv/lib/python3.7/site-packages/setuptools/command/install_lib.py", line 17, in finalize_options
self.set_undefined_options('install',('install_layout','install_layout'))
File "/var/opt/redislabs/lib/modules/python3/lib/python3.7/distutils/cmd.py", line 290, in set_undefined_options
setattr(self, dst_option, getattr(src_cmd_obj, src_option))
File "/var/opt/redislabs/lib/modules/python3/lib/python3.7/distutils/cmd.py", line 103, in getattr
raise AttributeError(attr)
AttributeError: install_layout

ERROR: Failed building wheel for gearsclient
ERROR: Command errored out with exit status 1:
command: /root/RedisGears/bin/linux-x64-release/redislabs/lib/modules/python3/.venv/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-q0i01o71/redisgears-py/setup.py'"'"'; file='"'"'/tmp/pip-install-q0i01o71/redisgears-py/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-pffzz733
cwd: /tmp/pip-install-q0i01o71/redisgears-py/
Complete output (48 lines):
running bdist_wheel
running build
running build_py
creating build
creating build/lib
creating build/lib/gearsclient
copying gearsclient/init.py -> build/lib/gearsclient
copying gearsclient/redisgears_builder.py -> build/lib/gearsclient
warning: build_py: byte-compiling is disabled, skipping.

installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
Traceback (most recent call last):
File "", line 1, in
File "/tmp/pip-install-q0i01o71/redisgears-py/setup.py", line 28, in
author_email='[email protected]'
File "/root/RedisGears/bin/linux-x64-release/redislabs/lib/modules/python3/.venv/lib/python3.7/site-packages/setuptools/init.py", line 145, in setup
return distutils.core.setup(**attrs)
File "/var/opt/redislabs/lib/modules/python3/lib/python3.7/distutils/core.py", line 148, in setup
dist.run_commands()
File "/var/opt/redislabs/lib/modules/python3/lib/python3.7/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/var/opt/redislabs/lib/modules/python3/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/root/RedisGears/bin/linux-x64-release/redislabs/lib/modules/python3/.venv/lib/python3.7/site-packages/wheel/bdist_wheel.py", line 259, in run
self.run_command('install')
File "/var/opt/redislabs/lib/modules/python3/lib/python3.7/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/var/opt/redislabs/lib/modules/python3/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/root/RedisGears/bin/linux-x64-release/redislabs/lib/modules/python3/.venv/lib/python3.7/site-packages/setuptools/command/install.py", line 61, in run
return orig.install.run(self)
File "/var/opt/redislabs/lib/modules/python3/lib/python3.7/distutils/command/install.py", line 557, in run
self.run_command(cmd_name)
File "/var/opt/redislabs/lib/modules/python3/lib/python3.7/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/var/opt/redislabs/lib/modules/python3/lib/python3.7/distutils/dist.py", line 984, in run_command
cmd_obj.ensure_finalized()
File "/var/opt/redislabs/lib/modules/python3/lib/python3.7/distutils/cmd.py", line 107, in ensure_finalized
self.finalize_options()
File "/root/RedisGears/bin/linux-x64-release/redislabs/lib/modules/python3/.venv/lib/python3.7/site-packages/setuptools/command/install_lib.py", line 17, in finalize_options
self.set_undefined_options('install',('install_layout','install_layout'))
File "/var/opt/redislabs/lib/modules/python3/lib/python3.7/distutils/cmd.py", line 290, in set_undefined_options
setattr(self, dst_option, getattr(src_cmd_obj, src_option))
File "/var/opt/redislabs/lib/modules/python3/lib/python3.7/distutils/cmd.py", line 103, in getattr
raise AttributeError(attr)
AttributeError: install_layout

ERROR: Failed building wheel for gearsclient
ERROR: Command errored out with exit status 1:
command: /root/RedisGears/bin/linux-x64-release/redislabs/lib/modules/python3/.venv/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-q0i01o71/redisgears-py/setup.py'"'"'; file='"'"'/tmp/pip-install-q0i01o71/redisgears-py/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /tmp/pip-record-1d64n_pe/install-record.txt --single-version-externally-managed --compile --install-headers /root/RedisGears/bin/linux-x64-release/redislabs/lib/modules/python3/.venv/include/site/python3.7/gearsclient
cwd: /tmp/pip-install-q0i01o71/redisgears-py/
Complete output (40 lines):
running install
running build
running build_py
creating build
creating build/lib
creating build/lib/gearsclient
copying gearsclient/init.py -> build/lib/gearsclient
copying gearsclient/redisgears_builder.py -> build/lib/gearsclient
warning: build_py: byte-compiling is disabled, skipping.

running install_lib                                                                                                                        
Traceback (most recent call last):                                                                                                         
  File "<string>", line 1, in <module>                                                                                                     
  File "/tmp/pip-install-q0i01o71/redisgears-py/setup.py", line 28, in <module>                                                            
    author_email='[email protected]'                                                                                                       
  File "/root/RedisGears/bin/linux-x64-release/redislabs/lib/modules/python3/.venv/lib/python3.7/site-packages/setuptools/__init__.py", line 145, in setup                                                                                                                                
    return distutils.core.setup(**attrs)                                                                                                   
  File "/var/opt/redislabs/lib/modules/python3/lib/python3.7/distutils/core.py", line 148, in setup                                        
    dist.run_commands()                                                                                                                    
  File "/var/opt/redislabs/lib/modules/python3/lib/python3.7/distutils/dist.py", line 966, in run_commands                                 
    self.run_command(cmd)
  File "/var/opt/redislabs/lib/modules/python3/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/root/RedisGears/bin/linux-x64-release/redislabs/lib/modules/python3/.venv/lib/python3.7/site-packages/setuptools/command/install.py", line 61, in run
    return orig.install.run(self)
  File "/var/opt/redislabs/lib/modules/python3/lib/python3.7/distutils/command/install.py", line 557, in run
    self.run_command(cmd_name)
  File "/var/opt/redislabs/lib/modules/python3/lib/python3.7/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/var/opt/redislabs/lib/modules/python3/lib/python3.7/distutils/dist.py", line 984, in run_command
    cmd_obj.ensure_finalized()
  File "/var/opt/redislabs/lib/modules/python3/lib/python3.7/distutils/cmd.py", line 107, in ensure_finalized
    self.finalize_options()
  File "/root/RedisGears/bin/linux-x64-release/redislabs/lib/modules/python3/.venv/lib/python3.7/site-packages/setuptools/command/install_lib.py", line 17, in finalize_options
    self.set_undefined_options('install',('install_layout','install_layout'))
  File "/var/opt/redislabs/lib/modules/python3/lib/python3.7/distutils/cmd.py", line 290, in set_undefined_options
    setattr(self, dst_option, getattr(src_cmd_obj, src_option))
  File "/var/opt/redislabs/lib/modules/python3/lib/python3.7/distutils/cmd.py", line 103, in __getattr__
    raise AttributeError(attr)
AttributeError: install_layout
----------------------------------------

ERROR: Command errored out with exit status 1: /root/RedisGears/bin/linux-x64-release/redislabs/lib/modules/python3/.venv/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-q0i01o71/redisgears-py/setup.py'"'"'; file='"'"'/tmp/pip-install-q0i01o71/redisgears-py/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /tmp/pip-record-1d64n_pe/install-record.txt --single-version-externally-managed --compile --install-headers /root/RedisGears/bin/linux-x64-release/redislabs/lib/modules/python3/.venv/include/site/python3.7/gearsclient Check the logs for full command output.

☤ ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 0/1 — 00:00:03 make[1]: *** [Makefile:86: /var/opt/redislabs/lib/modules/python3/.venv/bin/python3] Error 1 make: *** [Makefile:262: pyenv] Error 2

[MeirShpilraien]

@irthomasthomas did you try with v0.99.1 or with master?

[AlexMikhalev]

I am trying to compile on Pop os 20.04 which is focal derivative and build fails for both master and 1.0 tag:

cat /etc/lsb-release
DISTRIB_ID=Pop
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Pop!_OS 20.04 LTS"
$ sudo make setup SHOW=1
make all SHOW=1
make fetch
make clean
make all SHOW=1
make all SHOW=1     
make: *** No rule to make target '/home/alex/infrastructure/RedisGears/bin/linux-x64-release/python3_1.0.3', needed by 'artifacts/release/redisgears-dependencies.linux-pop20.04-x64.1.0.3.tgz'. Stop.

[MeirShpilraien]

Hey @AlexMikhalev Can you try : make clean ALL=1 DEPS=1; make all

[AlexMikhalev]

@MeirShpilraien thank you for suggestion so quickly. Failed on packing:

Packing dependencies...
Created artifacts/release/redisgears-dependencies.linux-pop20.04-x64.1.0.3.tgz
Created artifacts/snapshot/redisgears-dependencies.linux-pop20.04-x64.1.0.tgz
Linking bin/linux-x64-release/redisgears.so...
Packing module...
Failed to pack release
./pack.sh: line 74: -m: command not found
make: *** [Makefile:369: artifacts/release/redisgears.linux-pop20.04-x64.1.0.3.zip] Error 1

git status On branch 1.0 Your branch is up-to-date with 'origin/1.0'.

nothing to commit, working tree clean

[MeirShpilraien]

Interesting, @rafie any idea why the pack failed?

Regardless you should be good to go even without packing, the module was already built successfully and you should be able to run it.

[AlexMikhalev]

I tried master for the sake of completeness:

git checkout master
git submodule update --init --recursive
make clean ALL=1 DEPS=1; make all

....
gcc -shared -o bin/linux-x64-release//snapshot/redisgears.so ./bin/linux-x64-release/src/utils/adlist.o ./bin/linux-x64-release/src/utils/buffer.o ./bin/linux-x64-release/src/utils/dict.o ./bin/linux-x64-release/src/module.o ./bin/linux-x64-release/src/execution_plan.o ./bin/linux-x64-release/src/mgmt.o ./bin/linux-x64-release/src/readers/keys_reader.o ./bin/linux-x64-release/src/utils/thpool.o ./bin/linux-x64-release/src/mappers.o ./bin/linux-x64-release/src/record.o ./bin/linux-x64-release/src/cluster.o ./bin/linux-x64-release/src/commands.o ./bin/linux-x64-release/src/readers/streams_reader.o ./bin/linux-x64-release/src/globals.o ./bin/linux-x64-release/src/config.o ./bin/linux-x64-release/src/lock_handler.o ./bin/linux-x64-release/src/module_init.o ./bin/linux-x64-release/src/slots_table.o ./bin/linux-x64-release/src/common.o ./bin/linux-x64-release/src/readers/command_reader.o ./bin/linux-x64-release/src/readers/shardid_reader.o ./bin/linux-x64-release/src/crc16.o ./bin/linux-x64-release/src/redisgears_python.o ./bin/linux-x64-release/src/snapshot-deps.o  -Wl,-Bstatic bin/linux-x64-release/hiredis/libhiredis.a bin/linux-x64-release/libevent/.libs/libevent.a -Wl,-Bdynamic -Wl,--whole-archive bin/linux-x64-release/cpython/libpython3.7m-fixed.a -lutil -luuid -Wl,--no-whole-archive
strip --strip-debug --strip-unneeded bin/linux-x64-release/redisgears.so
strip --strip-debug --strip-unneeded bin/linux-x64-release//snapshot/redisgears.so
ln -sf ./bin/linux-x64-release/redisgears.so redisgears.so
Packing module...
RAMP=1 DEPS=0 VARIANT= ./pack.sh ./bin/linux-x64-release/redisgears.so
Failed to pack release
./pack.sh: line 74: -m: command not found
make: *** [Makefile:369: artifacts/release/redisgears.linux-focal-x64.99.99.99.zip] Error 1

[rafie]

@MeirShpilraien There's an error on pack.sh line 66:

local ramp="$(command -v python) -m RAMP.ramp"

We assume python is a command, which is true inside a virtualenv and on ubuntu:focal docker image, but apparently not on every Ubuntu Focal installation. It should have been python2. Will fix.

[AlexMikhalev]

@rafie if I follow quickstart doc ./deps/readies/bin/getpy2 is installing python2 on my machine, perhaps python interpreter name should come as a parameter into pack from readies?

[rafie]

@AlexMikhalev The python interpreter name is a funny thing. Most systems are playing well with python2 and python3. Some lack python2 in favor of python27. Other are have different defaults for python. A point of stability is virtualenv, which always calls its python python. My assumption is that there always be python2, and if it's missing, the user can create a symlink to establish it. We even support FIX=1 ./deps/readies/bin/getpy2 to achieve just that, although it not the default behavior. We also recommend of working within a virtualenv for the sake of system stability, but we don't demand it. PS if system stability is a concern, it's possible to build Gears inside a container that's similar to the host system, and to pull it from there (so one does not to install the world in the host system). Also: I added support for PopOS build.

[AlexMikhalev]

I had an opportunity to check on the new server - the master build still fails:

Packing dependencies...
Created artifacts/release/redisgears-dependencies.linux-focal-x64.99.99.99.tgz
Created artifacts/snapshot/redisgears-dependencies.linux-focal-x64.master.tgz
Linking bin/linux-x64-release/redisgears.so...
Packing module...
Created artifacts/release/artifacts/release/redisgears.linux-focal-x64.99.99.99.zip
Created artifacts/snapshot/artifacts/snapshot/redisgears.linux-focal-x64.master.zip
make: *** [Makefile:381: verify-packs] Error 1
alex@hetznersb100-bigbox:~/infrastructure/RedisGears$ cat /etc/issue
Ubuntu 20.04.1 LTS \n \l

alex@bigbox:~/infrastructure/RedisGears$ python
Python 2.7.18 (default, Aug  4 2020, 11:16:42) 
[GCC 9.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> 
alex@bigbox:~/infrastructure/RedisGears$ python3
Python 3.8.5 (default, Jul 28 2020, 12:59:40) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 
alex@bigbox:~/infrastructure/RedisGears$ command -v python
/usr/bin/python

[MeirShpilraien]

@rafie any idea what's wrong?

[rafie]

@MeirShpilraien @AlexMikhalev Just built master on Ubuntu 20 docker, build passed Ok. Just make sure you've got Redis installed (./deps/readies/bin/getredis) before you build. Afterwards, it's possible to compare the non-Docker environment to the Docker one.

[AlexMikhalev]

@rafie Redis Server installed redis-server -v Redis server v=6.0.9 sha=25214bd7:0 malloc=jemalloc-5.1.0 bits=64 build=476d782f220df0d3

how can I help you check the difference between hardware and docker image? This server only has Redis, RedisCluster + gears, and RedisGraph (in docker).

[rafie]

@AlexMikhalev I first would like to pinpoint where exactly does your build fail. It obviously bails out on the verify-packs make target, so we might want to run make verify-packs -n and see which command is the problematic one. If the problem is the comparison of SHA values, we can try removing all built packages (from artifacts/ dir) and building with make pack.

[AlexMikhalev]

Full log from git checkout

make verify-packs -n
set -e ;\
MOD=`./deps/readies/bin/redis-cmd --loadmodule ./bin/linux-x64-release/redisgears.so -- RG.CONFIGGET dependenciesSha256 2> /tmp/redisgears-sha.err` ;\
if [[ -z OD ]]; then \
	>2& cat /tmp/redisgears-sha.err ;\
	exit 1 ;\
fi ;\
REL=`cat artifacts/release/redisgears-dependencies.linux-focal-x64.99.99.99.tgz.sha256` ;\
SNAP=`cat artifacts/snapshot/redisgears-dependencies.linux-focal-x64.master.tgz.sha256` ;\
if [[ $MOD != $REL || $REL != $SNAP ]]; then \
	echo "Module and package SHA256 don't match." ;\
	echo "\"$MOD\"" ;\
	echo "\"$REL\"" ;\
	echo "\"$SNAP\"" ;\
	exit 1 ;\
else \
	echo "Signatures match." ;\
fi

I checked /tmp/redisgears-sha.err

cat /tmp/redisgears-sha.err
857836:C 06 Jan 2021 11:00:35.093 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
857836:C 06 Jan 2021 11:00:35.093 # Redis version=6.0.9, bits=64, commit=25214bd7, modified=0, pid=857836, just started
857836:C 06 Jan 2021 11:00:35.093 # Configuration loaded
857836:M 06 Jan 2021 11:00:35.094 * Increased maximum number of open files to 10032 (it was originally set to 1024).
857836:M 06 Jan 2021 11:00:35.095 # Could not create server TCP listening socket *:6379: bind: Address already in use

As I already have the server running.

[iakkus]

I followed the same steps here (including the error messages regarding lack of python2 on the machine). The final step regarding the verification was also failing. However, then make clean make all successfully passed also that step.

However, something else must be missing: in my redis-server, even though the rg module is loaded, i can't do RG.PYEXECUTE, although all other commands seem to be available.

Edit: It seems like with default configuration, that command is available (e.g., when make/pack.sh starts the redis-server). However, with a custom configuration to load the module (e.g., when redis-server is manually started), it fails, even though the module itself is successfully loaded (according to the redis-server logs). Perhaps, I am missing some other dependency?

[MeirShpilraien]

@iakkus you are probably running with the master version, on master the python was extracted to be a gears plugin and you need to tell RedisGears to load it by specifying Plugin ./gears_python.so as an argument right after the --loadmodule. The gears_python.so should be on the root directory. So to sum up you need to run it like this (from the root directory): redis-server --loadmodule ./redisgear.so Plugin gears_python.so (edited)

The reason we did it is because we plan to support more languages in future versions, one of them is the JVM (still under development but already at a testable state: https://github.com/RedisGears/JVMPlugin)

[iakkus]

Ah, that makes more sense now. Thanks!