ceph-iscsi
ceph-iscsi copied to clipboard
Published
20 hours ago •
ceph
ceph
ceph-iscsi on Ubuntu / no attribute cluster
Trying to get it running on Ubuntu 19.10 / Eoan.
gw service is running but the api makes some problems:
server03 /opt/ceph-iscsi # /usr/bin/rbd-target-api
Traceback (most recent call last):
File "/usr/bin/rbd-target-api", line 4, in <module>
__import__('pkg_resources').run_script('ceph-iscsi==3.3', 'rbd-target-api')
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 666, in run_script
self.require(requires)[0].run_script(script_name, ns)
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 1469, in run_script
exec(script_code, namespace, namespace)
File "/usr/local/lib/python3.7/dist-packages/ceph_iscsi-3.3-py3.7.egg/EGG-INFO/scripts/rbd-target-api", line 2916, in <module>
File "/usr/local/lib/python3.7/dist-packages/ceph_iscsi-3.3-py3.7.egg/ceph_iscsi_config/common.py", line 80, in __init__
File "/usr/local/lib/python3.7/dist-packages/ceph_iscsi-3.3-py3.7.egg/ceph_iscsi_config/common.py", line 34, in __init__
File "rados.pyx", line 625, in rados.Rados.__init__
File "rados.pyx", line 516, in rados.requires.wrapper.validate_func
File "rados.pyx", line 665, in rados.Rados.__setup
rados.Error: rados_initialize failed with error code: -22
Exception ignored in: <function CephCluster.__del__ at 0x7f28c80210e0>
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/ceph_iscsi-3.3-py3.7.egg/ceph_iscsi_config/common.py", line 42, in __del__
AttributeError: 'CephCluster' object has no attribute 'cluster'
Install Log:
server03 /opt/ceph-iscsi # python3 setup.py install --install-scripts=/usr/bin
running install
running bdist_egg
running egg_info
writing ceph_iscsi.egg-info/PKG-INFO
writing dependency_links to ceph_iscsi.egg-info/dependency_links.txt
writing top-level names to ceph_iscsi.egg-info/top_level.txt
reading manifest file 'ceph_iscsi.egg-info/SOURCES.txt'
writing manifest file 'ceph_iscsi.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/ceph_iscsi_config
copying build/lib/ceph_iscsi_config/gateway_object.py -> build/bdist.linux-x86_64/egg/ceph_iscsi_config
copying build/lib/ceph_iscsi_config/group.py -> build/bdist.linux-x86_64/egg/ceph_iscsi_config
copying build/lib/ceph_iscsi_config/settings.py -> build/bdist.linux-x86_64/egg/ceph_iscsi_config
copying build/lib/ceph_iscsi_config/common.py -> build/bdist.linux-x86_64/egg/ceph_iscsi_config
copying build/lib/ceph_iscsi_config/target.py -> build/bdist.linux-x86_64/egg/ceph_iscsi_config
copying build/lib/ceph_iscsi_config/client.py -> build/bdist.linux-x86_64/egg/ceph_iscsi_config
copying build/lib/ceph_iscsi_config/lio.py -> build/bdist.linux-x86_64/egg/ceph_iscsi_config
copying build/lib/ceph_iscsi_config/alua.py -> build/bdist.linux-x86_64/egg/ceph_iscsi_config
copying build/lib/ceph_iscsi_config/__init__.py -> build/bdist.linux-x86_64/egg/ceph_iscsi_config
copying build/lib/ceph_iscsi_config/backstore.py -> build/bdist.linux-x86_64/egg/ceph_iscsi_config
copying build/lib/ceph_iscsi_config/discovery.py -> build/bdist.linux-x86_64/egg/ceph_iscsi_config
copying build/lib/ceph_iscsi_config/gateway.py -> build/bdist.linux-x86_64/egg/ceph_iscsi_config
copying build/lib/ceph_iscsi_config/metrics.py -> build/bdist.linux-x86_64/egg/ceph_iscsi_config
copying build/lib/ceph_iscsi_config/utils.py -> build/bdist.linux-x86_64/egg/ceph_iscsi_config
copying build/lib/ceph_iscsi_config/lun.py -> build/bdist.linux-x86_64/egg/ceph_iscsi_config
copying build/lib/ceph_iscsi_config/gateway_setting.py -> build/bdist.linux-x86_64/egg/ceph_iscsi_config
creating build/bdist.linux-x86_64/egg/gwcli
copying build/lib/gwcli/ceph.py -> build/bdist.linux-x86_64/egg/gwcli
copying build/lib/gwcli/client.py -> build/bdist.linux-x86_64/egg/gwcli
copying build/lib/gwcli/node.py -> build/bdist.linux-x86_64/egg/gwcli
copying build/lib/gwcli/__init__.py -> build/bdist.linux-x86_64/egg/gwcli
copying build/lib/gwcli/storage.py -> build/bdist.linux-x86_64/egg/gwcli
copying build/lib/gwcli/gateway.py -> build/bdist.linux-x86_64/egg/gwcli
copying build/lib/gwcli/utils.py -> build/bdist.linux-x86_64/egg/gwcli
copying build/lib/gwcli/hostgroup.py -> build/bdist.linux-x86_64/egg/gwcli
byte-compiling build/bdist.linux-x86_64/egg/ceph_iscsi_config/gateway_object.py to gateway_object.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/ceph_iscsi_config/group.py to group.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/ceph_iscsi_config/settings.py to settings.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/ceph_iscsi_config/common.py to common.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/ceph_iscsi_config/target.py to target.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/ceph_iscsi_config/client.py to client.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/ceph_iscsi_config/lio.py to lio.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/ceph_iscsi_config/alua.py to alua.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/ceph_iscsi_config/__init__.py to __init__.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/ceph_iscsi_config/backstore.py to backstore.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/ceph_iscsi_config/discovery.py to discovery.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/ceph_iscsi_config/gateway.py to gateway.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/ceph_iscsi_config/metrics.py to metrics.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/ceph_iscsi_config/utils.py to utils.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/ceph_iscsi_config/lun.py to lun.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/ceph_iscsi_config/gateway_setting.py to gateway_setting.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/gwcli/ceph.py to ceph.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/gwcli/client.py to client.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/gwcli/node.py to node.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/gwcli/__init__.py to __init__.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/gwcli/storage.py to storage.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/gwcli/gateway.py to gateway.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/gwcli/utils.py to utils.cpython-37.pyc
byte-compiling build/bdist.linux-x86_64/egg/gwcli/hostgroup.py to hostgroup.cpython-37.pyc
installing package data to build/bdist.linux-x86_64/egg
running install_data
creating build/bdist.linux-x86_64/egg/EGG-INFO
installing scripts to build/bdist.linux-x86_64/egg/EGG-INFO/scripts
running install_scripts
running build_scripts
creating build/bdist.linux-x86_64/egg/EGG-INFO/scripts
copying build/scripts-3.7/rbd-target-gw.py -> build/bdist.linux-x86_64/egg/EGG-INFO/scripts
copying build/scripts-3.7/rbd-target-api.py -> build/bdist.linux-x86_64/egg/EGG-INFO/scripts
copying build/scripts-3.7/gwcli.py -> build/bdist.linux-x86_64/egg/EGG-INFO/scripts
changing mode of build/bdist.linux-x86_64/egg/EGG-INFO/scripts/rbd-target-gw.py to 755
changing mode of build/bdist.linux-x86_64/egg/EGG-INFO/scripts/rbd-target-api.py to 755
changing mode of build/bdist.linux-x86_64/egg/EGG-INFO/scripts/gwcli.py to 755
copying ceph_iscsi.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying ceph_iscsi.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying ceph_iscsi.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying ceph_iscsi.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
creating 'dist/ceph_iscsi-3.3-py3.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing ceph_iscsi-3.3-py3.7.egg
Removing /usr/local/lib/python3.7/dist-packages/ceph_iscsi-3.3-py3.7.egg
Copying ceph_iscsi-3.3-py3.7.egg to /usr/local/lib/python3.7/dist-packages
ceph-iscsi 3.3 is already the active version in easy-install.pth
Installing gwcli script to /usr/bin
Installing rbd-target-api script to /usr/bin
Installing rbd-target-gw script to /usr/bin
Installed /usr/local/lib/python3.7/dist-packages/ceph_iscsi-3.3-py3.7.egg
Processing dependencies for ceph-iscsi==3.3
Finished processing dependencies for ceph-iscsi==3.3
For the initial issue it looks like the cluster_client_name or cephconf is not setup correctly. Did you modify those values in /etc/ceph/iscsi-gateway.cfg?
After we hit that, then we hit a bug in the ceph-iscsi code where we assume cluster is always set.
Yes, the values are set (Default Cluster Name, running on same Node, Nautilus 12.2.2):
server03 ~ # cat /etc/ceph/iscsi-gateway.cfg
[config]
# name of the *.conf file. A suitable conf file allowing access to the ceph
# cluster from the gateway node is required.
cluster_name = ceph
# Pool name where internal `gateway.conf` object is stored
pool = config_iscsi
# CephX client name
cluster_client_name = ceph.client.admin.keyring
# API settings.
# The api supports a number of options that allow you to tailor it to your
# local environment. If you want to run the api under https, you will need to
# create crt/key files that are compatible for each gateway node (i.e. not
# locked to a specific node). SSL crt and key files *must* be called
# iscsi-gateway.crt and iscsi-gateway.key and placed in /etc/ceph on *each*
# gateway node. With the SSL files in place, you can use api_secure = true
# to switch to https mode.
# To support the api, the bear minimum settings are;
api_secure = false
# Additional API configuration options are as follows (defaults shown);
api_user = admin
api_password = admin
api_port = 5000
trusted_ip_list = 127.0.0.1
# Refer to the ceph-iscsi-config/settings module for more options
server03 ~ # cat /etc/ceph/ceph.conf
[global]
fsid = 34f09a6f-7aa4-4645-beaa-87cfcce6ad08
mon_initial_members = server03
mon_host = 172.16.0.202
auth_cluster_required = cephx
auth_service_required = cephx
auth_client_required = cephx
osd pool default size = 2
crush chooseleaf type = 0
osd pool default pg num = 64
osd pool default pgp num = 64
[client]
rbd cache = true
rbd cache policy = writeback
rbd cache size = 24Gb
rbd cache max dirty = 20Gb
rbd cache target dirty = 16Gb
When i change the keyring to something different, the connection fails:
server03 ~ # cat /var/log/rbd-target-api/rbd-target-api.log
...
2019-10-10 07:56:38,082 ERROR [rbd-target-api:2918:<module>()] - Unable to connect to the cluster (keyring missing?) - [errno 2] error connecting to the cluster
2019-10-10 07:56:38,084 CRITICAL [rbd-target-api:2736:halt()] - Unable to open/read the configuration object
But when i change the cluster_name to something different, the error remains the same.
You have a file
/etc/ceph/ceph.client.admin.keyring
right?
If so, leave cluster_client_name unset in iscsi-gateway.conf or set it to client.admin (this is the default).
Yay, after unsetting cluster_client_name it's working as expected,thanks :)
But now in Ceph Dashboard:
Unsupported ceph-iscsi config version. Expected 9 but found 11.
Which ceph-iscsi Release is providing a Version 9 Config? Ceph Release is Nautilus 14.2.2: ceph version 14.2.2 (4f8fa0a0024755aae7d95567c63f11d6862d55be) nautilus (stable)
Sadly there is corrently no newer apt source fpr Ubuntu Eoan.
The PR that adds support for ceph-iscsi config v11 in Nautilus Ceph Dashboard is still open https://github.com/ceph/ceph/pull/30545 so I'm afraid you have to wait for Ceph Nautilus 14.2.5 to be released.
