higlass-manage
higlass-manage copied to clipboard
higlass
Permission errors with docker while launching higlass-manage
Dear higlass team,
I followed instructions to install higlass-manage: Prerequisites:
$ python3 --version
Python 3.10.6
Docker installed following https://docs.docker.com/desktop/install/ubuntu/ and I can launch Docker desktop
$ docker compose version
Docker Compose version v2.15.1
$ docker --version
Docker version 20.10.22, build 3a2c30b
$ docker version
Client: Docker Engine - Community
Cloud integration: v1.0.29
Version: 20.10.22
API version: 1.41
Go version: go1.18.9
Git commit: 3a2c30b
Built: Thu Dec 15 22:28:04 2022
OS/Arch: linux/amd64
Context: desktop-linux
Experimental: true
Server: Docker Desktop 4.16.1 (95567)
Engine:
Version: 20.10.22
API version: 1.41 (minimum version 1.12)
Go version: go1.18.9
Git commit: 42c8b31
Built: Thu Dec 15 22:26:14 2022
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.6.14
GitCommit: 9ba4b250366a5ddde94bb7c9d1def331423aa323
runc:
Version: 1.1.4
GitCommit: v1.1.4-0-g5fd4c4d
docker-init:
Version: 0.19.0
GitCommit: de40ad0
Higlass is installed with pip install higlass-manage
$ higlass-manage version
0.8.0
But when I try to launch higlass, I hav the following errors:
$ higlass-manage start
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
httplib_response = self._make_request(
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 394, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/lib/python3.10/http/client.py", line 1282, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/lib/python3.10/http/client.py", line 1328, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/lib/python3.10/http/client.py", line 1277, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib/python3.10/http/client.py", line 1037, in _send_output
self.send(msg)
File "/usr/lib/python3.10/http/client.py", line 975, in send
self.connect()
File "/home/david/.local/lib/python3.10/site-packages/docker/transport/unixconn.py", line 30, in connect
sock.connect(self.unix_socket)
PermissionError: [Errno 13] Permission denied
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/david/.local/lib/python3.10/site-packages/requests/adapters.py", line 489, in send
resp = conn.urlopen(
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
retries = retries.increment(
File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 532, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/usr/lib/python3/dist-packages/six.py", line 718, in reraise
raise value.with_traceback(tb)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
httplib_response = self._make_request(
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 394, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/lib/python3.10/http/client.py", line 1282, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/lib/python3.10/http/client.py", line 1328, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/lib/python3.10/http/client.py", line 1277, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib/python3.10/http/client.py", line 1037, in _send_output
self.send(msg)
File "/usr/lib/python3.10/http/client.py", line 975, in send
self.connect()
File "/home/david/.local/lib/python3.10/site-packages/docker/transport/unixconn.py", line 30, in connect
sock.connect(self.unix_socket)
urllib3.exceptions.ProtocolError: ('Connection aborted.', PermissionError(13, 'Permission denied'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/david/.local/lib/python3.10/site-packages/docker/api/client.py", line 214, in _retrieve_server_version
return self.version(api_version=False)["ApiVersion"]
File "/home/david/.local/lib/python3.10/site-packages/docker/api/daemon.py", line 181, in version
return self._result(self._get(url), json=True)
File "/home/david/.local/lib/python3.10/site-packages/docker/utils/decorators.py", line 46, in inner
return f(self, *args, **kwargs)
File "/home/david/.local/lib/python3.10/site-packages/docker/api/client.py", line 237, in _get
return self.get(url, **self._set_request_timeout(kwargs))
File "/home/david/.local/lib/python3.10/site-packages/requests/sessions.py", line 600, in get
return self.request("GET", url, **kwargs)
File "/home/david/.local/lib/python3.10/site-packages/requests/sessions.py", line 587, in request
resp = self.send(prep, **send_kwargs)
File "/home/david/.local/lib/python3.10/site-packages/requests/sessions.py", line 701, in send
r = adapter.send(request, **kwargs)
File "/home/david/.local/lib/python3.10/site-packages/requests/adapters.py", line 547, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', PermissionError(13, 'Permission denied'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/david/.local/bin/higlass-manage", line 8, in <module>
sys.exit(cli())
File "/usr/lib/python3/dist-packages/click/core.py", line 1128, in __call__
return self.main(*args, **kwargs)
File "/usr/lib/python3/dist-packages/click/core.py", line 1053, in main
rv = self.invoke(ctx)
File "/usr/lib/python3/dist-packages/click/core.py", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/lib/python3/dist-packages/click/core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/lib/python3/dist-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "/home/david/.local/lib/python3.10/site-packages/higlass_manage/start.py", line 134, in start
_start(
File "/home/david/.local/lib/python3.10/site-packages/higlass_manage/start.py", line 177, in _start
client = docker.from_env()
File "/home/david/.local/lib/python3.10/site-packages/docker/client.py", line 96, in from_env
return cls(
File "/home/david/.local/lib/python3.10/site-packages/docker/client.py", line 45, in __init__
self.api = APIClient(*args, **kwargs)
File "/home/david/.local/lib/python3.10/site-packages/docker/api/client.py", line 197, in __init__
self._version = self._retrieve_server_version()
File "/home/david/.local/lib/python3.10/site-packages/docker/api/client.py", line 221, in _retrieve_server_version
raise DockerException(
docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', PermissionError(13, 'Permission denied'))
Could you help me and tell me what did I miss ? Thanks! David
You probably need to allow docker run as a non-root user. Have you tried this?
https://docs.docker.com/engine/install/linux-postinstall/#manage-docker-as-a-non-root-user
Ok thanks, it seems to work better but I had an other issue that seemed to come from cython:
$ higlass-manage view /tmp/sample.mcool
state True
temp_dir: /tmp/higlass-docker
Inferred filetype: cooler
Inferred datatype: matrix
Traceback (most recent call last):
File "/home/david/.local/bin/higlass-manage", line 8, in <module>
sys.exit(cli())
File "/usr/lib/python3/dist-packages/click/core.py", line 1128, in __call__
return self.main(*args, **kwargs)
File "/usr/lib/python3/dist-packages/click/core.py", line 1053, in main
rv = self.invoke(ctx)
File "/usr/lib/python3/dist-packages/click/core.py", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/lib/python3/dist-packages/click/core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/lib/python3/dist-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "/home/david/.local/lib/python3.10/site-packages/higlass_manage/view.py", line 157, in view
from higlass.client import Track, View, ViewConf
File "/home/david/.local/lib/python3.10/site-packages/higlass/__init__.py", line 3, in <module>
from .server import Server
File "/home/david/.local/lib/python3.10/site-packages/higlass/server.py", line 5, in <module>
import cytoolz as toolz
File "/home/david/.local/lib/python3.10/site-packages/cytoolz/__init__.py", line 3, in <module>
from .functoolz import *
ImportError: /home/david/.local/lib/python3.10/site-packages/cytoolz/functoolz.cpython-310-x86_64-linux-gnu.so: undefined symbol: _PyGen_Send
So I updated cython as suggested here https://github.com/Toblerity/Fiona/issues/1043, which may be incompatible with cooler but it allows me to open an higlass instance and visualise the sample.cool.
pip install --upgrade cytoolz
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: cytoolz in /home/david/.local/lib/python3.10/site-packages (0.10.1)
Collecting cytoolz
Downloading cytoolz-0.12.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.8 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/1.8 MB 7.7 MB/s eta 0:00:00
Requirement already satisfied: toolz>=0.8.0 in /home/david/.local/lib/python3.10/site-packages (from cytoolz) (0.12.0)
Installing collected packages: cytoolz
Attempting uninstall: cytoolz
Found existing installation: cytoolz 0.10.1
Uninstalling cytoolz-0.10.1:
Successfully uninstalled cytoolz-0.10.1
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
cooler 0.8.11 requires cytoolz<0.11, but you have cytoolz 0.12.1 which is incompatible.
Successfully installed cytoolz-0.12.1
However, higlass-manage browse command does not seem to exist. So I could open the app with http://localhost:8989/app, find my track in the list, but can't visualise it (.cool file, 10kb res). I could visualise however public data listed but this would quite, even after increase docker memory limits https://www.thorsten-hans.com/limit-memory-for-docker-containers/ ( I guess it could change something but not sure, I'm super new to docker)
Also, can you confirm higlass is the right tool for a fast and smooth exploration Hi-C data in .cool or .mcool format on the fly, like IGV is for other -seq ? I was used to work with .hic file from Juicer tools, and visualisation with Juicebox is actually quite fast, even at high res while higlass seems lagging a lot. Would it be faster running through python/jupyter notebook ?
Thanks for your support, David
So I could open the app with http://localhost:8989/app, find my track in the list, but can't visualise it (.cool file, 10kb res).
HiGlass requires a multi-res cooler file. To go from single resolution to multires is easy: cooler zoomify my.cool.
Hard to say if higlass is the right tool. It's a tool that some people like and others don't. As you said, Juicer works well as well. The main factor that affects higlass performance is the resolutions that are present in your mcool file. If you use cooler zoomify you'll get the best performance.