CloudFusion
CloudFusion copied to clipboard
joe42
Failing on OS X 10.10
Any idea what's going wrong here? (The installation succeeded, with no warnings.)
$ cloudfusion -h
Traceback (most recent call last):
File "/usr/local/Homebrew/bin/cloudfusion", line 9, in <module>
load_entry_point('CloudFusion==6.4.16', 'console_scripts', 'cloudfusion')()
File "build/bdist.macosx-10.10-x86_64/egg/pkg_resources.py", line 356, in load_entry_point
"""The named metadata resource as a string"""
File "build/bdist.macosx-10.10-x86_64/egg/pkg_resources.py", line 2476, in load_entry_point
"""Insert self.location in path before its nearest parent directory"""
File "build/bdist.macosx-10.10-x86_64/egg/pkg_resources.py", line 2190, in load
Entry point syntax follows the form::
File "/usr/local/Homebrew/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/CloudFusion-6.4.16-py2.7.egg/cloudfusion/main.py", line 5, in <module>
from cloudfusion.fuse import FUSE
File "/usr/local/Homebrew/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/CloudFusion-6.4.16-py2.7.egg/cloudfusion/fuse.py", line 245, in <module>
_libfuse = CDLL(_libfuse_path)
File "/usr/local/Homebrew/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ctypes/__init__.py", line 365, in __init__
self._handle = _dlopen(self._name, mode)
OSError: dlopen(/usr/local/Homebrew/lib/libfuse.dylib, 6): Library not loaded: /usr/local/lib/libosxfuse_i32.2.dylib
Referenced from: /usr/local/Homebrew/lib/libfuse.dylib
Reason: image not found
Hi alexreg,
I cannot reproduce the error since I don't have a mac.
A wild guess; Maybe the permissions set for the library /usr/local/lib/libosxfuse_i32.2.dylib do not allow access in this case so it cannot be loaded?
This seems to be a related issue:
http://apple.stackexchange.com/questions/114694/truecrypt-and-osx-fuse
Can you install the latest fusepy version and see if the same error shows up?
git clone https://github.com/terencehonles/fusepy.git fusepy
cd fusepy
cp examples/loopback.py .
mkdir back front
python loopback.py back front
If it works I will update it with the next release of CloudFusion.
joe
Ah, well /usr/local/lib/libosxfuse_i32.2.dylib does not exist as a file, probably because I'm on a 64-bit machine & OS. Odd that it's trying to load it still though?
Incidentally, I installed the latest fusepy and these are the errors I get when running that little snippet (the loopback.py script). https://gist.github.com/4cb06ee46b54792757ef
You may try creating a link to the actual library with:
ln -s /usr/local/lib/libosxfuse.dylib /usr/local/lib/libosxfuse_i32.2.dylib
Or try the suggestion of the last comment in the following thread and install a different Version: http://www.matthewratzloff.com/blog/2012/03/12/macfusion-on-os-x-lion/
I'm afraid that I'm on OS X 10.10, and since your software requires libfuse 2.8+, I have to use the 'master' branch of OSXFUSE, as this is the only recent enough version for Mac OS X.
Anyway, it seems this libosxfuse_i32.2.dylib file has been removed, as it only existed in OSXFUSE anyway for backwards compatibility with MacFUSE (which is now defunct/absorbed into OSXFUSE. See https://github.com/osxfuse/osxfuse/issues/183#issuecomment-64319419 for full info.
By the way, I tried your suggested symlink fix, but now I'm getting a new error:
Traceback (most recent call last):
File "/usr/local/Homebrew/bin/cloudfusion", line 9, in <module>
load_entry_point('CloudFusion==6.4.16', 'console_scripts', 'cloudfusion')()
File "build/bdist.macosx-10.10-x86_64/egg/pkg_resources.py", line 356, in load_entry_point
"""The named metadata resource as a string"""
File "build/bdist.macosx-10.10-x86_64/egg/pkg_resources.py", line 2476, in load_entry_point
"""Insert self.location in path before its nearest parent directory"""
File "build/bdist.macosx-10.10-x86_64/egg/pkg_resources.py", line 2190, in load
Entry point syntax follows the form::
File "/usr/local/Homebrew/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/CloudFusion-6.4.16-py2.7.egg/cloudfusion/main.py", line 5, in <module>
from cloudfusion.fuse import FUSE
File "/usr/local/Homebrew/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/CloudFusion-6.4.16-py2.7.egg/cloudfusion/fuse.py", line 245, in <module>
_libfuse = CDLL(_libfuse_path)
File "/usr/local/Homebrew/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ctypes/__init__.py", line 365, in __init__
self._handle = _dlopen(self._name, mode)
OSError: dlopen(/usr/local/Homebrew/lib/libfuse.dylib, 6): Symbol not found: _osxfuse_enable_macfuse_mode
Referenced from: /usr/local/Homebrew/lib/libfuse.dylib
Expected in: /usr/local/lib/libosxfuse_i32.2.dylib
It seems you support the obsolete MacFUSE and not the newer OSXFUSE?
It seems you support the obsolete MacFUSE and not the newer OSXFUSE?
Actually CloudFusion builds on fusepy, which in turn supports MacFuse. As I said, I have no way of testing CloudFusion on a Mac, so I can't even claim that it works with MacFUSE. According to https://osxfuse.github.io/:
OSXFUSE features a compatibility mode for file systems that have been built for MacFUSE. It can be a drop-in replacement for MacFUSE.
Maybe you need to install or activate the compatibility layer?
Depending on your use case, you could use CloudFusion directly, i.e. without the file system: http://joe42.github.io/CloudFusion/develop.html#using-the-api-directly
Basically, you just initialize a Store implementations, which could be any one of DropboxStore, WebDAVStore, AmazonStore... Then you can for instance store files with the method store_file, and retrieve them with get_file.
Well, the problem is the compatibility for MacFUSE is disappearing in the new version. Looks like the maintainer of pyfuse needs to update support on his end.
Thanks for the advice in any case.
On 25 Nov 2014, at 17:27, joe42 [email protected] wrote:
It seems you support the obsolete MacFUSE and not the newer OSXFUSE?
Actually CloudFusion builds on fusepy, which in turn supports MacFuse. As I said, I have no way of testing CloudFusion on a Mac, so I can't even claim that it works with MacFUSE. According to https://osxfuse.github.io/ https://osxfuse.github.io/:
OSXFUSE features a compatibility mode for file systems that have been built for MacFUSE. It can be a drop-in replacement for MacFUSE. Maybe you need to install or activate the compatibility layer?
Depending on your use case, you could use CloudFusion directly, i.e. without the file system: http://joe42.github.io/CloudFusion/develop.html#using-the-api-directly http://joe42.github.io/CloudFusion/develop.html#using-the-api-directly Basically, you just initialize a Store implementations, which could be any one of DropboxStore, WebDAVStore, AmazonStore... Then you can for instance store files with the method store_file, and retrieve them with get_file.
— Reply to this email directly or view it on GitHub https://github.com/joe42/CloudFusion/issues/17#issuecomment-64437803.
Seems this is an issue in pyfuse, according to https://github.com/osxfuse/osxfuse/issues/183#issuecomment-64446983. Will contact its author. Thanks.
See also #28 and terencehonles/fusepy#35. Recent versions of fusepy appear to be compatible with OSXFUSE. Note that ff4d allows mounting via fusepy and OSXFUSE without issue (although the results are somewhat flaky).
I am having trouble with this as well (see also #22). With my proposed patch, I can get the loopback example to work without issue, but I can't get my Dropbox to mount, either with the vendored fuse.py or with terencehonles/fusepy:
% git clone https://github.com/joe42/CloudFusion.git
...
% cd CloudFusion
% patch -p1 <../cloudfusion-28.patch # see my patch in joe42/CloudFusion#28
...
% virtualenv --python="$( which python2.7 )" .venv
...
% ./.venv/bin/pip install --editable .
...
% cp cloudfusion/config/Dropbox.ini config.ini
% # edit config.ini
...
% mkdir -p /Volumes/dropbox
% LIBFUSE_PATH=/usr/local/lib/libosxfuse.2.dylib ./.venv/bin/cloudfusion --config config.ini
/Volumes/dropbox foreground --log
fuse: unknown option `big_writes'
mount_osxfusefs: failed to mount /Volumes/dropbox@/dev/osxfuse1: Socket is not connected
Exception in statfs: 'NoneType' object has no attribute 'items'
Traceback (most recent call last):
File "/.../CloudFusion/cloudfusion/fuse.py", line 301, in _wrapper_
return func(*args, **kwargs) or 0
File "/.../CloudFusion/cloudfusion/fuse.py", line 382, in statfs
for key, val in attrs.items():
AttributeError: 'NoneType' object has no attribute 'items'
mount_osxfusefs: failed to mount /Volumes/dropbox@/dev/osxfuse2: Bad address
*** PROFILER RESULTS ***
_profiled_run (/.../CloudFusion/cloudfusion/store/store_sync_thread.py:209)
function called 0 times
0 function calls in 0.000 seconds
Ordered by: cumulative time, internal time, call count
ncalls tottime percall cumtime percall filename:lineno(function)
0 0.000 0.000 profile:0(profiler)
% git clone https://github.com/terencehonles/fusepy.git
...
% cd fusepy
% patch -p1 <../../fusepy-35.patch # see my patch in terencehonles/fusepy#35
% ../.venv/bin/pip install --editable .
...
% mkdir back front
% LIBFUSE_PATH=/usr/local/lib/libosxfuse.2.dylib ../.venv/bin/python ./examples/loopback.py back front
^C
% # The above works fine, changes to `back` appear in `front`
% rm -fr back front
% cd ..
% pwd
.../CloudFusion
% rm -f cloudfusion/fuse.py* # get rid of vendored fusepy
% patch -p1 <cf-use-installed-fusepy.patch
...
% rm -f cloudfusion/main.pyc cloudfusion/pyfusebox.pyc # just to be sure, since we just edited the corresponding .py files
% LIBFUSE_PATH=/usr/local/lib/libosxfuse.2.dylib ./.venv/bin/cloudfusion --config config.ini /Volumes/dropbox foreground --log
fuse: unknown option `big_writes'
mount_osxfusefs: failed to mount /Volumes/dropbox@/dev/osxfuse1: Socket is not connected
Traceback (most recent call last):
File "/.../CloudFusion/fusepy/fuse.py", line 422, in _wrapper
return func(*args, **kwargs) or 0
File "/.../CloudFusion/fusepy/fuse.py", line 528, in statfs
for key, val in attrs.items():
AttributeError: 'NoneType' object has no attribute 'items'
mount_osxfusefs: failed to mount /Volumes/dropbox@/dev/osxfuse2: Bad address
*** PROFILER RESULTS ***
_profiled_run (/.../CloudFusion/cloudfusion/store/store_sync_thread.py:209)
function called 0 times
0 function calls in 0.000 seconds
Ordered by: cumulative time, internal time, call count
ncalls tottime percall cumtime percall filename:lineno(function)
0 0.000 0.000 profile:0(profiler)
cf-use-installed-fusepy.patch is as follows:
diff --git a/cloudfusion/main.py b/cloudfusion/main.py
index 79de7f8..4001f67 100644
--- a/cloudfusion/main.py
+++ b/cloudfusion/main.py
@@ -2,7 +2,7 @@
Created on 12.05.2011
'''
-from cloudfusion.fuse import FUSE
+from fuse import FUSE
import os, sys
import logging.config
from mylogging import db_logging_thread
diff --git a/cloudfusion/pyfusebox/pyfusebox.py b/cloudfusion/pyfusebox/pyfusebox.py
index 52dfa9e..229b39b 100644
--- a/cloudfusion/pyfusebox/pyfusebox.py
+++ b/cloudfusion/pyfusebox/pyfusebox.py
@@ -3,7 +3,7 @@ from cloudfusion.store.store import NoSuchFilesytemObjectError,\
from cloudfusion.util import file_util
import os, stat, time
from errno import *
-from cloudfusion.fuse import FuseOSError, Operations
+from fuse import FuseOSError, Operations
import tempfile
import logging
CloudFusion first tries to mount with big_writes option, which is an optimization. But I think it is outdated, so I might remove this in the future. This should explain the first error message.
The second error complains about an ill specified mountpath. And /Volumes/dropbox@/dev/osxfuse2 is obviously not a directory. Though I don't know where the @/dev/oxfuse2 part comes from. Also at first glance I cannot make out a differences that allows ff4d to work as expected.
Hmmm...I had (momentarily) considered an improper path specification, but assumed the @/dev/osxfuse2 was some logging oddity and not reflective of the actual mount point (since I too couldn't figure out where it came from). :disappointed_relieved:
I cannot make out a differences that allows ff4d to work as expected.
If @/dev/osxfuse2 is actually being added to the mount point, that might be the cause?
Sure, that would be the cause. Though I am certain that it isn't added by CloudFusion. You could of course verify that by printing out the mountpoint variable right before the call to FUSE. I will ask Google, though I guess that it has something to do with OSXFUSE, as the path suggests.
I would probably replace the first call to FUSE with the big_writes option with:
FUSE(fuse_operations, mountpoint, foreground=foreground, nothreads=True)
The problem might be about trying to mount the file system twice, which as I think about it is also a difference to the implementation of ff4d.
I tried your suggestions. This is what I got (note the second line, where I print out the value of mountpoint before passing it to the TransparentConfigurablePyFuseBox constructor:
% LIBFUSE_PATH=/usr/local/lib/libosxfuse.2.dylib ./.venv/bin/cloudfusion --config config.ini /Volumes/dropbox foreground --log
mountpoint: '/Volumes/dropbox'
Exception in statfs: 'NoneType' object has no attribute 'items'
Traceback (most recent call last):
File "/.../CloudFusion/cloudfusion/fuse.py", line 301, in _wrapper_
return func(*args, **kwargs) or 0
File "/.../CloudFusion/cloudfusion/fuse.py", line 382, in statfs
for key, val in attrs.items():
AttributeError: 'NoneType' object has no attribute 'items'
mount_osxfusefs: failed to mount /Volumes/dropbox@/dev/osxfuse0: Bad address
*** PROFILER RESULTS ***
_profiled_run (/.../CloudFusion/cloudfusion/store/store_sync_thread.py:209)
function called 0 times
0 function calls in 0.000 seconds
Ordered by: cumulative time, internal time, call count
ncalls tottime percall cumtime percall filename:lineno(function)
0 0.000 0.000 profile:0(profiler)
Here are my modifications:
diff --git a/cloudfusion/fuse.py b/cloudfusion/fuse.py
index a062031..de6252a 100644
--- a/cloudfusion/fuse.py
+++ b/cloudfusion/fuse.py
@@ -18,7 +18,7 @@ from ctypes import *
from ctypes.util import find_library
from errno import *
from functools import partial
-from os import strerror
+from os import environ, strerror
from platform import machine, system
from stat import S_IFDIR
from traceback import print_exc
@@ -239,7 +239,7 @@ def set_st_attrs(st, attrs):
setattr(st, key, val)
-_libfuse_path = find_library('fuse')
+_libfuse_path = environ.get('LIBFUSE_PATH', find_library('fuse'))
if not _libfuse_path:
raise EnvironmentError('Unable to find libfuse')
_libfuse = CDLL(_libfuse_path)
diff --git a/cloudfusion/main.py b/cloudfusion/main.py
index 79de7f8..0bb07f6 100644
--- a/cloudfusion/main.py
+++ b/cloudfusion/main.py
@@ -131,11 +131,8 @@ def main():
if isinstance(fn, types.UnboundMethodType):
if not name.startswith('_'):
setattr(TransparentConfigurablePyFuseBox, name, profile(fn, filename='/tmp/cloudfusion_profile'))
+ print('mountpoint: {!r}'.format(mountpoint))
fuse_operations = TransparentConfigurablePyFuseBox(mountpoint)
- try:
- #first try to mount file system with big_writes option (more performant)
- FUSE(fuse_operations, mountpoint, foreground=foreground, nothreads=True, big_writes=True, max_read=131072, max_write=131072)
- except RuntimeError, e:
- FUSE(fuse_operations, mountpoint, foreground=foreground, nothreads=True)
+ FUSE(fuse_operations, mountpoint, foreground=foreground, nothreads=True)
if __name__ == '__main__':
main()