mmap.error: [Errno 12] Cannot allocate memory

[stigger]

Hi. I'm trying to upload a 4G file, but glacier-cmd uploads first 54.8 megabytes, then fails. I tried multiple times, but it fails always after exactly 54.8 megabytes. Here is the stacktrace:

Wrote 54.8 MB. Rate 443.97 KB/s.te 525.79 KB/s, average 452.05 KB/s, eta 20:45:21.                                                                                         
Traceback (most recent call last):                                                                                                                
  File "/usr/bin/glacier-cmd", line 9, in <module>
    load_entry_point('glacier==0.2dev', 'console_scripts', 'glacier-cmd')()
  File "/usr/lib/python2.6/site-packages/glacier-0.2dev-py2.6.egg/glacier/glacier.py", line 751, in main
    args.func(args)
  File "/usr/lib/python2.6/site-packages/glacier-0.2dev-py2.6.egg/glacier/glacier.py", line 147, in wrapper
    return fn(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/glacier-0.2dev-py2.6.egg/glacier/glacier.py", line 300, in upload
    args.name, args.partsize, args.uploadid, args.resume)
  File "/usr/lib/python2.6/site-packages/glacier-0.2dev-py2.6.egg/glacier/GlacierWrapper.py", line 62, in wrapper
    ret = fn(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/glacier-0.2dev-py2.6.egg/glacier/GlacierWrapper.py", line 204, in glacier_connect_wrap
    return func(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/glacier-0.2dev-py2.6.egg/glacier/GlacierWrapper.py", line 62, in wrapper
    ret = fn(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/glacier-0.2dev-py2.6.egg/glacier/GlacierWrapper.py", line 225, in sdb_connect_wrap
    return func(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/glacier-0.2dev-py2.6.egg/glacier/GlacierWrapper.py", line 62, in wrapper
    ret = fn(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/glacier-0.2dev-py2.6.egg/glacier/GlacierWrapper.py", line 925, in upload
    mmapped_file = mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ)
mmap.error: [Errno 12] Cannot allocate memory

[offlinehacker]

How much free memory do you have? And what system do you use?

On Fri, Oct 26, 2012 at 2:16 PM, Vyacheslav Karpukhin < [email protected]> wrote:

Hi. I'm trying to upload 4G file, but glacier-cmd uploads first 54.8 megabytes, then fails. I tried multiple times, but it fails always after exactly 54.8 megabytes. Here is the stacktrace:

Wrote 54.8 MB. Rate 443.97 KB/s.te 525.79 KB/s, average 452.05 KB/s, eta 20:45:21. Traceback (most recent call last): File "/usr/bin/glacier-cmd", line 9, in load_entry_point('glacier==0.2dev', 'console_scripts', 'glacier-cmd')() File "/usr/lib/python2.6/site-packages/glacier-0.2dev-py2.6.egg/glacier/glacier.py", line 751, in main args.func(args) File "/usr/lib/python2.6/site-packages/glacier-0.2dev-py2.6.egg/glacier/glacier.py", line 147, in wrapper return fn(_args, *_kwargs) File "/usr/lib/python2.6/site-packages/glacier-0.2dev-py2.6.egg/glacier/glacier.py", line 300, in upload args.name, args.partsize, args.uploadid, args.resume) File "/usr/lib/python2.6/site-packages/glacier-0.2dev-py2.6.egg/glacier/GlacierWrapper.py", line 62, in wrapper ret = fn(_args, *_kwargs) File "/usr/lib/python2.6/site-packages/glacier-0.2dev-py2.6.egg/glacier/GlacierWrapper.py", line 204, in glacier_connect_wrap return func(_args, *_kwargs) File "/usr/lib/python2.6/site-packages/glacier-0.2dev-py2.6.egg/glacier/GlacierWrapper.py", line 62, in wrapper ret = fn(_args, *_kwargs) File "/usr/lib/python2.6/site-packages/glacier-0.2dev-py2.6.egg/glacier/GlacierWrapper.py", line 225, in sdb_connect_wrap return func(_args, *_kwargs) File "/usr/lib/python2.6/site-packages/glacier-0.2dev-py2.6.egg/glacier/GlacierWrapper.py", line 62, in wrapper ret = fn(_args, *_kwargs) File "/usr/lib/python2.6/site-packages/glacier-0.2dev-py2.6.egg/glacier/GlacierWrapper.py", line 925, in upload mmapped_file = mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ) mmap.error: [Errno 12] Cannot allocate memory

— Reply to this email directly or view it on GitHubhttps://github.com/uskudnik/amazon-glacier-cmd-interface/issues/94.

[offlinehacker]

kernel version, distro and architecture.

On Fri, Oct 26, 2012 at 2:42 PM, Jaka Hudoklin [email protected]:

How much free memory do you have? And what system do you use?

On Fri, Oct 26, 2012 at 2:16 PM, Vyacheslav Karpukhin < [email protected]> wrote:

Hi. I'm trying to upload 4G file, but glacier-cmd uploads first 54.8 megabytes, then fails. I tried multiple times, but it fails always after exactly 54.8 megabytes. Here is the stacktrace:

Wrote 54.8 MB. Rate 443.97 KB/s.te 525.79 KB/s, average 452.05 KB/s, eta 20:45:21. Traceback (most recent call last): File "/usr/bin/glacier-cmd", line 9, in load_entry_point('glacier==0.2dev', 'console_scripts', 'glacier-cmd')() File "/usr/lib/python2.6/site-packages/glacier-0.2dev-py2.6.egg/glacier/glacier.py", line 751, in main args.func(args) File "/usr/lib/python2.6/site-packages/glacier-0.2dev-py2.6.egg/glacier/glacier.py", line 147, in wrapper return fn(_args, *_kwargs) File "/usr/lib/python2.6/site-packages/glacier-0.2dev-py2.6.egg/glacier/glacier.py", line 300, in upload args.name, args.partsize, args.uploadid, args.resume) File "/usr/lib/python2.6/site-packages/glacier-0.2dev-py2.6.egg/glacier/GlacierWrapper.py", line 62, in wrapper ret = fn(_args, *_kwargs) File "/usr/lib/python2.6/site-packages/glacier-0.2dev-py2.6.egg/glacier/GlacierWrapper.py", line 204, in glacier_connect_wrap return func(_args, *_kwargs) File "/usr/lib/python2.6/site-packages/glacier-0.2dev-py2.6.egg/glacier/GlacierWrapper.py", line 62, in wrapper ret = fn(_args, *_kwargs) File "/usr/lib/python2.6/site-packages/glacier-0.2dev-py2.6.egg/glacier/GlacierWrapper.py", line 225, in sdb_connect_wrap return func(_args, *_kwargs) File "/usr/lib/python2.6/site-packages/glacier-0.2dev-py2.6.egg/glacier/GlacierWrapper.py", line 62, in wrapper ret = fn(_args, *_kwargs) File "/usr/lib/python2.6/site-packages/glacier-0.2dev-py2.6.egg/glacier/GlacierWrapper.py", line 925, in upload mmapped_file = mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ) mmap.error: [Errno 12] Cannot allocate memory

— Reply to this email directly or view it on GitHubhttps://github.com/uskudnik/amazon-glacier-cmd-interface/issues/94.

[stigger]

I'm on CentOS 6.3, kernel 2.6.32-279.5.2.el6.i686. I have 2G of RAM, about 834 megabytes of it is free.

[wvmarle]

The problem is shown in the error message.

It's trying to mmap the complete file... should have been changed since but I just check my code and it's not there any more. Too much merging and so I'm afraid. Have to look into that. Sigh. Really have to go through that whole uploading code I'm afraid, it's quite complex now with the resumption and mmapping and whatnot.

[offlinehacker]

It's wierd, where the heck went commit that fixed that!