Zmbkpose icon indicating copy to clipboard operation
Zmbkpose copied to clipboard

Restauring from FULL curl: option --data-binary: out of memory

Open arekstuka opened this issue 11 years ago • 7 comments

Hi Guys,

I have a small problem :) When i try to restore mailbox from my full backup (~3GB ) i receiving this error:

2014-02-23 21:45:55 DEBUG: [email protected] - Restauring from 23/02/2014 21:08:03 FULL curl: option --data-binary: out of memory curl: try 'curl --help' or 'curl --manual' for more information

enybody has sollution for this?

p.s. when i try to make restore from another user (mailbox 200mb ) its working without any problems

arekstuka avatar Feb 23 '14 21:02 arekstuka

problem still exist.....any idea ?

arekstuka avatar Mar 09 '14 20:03 arekstuka

curl cannot get sufficient memory for build the request. Try save memory skipping the web service. Use zimbra user, in your zimbra server, and import the backup as follow : $ /opt/zimbra/bin/zmmailbox -z -m UserEmailAddress postRestURL "//?fmt=tgz&resolve=reset" backup.tgz replace "UserEmailAddress", and "backup.tgz" with the correct values.

You may get the following error: ERROR: zclient.IO_ERROR (Broken pipe) (cause: java.net.SocketException Broken pipe)

In this case, try the follow: Agree http://www.zimbra.com/forums/administrators/47949-recovery-tgz-mail-file-fails.html , you can try to import each message using $ zmmailbox -z -m @<yourmailserver.com> addMessage /Inbox *.eml

Note1: zmmailbox can use a directory argument, but it not examines recursivly, and not filters eml files. Note2: Each directory must exist on the destination mailbox, otherwise you get an error. See below for a script for filter eml files, and create each directory in destination mailbox. untar your backup.tgz file, and use zimbra user for execute the scritp as follow:

 $ import_backup.sh  [email protected] your_decompress_backup_dir

and patience .....




############ import_backup.sh ####################################
#!/bin/bash
set -e
email=$1
bkdir="$2"
zmmailbox_bin=/opt/zimbra/bin/zmmailbox

if [ ! -x $zmmailbox_bin ] ;then echo "$zmmailbox_bin does not exists" ; exit ; fi

mb_folders=/tmp/mbox_folders

cd "$bkdir" 

debug(){ echo "$(date ): $@" ; }
create_folder(){
  debug "Creating folder \"$1\" "
  $zmmailbox_bin -v -z -m $email createFolder "$1"
}

#Get actual folders
while read Id View Unread Msg_Count Path ;do
  [ -z "$Path" ] || echo $Path
done<<EOF |grep '^/.' > $mb_folders
   $($zmmailbox_bin -v -z -m  $email getAllFolders)
EOF

#Import messages
while read dir;do
  #Folder exists?
  egrep -q "^${dir#.}$" $mb_folders || create_folder "${dir#.}"

  #Import messages in dir
  find "$dir" -mindepth 1 -maxdepth 1 -type f -regex '.*\.eml' -print0 |\
     xargs -L 1 -0 -r $zmmailbox_bin -v -z -m $email addMessage "${dir#.}"
done<<EOF
   $(find . -mindepth 1 -type d )
EOF



liandros avatar Mar 25 '14 20:03 liandros

Hi, just tried the script of the last comment but there's something wrong with it near line 23-24, could you re-post it?

dusty73 avatar Jun 09 '14 14:06 dusty73

Sorry, I modified the previous post . This is a display problem because GitHub use Markdown parse for text styles, and ate some characters. Try now please.

liandros avatar Jun 13 '14 16:06 liandros

Hi, i tried your script but is extremely slow, I checked it and found the problem (and a solution). The message import is done with xargs, but zmmailbox is called for every message:

#Import messages in dir
find "$dir" -mindepth 1 -maxdepth 1 -type f -regex '.*\.eml' -print0 |\
  xargs -L 1 -0 -r $zmmailbox_bin -v -z -m $email addMessage "${dir#.}"

I swapped these previous line with:

$zmmailbox_bin -v -z -m $email addMessage "${dir#.}" "$dir"/*eml

In this way all the messages in a folder are imported in a single run of zmmailbox.

dusty73 avatar Aug 08 '14 13:08 dusty73

Hi, I'm trying the scripts but I get this error for some messages: Exception in thread "main" java.io.FileNotFoundException: ./Inbox!49/0000220914-CD SF LURIN_ ACTA DE REUNI??N 39.eml (No such file or directory) at java.base/java.io.FileInputStream.open0(Native Method) at java.base/java.io.FileInputStream.open(FileInputStream.java:219) at java.base/java.io.FileInputStream.(FileInputStream.java:157) at com.zimbra.cs.zclient.ZMailboxUtil.addMessage(ZMailboxUtil.java:1936) at com.zimbra.cs.zclient.ZMailboxUtil.doAddMessage(ZMailboxUtil.java:1979) at com.zimbra.cs.zclient.ZMailboxUtil.execute(ZMailboxUtil.java:1085) at com.zimbra.cs.zclient.ZMailboxUtil.main(ZMailboxUtil.java:2917) I think it is the word accent, is there any way to deal with word accent or special characters in an email in order to add the message?

jrios83 avatar Jun 08 '19 19:06 jrios83

The issue of lack of memory was resolved for me by adding physical memory to the server. The server consumed 7 GB of RAM, the archive took up 17 GB. 7+17=24 GB of RAM. I allocated 32 GB of RAM. The recovery went without problems.

Propovednik87 avatar Aug 12 '24 23:08 Propovednik87