borg icon indicating copy to clipboard operation
borg copied to clipboard

How to specify the segment file size?

Open AlphaJack opened this issue 3 years ago • 2 comments

Have you checked borgbackup docs, FAQ, and open Github issues?

Yes

Is this a BUG / ISSUE report or a QUESTION?

QUESTION

Describe the problem you're observing.

Is there a way to specify a chunk size for borg?

For relatively big collections (hundreds of GB) it creates 500MB-max files, for smaller collection it varies.

In my case I would like all repos to have a large chunk size, in order to reduce the total number of files

With a big collection:

-- repo:/bigrepo/data/0 ---------------------------------------------------------------------------------------------
  507.900Mi [##########]  317                                                                                                                
  507.687Mi [##########]  99                                                                                                                 
  507.332Mi [##########]  57                                                                                                                 
  507.329Mi [##########]  208                                                                                                                
  507.157Mi [##########]  35                                                                                                                 
  507.077Mi [##########]  430                                                                                                                
  506.875Mi [##########]  687  

With a small collection:

-- remote:/smallrepo/data/0 --------------------------------------------------------------------------------------------
   19.037Ki [##########]  78                                                                                                                 
   17.421Ki [######### ]  50                                                                                                                 
   17.097Ki [######### ]  62                                                                                                                 
   16.778Ki [######### ]  70                                                                                                                 
   16.666Ki [######### ]  54                                                                                                                 
   11.455Ki [######    ]  6                                                                                                                  
   11.439Ki [######    ]  2                                                                                                                  
    9.740Ki [#####     ]  42                                                                                                                 
    9.605Ki [#####     ]  34                                                                                                                 
    9.510Ki [#####     ]  26                                                                                                                 
    9.445Ki [#####     ]  14                                                                                                                 
    9.430Ki [#####     ]  10                                                                                                                 
        869 [          ]  80                                                                                                                 
        854 [          ]  56                                                                                                                 
        854 [          ]  64                                                                                                                 
        854 [          ]  72                                                                                                                 
        838 [          ]  20                                                                                                                 
        800 [          ]  60                                                                                                                 
        799 [          ]  68                                                                                                                 
        799 [          ]  76                                                                                                                 
        786 [          ]  52                                                                                                                 
        783 [          ]  36                                                                                                                 
        782 [          ]  16

AlphaJack avatar Dec 07 '22 23:12 AlphaJack

If you mean the max size of the files below borg_repo_dir/data, that is the so called segment file size and you can set that in the repo config.

ThomasWaldmann avatar Dec 08 '22 02:12 ThomasWaldmann

TODO: check if our docs talk about that.

ThomasWaldmann avatar Apr 02 '23 00:04 ThomasWaldmann