cubed icon indicating copy to clipboard operation
cubed copied to clipboard

Find good defaults for reduction parameters

Open tomwhite opened this issue 1 year ago • 0 comments

There are a couple of parameters we can set:

  • split_every, which controls the number of blocks in each stage of tree reduce (defaults to 4, but set to 10 in the quadratic means benchmark)
  • max_total_num_input_blocks, which controls how many blocks can be read by one task when doing optimization (not enabled by default, but set to 40 in the quadratic means benchmark)

This comment discusses the trade off to optimize: https://github.com/cubed-dev/cubed/issues/331#issuecomment-1850978608

tomwhite avatar Mar 11 '24 10:03 tomwhite