WarpX icon indicating copy to clipboard operation
WarpX copied to clipboard

How to roughly control ADIOS2 block number and WarpX particle number

Open Change72 opened this issue 1 year ago • 3 comments

  1. ADIOS2 block number:

I want to increase the ADIOS2 block number. I notice the field number is matched with grids number. For example, key = /data/600/fields/E/x. However, for particle blockNum, it seems a magic number: key = /data/600/particles/electrons/momentum/x. I didn't find any parameter to control it, and I guess there should be one in the openPMD layer.

n_cell max_grid_size blocking_factor grids field blockNum particle blockNum smallest grid biggest grid
32x32x256 64 32 4 4 128 32x32x64 32x32x64
256x256x512 64 32 128 128 128 64x64x64 64x64x64
64x64x256 64 32 4 4 4 64x64x64 64x64x64
256x256x256 8 8 32768 32768 738 8x8x8 8x8x8
    bpIO = adios.DeclareIO("ReadBP");
    bpIO.SetParameter("Threads", std::to_string(nThreads));
    bpReader = bpIO.Open(bpFileName, adios2::Mode::Read);

    adios2::Variable<double> x_meta_info = bpIO.InquireVariable<double>(key + "x");
    auto xBlocksInfo = bpReader.AllStepsBlocksInfo(x_meta_info);
    auto x_it = xBlocksInfo.begin();
    
    for (; x_it != xBlocksInfo.end(); ++x_it) {
        const auto &var_vec1 = x_it->second;
        std:cout << var_vec1.size() << std::endl;  // as ADIOS2 block number
    }
  1. for the particle number in bpls, the injection style I used is electrons.injection_style = "NRandomPerCell". The first two lines satisfy the exception, but the third and fourth lines are quite weird.
n_cell iteration num_particles_per_cell num_particles_in_last_iteration
32x32x256 500 1 116072
32x32x256 500 100 11607258
256x256x512 600 100 14069550
64x64x256 600 10000 45020389
256x256x256 600 1 64950

Change72 avatar Feb 04 '24 01:02 Change72

@guj do you want to comment first? :)

ax3l avatar Feb 08 '24 00:02 ax3l

Sorry I missed this one.

For 1, yes, the number of ADIOS blocks for field related variables equals the number of sub grids. Looks like you know how to adjust number of sub grids via blocking factor/max_grid_size. I am aware of any way to predict number of blocks for particles. @ax3l can comment.

The keys you mentioned are names of ADIOS variables. Looks like you are using openPMD's group based outputs, so the time steps are embedded in the names. It has no relation with number of blocks.

For 2, Sorry I am not sure I understand the question. Maybe @ax3l can help.

guj avatar Feb 08 '24 06:02 guj

@ax3l any idea? :)

Change72 avatar Feb 14 '24 20:02 Change72