openmc icon indicating copy to clipboard operation
openmc copied to clipboard
verified publisher icon openmc-dev

Global Variance Reduction & Local Variance Reduction in OpenMC

Open YuanHu-PKU-KIT opened this issue 3 years ago • 10 comments

The PS-GVR method and the WWG function as LVR method have been implemented in this branch.

The modification of <settings.xml>:

<weight_windows>
  //--- this part remain the same as WWM inputs
  <id> </id>
  <mesh> </mesh>
  <particle_type> </particle_type>
  <energy_bins> </energy_bins>
  <lower_ww_bounds> </lower_ww_bounds>
  <upper_ww_bounds> </upper_ww_bounds>
  <survival> </survival>
  <max_lower_bound_ratio> </max_lower_bound_ratio>
  <max_split> </max_split>
  <weight_cutoff> </weight_cutoff>	
  // --- This part is the input for GVR
  <global>
    <tally> </tally>                       //!< id of the tally which used for flux-based WW calculation (PS-GVR)
    <iteration> </iteration>               //!< the iteration series for WW calculation
    <source_space> </source_space>         //!< S value in PS-GVR
    <n_statistics> </n_statistics>         //!< n_statistics value in PS-GVR
    <max_nps_per_task> </max_nps_per_task>  //!< the max number of particle histories in each task in the asynchronous scheduling	
  </global>
  //--- This part is the input for WWG (LVR)
  <local>
    <target_lower_left> </target_lower_left>      //!<  the lower left point of the target region
    <target_upper_right> </target_upper_right>    //!<   the upper right point of the target region
  </local>
</weight_windows>

The modification for GVR:

1. in simulation.cpp:   
- openmc_generate_WW()   //!<  generate the WWM during iteration
- update_WW()            //!<  call the calculate_WW() in weight_windows.cpp to calculate the WWM based on the PS-GVR method
- asynchronous_scheduling()  //!< asynchronous scheduling for parallel calculation in GVR simulation
- initialize_history_asynchronous()  //!< initialize_history() which used in asynchronous scheduling
- transport_history_based_asynchronous() //!< transport_history_based()  which used in asynchronous scheduling

2. in weight_windows.cpp:  
- the inputs and global variables for GVR & LVR
- calculate_WW()  //!< calculate the WWM based on the PS-GVR method

The modification for LVR:

1. in particle_data.h:  
- two vectors (crossed_mesh, crossed_weight) to recode the mesh bin and the corresponding weight that particle crossed
- a few functions to operate these two vectors

2. in particle.cpp:  
- copy or reset the two vectors above for secondary or source particle

3. in mesh.cpp:  
- mesh_bins()          //!<  record the mesh bins intersected by the track         
- arrived_target()    //!< check if the track arrived target region

4. in simulation.cpp:  
- record the weight & score during particle transporting
- output the WWM into files
- add_weight()  //!< accumulate the weight
- add_score()   //!< accumulate the score

For more detail about the PS-GVR and the asynchronous scheduling, please see

https://doi.org/10.1016/j.fusengdes.2021.112829

The work on LVR is from (https://doi.org/10.1016/j.net.2022.04.021).

Currently, there is still a lack of tests for this GVR & LVR. Hope to see your advice on improving this code and the tests for this GVR & LVR.

Comments from my side. When I wrote this GVR & LVR, I tried to not do any modifications to the original code, and make a bypass for me to achieve those functions I want to realize. This is quite useful for my own usage, but it may make the code redundant. I have a few thoughts about merging my "bypass" into the original code below：

  1. For the asynchronous scheduling, I made a copy of initialize_history() and transport_history_based(), and modified it for the asynchronous scheduling. But the original initialize_history() and transport_history_based() could be modified to meet the requirement for both static and asynchronous scheduling.
  2. This asynchronous scheduling is actually not directly related to the GVR. It could be used in any simulation. So separating this asynchronous scheduling out may be better.
  3. About the two vectors used in WWG, I wrote the MPI_Reduce() to accumulate the results on different ranks. If these two vectors could be assigned as tallies, it will be more consistent with the current codes.
  4. About the WWM output in WWG, I wrote this part in the broadcast_results(). It definitely will be much better if this part is in the output.cpp.
  5. About the tally used in GVR. I currently use a default that the order of filters is energy filter first and then mesh filter. This could be more flexible for different tally inputs.

YuanHu-PKU-KIT avatar Apr 11 '22 08:04 YuanHu-PKU-KIT

Thank you for this @YuanHu-PKU-KIT! I'll be doing an initial review of this today.

One thing that jumped out at me immediately was that there's a large binary file, plot_settings.pkl, that is committed in this branch. This is a mistake of mine from a while back, so I apologize for the extra step needed here. We'll want to make sure we remove it from the history of this branch to keep it out of the repo history.

First, I'd create a backup of this branch for safety. Just good practice to do this if you're working with git commands you haven't used before. So from this branch start with

$ git branch GVR-LVR_in_OpenMC_backup

to create a backup branch.

Then the following commands will update your branch:

$ git rebase --onto upstream/develop 9897446

I didn't see any conflicts when I did this, so it should go smoothly.

Once you've updated your branch locally, you'll need to force push the branch to override the history on the remote.

pshriwise avatar Apr 12 '22 17:04 pshriwise

I'm not familiar with the git command, so there may be some silly questions below.

  1. Should I clone my branch with git clone -b GVR-LVR_in_OpenMC https://github.com/YuanHu-PKU-KIT/openmc.git in the Git Bash software first? I got this error:
Cloning into 'openmc'...
fatal: unable to access 'https://github.com/YuanHu-PKU-KIT/openmc.git/': OpenSSL SSL_read: Connection was reset, errno 10054

I also tried git clone -b GVR-LVR_in_OpenMC git://github.com/YuanHu-PKU-KIT/openmc.git, and I got this error:

Cloning into 'openmc'...
fatal: remote error:
  The unauthenticated git protocol on port 9418 is no longer supported.
Please see https://github.blog/2021-09-01-improving-git-protocol-security-github/ for more information.

  1. And should I use git branch GVR-LVR_in_OpenMC_backup in the folder of GVR-LVR_in_OpenMC or its upper folder to create a backup branch? (in short, the path to use this command should be ~/GVR-LVR_in_OpenMC or ~/ ?)

  2. Then I should use git rebase --onto upstream/develop 9897446 in the folder GVR-LVR_in_OpenMC to update this branch, right?

YuanHu-PKU-KIT avatar Apr 13 '22 02:04 YuanHu-PKU-KIT

I'm not familiar with the git command, so there may be some silly questions below.

  1. Should I clone my branch with git clone -b GVR-LVR_in_OpenMC https://github.com/YuanHu-PKU-KIT/openmc.git in the Git Bash software first? I got this error:
Cloning into 'openmc'...
fatal: unable to access 'https://github.com/YuanHu-PKU-KIT/openmc.git/': OpenSSL SSL_read: Connection was reset, errno 10054

I also tried git clone -b GVR-LVR_in_OpenMC git://github.com/YuanHu-PKU-KIT/openmc.git, and I got this error:

Cloning into 'openmc'...
fatal: remote error:
  The unauthenticated git protocol on port 9418 is no longer supported.
Please see https://github.blog/2021-09-01-improving-git-protocol-security-github/ for more information.
  1. And should I use git branch GVR-LVR_in_OpenMC_backup in the folder of GVR-LVR_in_OpenMC or its upper folder to create a backup branch? (in short, the path to use this command should be ~/GVR-LVR_in_OpenMC or ~/ ?)
  2. Then I should use git rebase --onto upstream/develop 9897446 in the folder GVR-LVR_in_OpenMC to update this branch, right?

Sorry, I was making some assumptions about how your connections to git repos are setup. Here's a full set of commands that will update your branch starting from scratch:

$ git clone https://github.com/YuanHu-PKU-KIT/openmc
$ cd openmc
$ git remote add upstream https://github.com/openmc-dev/openmc
$ git fetch origin
$ git fetch upstream
$ git checkout GVR-LVR_in_OpenMC
$ git branch GVR-LVR_in_OpenMC_backup
$ git rebase --onto upstream/develop 9897446

and then once you've verified that you're happy with the new branch

$ git push -f origin

You'll then want to use that updated branch to make any adjustments to the PR.

pshriwise avatar Apr 13 '22 02:04 pshriwise

I still got error with this:

$ git clone https://github.com/YuanHu-PKU-KIT/openmc
Cloning into 'openmc'...
fatal: unable to access 'https://github.com/YuanHu-PKU-KIT/openmc/': OpenSSL SSL_read: Connection was reset, errno 10054

I also tried ping github.com, and the results:

Pinging github.com with 32 bytes of data:
Request timed out.
Request timed out.
Request timed out.
Request timed out.

Ping statistics :
    Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),

Is there any problem with my internet?

YuanHu-PKU-KIT avatar Apr 13 '22 02:04 YuanHu-PKU-KIT

I still got error with this:

$ git clone https://github.com/YuanHu-PKU-KIT/openmc
Cloning into 'openmc'...
fatal: unable to access 'https://github.com/YuanHu-PKU-KIT/openmc/': OpenSSL SSL_read: Connection was reset, errno 10054

I also tried ping github.com, and the results:

Pinging github.com with 32 bytes of data:
Request timed out.
Request timed out.
Request timed out.
Request timed out.

Ping statistics :
    Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),

Is there any problem with my internet?

That part I'm not entirely sure about. It's possible that there's a problem with your connection, yes. Everything seems fine on github's side https://www.githubstatus.com/.

pshriwise avatar Apr 13 '22 02:04 pshriwise

I solved this problem:

$ git clone https://github.com/YuanHu-PKU-KIT/openmc
Cloning into 'openmc'...
remote: Enumerating objects: 100936, done.
remote: Counting objects: 100% (2167/2167), done.
remote: Compressing objects: 100% (862/862), done.
remote: Total 100936 (delta 1393), reused 1857 (delta 1226), pack-reused 98769
Receiving objects: 100% (100936/100936), 62.88 MiB | 2.65 MiB/s, done.
Resolving deltas: 100% (74381/74381), done.
Updating files: 100% (1291/1291), done.

And with step 3, I got this:

$ git remote add upstream
usage: git remote add [<options>] <name> <url>

    -f, --fetch           fetch the remote branches
    --tags                import all tags and associated objects when fetching
                          or do not fetch any tag at all (--no-tags)
    -t, --track <branch>  branch(es) to track
    -m, --master <branch>
                          master branch
    --mirror[=(push|fetch)]
                          set up remote as a mirror to push to or fetch from

Should I add an url follow this command?

YuanHu-PKU-KIT avatar Apr 13 '22 02:04 YuanHu-PKU-KIT

I solved this problem:

$ git clone https://github.com/YuanHu-PKU-KIT/openmc
Cloning into 'openmc'...
remote: Enumerating objects: 100936, done.
remote: Counting objects: 100% (2167/2167), done.
remote: Compressing objects: 100% (862/862), done.
remote: Total 100936 (delta 1393), reused 1857 (delta 1226), pack-reused 98769
Receiving objects: 100% (100936/100936), 62.88 MiB | 2.65 MiB/s, done.
Resolving deltas: 100% (74381/74381), done.
Updating files: 100% (1291/1291), done.

And with step 3, I got this:

$ git remote add upstream
usage: git remote add [<options>] <name> <url>

    -f, --fetch           fetch the remote branches
    --tags                import all tags and associated objects when fetching
                          or do not fetch any tag at all (--no-tags)
    -t, --track <branch>  branch(es) to track
    -m, --master <branch>
                          master branch
    --mirror[=(push|fetch)]
                          set up remote as a mirror to push to or fetch from

Should I add an url follow this command?

Argh, yeah. Sorry. In too much of a hurry. I've just updated the post above. Should have been

$ git remote add upstream https://github.com/openmc-dev/openmc

pshriwise avatar Apr 13 '22 02:04 pshriwise

I finished these steps, and I got:

$ git rebase --onto upstream/develop 9897446
Successfully rebased and updated refs/heads/GVR-LVR_in_OpenMC.

$ git push -f origin
Enumerating objects: 48, done.
Counting objects: 100% (48/48), done.
Delta compression using up to 8 threads
Compressing objects: 100% (32/32), done.
Writing objects: 100% (36/36), 13.29 KiB | 1.66 MiB/s, done.
Total 36 (delta 30), reused 1 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (30/30), completed with 10 local objects.
To https://github.com/YuanHu-PKU-KIT/openmc
 + 5815154be...4d5addc3d GVR-LVR_in_OpenMC -> GVR-LVR_in_OpenMC (forced update)

I think it's done, right?

YuanHu-PKU-KIT avatar Apr 13 '22 03:04 YuanHu-PKU-KIT

I finished these steps, and I got:

$ git rebase --onto upstream/develop 9897446
Successfully rebased and updated refs/heads/GVR-LVR_in_OpenMC.

$ git push -f origin
Enumerating objects: 48, done.
Counting objects: 100% (48/48), done.
Delta compression using up to 8 threads
Compressing objects: 100% (32/32), done.
Writing objects: 100% (36/36), 13.29 KiB | 1.66 MiB/s, done.
Total 36 (delta 30), reused 1 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (30/30), completed with 10 local objects.
To https://github.com/YuanHu-PKU-KIT/openmc
 + 5815154be...4d5addc3d GVR-LVR_in_OpenMC -> GVR-LVR_in_OpenMC (forced update)

I think it's done, right?

Yep! Looks good to me now!

pshriwise avatar Apr 13 '22 04:04 pshriwise

As discussed with @pshriwise, I separated this PR into 3 PRs (#2030 for LVR, #2031 for GVR, and #2032 for asynchronous). Thus we could be able to focus on the topics independently. In the meantime, this PR will remain until all 3 parts have been finished.

YuanHu-PKU-KIT avatar Apr 18 '22 04:04 YuanHu-PKU-KIT