EGSnrc icon indicating copy to clipboard operation
EGSnrc copied to clipboard

Add egs_quality application

Open rtownson opened this issue 6 years ago • 0 comments

This PR is still under development! Please post your comments and suggestions.

  • Adds a new application egs_quality for testing the consistency of an EGSnrc installation. It can run 3 kinds of tests, and can do dosimetric comparisons (within 2 sigma). This is controlled by the egs_quality input file, where you list which tests to perform.

  • All the tests require input files to be located in the egs_quality/inputs directory. These are then copied to the application directories for the run, and the egslst or output is recorded into the egs_quality/outputs/latest directory. A report is generated in egs_quality/outputs/latest/quality_report.txt.

  • When running egs_quality, results are compared with the corresponding outputs saved in egs_quality/outputs/benchmark.

  • Once merged, we should add these lines to the travis script:

egs_quality -i quality
cat $EGS_HOME/egs_quality/outputs/latest/quality_report.txt
(! grep -i fail $EGS_HOME/egs_quality/outputs/latest/quality_report.txt)
  • Note that this PR effectively duplicates quite a few old example input files. However, they will be custom tuned for the egs_quality application so I think this is ok. Currently the input files are set up for a "quick" test; we should consider adding input files that test the physics more thoughtfully.

Other changes: Fixes the number of regions assigned to egs_prism geometries. Now nreg is set to 1 instead of defaulting to 0. Previously, this would result in region labels not working. Also, two warning messages were commented out that always occurred for egs_planes and egs_pyramids.

Questions for you:

  • What quick tests are missing that test general functionality / consistency?

  • What long tests are missing that test physics consistency / accuracy? Maybe we run these on the cluster only after major changes. For example, say we rewrite electron transport in c++ and want to check that it matches the mortran algorithm. What tests would evaluate that?

Quality tests currently included:

  1. Application output. Full simulations are run and compared with scored output for statistical agreement. Each application must be coded into egs_quality to be supported, and the output format may depend on the input file that the test is run on. So as we add new input files for testing the applications, egs_quality may need some additional updates. Currently most applications are supported and one input file for each is set up for tests on a few dosimetric outputs.
:start quality test applications:
    :start test:
        application = tutor1
        pegs = tutor_data
    :stop test:
    :start test:
        application = egs_chamber
        inputs = quality_kerma
    :stop test:
:stop quality test applications:
  1. egs++ geometries, tested against one or more sources. Full simulations are run and compared with ausgab dose output for statistical agreement. Currently at least one input is provided for each geometry, but not necessarily all configurations.
:start quality test geometries:
    # TODO: missing fastenvelope, deformedxyz, octree, vhpgeometry

    geometry inputs = egs_gstack egs_box egs_cdgeometry egs_coneset egs_conestack egs_cylinders egs_elliptic_cylinders egs_genvelope egs_gtransformed egs_gunion egs_iplanes egs_parallelcones egs_planecollection egs_prism egs_pyramid egs_radialrepeater egs_roundrect_cylinders egs_simplecone egs_space egs_spheres egs_xyzgeometry egs_xyzrepeater
    source inputs = egs_isotropic_source
:stop quality test geometries:
  1. egs++ sources. A few particles (10) are generated from each source and checked for exact agreement of all properties.
:start quality test sources:
    # TODO: missing fanosource

    source inputs = egs_angular_spread_source egs_beam_source egs_collimated_source egs_isotropic_source egs_parallel_beam egs_phsp_source egs_point_source egs_radionuclide_source egs_source_collection egs_transformed_source iaea_phsp_source
:stop quality test sources:

Example report output:

EGSnrc quality report

Local time: Thu Aug 23 18:44:42 2018


This file contains a quality report for EGSnrc. It is generated automatically by the egs_quality application and can be used to test for consistency of results following changes to the EGSnrc codebase.


Geometry quality tests

These tests run full simulations using each geometry, optionally repeated with multiple sources.

egs_gstack / egs_isotropic_source

NOTE 19 lines did not match PASS exact dosimetry match


Source quality tests

These tests check for an exact particle match of each source, compared against benchmark data. Only expect success when the benchmark was calculated with the same machine, code version, compiler and random number seeds.

egs_angular_spread_source

PASS with exact match

rtownson avatar Aug 24 '18 21:08 rtownson