wrf-coop icon indicating copy to clipboard operation
wrf-coop copied to clipboard

Changes for finer granularity

Open davegill opened this issue 3 years ago • 1 comments

  1. We are back to chem tests 1, 2, 5. There was no timing performance advantage to split the tests apart. Howevr, using a BIGGER macine, with more cores for parallel compilation, was important.
  2. Zapped NMM.
  3. Split apart the 10, 13, 14, 15 tests. These all have serial components that run quite slowly. The may need to be further split.
  4. To support this whole splitting thing, remove the hard-coded "19", and actually count how many tests are being conducted.
  5. Introduce the FEATURE flag, in anticipation of usage with Kelly's feature (restart) tests.
  6. With Varun, rewrote the output "part.sh" file. This is to allow the code to be less hard-coded in the groovy scripts. This required calling the last_oinly_once script.
  7. The "last_only_once.csh" script now has an argument. This is the directory where the various SUCCESS* files are held.

modified: build.csh

davegill avatar Oct 22 '21 01:10 davegill

@vlakshmanan-scala Varun, Here are the proposed changes to build.csh.

  1. Note that now the last_only_once.csh file has this line:
cd $1 >& /dev/null
  1. The part.sh looks like this (for test 1 and 2). MUCH MUCH shorter. Good idea!
#########   Comparison.sh - autogenerated    ##############

        # This compares both serial vs openmp and serial vs mpi results


        sudo -S unzip /tmp/raw_output/OUTPUT_output_24.zip -d /tmp/raw_output/OUTPUT_1
        sudo -S unzip /tmp/raw_output/OUTPUT_output_25.zip -d /tmp/raw_output/OUTPUT_1
        sudo -S unzip /tmp/raw_output/OUTPUT_output_26.zip -d /tmp/raw_output/OUTPUT_1
        sudo -S cat /tmp/raw_output/output_24 /tmp/raw_output/output_25 /tmp/raw_output/output_26 | sudo tee -a /tmp/raw_output/final_output/output_1

        sudo -S ls -l /tmp/raw_output/OUTPUT_output_24/home/ubuntu/wrf-stuff/wrf-coop/OUTPUT | sudo tee -a /tmp/raw_output/final_output/output_1
        sudo -S ls -l /tmp/raw_output/OUTPUT_output_25/home/ubuntu/wrf-stuff/wrf-coop/OUTPUT | sudo tee -a /tmp/raw_output/final_output/output_1
        sudo -S ls -l /tmp/raw_output/OUTPUT_output_26/home/ubuntu/wrf-stuff/wrf-coop/OUTPUT | sudo tee -a /tmp/raw_output/final_output/output_1   

        ./last_only_once.csh /tmp/raw_output/OUTPUT_1 | sudo tee -a /tmp/raw_output/final_output/output_1

        sudo -S unzip /tmp/raw_output/OUTPUT_output_27.zip -d /tmp/raw_output/OUTPUT_2
        sudo -S unzip /tmp/raw_output/OUTPUT_output_28.zip -d /tmp/raw_output/OUTPUT_2
        sudo -S cat /tmp/raw_output/output_27 /tmp/raw_output/output_28 | sudo tee -a /tmp/raw_output/final_output/output_2

        sudo -S ls -l /tmp/raw_output/OUTPUT_output_27/home/ubuntu/wrf-stuff/wrf-coop/OUTPUT | sudo tee -a /tmp/raw_output/final_output/output_2
        sudo -S ls -l /tmp/raw_output/OUTPUT_output_28/home/ubuntu/wrf-stuff/wrf-coop/OUTPUT | sudo tee -a /tmp/raw_output/final_output/output_2   

        ./last_only_once.csh /tmp/raw_output/OUTPUT_2 | sudo tee -a /tmp/raw_output/final_output/output_2
  1. The chemistry tests are not consistently number with your re-ordering. I should probably do that internally, rather than in the groovy scripts.

davegill avatar Oct 22 '21 02:10 davegill