cmssw icon indicating copy to clipboard operation
cmssw copied to clipboard

Published 20 hours ago • verified publisher icon cms-sw

l1nano import location

Open vlimant opened this issue 11 months ago • 12 comments

PR description:

trivial move of the import

PR validation:

nano matrix

vlimant avatar Mar 13 '24 13:03 vlimant

enable nano

vlimant avatar Mar 13 '24 13:03 vlimant

cms-bot internal usage

cmsbuild avatar Mar 13 '24 13:03 cmsbuild

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-44391/39452

  • This PR adds an extra 16KB to repository

  • There are other open Pull requests which might conflict with changes you have proposed:

    • File PhysicsTools/NanoAOD/python/autoNANO.py modified in PR(s): #43485, #44087

cmsbuild avatar Mar 13 '24 13:03 cmsbuild

A new Pull Request was created by @vlimant for master.

It involves the following packages:

  • PhysicsTools/NanoAOD (xpog)

@cmsbuild, @vlimant, @hqucms can you please review it and eventually sign? Thanks. @AnnikaStein, @gpetruc this is something you requested to watch as well. @sextonkennedy, @antoniovilela, @rappoccio you are the release manager for this.

cms-bot commands are listed here

cmsbuild avatar Mar 13 '24 13:03 cmsbuild

please test

vlimant avatar Mar 13 '24 14:03 vlimant

-1

Failed Tests: RelVals-INPUT RelVals-NANO Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-352eb2/38105/summary.html COMMIT: 6784a28bfe4515f8573a461d8ffed29bc795d89e CMSSW: CMSSW_14_1_X_2024-03-13-1100/el8_amd64_gcc12 Additional Tests: NANO User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/44391/38105/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals-INPUT

  • 2500.2112500.211_NANOdata124Xrun3/step2_NANOdata124Xrun3.log
  • 2500.3112500.311_NANOdata130Xrun3/step2_NANOdata130Xrun3.log

RelVals-NANO

  • 2500.2112500.211_NANOdata124Xrun3/step2_NANOdata124Xrun3.log
  • 2500.3112500.311_NANOdata130Xrun3/step2_NANOdata130Xrun3.log

Comparison Summary

Summary:

  • You potentially added 91 lines to the logs
  • Reco comparison results: 49 differences found in the comparisons
  • DQMHistoTests: Total files compared: 48
  • DQMHistoTests: Total histograms compared: 3297383
  • DQMHistoTests: Total failures: 3
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3297360
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
  • Checked 202 log files, 165 edm output root files, 48 DQM output files
  • TriggerResults: no differences found

cmsbuild avatar Mar 13 '24 17:03 cmsbuild

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-44391/39470

  • This PR adds an extra 24KB to repository

  • There are other open Pull requests which might conflict with changes you have proposed:

    • File Configuration/DataProcessing/python/Reco.py modified in PR(s): #44368
    • File Configuration/PyReleaseValidation/python/relval_nano.py modified in PR(s): #43485, #44087
    • File PhysicsTools/NanoAOD/python/autoNANO.py modified in PR(s): #43485, #44087

cmsbuild avatar Mar 14 '24 08:03 cmsbuild

Pull request #44391 was updated. @subirsarkar, @davidlange6, @vlimant, @antoniovilela, @hqucms, @rappoccio, @sunilUIET, @cmsbuild, @miquork, @srimanob, @fabiocos, @AdrianoDee can you please check and sign again.

cmsbuild avatar Mar 14 '24 08:03 cmsbuild

please test

vlimant avatar Mar 14 '24 09:03 vlimant

@lathomas please have a look

vlimant avatar Mar 14 '24 09:03 vlimant

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-352eb2/38124/summary.html COMMIT: 2caf7f3080e0ef9fd1eeced6927bca055c3c125a CMSSW: CMSSW_14_1_X_2024-03-13-1100/el8_amd64_gcc12 Additional Tests: NANO User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/44391/38124/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • You potentially added 106 lines to the logs
  • Reco comparison results: 62 differences found in the comparisons
  • DQMHistoTests: Total files compared: 48
  • DQMHistoTests: Total histograms compared: 3297383
  • DQMHistoTests: Total failures: 9
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3297354
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
  • Checked 202 log files, 165 edm output root files, 48 DQM output files
  • TriggerResults: no differences found

NANO Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 15
  • DQMHistoTests: Total histograms compared: 16402
  • DQMHistoTests: Total failures: 0
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 16402
  • DQMHistoTests: Total skipped: 0
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 14 files compared)
  • Checked 45 log files, 25 edm output root files, 15 DQM output files

Nano size comparison Summary:

Sample kb/ev ref kb/ev diff kb/ev ev/s/thd ref ev/s/thd diff rate mem/thd ref mem/thd
2500.0 2.548 2.548 0.000 ( +0.0% ) 5.22 5.10 +2.3% 2.218 2.220
2500.001 2.698 2.698 0.000 ( +0.0% ) 4.69 4.60 +1.9% 2.646 2.658
2500.002 2.638 2.638 0.000 ( +0.0% ) 4.88 4.72 +3.3% 2.626 2.650
2500.01 1.318 1.318 0.000 ( +0.0% ) 9.55 9.42 +1.3% 2.270 2.331
2500.011 1.740 1.740 0.000 ( +0.0% ) 5.22 5.16 +1.2% 2.441 2.534
2500.012 1.581 1.581 0.000 ( +0.0% ) 7.42 7.25 +2.3% 2.330 2.441
2500.1 2.192 2.192 0.000 ( +0.0% ) 5.28 5.30 -0.3% 2.061 2.061
2500.2 2.308 2.308 0.000 ( +0.0% ) 6.01 6.04 -0.4% 1.969 1.968
2500.21 1.181 1.181 0.000 ( +0.0% ) 4.30 4.31 -0.4% 2.267 2.272
2500.211 1.545 1.545 0.000 ( +0.0% ) 3.70 3.71 -0.3% 2.357 2.353
2500.3 2.058 2.058 0.000 ( +0.0% ) 12.42 12.12 +2.5% 1.958 1.959
2500.301 2.631 2.631 0.000 ( +0.0% ) 10.07 9.90 +1.7% 1.937 1.941
2500.31 1.254 1.254 0.000 ( +0.0% ) 19.48 19.43 +0.3% 2.328 2.357
2500.311 1.640 1.640 0.000 ( +0.0% ) 13.07 13.73 -4.8% 2.408 2.433
2500.312 7.159 7.159 0.000 ( +0.0% ) 1.37 1.42 -3.1% 1.683 1.679
2500.313 1.564 1.564 0.000 ( +0.0% ) 6.35 6.86 -7.5% 1.037 1.030
2500.314 1.163 1.163 0.000 ( +0.0% ) 13.77 14.64 -5.9% 2.057 2.201
2500.315 1.758 1.758 0.000 ( +0.0% ) 12.83 13.13 -2.3% 2.313 2.351
2500.316 1.787 1.787 0.000 ( +0.0% ) 13.26 13.18 +0.6% 2.183 2.350
2500.4 2.058 2.058 0.000 ( +0.0% ) 12.38 12.39 -0.1% 1.853 1.965
2500.401 1.824 1.824 0.000 ( +0.0% ) 10.16 9.56 +6.3% 1.808 1.812
2500.402 2.631 2.631 0.000 ( +0.0% ) 10.13 9.81 +3.2% 1.944 1.952
2500.403 2.620 2.620 0.000 ( +0.0% ) 10.29 10.02 +2.7% 1.948 1.950
2500.5 19.575 19.575 0.000 ( +0.0% ) 1.05 1.07 -2.5% 1.343 1.343

cmsbuild avatar Mar 14 '24 11:03 cmsbuild

+1

vlimant avatar Mar 14 '24 12:03 vlimant

+pdmv

AdrianoDee avatar Mar 22 '24 14:03 AdrianoDee

can we please have the remaining signatures to this ? @cms-sw/operations-l2 @cms-sw/upgrade-l2

vlimant avatar Mar 27 '24 17:03 vlimant

I honestly don’t know what turn over we can expect for signatures, but I have the feeling it’s taking much too much time for trivial things …

vlimant avatar Mar 27 '24 17:03 vlimant

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-44391/39704

  • This PR adds an extra 24KB to repository

  • There are other open Pull requests which might conflict with changes you have proposed:

    • File Configuration/DataProcessing/python/Reco.py modified in PR(s): #44368
    • File Configuration/PyReleaseValidation/python/relval_nano.py modified in PR(s): #43485, #43957, #44492
    • File PhysicsTools/NanoAOD/python/autoNANO.py modified in PR(s): #43485, #43957, #44492
    • File PhysicsTools/NanoAOD/python/nano_cff.py modified in PR(s): #44507

cmsbuild avatar Mar 27 '24 17:03 cmsbuild

Pull request #44391 was updated. @antoniovilela, @rappoccio, @fabiocos, @srimanob, @davidlange6, @cmsbuild, @subirsarkar can you please check and sign again.

cmsbuild avatar Mar 27 '24 17:03 cmsbuild

please test

vlimant avatar Mar 28 '24 08:03 vlimant

-1

Failed Tests: UnitTests Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-352eb2/38482/summary.html COMMIT: a4685222f6eff192af0fdb83acb30da7233a780f CMSSW: CMSSW_14_1_X_2024-03-27-2300/el8_amd64_gcc12 Additional Tests: NANO User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/44391/38482/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found 1 errors in the following unit tests:

---> test TestConfigDP_8 had ERRORS

Comparison Summary

Summary:

  • You potentially added 107 lines to the logs
  • Reco comparison results: 46 differences found in the comparisons
  • DQMHistoTests: Total files compared: 48
  • DQMHistoTests: Total histograms compared: 3297437
  • DQMHistoTests: Total failures: 6
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3297411
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
  • Checked 202 log files, 165 edm output root files, 48 DQM output files
  • TriggerResults: no differences found

NANO Comparison Summary

The workflows 2500.2 have different files in step1_dasquery.log than the ones found in the baseline. You may want to check and retrigger the tests if necessary. You can check it in the "files" directory in the results of the comparisons

Summary:

  • You potentially removed 1 lines from the logs
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 15
  • DQMHistoTests: Total histograms compared: 16402
  • DQMHistoTests: Total failures: 0
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 16402
  • DQMHistoTests: Total skipped: 0
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 14 files compared)
  • Checked 47 log files, 27 edm output root files, 15 DQM output files

Nano size comparison Summary:

Sample kb/ev ref kb/ev diff kb/ev ev/s/thd ref ev/s/thd diff rate mem/thd ref mem/thd
2500.0 2.548 2.548 0.000 ( +0.0% ) 4.67 4.53 +2.9% 2.230 2.173
2500.001 2.697 2.697 0.000 ( +0.0% ) 3.97 4.06 -2.2% 2.636 2.598
2500.002 2.637 2.637 0.000 ( +0.0% ) 4.34 4.22 +2.7% 2.623 2.591
2500.01 1.318 1.318 0.000 ( +0.0% ) 8.43 8.07 +4.5% 2.275 2.291
2500.011 1.739 1.739 0.000 ( +0.0% ) 4.63 4.53 +2.2% 2.433 2.430
2500.012 1.580 1.580 0.000 ( +0.0% ) 6.56 6.35 +3.4% 2.404 2.377
2500.1 2.192 2.192 0.000 ( +0.0% ) 4.68 4.57 +2.4% 2.058 2.065
2500.2 2.308 2.308 0.000 ( +0.0% ) 5.34 5.17 +3.1% 1.979 1.974
2500.21 1.181 1.181 0.000 ( +0.0% ) 3.55 3.49 +1.5% 2.266 2.256
2500.211 1.545 1.545 0.000 ( +0.0% ) 3.19 3.03 +5.5% 2.340 2.337
2500.3 2.064 2.064 0.000 ( +0.0% ) 9.62 9.63 -0.1% 1.966 1.959
2500.301 2.637 2.637 0.000 ( +0.0% ) 8.78 8.02 +9.4% 1.952 1.947
2500.31 1.255 1.255 0.000 ( +0.0% ) 13.23 12.25 +8.0% 2.343 2.344
2500.311 1.642 1.642 0.000 ( +0.0% ) 10.24 9.57 +7.0% 2.405 2.412
2500.312 7.159 7.159 0.000 ( +0.0% ) 1.45 1.34 +7.8% 1.708 1.705
2500.313 1.564 1.564 0.000 ( +0.0% ) 7.08 6.65 +6.4% 1.058 1.059
2500.314 1.163 1.163 0.000 ( +0.0% ) 10.45 9.29 +12.4% 2.260 2.236
2500.315 1.758 1.758 0.000 ( +0.0% ) 10.37 9.68 +7.2% 2.283 2.294
2500.316 3.093 3.093 0.000 ( +0.0% ) 1.84 1.73 +6.4% 2.256 2.248
2500.317 1.789 1.789 0.000 ( +0.0% ) 10.08 9.36 +7.7% 2.157 2.159
2500.4 2.221 2.221 0.000 ( +0.0% ) 9.82 9.34 +5.2% 1.892 1.878
2500.401 1.755 1.755 0.000 ( +0.0% ) 8.05 7.85 +2.5% 1.690 1.690
2500.402 2.766 2.766 0.000 ( +0.0% ) 8.54 7.91 +8.0% 1.905 1.843
2500.403 5.068 5.068 0.000 ( +0.0% ) 1.34 1.28 +4.8% 1.805 1.829
2500.404 2.774 2.774 0.000 ( +0.0% ) 8.22 7.28 +12.8% 1.698 1.700
2500.5 4.936 4.936 0.000 ( +0.0% ) 16.03 15.19 +5.5% 1.527 1.599
2500.51 8.960 8.960 0.000 ( +0.0% ) 9.56 9.33 +2.5% 1.368 1.441

cmsbuild avatar Mar 28 '24 10:03 cmsbuild

please test

vlimant avatar Mar 28 '24 16:03 vlimant

N.B git cms-rebase-topic vlimant:l1nano_include gives me all sorts of conflicts that are not detected here, therefore I cannot test anything locally and going blind

vlimant avatar Mar 28 '24 17:03 vlimant

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-44391/39725

  • This PR adds an extra 20KB to repository

  • There are other open Pull requests which might conflict with changes you have proposed:

    • File Configuration/DataProcessing/python/Reco.py modified in PR(s): #44368
    • File Configuration/PyReleaseValidation/python/relval_nano.py modified in PR(s): #43485, #43957
    • File PhysicsTools/NanoAOD/python/autoNANO.py modified in PR(s): #43485, #43957

cmsbuild avatar Mar 28 '24 17:03 cmsbuild

Pull request #44391 was updated. @davidlange6, @rappoccio, @fabiocos, @antoniovilela, @subirsarkar, @srimanob can you please check and sign again.

cmsbuild avatar Mar 28 '24 17:03 cmsbuild

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-352eb2/38494/summary.html COMMIT: 310cc3efb3bee5d01837d72d1d64a061f4b54203 CMSSW: CMSSW_14_1_X_2024-03-28-1100/el8_amd64_gcc12 Additional Tests: NANO User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/44391/38494/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • You potentially added 97 lines to the logs
  • Reco comparison results: 40 differences found in the comparisons
  • DQMHistoTests: Total files compared: 48
  • DQMHistoTests: Total histograms compared: 3297437
  • DQMHistoTests: Total failures: 3
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3297414
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
  • Checked 202 log files, 165 edm output root files, 48 DQM output files
  • TriggerResults: no differences found

NANO Comparison Summary

Summary:

  • You potentially removed 1 lines from the logs
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 15
  • DQMHistoTests: Total histograms compared: 16402
  • DQMHistoTests: Total failures: 0
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 16402
  • DQMHistoTests: Total skipped: 0
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 14 files compared)
  • Checked 47 log files, 27 edm output root files, 15 DQM output files

Nano size comparison Summary:

Sample kb/ev ref kb/ev diff kb/ev ev/s/thd ref ev/s/thd diff rate mem/thd ref mem/thd
2500.0 2.548 2.548 0.000 ( +0.0% ) 4.50 4.63 -2.8% 2.163 2.134
2500.001 2.697 2.697 0.000 ( +0.0% ) 4.04 4.13 -2.1% 2.542 2.553
2500.002 2.637 2.637 0.000 ( +0.0% ) 4.17 4.30 -3.0% 2.533 2.546
2500.01 1.318 1.318 0.000 ( +0.0% ) 7.93 8.35 -5.0% 2.267 2.219
2500.011 1.739 1.739 0.000 ( +0.0% ) 4.49 4.60 -2.3% 2.405 2.353
2500.012 1.580 1.580 0.000 ( +0.0% ) 6.31 6.57 -3.9% 2.402 2.318
2500.1 2.192 2.192 0.000 ( +0.0% ) 4.54 4.62 -1.9% 2.063 1.958
2500.2 2.308 2.308 0.000 ( +0.0% ) 5.13 5.29 -3.0% 1.974 1.860
2500.21 1.181 1.181 0.000 ( +0.0% ) 3.48 3.53 -1.5% 2.255 2.157
2500.211 1.545 1.545 0.000 ( +0.0% ) 2.94 3.12 -5.8% 2.231 2.170
2500.3 2.064 2.064 0.000 ( +0.0% ) 9.33 9.98 -6.5% 1.962 1.859
2500.301 2.637 2.637 0.000 ( +0.0% ) 8.00 8.28 -3.4% 1.946 1.791
2500.31 1.255 1.255 0.000 ( +0.0% ) 12.32 13.28 -7.2% 2.333 2.207
2500.311 1.642 1.642 0.000 ( +0.0% ) 9.47 9.74 -2.8% 2.301 2.245
2500.312 7.159 7.159 0.000 ( +0.0% ) 1.36 1.42 -4.2% 1.709 1.708
2500.313 1.564 1.564 0.000 ( +0.0% ) 6.44 6.85 -6.0% 1.064 1.054
2500.314 1.163 1.163 0.000 ( +0.0% ) 9.11 10.32 -11.7% 2.225 2.215
2500.315 1.758 1.758 0.000 ( +0.0% ) 9.37 10.45 -10.4% 2.285 2.280
2500.316 3.093 3.093 0.000 ( +0.0% ) 1.67 1.82 -8.8% 2.252 2.056
2500.317 1.789 1.789 0.000 ( +0.0% ) 9.02 10.24 -11.9% 2.211 2.258
2500.4 2.221 2.221 0.000 ( +0.0% ) 9.23 9.89 -6.6% 1.886 1.838
2500.401 1.755 1.755 0.000 ( +0.0% ) 7.86 8.54 -7.9% 1.700 1.684
2500.402 2.766 2.766 0.000 ( +0.0% ) 7.89 8.52 -7.4% 1.780 1.766
2500.403 5.068 5.068 0.000 ( +0.0% ) 1.26 1.34 -5.5% 1.829 1.800
2500.404 2.774 2.774 0.000 ( +0.0% ) 7.53 8.16 -7.7% 1.699 1.697
2500.5 4.936 4.936 0.000 ( +0.0% ) 14.90 16.15 -7.7% 1.599 1.580
2500.51 8.960 8.960 0.000 ( +0.0% ) 9.20 9.65 -4.6% 1.439 1.404

cmsbuild avatar Mar 28 '24 19:03 cmsbuild

+pdmv

  • re-sign

AdrianoDee avatar Mar 28 '24 21:03 AdrianoDee

+Upgrade

srimanob avatar Mar 29 '24 14:03 srimanob

could we bypass the operations signature to merge this in ?

vlimant avatar Mar 30 '24 09:03 vlimant

N.B git cms-rebase-topic vlimant:l1nano_include gives me all sorts of conflicts that are not detected here, therefore I cannot test anything locally and going blind

@vlimant Is it working locally now?

antoniovilela avatar Mar 31 '24 23:03 antoniovilela

could we bypass the operations signature to merge this in ?

@vlimant Do not worry about the @cms-sw/operations-l2 signature. ORP signs for operations, and usually we will sign and merge simultaneously, after the other signatures are cleared (as previously mentioned, you can raise PRs in the weekly meeting requesting missing signatures).

antoniovilela avatar Mar 31 '24 23:03 antoniovilela

all good on testing

vlimant avatar Apr 02 '24 06:04 vlimant