data-curation
data-curation copied to clipboard
cernopendata
Data ingestion and curation tools
The release info json has been updated with HI-related runs. This means that there will be more than one entry / year and they can be distinguished with a new...
from #124 Run categorization script: - [ ] update the early MC listing in [CMS-2016-mc-datasets.txt](https://github.com/cernopendata/data-curation/blob/master/cms-YYYY-simulated-datasets/inputs/CMS-2016-mc-datasets.txt) with the current listing #156 - [ ] run the[ categorization script](https://github.com/cernopendata/data-curation/blob/master/cms-YYYY-simulated-datasets/run_categorisation.sh) and check if...
## Setup ``` source /cvmfs/cms.cern.ch/cmsset_default.sh source /cvmfs/cms.cern.ch/rucio/setup-py3.sh voms-proxy-init -voms cms -rfc -valid 192:00 export RUCIO_ACCOUNT=`whoami` ``` ## Quota Check: ``` rucio list-account-limits $RUCIO_ACCOUNT ``` Add (as manager): ``` rucio-admin account...
CMS has now some quite extensive and systematically produced derived datasets under ``` /eos/opendata/cms/derived-data/POET/23-Jul-22/ /eos/opendata/cms/derived-data/NanoAODRun1/01-Jul-22/ ``` produced, respectively with - https://github.com/cms-opendata-analyses/PhysObjectExtractorTool/tree/odws2022-ttbaljets-prod - https://github.com/cms-opendata-analyses/NanoAODRun1ProducerTool These were used at the CMS OD...
When running the categorisation for [the 2015 MC list](https://github.com/cernopendata/data-curation/blob/master/cms-YYYY-simulated-datasets/inputs/CMS-2015-mc-datasets.txt), there are > 600 datasets in the "Miscellaneous" category which collects those datasets that have not been directed to any existing...
Complete `cms-release-info/run_ranges.json` with a list of datasets for each run era. See https://github.com/cernopendata/data-curation/issues/136 for an example listing for 2013 pPb.
Test the updates of `cms-YYYY-simulated-datasets` by applying it to a new 2012 dataset `/MinBias_TuneZ2star_8TeV-pythia6/Summer12_DR53X-PU_S10_START53_V7A-v1/AODSIM` see https://github.com/cernopendata/opendata.cern.ch/issues/3150 The files are located under `/eos/opendata/cms/mc/Summer12_DR53X/MinBias_TuneZ2star_8TeV-pythia6/` The "fixed" fields for 2012 simulated data are:...
(from #65) The trigger listings for Run2 data taking are now available in https://twiki.cern.ch/twiki/bin/viewauth/CMS/HLTPathsRunIIListWith2015 NB (from that page) > Active lumi is the luminosity when the trigger prescale is not...
270 of the 2015 MC datasets have `_ext` in the dataset name, e.g.: ``` /DYJetsToLL_M-1000to1500_TuneCUETP8M1_13TeV-amcatnloFXFX-pythia8/RunIIFall15MiniAODv2-PU25nsData2015v1_76X_mcRun2_asymptotic_v12-v1/MINIAODSIM /DYJetsToLL_M-1000to1500_TuneCUETP8M1_13TeV-amcatnloFXFX-pythia8/RunIIFall15MiniAODv2-PU25nsData2015v1_76X_mcRun2_asymptotic_v12_ext1-v1/MINIAODSIM /DYJetsToLL_M-100to200_TuneCUETP8M1_13TeV-amcatnloFXFX-pythia8/RunIIFall15MiniAODv2-PU25nsData2015v1_76X_mcRun2_asymptotic_v12-v1/MINIAODSIM /DYJetsToLL_M-100to200_TuneCUETP8M1_13TeV-amcatnloFXFX-pythia8/RunIIFall15MiniAODv2-PU25nsData2015v1_76X_mcRun2_asymptotic_v12_ext1-v1/MINIAODSIM /DYJetsToLL_M-10to50_TuneCUETP8M1_13TeV-amcatnloFXFX-pythia8/RunIIFall15MiniAODv2-PU25nsData2015v1_76X_mcRun2_asymptotic_v12-v1/MINIAODSIM /DYJetsToLL_M-10to50_TuneCUETP8M1_13TeV-amcatnloFXFX-pythia8/RunIIFall15MiniAODv2-PU25nsData2015v1_76X_mcRun2_asymptotic_v12_ext1-v1/MINIAODSIM /DYJetsToLL_M-10to50_TuneCUETP8M1_13TeV-amcatnloFXFX-pythia8/RunIIFall15MiniAODv2-PU25nsData2015v1_76X_mcRun2_asymptotic_v12_ext3-v1/MINIAODSIM /DYJetsToLL_M-1500to2000_TuneCUETP8M1_13TeV-amcatnloFXFX-pythia8/RunIIFall15MiniAODv2-PU25nsData2015v1_76X_mcRun2_asymptotic_v12-v1/MINIAODSIM /DYJetsToLL_M-1500to2000_TuneCUETP8M1_13TeV-amcatnloFXFX-pythia8/RunIIFall15MiniAODv2-PU25nsData2015v1_76X_mcRun2_asymptotic_v12_ext1-v1/MINIAODSIM /DYJetsToLL_M-150_TuneCUETP8M1_13TeV-madgraphMLM-pythia8/RunIIFall15MiniAODv2-PU25nsData2015v1_76X_mcRun2_asymptotic_v12-v1/MINIAODSIM ``` These are very likely more...
Release guidelines; see https://twiki.cern.ch/twiki/bin/view/CMS/DPOAMLSampleReleaseGuidelines ## Agreements - [ ] the ML group agrees that these samples are of interest for a public release - presented/discussed in (meeting/presentation link) - [...