pycbc icon indicating copy to clipboard operation
pycbc copied to clipboard

Rework trigger file finding in pycbc_live_single_trigger_fits

Open GarethCabournDavies opened this issue 1 year ago • 10 comments

Use gps start and end times to find files for triggers, rather than a single directory where we find all files

This should be something fairly easily changed in future to allow us to use file attributes rather than filenames

~Long term thing, not needed for O4 analysis~ Needed for the trigger collation script

GarethCabournDavies avatar May 05 '23 14:05 GarethCabournDavies

All output datasets are the same to within a numpy.isclose, some are re-ordered but the same once sorted. Keys change, but that is largely in the addition of fit-start and fit-end, as well as the input as arguments have changed

GarethCabournDavies avatar May 05 '23 14:05 GarethCabournDavies

Might be worth rebasing this given all the work that has happened in the mean time.

titodalcanton avatar Apr 11 '24 09:04 titodalcanton

I've rebased and added some changes to this.

Mainly the changes are to move the argument parsing into functions in a module so they can be reused in https://github.com/gwastro/pycbc/pull/4697

GarethCabournDavies avatar Apr 17 '24 10:04 GarethCabournDavies

All output datasets are the same to within a numpy.isclose, some are re-ordered but the same once sorted. Keys change, but that is largely in the addition of fit-start and fit-end, as well as the input as arguments have changed

This is still the case with the updates since the rebase, for more concrete discussion of changes

  • pruned_trigger_times has changed order
  • All datasets under the triggers group change order
  • attributes fit-start, fit-end have been added, and analysis_date is removed
  • fit_coeff datasets change by small amounts (<1e-4)

GarethCabournDavies avatar Apr 17 '24 11:04 GarethCabournDavies

Ive realised that this will need an update to the supervisor script, which I will add in the next day or so

GarethCabournDavies avatar Apr 24 '24 09:04 GarethCabournDavies

quite a few changes since last review need looking at

GarethCabournDavies avatar Apr 24 '24 13:04 GarethCabournDavies

@titodalcanton Could you have a quick look over the changes Gareth has made to the supervise scripts? Then hopefully this can be merged 😁

ArthurTolley avatar Apr 30 '24 08:04 ArthurTolley

testing for latest changes is ongoing

GarethCabournDavies avatar May 03 '24 11:05 GarethCabournDavies

I think the remaining codeclimate complaints are previously-existing problems

GarethCabournDavies avatar May 03 '24 12:05 GarethCabournDavies

Didnt realise I hadn't re-requested review from @titodalcanton, oops!

GarethCabournDavies avatar May 15 '24 11:05 GarethCabournDavies