@myanvoos: Extend web.py and add local hot reload

[DonggeLiu]

Running experiment for #884, contributed by @myanvoos

[DonggeLiu]

/gcbrun exp -n mv -ag

[DonggeLiu]

/gcbrun exp -n mv -ag

[myanvoos]

Looks like we're missing the watchdog import. I'll quickly fix this

[DonggeLiu]

Looks like we're missing the watchdog import. I'll quickly fix this

Thanks! Let's do this in a new PR based on this branch (exp-884)

[DonggeLiu]

/gcbrun exp -n mv1 -ag

[DonggeLiu]

/gcbrun exp -n mv -ag

[DonggeLiu]

Hmm... the report is still empty, suggesting something wrong with the experiment. The GKE job was lost along with the log, I will rerun to reproduce the issue.

[DonggeLiu]

/gcbrun exp -n mv -ag

[myanvoos]

Hey @DonggeLiu, I'm just revisiting this watcher now and noticed that I should probably update the README to reference the -wf flag, not -w, and sync it with the main branch.

I will also add some extra logic to the watcher so that it makes sure results-report reflects exactly what's in results. Currently the watcher regenerates reports with every new benchmark added but doesn't clear it if we remove the benchmark from results, so results-report will still be populated with old 'ghost' benchmarks (that won't show up on reports, but they bloat up the folder)

This is more of a folder organisation/tidying up thing and isn't related to the contents of the reports, which should be updated regardless of if we add or remove benchmarks.

Also, were you able to reproduce the issue? The hot reloading is working locally for me but I'm keen to fix it!

[DonggeLiu]

The cloud log did not show much useful information, unfortunately.

Here are relevant lines: https://github.com/google/oss-fuzz-gen/blob/dc06767c72fdb5418da16ffde6953f79be530726/report/upload_report.sh#L63-L74

QQ: Do we need to modify this? https://github.com/google/oss-fuzz-gen/blob/dc06767c72fdb5418da16ffde6953f79be530726/report/upload_report.sh#L58

[myanvoos]

The cloud log did not show much useful information, unfortunately.

Here are relevant lines:

https://github.com/google/oss-fuzz-gen/blob/dc06767c72fdb5418da16ffde6953f79be530726/report/upload_report.sh#L63-L74

QQ: Do we need to modify this?

https://github.com/google/oss-fuzz-gen/blob/dc06767c72fdb5418da16ffde6953f79be530726/report/upload_report.sh#L58

Oh right I think I might know what's happening.

def main():
  args = _parse_arguments()
  logging.getLogger().setLevel(os.environ.get('LOGLEVEL', 'INFO').upper())

  watcher = ReportWatcher(args)
  watcher.start()

  try:
    should_continue = args.serve or args.watch_filesystem or args.watch_template

    while should_continue:
      generate_report(args)
      time.sleep(args.interval_seconds)

  except KeyboardInterrupt:
    watcher.stop()
    logging.info('Exiting.')
    os._exit(0)

Without a --serve or watcher flags in the args should_continue will be false so we won't even reach generate_report(). So the upload report Bash script will be looking for files in an output folder that don't exist.

I'll put in an initial call to generate the report before initialising the watcher, that should fix it.

Edit for QQ: I don't think we need to modify

https://github.com/google/oss-fuzz-gen/blob/dc06767c72fdb5418da16ffde6953f79be530726/report/upload_report.sh#L58

because we already re-generate the report periodically here

https://github.com/google/oss-fuzz-gen/blob/dc06767c72fdb5418da16ffde6953f79be530726/report/upload_report.sh#L105

(Unless you meant like for debugging -- oops)

[DonggeLiu]

/gcbrun exp -n mv -ag