botorch icon indicating copy to clipboard operation
botorch copied to clipboard

Update decoupled MOBO tutorial to use input transforms & Log NEHVI

Open saitcakmak opened this issue 1 year ago • 1 comments

Previously, this tutorial used normalize & unnormalize helpers to manually transform the inputs before feeding them to the model & acquisition functions. This requires transforming the data in many places and can be confusing to the users (see https://github.com/pytorch/botorch/discussions/2478). My original goal was to replace this with the Normalize input transform.

  • Replaced normalize & unnormalize with Normalize input transform.
  • Noticed that the tutorial was using non-log NEHVI, so replaced that with LogNEHVI.
  • Running the tutorial was producing extremely noisy plots that didn't show any improvement from any of the methods. This was due to pymoo imports silently failing (pymoo has gone through a BC breaking refactor) and the HV being computed using the brute force approach. The brute force approach seemed too inaccurate, so I removed it. The tutorial now requires pymoo (works with 0.6.1.3, should we fix the version?).

With all these changes, the plots are still sensitive to the optimization budget used in pymoo to compute the HV. I cannot produce plots that differentiate the methods as clearly as the originals.

The internal pymoo version is 0.6.0.1. We also need to upgrade that before this can land.

saitcakmak avatar Aug 26 '24 14:08 saitcakmak

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 99.98%. Comparing base (831072c) to head (2da31e8). Report is 118 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2488   +/-   ##
=======================================
  Coverage   99.98%   99.98%           
=======================================
  Files         193      193           
  Lines       16941    16941           
=======================================
  Hits        16939    16939           
  Misses          2        2           

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Aug 26 '24 14:08 codecov[bot]