Oracles not satisfying public benchmarks

[amva13]

          Hi @amva13 , thanks for the fix!

Checking with the other oracles in that specific version, something seems to break in deco hop. In the first example of the documentation (the same one I provided above) I went from getting 0.5338... to getting 0.0. Weird!

The rest of the oracles seem to work as expected, except for the ones in the issue I raised recently (#244).

see https://github.com/mims-harvard/TDC/blob/main/tutorials/User_Group/UserGroupMeeting_Wenhao.ipynb

Thanks again for the hard work.

Originally posted by @miguelgondu in https://github.com/mims-harvard/TDC/issues/238#issuecomment-2047673195

[amva13]

making note here. this and many similar issues probably due to

1 repository in your mims-harvard organization might be affected by a security vulnerability in nltk  ntlk unsafe deserialization vulnerability High severity  nltkCVE-2024-39705 View all alerts  mims-harvard/TDCexamples/generation/docking_generation/guacamol_tdc/guacamol_baselines/dockers/requirements.txtexamples/generation/docking_generation/guacamol_tdc/guacamol_baselines/requirements.txt | 1 repository in your mims-harvard organization might be affected by a security vulnerability in nltk | 1 repository in your mims-harvard organization might be affected by a security vulnerability in nltk |   | ntlk unsafe deserialization vulnerability High severity  nltkCVE-2024-39705 View all alerts  mims-harvard/TDCexamples/generation/docking_generation/guacamol_tdc/guacamol_baselines/dockers/requirements.txtexamples/generation/docking_generation/guacamol_tdc/guacamol_baselines/requirements.txt | ntlk unsafe deserialization vulnerability High severity  nltkCVE-2024-39705 View all alerts  mims-harvard/TDCexamples/generation/docking_generation/guacamol_tdc/guacamol_baselines/dockers/requirements.txtexamples/generation/docking_generation/guacamol_tdc/guacamol_baselines/requirements.txt | ntlk unsafe deserialization vulnerability High severity  nltkCVE-2024-39705 View all alerts  mims-harvard/TDCexamples/generation/docking_generation/guacamol_tdc/guacamol_baselines/dockers/requirements.txtexamples/generation/docking_generation/guacamol_tdc/guacamol_baselines/requirements.txt | ntlk unsafe deserialization vulnerability High severity  nltkCVE-2024-39705 View all alerts | ntlk unsafe deserialization vulnerability High severity  nltkCVE-2024-39705 View all alerts |   |   |   |   | View all alerts | View all alerts | View all alerts |   | mims-harvard/TDCexamples/generation/docking_generation/guacamol_tdc/guacamol_baselines/dockers/requirements.txtexamples/generation/docking_generation/guacamol_tdc/guacamol_baselines/requirements.txt |   | mims-harvard/TDCexamples/generation/docking_generation/guacamol_tdc/guacamol_baselines/dockers/requirements.txtexamples/generation/docking_generation/guacamol_tdc/guacamol_baselines/requirements.txt -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- 1 repository in your mims-harvard organization might be affected by a security vulnerability in nltk | 1 repository in your mims-harvard organization might be affected by a security vulnerability in nltk |   1 repository in your mims-harvard organization might be affected by a security vulnerability in nltk   ntlk unsafe deserialization vulnerability High severity  nltkCVE-2024-39705 View all alerts  mims-harvard/TDCexamples/generation/docking_generation/guacamol_tdc/guacamol_baselines/dockers/requirements.txtexamples/generation/docking_generation/guacamol_tdc/guacamol_baselines/requirements.txt | ntlk unsafe deserialization vulnerability High severity  nltkCVE-2024-39705 View all alerts  mims-harvard/TDCexamples/generation/docking_generation/guacamol_tdc/guacamol_baselines/dockers/requirements.txtexamples/generation/docking_generation/guacamol_tdc/guacamol_baselines/requirements.txt | ntlk unsafe deserialization vulnerability High severity  nltkCVE-2024-39705 View all alerts  mims-harvard/TDCexamples/generation/docking_generation/guacamol_tdc/guacamol_baselines/dockers/requirements.txtexamples/generation/docking_generation/guacamol_tdc/guacamol_baselines/requirements.txt | ntlk unsafe deserialization vulnerability High severity  nltkCVE-2024-39705 View all alerts | ntlk unsafe deserialization vulnerability High severity  nltkCVE-2024-39705 View all alerts |   |   |   |   | View all alerts | View all alerts | View all alerts |   | mims-harvard/TDCexamples/generation/docking_generation/guacamol_tdc/guacamol_baselines/dockers/requirements.txtexamples/generation/docking_generation/guacamol_tdc/guacamol_baselines/requirements.txt |   | mims-harvard/TDCexamples/generation/docking_generation/guacamol_tdc/guacamol_baselines/dockers/requirements.txtexamples/generation/docking_generation/guacamol_tdc/guacamol_baselines/requirements.txt ntlk unsafe deserialization vulnerability High severity  nltkCVE-2024-39705 View all alerts  mims-harvard/TDCexamples/generation/docking_generation/guacamol_tdc/guacamol_baselines/dockers/requirements.txtexamples/generation/docking_generation/guacamol_tdc/guacamol_baselines/requirements.txt | ntlk unsafe deserialization vulnerability High severity  nltkCVE-2024-39705 View all alerts  mims-harvard/TDCexamples/generation/docking_generation/guacamol_tdc/guacamol_baselines/dockers/requirements.txtexamples/generation/docking_generation/guacamol_tdc/guacamol_baselines/requirements.txt | ntlk unsafe deserialization vulnerability High severity  nltkCVE-2024-39705 View all alerts | ntlk unsafe deserialization vulnerability High severity  nltkCVE-2024-39705 View all alerts |   |   |   |   | View all alerts | View all alerts | View all alerts |   | mims-harvard/TDCexamples/generation/docking_generation/guacamol_tdc/guacamol_baselines/dockers/requirements.txtexamples/generation/docking_generation/guacamol_tdc/guacamol_baselines/requirements.txt |   | mims-harvard/TDCexamples/generation/docking_generation/guacamol_tdc/guacamol_baselines/dockers/requirements.txtexamples/generation/docking_generation/guacamol_tdc/guacamol_baselines/requirements.txt ntlk unsafe deserialization vulnerability High severity  nltkCVE-2024-39705 View all alerts  mims-harvard/TDCexamples/generation/docking_generation/guacamol_tdc/guacamol_baselines/dockers/requirements.txtexamples/generation/docking_generation/guacamol_tdc/guacamol_baselines/requirements.txt | ntlk unsafe deserialization vulnerability High severity  nltkCVE-2024-39705 View all alerts | ntlk unsafe deserialization vulnerability High severity  nltkCVE-2024-39705 View all alerts |   |   |   |   | View all alerts | View all alerts | View all alerts |   | mims-harvard/TDCexamples/generation/docking_generation/guacamol_tdc/guacamol_baselines/dockers/requirements.txtexamples/generation/docking_generation/guacamol_tdc/guacamol_baselines/requirements.txt |   | mims-harvard/TDCexamples/generation/docking_generation/guacamol_tdc/guacamol_baselines/dockers/requirements.txtexamples/generation/docking_generation/guacamol_tdc/guacamol_baselines/requirements.txt ntlk unsafe deserialization vulnerability High severity  nltkCVE-2024-39705 View all alerts | ntlk unsafe deserialization vulnerability High severity  nltkCVE-2024-39705 View all alerts |   |   |   |   | View all alerts | View all alerts | View all alerts |   ntlk unsafe deserialization vulnerability High severity  nltkCVE-2024-39705 View all alerts |   |   |   |   | View all alerts | View all alerts | View all alerts |           View all alerts | View all alerts | View all alerts View all alerts | View all alerts View all alerts   mims-harvard/TDCexamples/generation/docking_generation/guacamol_tdc/guacamol_baselines/dockers/requirements.txtexamples/generation/docking_generation/guacamol_tdc/guacamol_baselines/requirements.txt |   | mims-harvard/TDCexamples/generation/docking_generation/guacamol_tdc/guacamol_baselines/dockers/requirements.txtexamples/generation/docking_generation/guacamol_tdc/guacamol_baselines/requirements.txt   | mims-harvard/TDCexamples/generation/docking_generation/guacamol_tdc/guacamol_baselines/dockers/requirements.txtexamples/generation/docking_generation/guacamol_tdc/guacamol_baselines/requirements.txt

1 repository in your mims-harvard organization might be affected by a security vulnerability in nltk

ntlk unsafe deserialization vulnerability High severity

nltk

CVE-2024-39705

mims-harvard/TDC

examples/generation/docking_generation/guacamol_tdc/guacamol_baselines/dockers/requirements.txt examples/generation/docking_generation/guacamol_tdc/guacamol_baselines/requirements.txt

[amva13]

making this the general ticket for https://github.com/mims-harvard/TDC/issues/244 https://github.com/mims-harvard/TDC/issues/235 https://github.com/mims-harvard/TDC/issues/210

issue being most explored is https://github.com/mims-harvard/TDC/issues/238#issuecomment-2046067032

[amva13]

https://github.com/mims-harvard/TDC/issues/235#issuecomment-2207244304 this ticker and all associated will be closed with https://github.com/mims-harvard/TDC/pull/293

[amva13]

In addition to everything discussed regarding community vs TDC oracles, TDC-2 has introduced the Model Hub. We will migrate away from the current Oracle framework to the model hub, eliminating all the problems associated with pickling. https://huggingface.co/tdc

[amva13]

closed by https://github.com/mims-harvard/TDC/pull/293

[amva13]

for tracking, also closed https://github.com/mims-harvard/TDC/issues/291