Adrian Mirza comments

Results 24 comments of


                                            Adrian Mirza

chembench 2.0 - collecting ideas

> another framework https://ukgovernmentbeis.github.io/inspect_ai/ this one looks quite robust, but I see that we cannot add new models if using some of their code

feat: summary file for multiple models

Maybe add a function to run multiple models in comparison to each other on the same questions?

feat: add human subset, updated classification, plotting function

the cli works now, but I need to run this first in the home directory. `PYTHONPATH=$(pwd):$PYTHONPATH`

feat: add human subset, updated classification, plotting function

> > the cli works now, but I need to run this first in the home directory. `PYTHONPATH=$(pwd):$PYTHONPATH` > > hm, this should not be needed if you installed the...

feat: add human subset, updated classification, plotting function

@sourcery-ai review

feat: add human subset, updated classification, plotting function

@kjappelbaum should I add `scripts` to `MANIFEST`?

feat: add human subset, updated classification, plotting function

> @kjappelbaum should I add `scripts` to `MANIFEST`? what do you say?

feat: add human subset, updated classification, plotting function

@kjappelbaum @n0w0f should we merge this?

feat: add human subset, updated classification, plotting function

The code is final. I can delete the csvs from this PR if changes were made to them for the app? And keep only the full classification in?

feat: add human subset, updated classification, plotting function

> I think the deletion [here](https://github.com/lamalab-org/chem-bench/pull/478/commits/ae18772197e6498bbba4c6d45b5662ce521424b3) will actually delete the file when we merge The order seems correct no? Since @MrtinoRG merged main into this branch after removing them from...