medperf
medperf copied to clipboard
mlcommons
An open benchmarking platform for medical artificial intelligence using Federated Evaluation.
**Describe the bug** Only validates field after some lengthy operations, and error message doesn't say whether the command should be rerun or whether we can still change the field after...
This PR includes the necessary changes for running a preparation dashboard for their mlcube/experiment. Although at the moment is somewhat targeted to RANO, small tweaks may be done to make...
An ancestor of https://github.com/mlcommons/medperf/pull/550 (branch was renamed) ==== The goal is to make medperf client installable from pypi. This PR is the first part that allows installing package not from...
- now config.yaml location can be defined with custom value (env var) - integration tests use a new /tmp/ folder storage with every run - After every command db is...
Bumps the pip group with 1 update in the /cli directory: [pydantic](https://github.com/pydantic/pydantic). Bumps the pip group with 1 update in the /examples/ChestXRay/metrics/project directory: [scikit-learn](https://github.com/scikit-learn/scikit-learn). Bumps the pip group with 1...
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot] avatar"){kind=avatar}
status: waiting for experiment leads to decide on weather this is OK
Refactor entities and use inheritance to not repeat code. This is needed since for FL support we are adding 4 new entities with similar interface. This PR also removes the...
[In some places](https://github.com/mlcommons/medperf/blob/main/cli/medperf/entities/dataset.py#L93) we are using `{}` as default function param values. However this is error-prone as these default values are mutable and can be modified unexpectedly during the call:...
More information is needed, but it seems the filtering logic (at the comms layer, or at the entity layer) doesn't take into account models associated with multiple benchmarks.