specification icon indicating copy to clipboard operation
specification copied to clipboard

Update the "users" / "use cases" field in machine-learning-model component

Open bardenstein opened this issue 2 years ago • 2 comments

Proposal: based on research of model cards across HuggingFace, Google, and Meta, there seem to be a common way of breaking down usage: "direct/intended use," "out of scope" usage, and "misuse / malicious use."

Request: Remove the "users" and "use cases" fields, and replace them with the model below:

"usage" : [ "intended-use": "how is the model intended to be used, primary use case.", "out-of-scope": "out of scope usage.", "misuse-or-malicious": "examples of usage considered to be misuse or malicious use." ]

Required: True Type: Array of strings.

bardenstein avatar Aug 25 '23 17:08 bardenstein

Remove the "users" and "use cases" fields

That would be a breaking change which would have to wait until v2.0.

CycloneDX is aligned to the Tensorflow Model Toolkit and VerifyML, both of which support machine readable model cards supporting both "users" and "use cases".

See:

  • https://github.com/tensorflow/model-card-toolkit/blob/main/model_card_toolkit/schema/v0.0.2/model_card.schema.json#L104-L113
  • https://github.com/cylynx/verifyml/blob/main/verifyml/model_card_toolkit/schema/v0.0.4/model_card.schema.json#L61-L74

I'd be happy to add additional fields if necessary.

stevespringett avatar Aug 25 '23 21:08 stevespringett

@stevespringett here are some examples:

  • Llama2:

Model cards on HuggingFace, Meta, and beyond seem to break things down into Intended Use, Out of Scope, and Misuse/Malicious Use. No mention of "users," and I'm not sure what model developers would regularly put in that field.

bardenstein avatar Aug 28 '23 02:08 bardenstein