autolabel icon indicating copy to clipboard operation
autolabel copied to clipboard

[Feature Request]: Support label descriptions for classification and NER tasks

Open rishabh-bhargava opened this issue 11 months ago • 2 comments

Is your feature request related to a problem? Please describe. For some data labeling tasks, it could be useful to add a short description for what any given label means. This could be provided in the LLM prompt right next to the labels themselves and act as helpful context for the LLM.

Describe the solution you'd like Instead of supporting a simple list of strings under the prompt.labels key in the config, we could also support a list of objects, of the type. Example, instead of:

"labels": [
        "Location",
        "Organization",
        "Person"
]

it could say:

"labels": [
    {
        "name": "Location",
        "description": "A location such as a city, country, state, or province."
    },
    {
        "name": "Organization",
        "description": "An organization such as a company, school, or non-profit."
    },
    {
        "name": "Person",
        "description": "A person or fictional character."
    }
]

Additional context We'd want to benchmark this on a few tasks to see if/how much it helps. Ideally, classification and NER both, and especially a task that has many output labels.

rishabh-bhargava avatar Jul 13 '23 21:07 rishabh-bhargava