spacy-llm icon indicating copy to clipboard operation
spacy-llm copied to clipboard

Working dummy example for custom LLM endpoint integration

Open borhenryk opened this issue 1 year ago • 1 comments

How would a working dummy example work for the scenario of an own LLM like in this README.md https://github.com/explosion/spacy-llm/blob/main/usage_examples/README.md when running it with the proposed config file and adding the dummy RandomClassification.v1 I get this error:

Exception has occurred: AssertionError exception: no description

from spacy_llm.registry import registry
import random
from typing import Iterable

@registry.llm_models("RandomClassification.v1")
def random_textcat(labels: str):
    labels = labels.split(",")
    def _classify(prompts: Iterable[str]) -> Iterable[str]:
        for _ in prompts:
            yield random.choice(labels)

    return _classify
...
[components.llm.task]
@llm_tasks = "spacy.TextCat.v1"
labels = LABEL1,LABEL2,LABEL3


[components.llm.model]
@llm_models = "RandomClassification.v1"
labels = ${components.llm.task.labels}  # Make sure to use the same label

borhenryk avatar Jan 29 '24 13:01 borhenryk

Hi @borhenryk, can you provide your config and the entire script?

rmitsch avatar Feb 01 '24 11:02 rmitsch