autolabel
autolabel copied to clipboard
LLM label method should pass through exceptions instead of returning empty generations
Currently, in case of any exception from the LLM for generation, we return an empty generation, which will later be caught as a Invalid Response Parsing Error. This is misleading, and we should instead catch them as failures.
Error generating from LLM: ('post request failed with status code: 404', ''), returning empty result
2023-05-12 18:02:56.103 | INFO | autolabel.tasks.base:parse_json_llm_response:134 - Error parsing LLM response: . ValueError('No valid JSON substring found. Either increase the max_tokens or the model is not able to follow the output format instruction.')