autolabel icon indicating copy to clipboard operation
autolabel copied to clipboard

LLM label method should pass through exceptions instead of returning empty generations

Open yadavsahil197 opened this issue 2 years ago • 0 comments

Currently, in case of any exception from the LLM for generation, we return an empty generation, which will later be caught as a Invalid Response Parsing Error. This is misleading, and we should instead catch them as failures. Error generating from LLM: ('post request failed with status code: 404', ''), returning empty result 2023-05-12 18:02:56.103 | INFO | autolabel.tasks.base:parse_json_llm_response:134 - Error parsing LLM response: . ValueError('No valid JSON substring found. Either increase the max_tokens or the model is not able to follow the output format instruction.')

yadavsahil197 avatar May 13 '23 01:05 yadavsahil197