langkit icon indicating copy to clipboard operation
langkit copied to clipboard

Importing metrics issue since there is not a way to pass the model path if stored locally

Open Starignus opened this issue 5 months ago • 3 comments

If you want to import a metric or the metrics module like:

from langkit import toxicity,
from langkit import llm_metrics 

By default, it downloads the models from the Huggingfaces when you try to import the module. The issue is when your organisation blocks the connection for downloading big files, but the organisation hosts the models in a secure location. For your reference, see this issue on the Transformers page

I searched in Langkit documentation for a way that the user could indicate the path of the models, but I could not find anything. Besides, it is impossible to pass any variable to a module when importing it. The problem can be solved by letting the user provide a path in a configuration file (e.g. JSON) that could override the default path. For example, in the toxicity module, I can see that the option can be taken.

This can be a potential blocker if an organisation wants to try the package and cannot since it might have some security concerns. This will be a good enhancement.

Starignus avatar Feb 07 '24 14:02 Starignus