LLMLingua icon indicating copy to clipboard operation
LLMLingua copied to clipboard

[Question]: access to public storage https://openaipublic.blob.core.windows.net/ is prohibited in secure environments ,

Open amrosalehms opened this issue 1 year ago • 3 comments

Describe the issue

In certain environments where private connectivity prohibits access to public storage at https://openaipublic.blob.core.windows.net/, we encountered a connection timeout error when attempting to import the library. Is there a workaround to avoid accessing this public storage account?

amrosalehms avatar Mar 11 '24 11:03 amrosalehms

Hi @amrosalehms, I suspect the issue is due to loading the tokenizer for GPT-3.5-turbo. You can comment out this line and related code to resolve the issue.

iofu728 avatar Mar 11 '24 12:03 iofu728

Hi @amrosalehms, I suspect the issue is due to loading the tokenizer for GPT-3.5-turbo. You can comment out this line and related code to resolve the issue.

Hi @iofu728 ,

I'v met the same issue. Is it ok to just comment out the lines you suggested? Will it cause any side effect?

Dorish avatar Mar 28 '24 10:03 Dorish

Hi @Dorish, the code invoking self.oai_tokenizer, such as in #712, needs to be modified.

iofu728 avatar Mar 29 '24 07:03 iofu728