`VectorStoreIndex.init` broken when not using env variables

[parhammmm]

getEmbeddedModel initialises OpenAIEmbedding without any options passed to it when using:

    let vsi = await VectorStoreIndex.init({
      serviceContext: ...,
      nodes: ...,
    });

Trace:

Error: Set OpenAI Key in OPENAI_API_KEY env variable
    at new OpenAISession (webpack-internal:///(rsc)/../LlamaIndexTS/packages/core/dist/llm/openai.js:416:19)
    at getOpenAISession (webpack-internal:///(rsc)/../LlamaIndexTS/packages/core/dist/llm/openai.js:442:19)
    at new OpenAIEmbedding (webpack-internal:///(rsc)/../LlamaIndexTS/packages/core/dist/embeddings/OpenAIEmbedding.js:71:109)
    at getEmbeddedModel (webpack-internal:///(rsc)/../LlamaIndexTS/packages/core/dist/internal/settings/EmbedModel.js:15:31)
    at new VectorStoreBase (webpack-internal:///(rsc)/../LlamaIndexTS/packages/core/dist/storage/vectorStore/types.js:22:123)
    at new SimpleVectorStore (webpack-internal:///(rsc)/../LlamaIndexTS/packages/core/dist/storage/vectorStore/SimpleVectorStore.js:29:9)
    at storageContextFromDefaults (webpack-internal:///(rsc)/../LlamaIndexTS/packages/core/dist/storage/StorageContext.js:25:100)
    at VectorStoreIndex.init (webpack-internal:///(rsc)/../LlamaIndexTS/packages/core/dist/indices/vectorStore/index.js:417

https://github.com/run-llama/LlamaIndexTS/blob/fb2c1fa91765e6319677274383742bef664d071e/packages/core/src/internal/settings/EmbedModel.ts#L8

[marcusschiesser]

@parhammmm thanks for the report. Can you send a full example, then I can have a closer look at it

[parhammmm]

@marcusschiesser no problem and thanks for replying!

Above would be the full example. If you just do that without setting OPENAI_API_KEY it can be replicated.

Our workaround was to provide the storage context.

    const storageContext = await storageContextFromDefaults({
      vectorStore: new SimpleVectorStore({
        embedModel: getEmbeddingModel(this.embeddingModel),
      }),
    });
    let vsi = await VectorStoreIndex.init({
      storageContext,
      serviceContext: serviceContextForEmbedding(this.embeddingModel),
      nodes: parsedNodes,
    });

serviceContextForEmbedding just returns a service context object with our embedding and llm

Side note: I know the project now includes Settings etc but that would not work at all for our use case, hence the above. Please don't deprecate serviceContextFromDefaults it is very useful because Settings is global from what I understand which is not usable for us.

[himself65]

I know the project now includes Settings etc but that would not work at all for our use case, hence the above. Please don't deprecate serviceContextFromDefaults it is very useful because Settings is global from what I understand which is not usable for us.

Is Setting.withXXX API good for your case?

[cdll]

same issue here with creating Ollama embedModel in a serviceContext for the VectorStoreIndex.

[parhammmm]

Same issue exists with ChatMemoryBuffer in SimpleChatEngine because ChatMemoryBuffer has it's own llm context instead of using SimpleChatEngine's

https://github.com/run-llama/LlamaIndexTS/blob/main/packages/llamaindex/src/engines/chat/SimpleChatEngine.ts#L25

and

https://github.com/run-llama/LlamaIndexTS/blob/main/packages/llamaindex/src/engines/chat/SimpleChatEngine.ts#L39

and

https://github.com/run-llama/LlamaIndexTS/blob/main/packages/llamaindex/src/engines/chat/ContextChatEngine.ts#L53

[parhammmm]

I know the project now includes Settings etc but that would not work at all for our use case, hence the above. Please don't deprecate serviceContextFromDefaults it is very useful because Settings is global from what I understand which is not usable for us.

Is Setting.withXXX API good for your case?

Kind of helps but it feels odd, currently using it to patch the above errors. Will keep posted on what happens but we do use multiple llms so curious how it will work there... will keep posted

[himself65]

I see, I think we didn't pass some context properly, I will check that later

[AVtheking]

is this fixed ?

[himself65]

I think no

[AVtheking]

I was checking code of getEmbeddibgModel and now we do not use OpenAI directly there

[himself65]

if so, would u please add a test case for this? I want to make repo stable

[AVtheking]

Could you tell me what exactly to add test for 😅 , is it should be for not initializing OpenAIEmbedding model in the getEmbeddingModel function ?

[marcusschiesser]

@parhammmm sorry for taking so long to look at this!

The problem is that VectorStoreIndex requires a vector store and a vector store requires an embedding model which defaults to Settings.embedModel, so calling:

await VectorStoreIndex.init({
      nodes: ...,
    })

will use the OpenAI embedding model which is the default value of Settings.embedModel. So either you specify a vector store with embedding model or you change the Settings.embedModel value.

I think we should remove this OpenAI default, see https://github.com/run-llama/LlamaIndexTS/issues/1797

I'll close this ticket for now. Please feel free to reopen if that doesn't work for you.

[AVtheking]

@marcusschiesser I think he also meant that we didn't use the llms passed locally in individual classes for eg here

we didn't pass the llm passed in the SimpleChatEngine to the ChatMemoryBuffer class , it uses from the global settings.

[marcusschiesser]

yes, that's a bug, i'll send a PR

[marcusschiesser]

https://github.com/run-llama/LlamaIndexTS/pull/1798