UnsupportedOperationException on build() call of HuggingFaceChatModel

[ginccc]

trafficstars

Hi there,

I am encountering the following exception when trying to build a dev.langchain4j.model.huggingface.HuggingFaceChatModel

dev.langchain4j.model.huggingfacejava.lang.UnsupportedOperationException: Should not be called
	at io.quarkiverse.langchain4j.huggingface.QuarkusHuggingFaceClientFactory.create(QuarkusHuggingFaceClientFactory.java:32)
	at dev.langchain4j.model.huggingface.HuggingFaceChatModel.<init>(HuggingFaceChatModel.java:45)
	at dev.langchain4j.model.huggingface.HuggingFaceChatModel$Builder.build(HuggingFaceChatModel.java:153)

Code to reproduce:

var builder = HuggingFaceChatModel.builder().accessToken("<token>");
HuggingFaceChatModel chatModel = builder.build(); // Exception on calling build()

On another note: Missing logRequests and logResponses methods on the HuggingFace builder (OpenAI, Anthropic, Gemini and Ollama builders have them defined)

[geoand]

Thanks for reporting.

This is definitely a bug, I'll have a look soon.

[geoand]

You can however access use CDI to access ChatLanguageModel or use QuarkusHuggingFaceChatModel programmatically.

[ginccc]

I have tried it with QuarkusHuggingFaceModel, but it seems to require a url. Shouldn't there be any default for this, so that only the accessToken is needed?

java.lang.IllegalStateException: No URL specified. Cannot build a rest client without URL
	at io.quarkus.rest.client.reactive.runtime.RestClientBuilderImpl.build(RestClientBuilderImpl.java:338)
	at io.quarkus.rest.client.reactive.runtime.QuarkusRestClientBuilderImpl.build(QuarkusRestClientBuilderImpl.java:260)
	at io.quarkiverse.langchain4j.huggingface.QuarkusHuggingFaceClientFactory.create(QuarkusHuggingFaceClientFactory.java:48)
	at io.quarkiverse.langchain4j.huggingface.QuarkusHuggingFaceChatModel.<init>(QuarkusHuggingFaceChatModel.java:46)
	at io.quarkiverse.langchain4j.huggingface.QuarkusHuggingFaceChatModel$Builder.build(QuarkusHuggingFaceChatModel.java:194)

[ginccc]

The builder of QuarkusHuggingFaceChatModel seems to be missing the modelId setter which is available on the HuggingFaceChatModel builder

[geoand]

#604 will provide a default URL

[geoand]

The builder of QuarkusHuggingFaceChatModel seems to be missing the modelId setter which is available on the HuggingFaceChatModel builder

Right, it's part of the URL really.

[geoand]

Which I should not is not ideal, but we'll have to take a closer look at what we do with HuggingFace at some point in the future.

[ginccc]

As I can see, the model is part of the url, however, from a library usage point of view it would be great if I could just hand over the modelId, and the url will be constructed for me. I could of course also do it in our implementation code, but solving this within the library would be better as it would be more consistent with HuggingFaceChatModel which has modelId setter implemented

[geoand]

I agree, we need to redo this as it is not optimal

[geoand]

That said, if you use the CDI bean, you need to deal with creating the LangChain4j models at all 😀

[ginccc]

I need it for being able to configure it at runtime, so CDI bean will not be ideal as I would like to be able to create multiple ones with different parameters (potentially for chaining them).

@ApplicationScoped
public class HuggingFaceLanguageModelBuilder implements ILanguageModelBuilder {
    private static final String KEY_ACCESS_TOKEN = "accessToken";
    private static final String KEY_MODEL_ID = "modelId";
    private static final String KEY_TEMPERATURE = "temperature";
    private static final String KEY_TIMEOUT = "timeout";

    @Override
    public ChatLanguageModel build(Map<String, String> parameters) {
        var builder = HuggingFaceChatModel.builder();
        if (!isNullOrEmpty(parameters.get(KEY_ACCESS_TOKEN))) {
            builder.accessToken(parameters.get(KEY_ACCESS_TOKEN));
        }
        
        if (!isNullOrEmpty(parameters.get(KEY_MODEL_ID))) {
            builder.modelId(parameters.get(KEY_MODEL_ID));
        }

        if (!isNullOrEmpty(parameters.get(KEY_TIMEOUT))) {
            builder.timeout(Duration.ofMillis(Long.parseLong(parameters.get(KEY_TIMEOUT))));
        }

        if (!isNullOrEmpty(parameters.get(KEY_TEMPERATURE))) {
            builder.temperature(Double.parseDouble(parameters.get(KEY_TEMPERATURE)));
        }

        return builder.build();
    }
}

[geoand]

I see

[geoand]

I'll reopen since we need to improve here