[BUG] Mistral Codestral FIM 400 Response

[owpk]

Describe the bug Getting 400 code response from https://codestral.mistral.ai/v1/fim/completions According to the codestral api documentation, the n_prediction field is passed, which should not be in the request body

• provider settings: (api key hidden by me, but it refreshed and checked)

Logging

***Twinny Stream Debug***
    Streaming response from codestral.mistral.ai.
    Request body:
    {
  "prompt": "\n\n[SUFFIX]\n            return callable.call();\n        } catch (StatusRuntimeException e) {\n            log.info(\"Catch grpc status exception: \" + e.getStatus());\n            if (e.getStatus().getCode().equals(Status.UNAUTHENTICATED.getCode())) {\n                log.info(\"Seems jwt token is corrupted. Refreshing token...\");\n                try {\n                    jwtTokenProvider.refreshToken();\n                    return callable.call();\n                } catch (Exception ex) {\n                    jwtTokenProvider.cleareToken();\n                    throw new RuntimeException(\"Error while refreshing token\", ex);\n                }\n            } else\n                throw e;\n        } catch (Throwable e) {\n            log.error(\"Catch retrying exception: \" + e);\n            throw new RuntimeException(e);\n        }\n    }\n}\n[PREFIX]\n/* Language: Java (java) */\n/* File uri: file:///Users/slavavorobev/gh/gigachat-grpc-client/src/main/java/owpk/grpc/GigaChatGRpcClient.java (java) */\npackage owpk.grpc;\n\nimport java.util.Iterator;\nimport java.util.concurrent.Callable;\nimport java.util.concurrent.TimeUnit;\n\nimport javax.net.ssl.SSLException;\n\nimport gigachat.v1.ChatServiceGrpc;\nimport gigachat.v1.Gigachatv1;\nimport gigachat.v1.ModelsServiceGrpc;\nimport io.grpc.CallCredentials;\nimport io.grpc.Channel;\nimport io.grpc.Status;\nimport io.grpc.StatusRuntimeException;\nimport io.grpc.netty.GrpcSslContexts;\nimport io.grpc.netty.NettyChannelBuilder;\nimport io.netty.handler.ssl.util.InsecureTrustManagerFactory;\nimport lombok.Getter;\nimport lombok.extern.slf4j.Slf4j;\nimport owpk.properties.concrete.MainProps;\n\n@Slf4j\npublic class GigaChatGRpcClient extends ModelsServiceGrpc.ModelsServiceImplBase {\n    private static final Integer DEFAULT_SSL_PORT = 443;\n    @Getter\n    private final JwtTokenProvider jwtTokenProvider;\n    private final ChatServiceGrpc.ChatServiceBlockingStub stub;\n    private final ModelsServiceGrpc.ModelsServiceBlockingStub modelStub;\n    private final CallCredentials callCredentials;\n\n    public GigaChatGRpcClient(MainProps props, JwtTokenProvider jwtTokenProvider) throws SSLException {\n        this.jwtTokenProvider = jwtTokenProvider;\n        var channel = configureNettyChannel(props.getProperty(MainProps.DEF_TARGET));\n        callCredentials = new GigaChatCreds(jwtTokenProvider);\n        stub = ChatServiceGrpc.newBlockingStub(channel).withCallCredentials(callCredentials);\n        modelStub = ModelsServiceGrpc.newBlockingStub(channel).withCallCredentials(callCredentials);\n    }\n\n    private Channel configureNettyChannel(String target) throws SSLException {\n        var ssl = GrpcSslContexts.forClient()\n                .trustManager(InsecureTrustManagerFactory.INSTANCE).build();\n        var builder = NettyChannelBuilder.forAddress(target, DEFAULT_SSL_PORT)\n                .enableRetry()\n                .keepAliveTime(10, TimeUnit.SECONDS);\n        return builder\n                .sslContext(ssl)\n                .build();\n    }\n\n    public Gigachatv1.ListModelsResponse modelList() {\n        return modelStub.listModels(Gigachatv1.ListModelsRequest.newBuilder().build());\n    }\n\n    public Gigachatv1.ChatResponse chat(Gigachatv1.ChatRequest chatRequest) {\n        return catchUnauthorized(() -> stub.chat(chatRequest));\n    }\n\n    public Iterator<Gigachatv1.ChatResponse> chatStream(Gigachatv1.ChatRequest chatRequest) {\n        return catchUnauthorized(() -> stub.chatStream(chatRequest));\n    }\n\n    private <T> T catchUnauthorized(Callable<T> callable) {\n        try {\n            ",
  "stream": true,
  "temperature": 0.2,
  "n_predict": -1
}

    Request options:
    {
  "hostname": "codestral.mistral.ai",
  "path": "/v1/fim/completions",
  "protocol": "https",
  "method": "POST",
  "headers": {
    "Content-Type": "application/json",
    "Authorization": "Bearer ###### (hidden)"
  }
}

ERR [Extension Host] [0m [ERROR_twinny] [32m Message: Fetch error Error Type: Error Error Message: Server responded with status code: 400

API Provider Mistral

Chat or Auto Complete? Autocomplete

Model Name codestral-latest

Additional info https://docs.mistral.ai/api/#tag/fim/operation/fim_completion_v1_fim_completions_post

[rjmacarthy]

Hello FIM for Mistral API is not currently supported.