funcchain

funcchain copied to clipboard

I'd like to get the exact string sent to the backend, after everything is filled in and processed and formatted etc. How can I do that please?

AriMKatz

TypedDict is not supported

1

Would it be possible to support it to avoid pydantic import?

ahuang11

How does Funcchain handle different prompt templates?

3

Hi again. I'm wondering, does funcchain handle different chat templates internally? e.g. for Llama ``` [INST] You are a helpful, respectful and honest assistant. Always answer as helpfully as possible....

ahuang11

Docstring for get_gguf_model is inaccurate

1

I don't think it matches the signature. ```python get_gguf_model( name: str, label: str, settings: FuncchainSettings, ) -> Path: """ Gather GGUF model from huggingface/TheBloke possible input: - DiscoLM-mixtral-8x7b-v2-GGUF - TheBloke/DiscoLM-mixtral-8x7b-v2...

ahuang11

Document how to reuse LLM (without having it restart on every run)

1

So that `llama_model_loader: loaded meta data with 20 key-value pairs and 291 tensors from .models/mistral-7b-instruct-v0.1.Q4_K_M.gguf...` is only shown once ```python from funcchain import chain, settings from funcchain.model.defaults import ChatLlamaCpp, get_gguf_model...

ahuang11