Andrei comments

Results 177 comments of


                                            Andrei

Workflow speed up - POST PART 1

I tried setting `RUST_LOG=trace` locally but I still don't see the actual logs from the cmake build system.

Workflow speed up - POST PART 1

@Smartappli thank you for your contribution and patience on this one, merging now that uv has support for the build logs with `--verbose`

Allow relative paths at model initialization

@andreakiro what's your operating system? I only ask because relative paths have always worked for me.

Allow relative paths at model initialization

@andreakiro ah gotcha. So strictly speaking that isn't a relative path, it's a shortcut that's expanded through an environment environment variable, this is usually done by the user shell (for...

When using LlamaCppEmbeddings to embed a gguf type model, an error is reported

Sorry to get to this so late, I'll take a look!

flash attention on Nvidia Tesla P100s results in the `CUDA error: unspecified launch failure` - (`CUDA kernel flash_attn_tile_ext_f16 has no device code compatible with CUDA arch 520`)

@AlHering two suggestions on debugging this: - To temporarily fix this you may need to modify the `CUDA_ARCHITECTURES` flag for cmake using cmake args to target a higher cuda arch....

WIP: Mechanism to retrieve all logprobs on completion

@tristanvdb thanks for the PR, I think one of the next pieces of development is going to be to make the high-level API a little more functional / modular to...