Dulany W

icon indicating a website link https://www.linkedin.com/in/dulanyweaver

icon location Ontario, Canada icon location Strategy Leader | Engineering Manager | Software Engineer, focused on Javascript, Python, and Ruby on Rails. Passionate about making businesses better. He/Him.

Results 6 comments of Dulany W

Expose API

Agree with all comments above. The highlight of this awesome effort is the ability to use a range of models cross-platform at high performance - but doing so requires a...

Expose API

Related: https://github.com/mlc-ai/mlc-llm/issues/74#issuecomment-1533909744

win 10 not work

This project may not be the tool you're looking for. This project provides a chat system you can run on your own computer locally. This is not OpenAI's ChatGPT system,...

Stops after sampling parameters: temp = 0.100000, top_k = 40, top_p = 0.950000, repeat_last_n = 64, repeat_penalty = 1.300000

Same issue. Based on some quick troubleshooting, issue is silent failure somewhere in the llama_eval function. This is also on the Windows build, also observed after building from source, also...

Stops after sampling parameters: temp = 0.100000, top_k = 40, top_p = 0.950000, repeat_last_n = 64, repeat_penalty = 1.300000

Update: Traced it down to a silent failure in the function "ggml_graph_compute" in ggml.c. Wonder if it might be a multi-threading issue? However, still failed when number of threads set...

Stops after sampling parameters: temp = 0.100000, top_k = 40, top_p = 0.950000, repeat_last_n = 64, repeat_penalty = 1.300000

Final update: Was able to at least reach the prompt by updating the CMAKELists.txt file so that the third line under "if (MSVC)" was "set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -march=native")" (per GG's recommendation...