Alpaca-LoRA-Serve

Alpaca-LoRA-Serve copied to clipboard

While the model t5-vicuna loads, with the first prompt I get the following error:

determined model type: t5-vicuna
...
...
...

loc("mps_clamp"("(mpsFileLoc): /AppleInternal/Library/BuildRoots/c2cb9645-dafc-11ed-aa26-6ec1e3b3f7b3/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphUtilities.mm":268:0)): error: input types 'tensor<1x78x2048xf16>' and 'tensor<1x1x1xf32>' are not broadcast compatible
LLVM ERROR: Failed to infer result type(s).
[1]    76583 abort      python3 app.py
/opt/homebrew/Caskroom/miniconda/base/envs/llm-serve/lib/python3.9/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '

I found a simillar report here (stable-diffusion-webui) but not sure how to set the no-half flag if thats the case.

Oh will fix it by setting no half in case of t5 vicuna. Thanks for reporting this

Hi all, i have the same issue when using SD.Next with Controlnet => python crashes. (mac Studio, M2 ultra, 64gb). SD.Next does not allow --no-half but --no-half-controlnet. Unfortunately this does not solve this issue either. Any suggestions on that ?