Apple Silicon : macOS 14 - error: ANEProgramProcessRequestDirect()

[criskb]

Im running latest macOS 14 beta Apple M1 Mini 16GB

Getting this error and crash if I try to generate something

/AppleInternal/Library/BuildRoots/1a7a4148-f669-11ed-9d56-f6357a1003e8/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Runtimes/MPSRuntime/Operations/GPUANERegionOps.mm:332: failed assertion `ANE Evaluation Error = Error Domain=com.apple.appleneuralengine Code=3 "processRequest:model:qos:qIndex:modelStringID:options:error:: ANEProgramProcessRequestDirect() Failed with status=0xf : statusType=0x9: Program Inference timeout: timed out" UserInfo={NSLocalizedDescription=processRequest:model:qos:qIndex:modelStringID:options:error:: ANEProgramProcessRequestDirect() Failed with status=0xf : statusType=0x9: Program Inference timeout: timed out}' [2] 24037 abort ./webui.sh

/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown warnings.warn('resource_tracker: There appear to be %d '

(--no-half) seems to work for now but im getting worse speeds with this enabled

[jrittvo]

The "leaked semaphore" seems to be a generic warning when python crashes because of memory issues. The smaller print that you copied above that is the real error, but I'm in the dark as to what it means.

[atiorh]

Hi @criskb, couple of notes:

• I am unsure what you are referring to by --no-half, could you please clarify?
• Looks like you are using a webui which is not a part of this repository. Are you able to reproduce using our Swift or Python CLIs?
• Python 3.10 is not tested for this repo. I recommend 3.8 or 3.9 but I don't yet think this is related to your issue
• "leaked semaphore" is also highly unlikely to be root cause of your issue

[criskb]

@atiorh Just edited the (webui-macos-env.sh) to contain --no-half under commandline_args

im using https://github.com/brkirch/stable-diffusion-webui fork but I was able to make this work on the main project on automatic1111 page too

[jrittvo]

Automatic1111 does not use any of the CoreML python packages, so this is really the wrong place to be discussing this, but I have been using these command arguments for a while, with both the brkirch build and the main branch:

export COMMANDLINE_ARGS="--skip-torch-cuda-test --upcast-sampling --use-cpu interrogate --opt-sub-quad-attention --precision full --no-half-vae --disable-nan-check"

I also use only FP16 models. No FP32 models.

On my laptop, the generation speed running the current main branch is almost the same as using an efficient Swift app that does use CoreML. Recently, the brkirch build has been running noticeably slower for me than the main branch. It needs a new build, or perhaps all his newest optimizations are already merged into main and his experiments are finished.

I am still on macOS 13.5 beta. I have not tried Automatic1111 with the macOS 14 beta yet. I don't know if there are any important changes there in the MPS backend that Automatic1111 does use.