AttributeError: 'XGLMForCausalLM' object has no attribute 'kai_scanner'

[briankendall]

I've installed KoboltAI-Client on my MacBook Pro with 32 GB of RAM running macOS 10.14.6. Not the ideal environment for it, granted, since everything will have to be done on the CPU, but it's what I've got.

I can get either the huggingface or rocm conda environments to install (though I have to remove cudatoolkit from huggingface.yml) and then launch the app using python aiserver.py -v --cpu. However, after I load a model and give it a prompt, I get this exception:

ERROR      | __main__:generate:4947 - Traceback (most recent call last):
  File "aiserver.py", line 4936, in generate
    genout, already_generated = tpool.execute(_generate, txt, minimum, maximum, found_entries)
  File "/path/to/koboldai/env/lib/python3.8/site-packages/eventlet/tpool.py", line 132, in execute
    six.reraise(c, e, tb)
  File "/path/to/koboldai/env/lib/python3.8/site-packages/six.py", line 719, in reraise
    raise value
  File "/path/to/koboldai/env/lib/python3.8/site-packages/eventlet/tpool.py", line 86, in tworker
    rv = meth(*args, **kwargs)
  File "aiserver.py", line 4870, in _generate
    if(model.kai_scanner.halt or not model.kai_scanner.regeneration_required):
  File "/path/to/koboldai/env/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1269, in __getattr__
    raise AttributeError("'{}' object has no attribute '{}'".format(
AttributeError: 'XGLMForCausalLM' object has no attribute 'kai_scanner'

The model I'm loading is the GPU version of Nerys. Any idea how I can get this working?

[michaelbogdan]

Hi, I am trying to get this to run on my Apple M1 Pro with macOS 13.2, can you tell me the steps you followed to install it? I just want to connect to the Kobold Horde and check out the interface.

[briankendall]

At this point I can't really remember the finer details of what I did. But I believe I set up the conda environment using rocm.yml, which allowed me to run it in CPU-only mode.

That said, it was so painfully slow running it on either an Intel or M1 mac that it was pretty much unusable. You have a faster CPU than I used, so maybe you'll have more luck, but I wouldn't get your hopes up!