Jeff Waller

can you run instead? DEBUG=1 ollama serve Or equivalent to get it into debug mode? Also how many MIG instances, host GPUs?

hmm I see only the text `[image: image.png]` or were you referring to the previous message? Could you cut-paste the text instead of a screenshot? Also, if you indent the...

Ok first, sorry to mislead you, it's not `DEBUG=1` it's `OLLAMA_DEBUG=1` What I'm looking for is something like this: MIG Mode is 1 MIG Device Intance 0:0 found [. ]...

ok yea, this PR covers this: MIG Mode is 1 MIG Device Intance 0:0 found [0] CUDA device name: NVIDIA A100-SXM4-80GB MIG 7g.80gb [0] CUDA part number: 692-2G506-0210-002 [0] CUDA...

say, maybe check out my PR, what testing beyond what I've done (if any) is needed?

check me if I'm wrong on this, but you can't just add up the VRAM (e.g. use 10 5G instances), each MIG instance must have at least as much of...

That is certainly true and unless it's implicitly included in the Scope/Closure, then the only thing that prevents the synchronous case from failing as well is that it's throws to...

I've looked at the various blogs/documentation about how this is supposed to work, and supposedly it's all happen automatically so long as MakeCallback is used, but of course, that's not...

That's possible. However, [this slideware](http://www.slideshare.net/domenicdenicola/domains-20010482) especially slide 14 and in the node src [here](https://github.com/joyent/node/blob/master/src/node.cc#L981) and [here](https://github.com/joyent/node/blob/master/src/node.cc#L981), this appears to more closely associated with the Environment (node internal) and derived from...

Attempted that, and various other things but so far have not obtained the desired result. I feel that's very close to what needs to happen however.