TabbyML
Linux arm64/v8 support
docker run -it -p 8080:8080 -v $HOME/.tabby:/data tabbyml/tabby serve --model TabbyML/StarCoder-1B Unable to find image 'tabbyml/tabby:latest' locally latest: Pulling from tabbyml/tabby docker: no matching manifest for linux/arm64/v8 in the manifest list entries. See 'docker run --help'.
Do you have plans or the capability to build for ARM64?thanks
The Dockerfile at https://github.com/TabbyML/tabby/blob/main/contrib/jetson/Dockerfile demonstrates how to add support for the Jetson (ARM) platform with CUDA. Adding support for ARM64 should be similar, although it will only run on the CPU.
Sure - note with refactoring checked-in yesterday, a arm version should be pretty easy - you might wanna fork from https://github.com/TabbyML/tabby/blob/main/Dockerfile directly
I get this error as well on Mac M2Pro when running:
docker run -it -p 8080:8080 -v $HOME/.tabby:/data tabbyml/tabby serve --model TabbyML/StarCoder-1B --device metal
@edith007 , let me know if you have something, I can test and share feedback
For Mac silicons, please follow instructions at https://tabby.tabbyml.com/docs/installation/apple
This issue is for linux arm64/v8 support
Hi all - have not seen any activity on this issue for a month+, so I opened a PR that should fix arm64 Docker builds: https://github.com/TabbyML/tabby/pull/1022
I just tried to build the ARM one using the dockerfile provided but it is failing on my side
154.5 Compiling time v0.3.26
156.7 error[E0282]: type annotations needed for `Box<_>`
156.7 --> /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/time-0.3.26/src/format_description/parse/mod.rs:83:9
156.7 |
156.7 83 | let items = format_items
156.7 | ^^^^^
156.7 ...
156.7 86 | Ok(items.into())
156.7 | ---- type must be known at this point
156.7 |
156.7 help: consider giving `items` an explicit type, where the placeholders `_` are specified
156.7 |
156.7 83 | let items: Box<_> = format_items
156.7 | ++++++++
156.7
159.7 For more information about this error, try `rustc --explain E0282`.
159.7 error: could not compile `time` (lib) due to 1 previous error
159.7 warning: build failed, waiting for other jobs to finish...
------
Dockerfile:38
--------------------
37 |
38 | >>> RUN --mount=type=cache,target=/usr/local/cargo/registry \
39 | >>> --mount=type=cache,target=/root/workspace/target \
40 | >>> cargo build --features cuda --release --package tabby && \
41 | >>> cp target/release/tabby /opt/tabby/bin/
42 |
--------------------
ERROR: failed to solve: process "/bin/sh -c cargo build --features cuda --release --package tabby && cp target/release/tabby /opt/tabby/bin/" did not complete successfully: exit code: 101
I just tried to build the ARM one using the dockerfile provided but it is failing on my side
154.5 Compiling time v0.3.26 156.7 error[E0282]: type annotations needed for `Box<_>` 156.7 --> /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/time-0.3.26/src/format_description/parse/mod.rs:83:9 156.7 | 156.7 83 | let items = format_items 156.7 | ^^^^^ 156.7 ... 156.7 86 | Ok(items.into()) 156.7 | ---- type must be known at this point 156.7 | 156.7 help: consider giving `items` an explicit type, where the placeholders `_` are specified 156.7 | 156.7 83 | let items: Box<_> = format_items 156.7 | ++++++++ 156.7 159.7 For more information about this error, try `rustc --explain E0282`. 159.7 error: could not compile `time` (lib) due to 1 previous error 159.7 warning: build failed, waiting for other jobs to finish... ------ Dockerfile:38 -------------------- 37 | 38 | >>> RUN --mount=type=cache,target=/usr/local/cargo/registry \ 39 | >>> --mount=type=cache,target=/root/workspace/target \ 40 | >>> cargo build --features cuda --release --package tabby && \ 41 | >>> cp target/release/tabby /opt/tabby/bin/ 42 | -------------------- ERROR: failed to solve: process "/bin/sh -c cargo build --features cuda --release --package tabby && cp target/release/tabby /opt/tabby/bin/" did not complete successfully: exit code: 101
I also encountered a similar problem, has it been fixed now?
