Availble DeepseekCoder-6.7B model doesn't exist

[akinlong]

Describe the bug According to the https://tabby.tabbyml.com/docs/models/, tabbyML supports TabbyML/DeepseekCoder-6.7B, but it links to deepseek-coder-6.7b-base. In the meantime, I can't find the deepseek-coder-6.7B in https://huggingface.co/TabbyML.

So I downloaded the https://huggingface.co/deepseek-ai/deepseek-coder-6.7b-base and tried to load it, unfortunately, it fails due to gguf file missing.

Where can I find the available model? Or is there any other instruction about how to use the deepseek-coder-6.7b-base? Thanks!

Information about your version 0.7.0

Information about your GPU Nvidia A40

Additional context Add any other context about the problem here.

[wsxiaoys]

Hi, since 0.5.0, our model registry is no longer maintained on huggingface. It's now maintained as json file locates at https://github.com/TabbyML/registry-tabby/blob/main/models.json

Regardless, you should able to download the model with tabby download --model TabbyML/DeepseekCoder-6.7B, any problems you encountered?

[akinlong]

Hi, since 0.5.0, our model registry is no longer maintained on huggingface. It's now maintained as json file locates at https://github.com/TabbyML/registry-tabby/blob/main/models.json

Regardless, you should able to download the model with tabby download --model TabbyML/DeepseekCoder-6.7B, any problems you encountered?

Thanks for your information.

As known, the model's git repository is huge, in the meantime, I am working to deploy the repository to an offline environment, that's why I am trying to download the model and serve it manually. Does that mean I should only download "deepseek-coder-6.7B-base-GGUF" repository and serve it manually?

[wsxiaoys]

You could just run tabby download --model TabbyML/DeepseekCoder-6.7B to download the model

[akinlong]

You could just run tabby download --model TabbyML/DeepseekCoder-6.7B to download the model

I tried, but the download option seems not working without GPU device.

[wsxiaoys]

could you share the cpu information? you can aquire it by doing cat /proc/cpuinfo

[akinlong]

I am using Mac, below is the sysctl output ➞ sysctl machdep.cpu machdep.cpu.address_bits.physical: 39 machdep.cpu.address_bits.virtual: 48 machdep.cpu.tsc_ccc.numerator: 104 machdep.cpu.tsc_ccc.denominator: 2 machdep.cpu.mwait.linesize_min: 64 machdep.cpu.mwait.linesize_max: 64 machdep.cpu.mwait.extensions: 3 machdep.cpu.mwait.sub_Cstates: 286396448 machdep.cpu.thermal.sensor: 1 machdep.cpu.thermal.dynamic_acceleration: 1 machdep.cpu.thermal.invariant_APIC_timer: 1 machdep.cpu.thermal.thresholds: 2 machdep.cpu.thermal.ACNT_MCNT: 1 machdep.cpu.thermal.core_power_limits: 1 machdep.cpu.thermal.fine_grain_clock_mod: 1 machdep.cpu.thermal.package_thermal_intr: 1 machdep.cpu.thermal.hardware_feedback: 0 machdep.cpu.thermal.energy_policy: 1 machdep.cpu.xsave.extended_state: 743 2688 2696 0 machdep.cpu.xsave.extended_state1: 15 2432 8448 0 machdep.cpu.arch_perf.version: 5 machdep.cpu.arch_perf.number: 8 machdep.cpu.arch_perf.width: 48 machdep.cpu.arch_perf.events_number: 8 machdep.cpu.arch_perf.events: 0 machdep.cpu.arch_perf.fixed_number: 4 machdep.cpu.arch_perf.fixed_width: 48 machdep.cpu.cache.linesize: 64 machdep.cpu.cache.L2_associativity: 8 machdep.cpu.cache.size: 256 machdep.cpu.max_basic: 27 machdep.cpu.max_ext: 2147483656 machdep.cpu.vendor: GenuineIntel machdep.cpu.brand_string: Intel(R) Core(TM) i5-1038NG7 CPU @ 2.00GHz machdep.cpu.family: 6 machdep.cpu.model: 126 machdep.cpu.extmodel: 7 machdep.cpu.extfamily: 0 machdep.cpu.stepping: 5 machdep.cpu.feature_bits: 9221959987971750911 machdep.cpu.leaf7_feature_bits: 4072613871 1077960526 machdep.cpu.leaf7_feature_bits_edx: 3154118160 machdep.cpu.extfeature_bits: 1241984796928 machdep.cpu.signature: 460517 machdep.cpu.brand: 0 machdep.cpu.features: FPU VME DE PSE TSC MSR PAE MCE CX8 APIC SEP MTRR PGE MCA CMOV PAT PSE36 CLFSH DS ACPI MMX FXSR SSE SSE2 SS HTT TM PBE SSE3 PCLMULQDQ DTES64 MON DSCPL VMX EST TM2 SSSE3 FMA CX16 TPR PDCM SSE4.1 SSE4.2 x2APIC MOVBE POPCNT AES PCID XSAVE OSXSAVE SEGLIM64 TSCTMR AVX1.0 RDRAND F16C machdep.cpu.leaf7_features: RDWRFSGS TSC_THREAD_OFFSET SGX BMI1 AVX2 FDPEO SMEP BMI2 ERMS INVPCID FPU_CSDS AVX512F AVX512DQ RDSEED ADX SMAP AVX512IFMA CLFSOPT IPT AVX512CD SHA AVX512BW AVX512VL AVX512VBMI UMIP PKU GFNI VAES VPCLMULQDQ AVX512VNNI AVX512BITALG AVX512VPOPCNTDQ RDPID SGXLC FSREPMOV MDCLEAR IBRS STIBP L1DF ACAPMSR SSBD machdep.cpu.extfeatures: SYSCALL XD 1GBPAGE EM64T LAHF LZCNT PREFETCHW RDTSCP TSCI machdep.cpu.logical_per_package: 16 machdep.cpu.cores_per_package: 8 machdep.cpu.microcode_version: 184 machdep.cpu.processor_flag: 7 machdep.cpu.core_count: 4 machdep.cpu.thread_count: 8

[wsxiaoys]

ok, so the real problem here is that we don't provide an distribution for x86 mac at the moment. Do you have an linux machine / vm to help downloading the model?

[akinlong]

I tried Window10 and RHEL8 without GPU, all fails for different reason.

[wsxiaoys]

could you try tabby_x86_64-windows-msvc.exe, since you don't have cuda installed?

[akinlong]

Fails again

[wsxiaoys]

hey @blob42, this might not be relevant here.

Currently, the Tabby model cannot be downloaded because Hugging Face is down. Here's the issue for reference: https://github.com/TabbyML/tabby/issues/1575

[blob42]

Sorry I noticed and removed my comment right away.

On February 29, 2024 12:38:58 AM GMT+01:00, Meng Zhang @.***> wrote:

hey @blob42, this might not be relevant here.

Currently, the Tabby model cannot be downloaded because Hugging Face is down. Here's the issue for reference: https://github.com/TabbyML/tabby/issues/1575

-- Reply to this email directly or view it on GitHub: https://github.com/TabbyML/tabby/issues/1188#issuecomment-1970098619 You are receiving this because you were mentioned.

Message ID: @.***>