mlx-vlm Regression with DeepSeek-OCR

I successfully ran deepseek ocr when it was originally ported to MLX. Now, when I try to run it, I get an error

Commands

uv run mlx_vlm.server

curl --request POST \
  --url http://localhost:8080/chat/completions \
  --header 'content-type: application/json' \
  --data '{
  "model": "mlx-community/DeepSeek-OCR-8bit",
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "What are the days of the week?"
        }
      ]
    }
  ],
  "stream": false,
  "max_tokens": 10000
}'

Server log:

Error loading model mlx-community/DeepSeek-OCR-8bit: 'dict' object has no attribute 'model_type'

Am I doing something wrong? Or did this break at some point recently? Thanks for the help!

Nov 24 '25 23:11 silenceisgolden

Update your transformers

Nov 25 '25 19:11 jrp2014

Maybe related with removal of load from AutoConfig.

I will check and comeback to you.

Meanwhile test out updating your Transformers

Nov 25 '25 20:11 Blaizzy

Pardon the delayed response:

Updating transformers got the model to load and serve, but now I can only get looping answers that fill the max tokens limit:

curl --request POST \
  --url http://localhost:8080/chat/completions \
  --header 'content-type: application/json' \
  --data '{
  "model": "mlx-community/DeepSeek-OCR-8bit",
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "What are the days of the week?"
        }
      ]
    }
  ],
  "stream": false,
  "max_tokens": 2048,
  "temperature": 0.0,
  "top_p": 0.95
}'

{"model":"mlx-community/DeepSeek-OCR-8bit","choices":[{"finish_reason":"stop","message":{"role":"assistant","content":"1. What are the days of the week? 2. What are the days of the week? 3. What are the days of the week? 4. What are the days of the week? 5. What are the days of the week? 6. What are the days of the week? 7. What are the days of the week? 8. What are the days of the week? 9. What are the days of the week? 10. What are the days of the week? 11. What are the days of the week? 12. What are the days of the week? 13. What are the days of the week? 14. What are the days of the week? 15. What are the days of the week? 16. What are the days of the week? 17. What are the days of the week? 18. What are the days of the week? 19. What are the days of the week? 20. What are the days of the week? 21. What are the days of the week? 22. What are the days of the week? 23. What are the days of the week? 24. What are the days of the week? 25. What are the days of the week? 26. What are the days of the week? 27. What are the days of the week? 28. What are the days of the week? 29. What are the days of the week? 30. What are the days of the week? 31. What are the days of the week? 32. What are the days of the week? 33. What are the days of the week? 34. What are the days of the week? 35. What are the days of the week? 36. What are the days of the week? 37. What are the days of the week? 38. What are the days of the week? 39. What are the days of the week? 40. What are the days of the week? 41. What are the days of the week? 42. What are the days of the week? 43. What are the days of the week? 44. What are the days of the week? 45. What are the days of the week? 46. What are the days of the week? 47. What are the days of the week? 48. What are the days of the week? 49. What are the days of the week? 50. What are the days of the week? 51. What are the days of the week? 52. What are the days of the week? 53. What are the days of the week? 54. What are the days of the week? 55. What are the days of the week? 56. What are the days of the week? 57. What are the days of the week? 58. What are the days of the week? 59. What are the days of the week? 60. What are the days of the week? 61. What are the days of the week? 62. What are the days of the week? 63. What are the days of the week? 64. What are the days of the week? 65. What are the days of the week? 66. What are the days of the week? 67. What are the days of the week? 68. What are the days of the week? 69. What are the days of the week? 70. What are the days of the week? 71. What are the days of the week? 72. What are the days of the week? 73. What are the days of the week? 74. What are the days of the week? 75. What are the days of the week? 76. What are the days of the week? 77. What are the days of the week? 78. What are the days of the week? 79. What are the days of the week? 80. What are the days of the week? 81. What are the days of the week? 82. What are the days of the week? 83. What are the days of the week? 84. What are the days of the week? 85. What are the days of the week? 86. What are the days of the week? 87. What are the days of the week? 88. What are the days of the week? 89. What are the days of the week? 90. What are the days of the week? 91. What are the days of the week? 92. What are the days of the week? 93. What are the days of the week? 94. What are the days of the week? 95. What are the days of the week? 96. What are the days of the week? 97. What are the days of the week? 98. What are the days of the week? 99. What are the days of the week? 100. What are the days of the week? 101. What are the days of the week? 102. What are the days of the week? 103. What are the days of the week? 104. What are the days of the week? 105. What are the days of the week? 106. What are the days of the week? 107. What are the days of the week? 108. What are the days of the week? 109. What are the days of the week? 110. What are the days of the week? 111. What are the days of the week? 112. What are the days of the week? 113. What are the days of the week? 114. What are the days of the week? 115. What are the days of the week? 116. What are the days of the week? 117. What are the days of the week? 118. What are the days of the week? 119. What are the days of the week? 120. What are the days of the week? 121. What are the days of the week? 122. What are the days of the week? 123. What are the days of the week? 124. What are the days of the week? 125. What are the days of the week? 126. What are the days of the week? 127. What are the days of the week? 128. What are the days of the week? 129. What are the days of the week? 130. What are the days of the week? 131. What are the days of the week? 132. What are the days of the week? 133. What are the days of the week? 134. What are the days of the week? 135. What are the days of the week? 136. What are the days of the week? 137. What are the days of the week? 138. What are the days of the week? 139. What are the days of the week? 140. What are the days of the week? 141. What are the days of the week? 142. What are the days of the week? 143. What are the days of the week? 144. What are the days of the week? 145. What are the days of the week? 146. What are the days of the week? 147. What are the days of the week? 148. What are the days of the week? 149. What are the days of the week? 150. What are the days of the week? 151. What are the days of the week? 152. What are the days of the week? 153. What are the days of the week? 154. What are the days of the week? 155. What are the days of the week? 156. What are the days of the week? 157. What are the days of the week? 158. What are the days of the week? 159. What are the days of the week? 160. What are the days of the week? 161. What are the days of the week? 162. What are the days of the week? 163. What are the days of the week? 164. What are the days of the week? 165. What are the days of the week? 166. What are the days of the week? 167. What are the days of the week? 168. What are the days of the week? 169. What are the days of the week? 170. What are the days of the week? 171. What are the days of the week? 172. What are the days of the week? 173. What are the days of the week? 174. What are the days of the week? 175. What are the days of the week? 176. What are the days of the week? 177. What are the days of the week? 178. What are the days of the week? 179. What are the days of the week? 180. What are the days of the week? 181. What are the days of the week? 182. What are the days of the week? 183. What are the days of the week? 184. What are the days of the week? 185. What are the days of the week? 186. What are the days of the week? 187."}}],"usage":{"input_tokens":23,"output_tokens":2048,"total_tokens":2071,"prompt_tps":1175.1231326079424,"generation_tps":331.3006746643613,"peak_memory":4.035066494}}

Any advice would be appreciated!

Dec 03 '25 10:12 silenceisgolden

The issue is that DeepSeekOCR is a trained and behaves as a traditional OCR model.

It requires specific prompts and doesn't respond well to prompts outside the ones it was trained on.

https://huggingface.co/deepseek-ai/DeepSeek-OCR

For instance, it won't work on LM only mode.

You need to pass an image as input.

Dec 03 '25 22:12 Blaizzy

Or, if this input worked before please let me know at which version and I will reproduce the issue

Dec 03 '25 22:12 Blaizzy

This input works when running this model with other model runners. For example, with Ollama:

curl -X POST http://localhost:8092/api/chat \
-H "Content-Type: application/json" \
-d '{
    "model": "deepseek-ocr:3b",
    "messages": [{
    "role": "user",
    "content": "What are the days of the week?"
    }],
    "stream": false
}'

{"model":"deepseek-ocr:3b","created_at":"2025-12-04T17:30:09.30097Z","message":{"role":"assistant","content":"The days of the week are:\n\n- Monday\n- Tuesday\n- Wednesday\n- Thursday\n- Friday\n- Saturday\n- Sunday"},"done":true,"done_reason":"stop","total_duration":3343549917,"load_duration":2940373042,"prompt_eval_count":9,"prompt_eval_duration":215365416,"eval_count":28,"eval_duration":177957543}

Dec 04 '25 17:12 silenceisgolden