Please add the memory requirement estimate if run with cpu and vram request for run with GPU for each model in model list.

[JerryYao75]

What are you trying to do?

I want to know if my computer can support the model or not, but currently no one can tell me.

How should we solve this?

Add the memory needed for each model tag if run on cpu. Add the vram needed for each model tag if run on GPU.

What is the impact of not solving this?

All users had to download each model and test, this will be a big waste of resources.

Anything else?

No response

[mkos11]

To provide memory (RAM) and video memory (VRAM) requests for each model in a model list, I'll assume you are referring to machine learning or deep learning models commonly used in AI applications. Keep in mind that these are approximate values and actual resource requirements may vary based on specific implementations, dataset sizes, and hardware configurations. Also, GPU memory requirements can vary based on the batch size used during training.

Here is an example list of models along with their approximate memory and VRAM requests:

1. ResNet50:

   • CPU Memory Request: Around 4-8 GB RAM
   • GPU VRAM Request: Around 2-4 GB VRAM

2. MobileNetV2:

   • CPU Memory Request: Around 2-4 GB RAM
   • GPU VRAM Request: Around 1-2 GB VRAM

3. BERT (Base):

   • CPU Memory Request: Around 8-16 GB RAM
   • GPU VRAM Request: Around 6-12 GB VRAM

4. YOLOv3:

   • CPU Memory Request: Around 4-8 GB RAM
   • GPU VRAM Request: Around 4-8 GB VRAM

5. GPT-3 (Small):

   • CPU Memory Request: Around 16-32 GB RAM
   • GPU VRAM Request: Around 8-16 GB VRAM

6. InceptionV3:

   • CPU Memory Request: Around 4-8 GB RAM
   • GPU VRAM Request: Around 2-4 GB VRAM

7. VGG16:

   • CPU Memory Request: Around 8-16 GB RAM
   • GPU VRAM Request: Around 4-8 GB VRAM

8. LSTM (Long Short-Term Memory):

   • CPU Memory Request: Around 8-16 GB RAM
   • GPU VRAM Request: Around 4-8 GB VRAM

These are rough estimates and can vary based on factors such as batch size, input data size, model complexity, and specific hardware configurations. It's always a good practice to monitor resource usage during model training to optimize resource allocation and performance.

[santiago-afonso]

Some integration with an LLM VRAM calculator like this https://huggingface.co/spaces/NyxKrage/LLM-Model-VRAM-Calculator would be very helpful. A simple indication of whether one can run a model either on GPU or CPU in personal devices.

(Btw the ollama readme does have some model size estimates https://github.com/ollama/ollama, but the VRAM calc seems more complete.)