Please consider adding support for RISC-V's rvv build, compilation optimization, and related features.

[HougeLangley]

As we know. 😊

RVV (RISC-V Vector) is a vector processing extension for the RISC-V instruction set architecture (ISA). It's designed to provide high-performance computing capabilities for applications that require massive parallelism and data-level parallelism.

RVV is based on the concept of Single-Instruction, Multiple-Data (SIMD) processing, where a single instruction can operate on multiple data elements simultaneously. This allows RVV to perform complex computations on large datasets efficiently.

Some key features of RVV include:

1. Vector registers: RVV introduces 32-vector register files, each with a width of 128 bits or 256 bits. These registers can hold multiple values and be used for parallel processing.
2. Vector instructions: RVV provides a set of vector instructions that operate on the vector registers. These instructions include basic arithmetic operations (e.g., add, mul), comparisons, and bit-manipulation operations.
3. Parallel execution: RVV allows multiple vector instructions to execute in parallel, enabling high throughput for compute-intensive workloads.

RVV is designed to be compatible with other RISC-V extensions, such as the integer and floating-point units (INT and FPU). It's intended for use in a wide range of applications, including:

1. High-performance computing: RVV can accelerate scientific simulations, machine learning, and data analytics workloads.
2. Graphics processing: RVV can be used for graphics rendering, game development, and other graphical applications that require intense parallel processing.
3. Data processing: RVV can improve the performance of data-intensive applications, such as data compression, encryption, and filtering.

By providing a vector processing extension, RVV aims to enable developers to take advantage of the massive parallelism offered by modern computing architectures, while maintaining the flexibility and portability of the RISC-V ISA. 💻

So, how about considering adding support for RVV instruction set optimizations to future versions of Ollama, particularly in terms of their build and support capabilities?

[HougeLangley]

https://github.com/ollama/ollama/assets/1161594/6a08da6c-b5fd-44f6-aba8-421a07a4e7a1

[camel-cdr]

Vector registers: RVV introduces 32-vector register files, each with a width of 128 bits or 256 bits

That's not true, RVV vector registers can have any power of two from 128 up to 65,536 bits. (Assuming the standard V extension, there are variants that allow smaller vector length, but VLEN>=128 is what the binary app market should target)

RVV is designed to be compatible with other RISC-V extensions, such as the integer and floating-point units (INT and FPU).

Duh

Btw, RVV support won't help you with the Lichee Pi 4A, since it doesn't support RVV. gcc-14 is working on allowing you to compile RVV intrinsics to the incompatible non-standard XTheadVector extension present on that chip, but that isn't fully functional yet.

[HougeLangley]

Vector registers: RVV introduces 32-vector register files, each with a width of 128 bits or 256 bits

That's not true, RVV vector registers can have any power of two from 128 up to 65,536 bits. (Assuming the standard V extension, there are variants that allow smaller vector length, but VLEN>=128 is what the binary app market should target)

RVV is designed to be compatible with other RISC-V extensions, such as the integer and floating-point units (INT and FPU).

Duh

Btw, RVV support won't help you with the Lichee Pi 4A, since it doesn't support RVV. gcc-14 is working on allowing you to compile RVV intrinsics to the incompatible non-standard XTheadVector extension present on that chip, but that isn't fully functional yet.

Thanks