PPL LLM Kernel CUDA

Overview

ppl.llm.kernel.cuda is a part of PPL.LLM system.

We recommend users who are new to this project to read the Overview of system.

Primitive cuda kernel library for ppl.nn.llm

Currently, only Ampere and Hopper have been tested.

Prerequisites

• Linux running on x86_64 or arm64 CPUs
• GCC >= 9.4.0
• CMake >= 3.18
• Git >= 2.7.0
• CUDA Toolkit >= 11.4. 11.6 recommended. (for CUDA)

Quick Start

• Installing Prerequisites(on Debian or Ubuntu for example)

  apt-get install build-essential cmake git
  

• Cloning Source Code

  git clone https://github.com/openppl-public/ppl.llm.kernel.cuda.git
  

• Building from Source

  ./build.sh -DPPLNN_CUDA_ENABLE_NCCL=ON -DPPLNN_ENABLE_CUDA_JIT=OFF -DPPLNN_CUDA_ARCHITECTURES="'80;86;87'" -DPPLCOMMON_CUDA_ARCHITECTURES="'80;86;87'"
  

License

This project is distributed under the Apache License, Version 2.0.