About OPMX

Open PPL Model Exchange (OPMX) is an open ecosystem that empowers AI developers to choose the right tools as their project evolves. OPMX provides an open source format for AI models. It defines an extensible computation graph model, as well as definitions of built-in operators and standard data types.

Currently OPMX focus on the capabilities and hardware friendliness needed for Large Language Model(LLM) inferencing.

Important Notice

• PMX has changed to OPMX at 25/04/2024.
• And the domain of operators are also changed to opmx(refers to TOC).
• You can find the old code at llm_v1

Operator spec

Table of Contents: Link

About add new operator: Link

About update an operator's version: Link

Use OPMX Python API

OPMX provides functional API based on torch.autograd.Function.

Clone the OPMX repo, and import torch_function like this:

import pmx_llm.torch_function as OPMX

And then use it as Pytorch's functional API:

norm, skip_out = OPMX.skip_rms_norm(x, weight, skip_in, -1, eps)

We can use these API in pytorch to custom your own model.

All OPMX function could be exported as custom operators by torch.onnx.export.

Model Zoo

Some opensource model are provided in our model zoo.

Currently models:

• LLaMA 1/2/3
  • Exporting Facebook(Meta)'s LLaMA
  • Exporting Huggingfaces's LLaMA
• Baichuan 1/2 7B
  • Exporting Huggingfaces's Baichuan 7B
• Baichuan 1/2 13B
  • Exporting Huggingfaces's Baichuan 13B
• InternLM 1
  • Exporting Huggingfaces's InternLM
• InternLM 2
  • Exporting Huggingfaces's InternLM
• ChatGLM 2/3
  • Exporting Huggingfaces's ChatGLM
• Mixtral
  • Exporting Huggingfaces's Mixtral
• Qwen 1/1.5
  • Exporting Huggingfaces's Qwen
• Falcon
  • Exporting Huggingfaces's Falcon
• Bigcode
  • Exporting Huggingfaces's Bigcode