Cydral

This PR introduces a new RMS (Root Mean Square) Normalization layer to Dlib. RMS Normalization is a variant of Layer Normalization that has shown promising results in various deep learning...

Implementation of a new softmax version with PLANE_WISE mode support

3

This PR introduces a new implementation of the softmax layer, offering increased flexibility and better support for Large Language Models (LLMs) and other applications requiring 2D tensor processing. ### Main...

Add multm_prev_ layer and enhance gemm() function for PLANE_WISE operations

11

This pull request introduces a new layer, `multm_prev_`, and enhances the `gemm()` function to support PLANE_WISE operations. These changes aim to improve the flexibility and performance of matrix multiplications in...

Add positional_encodings_ layer to Dlib

8

This pull request introduces a new layer, positional_encodings_, to the Dlib library. The positional_encodings_ layer adds "positional encodings" to the input tensor, which is particularly useful for models processing sequential...

New model definition

Add transformer example with RoPE and MoE-like mechanisms

7

This PR introduces a new example demonstrating: 1. Rotary Positional Embeddings (RoPE) implementation 2. Experimental Mixture-of-Experts (MoE) layer showing how to: - Extend Dlib's capabilities without modifying core library -...

# Official transformer architecture support for Dlib This pull request represents the consolidation and stabilization of transformer-related layers and components developed throughout 2024~2025. This substantial commit introduces official support for...