feat(Story-2): Add IR Operations for ReLU Family (Group 1)

[ooples]

Story 2: IR Operations - ReLU Family

Added 8 IR operation classes for ReLU-family activations plus 2 additional common activations.

Activations Added:

• ReLU - Rectified Linear Unit (max(0, x))
• GELU - Gaussian Error Linear Unit (used in transformers)
• ELU - Exponential Linear Unit (parameterized with alpha)
• SELU - Scaled ELU with self-normalizing constants
• CELU - Continuously Differentiable ELU
• LeakyReLU - ReLU with small negative slope (default 0.01)
• PReLU - Parametric ReLU with learnable parameters
• RReLU - Randomized ReLU for regularization
• ThresholdedReLU - Sparse activation above threshold
• Sigmoid - Binary classification activation
• Tanh - Hyperbolic tangent activation

Pattern:

• Each class implements IROp interface
• Forward() uses IEngine methods where available (GELU, ELU, ReLU, Sigmoid, Tanh)
• Advanced variants (CELU, LeakyReLU, PReLU, RReLU, ThresholdedReLU) marked for future implementation
• Backward() placeholder for gradient support
• Proper null checks and XML documentation
• Comprehensive parameter validation

Files Modified:

• src/Interfaces/IJitCompilable.cs - New IROp marker interface for JIT-compilable operations
• src/JIT/ActivationOps.cs - New file with all activation IR operations

Build Status: ✅ Build succeeded with 0 warnings, 0 errors

🤖 Generated with Claude Code

[coderabbitai[bot]]

Summary by CodeRabbit

• New Features
  • Added 12 activation functions (ReLU, GELU, ELU, SELU, CELU, Leaky ReLU, PReLU, RReLU, Thresholded ReLU, Sigmoid, Tanh) with JIT compilation support.
  • Forward pass computation enabled with configurable hyperparameters and input validation.
  • Backward pass (gradient computation) implementation pending.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

Walkthrough

Adds a public marker interface IROp and implements eleven generic JIT-able activation operator classes (ReLU, GELU, ELU, SELU, CELU, LeakyReLU, PReLU, RReLU, ThresholdedReLU, Sigmoid, Tanh) whose forward methods delegate to an IEngine; backward methods are currently unimplemented.

Changes

Cohort / File(s)	Summary
Marker Interface src/Interfaces/IJitCompilable.cs	Added public marker interface IROp in AiDotNet.Interfaces with XML docs and a using AiDotNet.LinearAlgebra; directive to mark IR operations eligible for JIT compilation.
Activation Operations src/JIT/ActivationOps.cs	Added eleven generic activation operator classes (ReLUOp<T>, GeluOp<T>, EluOp<T>, SeluOp<T>, CeluOp<T>, LeakyReLUOp<T>, PReLUOp<T>, RReLUOp<T>, ThresholdedReLUOp<T>, SigmoidOp<T>, TanhOp<T>) implementing IROp. Each accepts an IEngine and optional hyperparameters; forward methods call engine routines and validate inputs; backward methods throw NotImplementedException.

Sequence Diagram

sequenceDiagram
    participant Client
    participant ActivationOp as Activation Operator
    participant Engine as IEngine
    participant Tensor

    Client->>ActivationOp: Forward(input Tensor)
    ActivationOp->>ActivationOp: validate input
    ActivationOp->>Engine: call activation (e.g., ReLU/GELU/ELU...)
    Engine->>Tensor: compute output Tensor
    Tensor-->>ActivationOp: return output
    ActivationOp-->>Client: return output Tensor

    Note right of ActivationOp: Backward currently throws NotImplementedException

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

• Pay attention to input validation consistency across ops.
• Verify correct engine method names and expected tensor semantics.
• Review hyperparameter handling (defaults, ranges) and any scalar post-processing (e.g., SELU scaling).
• Confirm visibility (public) and marker interface placement/namespace.

"I hopped through code with eager paws,
New ops and marks without a pause,
Engines hum, forwards run,
Backwards wait for future fun 🐇"

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	You can run @coderabbitai generate docstrings to improve docstring coverage.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Title check	✅ Passed	The PR title clearly and specifically describes the main change: adding IR operations for ReLU family activations as part of Story 2.
Description check	✅ Passed	The PR description is directly related to the changeset, providing detailed information about the 11 activation functions added, the IROp interface pattern, implementation details, and the files modified.

✨ Finishing touches

• [ ] 📝 Generate docstrings

🧪 Generate unit tests (beta)

• [ ] Create PR with unit tests
• [ ] Post copyable unit tests in a comment
• [ ] Commit unit tests in branch feat/activation-ir-ops-group1

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}