feat(Story-4): Add IR Operations for Softmax Family and Special (Group 3)

[ooples]

Story 4: IR Operations - Softmax Family & Special

Added 16 IR operation classes for vector-based and special activations.

Activations Added:

• Softmax variants: Softmin, LogSoftmax, LogSoftmin, Sparsemax, SphericalSoftmax, GumbelSoftmax, TaylorSoftmax, HierarchicalSoftmax
• Special: Maxout, Sign, Gaussian, ISRU, LiSHT, SQRBF, Squash, BinarySpikingActivation

Features:

• Numerically stable implementations for Log* variants (using log-sum-exp trick)
• Axis parameter support for vector operations (Softmax family, Squash)
• Parameterized operations (GumbelSoftmax temperature, ISRU alpha, SQRBF beta, etc.)
• Clear TODOs for complex algorithms (Sparsemax, HierarchicalSoftmax, TaylorSoftmax)
• Comprehensive documentation with beginner-friendly examples

Technical Details:

• All operations inherit from IROp base class
• Proper validation in each Validate() method
• Custom ToString() implementations for ops with parameters
• Follows existing IR operation patterns

Build Status:

• Build succeeded with no errors or warnings
• All 21 activation operations now available (5 existing + 16 new)

Generated with Claude Code

[coderabbitai[bot]]

[!WARNING]

Rate limit exceeded

@ooples has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 5 minutes and 3 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

📥 Commits

Reviewing files that changed from the base of the PR and between fdc5dd725f20c9d38cb1d313755e61b2e4fc5174 and 57ce5475aaf03dfab35f0e4891f3d2e79234e596.

📒 Files selected for processing (5)

• src/JitCompiler/IR/IRGraph.cs (1 hunks)
• src/JitCompiler/IR/IROp.cs (1 hunks)
• src/JitCompiler/IR/IRType.cs (1 hunks)
• src/JitCompiler/IR/Operations/ActivationOps.cs (1 hunks)
• src/JitCompiler/IR/TensorShape.cs (1 hunks)

✨ Finishing touches

• [ ] 📝 Generate docstrings

🧪 Generate unit tests (beta)

• [ ] Create PR with unit tests
• [ ] Post copyable unit tests in a comment
• [ ] Commit unit tests in branch feat/activation-ir-ops-group3

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}