Implement Damped Lagrangian Formulation

[lucas-maes]

Changes

Hi 👋, I've implemented the Damped Lagrangian Formulation to enhance the stability of the optimization process by addressing oscillatory behaviors when constraints are suddenly satisfied or violated. This involves:

• Adding the DampedLagrangianFormulation class which extends the traditional LagrangianFormulation with a damping mechanism.
• Updating the documentation in lagrangian_formulation.rst to describe the new formulation and its implementation details.

Testing

Added tests in test_lagrangian_formulation.py to ensure:

• The damping effect is properly computed as detailed in this blogpost.

References

• John C. Platt and Alan Barr, "Constrained differential optimization," presented at Neural Information Processing Systems, 1987.
• Engraved Blog, "How We Can Make Machine Learning Algorithms Tunable," 2024. Available online: Engraved Blog