konev-artem

Results 1 comments of konev-artem

Same amount of VRAM is taken as in AdamW

In my setting, Lion takes less memory than AdamW (9.9 Gb vs 10.1Gb) but Lion is slower in terms of steps/sec. Has anyone noticed the same? I compare Lion with...