← A Fix for Softmax and Cross-Entropy
Preset:
Lots of Weight Decay & Noise-Free Labels
No Weight Decay & Noisy Labels
Precision:
bf16
fp32
fp64
CE Mult:
Weight decay:
Adam ε:
Label Noise:
β₁ = 0.5
β₂ = 0.9