Linear Warmup is a learning rate schedule where we linearly increase the learning rate from a low rate to a constant rate thereafter. This reduces volatility in the early stages of training.
Image Credit: Chengwei Zhang
Paper | Code | Results | Date | Stars |
---|
Task | Papers | Share |
---|---|---|
Language Modelling | 7 | 11.86% |
Text Generation | 4 | 6.78% |
Self-Supervised Learning | 3 | 5.08% |
Object Detection | 3 | 5.08% |
Backdoor Attack | 2 | 3.39% |
Semantic Segmentation | 2 | 3.39% |
Image Classification | 2 | 3.39% |
Graph Representation Learning | 1 | 1.69% |
Click-Through Rate Prediction | 1 | 1.69% |
Component | Type |
|
---|---|---|
🤖 No Components Found | You can add them if they exist; e.g. Mask R-CNN uses RoIAlign |