Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class-Incremental Learning
Few-shot class-incremental learning (FSCIL) has been a challenging problem as only a few training samples are accessible for each novel class in the new sessions. Finetuning the backbone or adjusting the classifier prototypes trained in the prior sessions would inevitably cause a misalignment between the feature and classifier of old classes, which explains the well-known catastrophic forgetting problem. In this paper, we deal with this misalignment dilemma in FSCIL inspired by the recently discovered phenomenon named neural collapse, which reveals that the last-layer features of the same class will collapse into a vertex, and the vertices of all classes are aligned with the classifier prototypes, which are formed as a simplex equiangular tight frame (ETF). It corresponds to an optimal geometric structure for classification due to the maximized Fisher Discriminant Ratio. We propose a neural collapse inspired framework for FSCIL. A group of classifier prototypes are pre-assigned as a simplex ETF for the whole label space, including the base session and all the incremental sessions. During training, the classifier prototypes are not learnable, and we adopt a novel loss function that drives the features into their corresponding prototypes. Theoretical analysis shows that our method holds the neural collapse optimality and does not break the feature-classifier alignment in an incremental fashion. Experiments on the miniImageNet, CUB-200, and CIFAR-100 datasets demonstrate that our proposed framework outperforms the state-of-the-art performances. Our code will be publicly available.
PDF AbstractCode
Datasets
Results from the Paper
Task | Dataset | Model | Metric Name | Metric Value | Global Rank | Benchmark |
---|---|---|---|---|---|---|
Few-Shot Class-Incremental Learning | CIFAR-100 | NC-FSCIL | Average Accuracy | 67.50 | # 2 | |
Last Accuracy | 56.11 | # 3 | ||||
Few-Shot Class-Incremental Learning | CUB-200-2011 | NC-FSCIL | Average Accuracy | 67.28 | # 2 | |
Last Accuracy | 59.44 | # 3 | ||||
Few-Shot Class-Incremental Learning | mini-Imagenet | NC-FSCIL | Average Accuracy | 67.82 | # 2 | |
Last Accuracy | 58.31 | # 3 |