Continual Learning with Pre-trained Language Models
Event Description
Abstract: Continual Learning (CL) aims to develop models that can sequentially learn from streams of data and tasks, which is an important need for many real-world applications. The main challenge in CL is to reduce catastrophic forgetting, where models forget knowledge obtained from previous tasks after learning new tasks. Nowadays, pre-trained foundation models have significantly improved models' learning capacity. However, they still suffer from forgetting in sequential fine-tuning. In this thesis, we tackle the forgetting problem of pre-trained models from representation and parameter perspectives. From the representation perspective, we first identify a representation interference between tasks caused by an attention sink phenomenon, which degrades models' CL capacity. To mitigate the effect of attention sinks, we propose a pre-scaling method that encourages models to learn diverse attentions for downstream tasks. Second, to further mitigate interference between representations, we develop a global alignment model that aligns hidden representations across tasks by composing task representations with pre-trained token representations. A probing-first strategy is also used to align class vectors. Experimental results show that alignment models produce well-correlated representations for different tasks and mitigate forgetting in CL. From the parameter perspective, we find that better cross-task generalization of model weights helps mitigate forgetting. Specifically, parameters with consistent updates in pre-training and fine-tuning are more generalizable across tasks and suffer less from forgetting. Inspired by this, we develop a double filtering method to filter out inconsistent updates in tasks with reference to pre-trained weights. This method significantly improves models' generalization ability and reduces forgetting, achieving comparable performance to the model that is fine-tuned with large-scale reference data. In conclusion, works in this thesis help to understand the properties of pre- trained models, and design methods to better utilize them in CL scenarios.
Location: NCS 220
Zoom link: https://stonybrook.zoom.us/j/95777169038?pwd=RzRfGG679KOkHPriIoYZ8DfrO14UpE.1
Meeting ID: 957 7716 9038
Passcode: 048977
Location: NCS 220
Zoom link: https://stonybrook.zoom.us/j/95777169038?pwd=RzRfGG679KOkHPriIoYZ8DfrO14UpE.1
Meeting ID: 957 7716 9038
Passcode: 048977