MIT researchers develop a method for LLMs to learn new skills without forgetting old ones

MIT researchers developed a new method allowing LLMs to learn new skills without forgetting old ones. The technique, Self-Distillation Fine-Tuning (SDFT), uses a model's own in-context learning to create a feedback loop, enabling continuous skill acquisition without performance regression. This innovation could consolidate enterprise "model zoos" into single, adaptable AI agents, reducing costs and improving dynamic learning capabilities.