Glossary
Multi-task Learning
Multi-task Learning
Multi-task learning (MTL) is a machine learning paradigm where a single model is trained to perform multiple related tasks simultaneously. Instead of building separate models for each task, a shared representation is learned, enabling the model to transfer knowledge across tasks. This often leads to better generalization, reduced overfitting, and improved overall performance.
Background and origins
The concept was popularized by Rich Caruana in the 1990s, who showed that multi-task learning acted as a form of inductive bias and natural regularization. With the advent of deep learning, MTL gained new momentum, especially in NLP and computer vision, where massive neural architectures can effectively learn features useful across several objectives at once.
Practical applications
- NLP: large language models fine-tuned to perform sentiment analysis, summarization, and question answering in a unified framework.
- Computer Vision: networks jointly trained for object detection, segmentation, and pose estimation.
- Healthcare: systems that simultaneously detect multiple diseases or biomarkers from a single medical image.
- Personal assistants: integrating intent detection, dialogue management, and response generation into one model.
Challenges and limitations
- Task interference: some tasks improve performance (positive transfer), while others may harm it (negative transfer).
- Loss balancing: combining multiple objectives requires careful weighting strategies.
- Resource demand: multi-task models are more complex and data-hungry than single-task counterparts.
