Convolution
In machine learning, convolution is a fundamental mathematical operation used in Convolutional Neural Networks (CNNs). It applies a kernel (filter) across the input data (such as an image) to extract meaningful patterns like edges, shapes, or textures.
Key idea
- Each kernel focuses on a different feature (e.g., vertical edges, color gradients).
- Stacking multiple convolutional layers enables CNNs to build hierarchical feature maps.
Example
In computer vision, an early convolutional layer may detect horizontal and vertical lines, while deeper layers recognize complex objects like faces or cars.
Another important aspect of convolution is parameter sharing. Unlike fully connected layers, where each weight is unique, convolutional layers reuse the same filter across the entire input. This dramatically reduces the number of parameters, making CNNs more efficient and less prone to overfitting when compared with dense architectures.
Pooling operations often accompany convolutions to further condense information. Max pooling, for example, selects the most prominent value in a region, creating a more abstract representation that is invariant to small translations. This property is crucial for tasks like image classification, where the exact position of an object is less important than its presence.
Convolutions are not limited to images. In natural language processing, 1-D convolutions can capture local word patterns such as n-grams, improving text classification or sentiment analysis. Similarly, in audio processing, convolutional layers can identify frequency components and temporal structures, supporting tasks like speech recognition or music genre classification.
Finally, modern research extends convolutions beyond their traditional scope. Dilated convolutions expand the receptive field without increasing computational cost, while depthwise separable convolutions (used in MobileNet) enable CNNs to run efficiently on mobile devices. These innovations illustrate how the core principle of convolution continues to evolve, adapting to the needs of real-world AI applications.
Reference
- Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning. MIT Press.
