Efficient CNNs with Depthwise Separable and Dilated Convolutions

What I Explored

I explored CNN architecture design using CIFAR-10, focusing specifically on depthwise separable convolutions and dilated convolutions.

Rather than treating a CNN as a fixed sequence of convolution and pooling layers, this exercise encouraged me to think about receptive fields, parameter efficiency, and how different convolution operations transform information.

What Confused Me

The relationship between receptive field, kernel size, dilation, and network depth was initially difficult to reason about.

Depthwise separable convolution also raised an important question: how can spatial filtering and channel mixing be separated while still producing useful representations?

What Clicked

I understood that a standard convolution performs spatial feature extraction and channel combination simultaneously.

Depthwise separable convolution decomposes these operations into depthwise spatial filtering followed by pointwise channel mixing, reducing computational cost.

Dilated convolution provides another useful idea: expanding the receptive field without proportionally increasing parameters.

My Mental Model Now

I think about convolution architecture along three dimensions:

Spatial feature extraction + Channel interaction + Receptive field

Different convolution variants make different trade-offs between these responsibilities.

Implementation / Example

I designed and trained a CIFAR-10 network incorporating depthwise separable convolution and dilated convolution while experimenting with augmentation and training strategies.

Repository: Depthwise Separable and Dilated Convolutions with CIFAR10

What I Want to Explore Next

I want to compare these ideas with MobileNet-style architectures, grouped convolutions, inverted residual blocks, and other efficient CNN designs.