Vision Transformers (ViT): Image Classification with Pure Transformers

Vision Transformers replace convolutional layers with pure attention. Divide the image into patches, embed them, and apply Transformer blocks. ViTs achieve state-of-the-art accuracy on ImageNet and scale well to large datasets.

ViT Architecture

import torch
import torch.nn as nn from torchvision.models import vision_transformer
# Load pretrained ViT model = vision_transformer.vit_b_16(pretrained=True)
# ViT divides image into patches (16×16) # Patches are flattened and embedded # Then standard Transformer blocks
x = torch.randn(4, 3, 224, 224)  # Batch of images output = model(x)  # (4, 1000) class logits 

Patch Embedding

class PatchEmbedding(nn.Module):
def __init__(self, patch_size=16, embed_dim=768): super().__init__() self.patch_size = patch_size # Linear projection of patches self.proj = nn.Linear(3 * patch_size * patch_size, embed_dim)
def forward(self, x): # x: (batch, 3, 224, 224) # Convert to patches patches = x.unfold(2, self.patch_size, self.patch_size) \ .unfold(3, self.patch_size, self.patch_size) # Reshape and embed patches = patches.contiguous().view(x.size(0), -1, 3 * self.patch_size ** 2) return self.proj(patches) 

Conclusion

Vision Transformers show that pure attention can replace convolution. Understanding ViT architecture enables building efficient vision models. Next: multimodal models that combine vision and language.