🧠 Meta-Learning From Scratch

Teaching models to learn how to learn — one algorithm at a time

Quick Start • Algorithms • Docs • Examples

---

👋 Hey there!

I'm Komil Parmar, a second-year undergrad at IIT Guwahati, and this is my journey into the fascinating world of meta-learning. Instead of just reading about algorithms, I decided to build them from scratch to really understand what's going on under the hood.

What's meta-learning? Think of it as teaching an AI to be a quick learner. Instead of training a model for one specific task, we train it to rapidly adapt to new tasks with just a handful of examples. It's like the difference between memorizing facts and learning how to learn.

This repo is my playground for experimenting with different meta-learning algorithms — all built from the ground up, heavily documented, and designed to be easily hackable.

📖 My Learning Philosophy

I'm following the book "Meta-Learning: Theory, Algorithms and Applications" along with papers from the community. But here's the thing — I don't just want to run existing code. I want to:

• Understand deeply: Build everything from scratch
• Experiment freely: Modular code that's easy to modify
• Share openly: Comprehensive docs so others can learn too
• Stay curious: Keep adding new algorithms as I learn them

If you're learning meta-learning too, I hope this helps you as much as building it helped me! 🚀

---

🎯 What's Implemented?

Algorithm	Accuracy	Speed	Best For	Docs
MAML	75-80%	1x	Strong baseline, solid theory	📖
FOMAML	75-80%	1.4x faster	Speed + performance balance	📖
Meta-SGD	79.5%	1x	Per-parameter learning rates	📖
ANIL	77-90.5%	3x faster	Pretrained models, speed	📖
LEO	95-98% 🏆	0.7x	Highest accuracy, 1-shot tasks	📖
Meta Networks	77-86%	Very fast	Fast inference, no gradients	📖
TAML	~73%	Medium	Task-agnostic learning	📖
Reptile	~75%	Fast	Simpler than MAML	📖

_{All numbers are for 5-way 1-shot classification on Omniglot}

🎨 Special Features

• Meta Dropout: A dropout technique that maintains consistent masks during task adaptation. Improves accuracy by ~1-2% across algorithms!
• Prefetched Dataset: Load Omniglot into RAM once → 10-50x faster training. Because life's too short to wait for data loading.
• Shared Components: All algorithms use the same EmbeddingNetwork for fair comparisons.

---

🗂️ How This Repo is Organized

meta-learning-from-scratch/
│
├── 🧪 algorithms/          # The core implementations
│   ├── maml.py              # MAML/FOMAML/Meta-SGD
│   ├── anil.py              # Almost No Inner Loop
│   ├── leo.py               # Latent Embedding Optimization
│   ├── *_meta_network.py    # Meta Networks variants
│   └── meta_dropout.py      # Our secret sauce
│
├── 📊 evaluation/          # Measure performance & visualize
│   ├── evaluate_*.py        # Algorithm-specific evaluation
│   └── eval_visualization.py # Pretty plots
│
├── 🔬 tests/               # Make sure nothing breaks
│   └── test_*.py            # Pytest test suites
│
├── 🛠️ utils/               # Dataset loading & helpers
│   ├── load_omniglot.py     # Standard + prefetched loaders
│   └── visualize_omniglot.py # See your data
│
├── 📚 docs/                # Deep dives into each algorithm
│   └── *.md                 # Comprehensive guides
│
└── 🎓 examples/            # Learn by doing
    └── *.ipynb              # Interactive tutorials

Philosophy: Each folder has one job. Want to add a new algorithm? Drop it in algorithms/. Need to change how evaluation works? Check evaluation/. Everything is self-contained and modular.

---

🚀 Quick Start

Installation

# Clone this bad boy
git clone https://github.com/Komil-parmar/meta-learning-from-scratch.git
cd meta-learning-from-scratch

# Install dependencies
pip install -r requirements.txt

Get the Dataset

# Download Omniglot (the "MNIST of few-shot learning")
wget https://github.com/brendenlake/omniglot/raw/master/python/images_background.zip
wget https://github.com/brendenlake/omniglot/raw/master/python/images_evaluation.zip

# Extract
unzip images_background.zip -d omniglot/
unzip images_evaluation.zip -d omniglot/

Train Your First Meta-Learner

from algorithms.maml import train_maml
from algorithms.cnn_maml import SimpleConvNet
from utils.load_omniglot import PrefetchedOmniglotDataset, OmniglotTaskDataset
from torch.utils.data import DataLoader

# Load dataset (prefetched = fast! ⚡)
dataset = PrefetchedOmniglotDataset("omniglot/images_background")
task_dataset = OmniglotTaskDataset(dataset, n_way=5, k_shot=1, k_query=15, num_tasks=2000)
dataloader = DataLoader(task_dataset, batch_size=4, shuffle=True)

# Create model
model = SimpleConvNet(num_classes=5)

# Train with MAML
model, maml, losses = train_maml(
    model, dataloader,
    inner_lr=0.01,      # How fast to adapt to each task
    outer_lr=0.001,     # How fast to meta-learn
    inner_steps=5,      # Adaptation steps per task
    first_order=False   # True for FOMAML (faster)
)

# That's it! You just trained a meta-learner 🎉

🎛️ Want to try different algorithms?

FOMAML (faster):

train_maml(model, dataloader, first_order=True)

Meta-SGD (learns learning rates):

train_maml(model, dataloader, meta_sgd=True)

ANIL (3x faster):

from algorithms.anil import train_anil
body, head = create_anil_network(num_classes=5)
train_anil(body, head, dataloader, first_order=True, freeze_body=True)

Check out the examples/ folder for complete tutorials on each algorithm!

---

🎓 Learning Resources

Each algorithm comes with:

• 📖 Comprehensive docs in the docs/ folder explaining theory + implementation
• 🎓 Jupyter notebooks with step-by-step walkthroughs
• 🧪 Tests that double as usage examples

Start here:

1. examples/maml_on_omniglot.ipynb — Great introduction to meta-learning
2. docs/MAML_vs_FOMAML_vs_MAMLpp.md — Compare different MAML variants
3. docs/ANIL.md — My favorite: fastest algorithm with great performance

---

🔧 Want to Use Your Own Dataset?

The beauty of this repo is everything is modular. Just follow this pattern:

# 1. Create your dataset (like OmniglotDataset)
class MyDataset(Dataset):
    def __getitem__(self, idx):
        return images, class_idx

# 2. Create task sampler (N-way K-shot)
task_dataset = MyTaskDataset(
    my_dataset,
    n_way=5,      # 5 classes per task
    k_shot=1,     # 1 example per class
    k_query=15    # 15 test examples per class
)

# 3. Use any algorithm — they all work the same way!
train_maml(model, DataLoader(task_dataset), ...)

That's literally it. The algorithms don't care what data you throw at them.

---

💡 Why This Repo?

For learners: Every line is documented. Every choice is explained. Read the code, run the notebooks, break things, and learn.

For researchers: Modular design means you can quickly test ideas. Want to try a new inner loop optimizer? Just modify inner_update(). Want to test on a new dataset? Plug it in.

For practitioners: Production-ready implementations with proper testing, type hints, and documentation. Not just research code that barely runs.

---

🤝 Contributing

Found a bug? Want to add a new algorithm? Have a cool idea?

I'd love your contributions! Check out CONTRIBUTING.md for guidelines. Some ideas:

• Add Prototypical Networks
• Add Matching Networks
• Improve documentation
• Add more datasets
• Optimize existing code

Just remember: clarity over cleverness. This repo is for learning, so keep it readable!

---

📚 References & Inspiration

• Book: "Meta-Learning: Theory, Algorithms and Applications"
• Course: Stanford CS330: Deep Multi-Task and Meta Learning
• Papers: MAML, ANIL, LEO, Meta Networks, and more
• Dataset: Omniglot by Brenden Lake

---

📬 Let's Connect!

I'm always happy to chat about meta-learning, answer questions, or hear about what you're building!

• LinkedIn: Komil Parmar
• GitHub Issues: Questions, bugs, or ideas? Open an issue

---

📄 License

MIT License — feel free to use this for learning, research, or building cool stuff!

---

Made with ❤️ and lots of ☕ by a curious undergrad

If this helped you learn something new, consider giving it a ⭐!

Happy meta-learning! 🚀