# Chapter 1: Introduction

**Source chapter (free, official):** https://www.deeplearningbook.org/contents/intro.html

## Core Idea

Deep learning is representation learning stacked: instead of hand-designing the features a
model consumes, you let the model learn a hierarchy of representations, each layer expressed
in terms of the one below it. Depth is the mechanism that makes composition cheap.

## Key Concepts

- **Representation learning**: learning the features, not just the mapping from features to output.
- **Depth as composition**: a deep model reuses sub-computations, so a function needing
  exponentially many units at depth 1 may need polynomially many at depth k.
- **The curse of dimensionality**: as input dimension grows, the number of distinguishable
  configurations grows exponentially, so local-interpolation methods run out of neighbours.
- **Prior of compositionality**: deep learning bets that the data was generated by a composition
  of simple factors. This is a prior, not a theorem — where the bet is wrong, depth does not help.
- **AI winters and rebrands**: cybernetics → connectionism → deep learning; the ideas outlived
  three funding cycles because the constraint was data and compute, not concept.

## Mental Models

- Think of each layer as **re-coordinatizing** the input so the next layer's job gets easier —
  not as "extracting meaning."
- Prefer depth over width when the target function plausibly **factors**; prefer width when it
  does not. Depth is a structural prior, and priors have to be earned.
- Treat "the model learns features automatically" as **shifting where the human judgment goes**
  (architecture, objective, data curation), not eliminating it.

## Anti-patterns

- **Reading this chapter as history and skipping it**: the three growth curves it draws
  (dataset size, model size, benchmark accuracy) are the argument for why anything else works.
- **Assuming depth is always better**: without the compositional structure, extra depth buys
  optimization difficulty and nothing else.

## What changed after 2016

The chapter's central claim — capability tracks dataset and model scale — held, and was made
quantitative by neural scaling laws (Hestness et al. 2017; Kaplan et al. 2020; Hoffmann et al.
2022 "Chinchilla"), which the 2016 text predates. Its examples of "what deep learning can do"
are now badly dated; its explanation of *why* depth helps is not. **Confidence: high** — the
scaling-law literature is public and replicated.

## Key Takeaways

1. Before adding depth, state the compositional structure you believe the data has.
2. Use the curse of dimensionality as the test for whether a nearest-neighbour-style baseline
   is even viable on your input.
3. Read the growth curves as a resource argument: most 2016-era "deep learning doesn't work
   here" results were data or compute verdicts, not architecture verdicts.

## Connects To

- **Ch 5**: the formal learning-theory machinery behind the claims made informally here.
- **Ch 15**: representation learning, promised here and delivered there.
- **references/book_to_2026_delta.md**: the scaling-law layer this chapter now needs.
