Week 1, Lecture 2: History

Week 1, Lecture 2: History#

NPTEL Deep Learning for Computer Vision | Prof. Vineeth N Balasubramanian, IIT Hyderabad

Companion notebook for §1.2 History.

What you will learn

How computer vision began, from neurophysiology and the first digital image.
The early attempts at 3D understanding, line labelling and shape-based recognition.
Marr’s hierarchical framework, and the Neocognitron that anticipated the ConvNet.
The low-level algorithms of the 1980s that are still in use today.

Contents

Early history: initial forays
Towards algorithms and practice: low-level understanding
What the history tells us

No code in this lecture. This is a discussion lecture, so there is nothing to run here. The page follows the slides and ends with questions to think about. Hands-on notebooks begin at Lecture 4.

A note on scope, from the slides: this is a history as captured from multiple sources (including Szeliski’s book), and it is deliberately biased toward the topics this course covers. There is much more history in geometry-based vision, physics-based vision, image and video compression, graphics and computational photography that is not covered here, and a slight preference throughout for work on images over video.

Review#

The previous lecture defined computer vision, surveyed where it is used, and argued that it is hard because it is an inverse problem solved from partial and noisy data. This lecture asks how the field got here.

1. Early history: initial forays#

1959 — Hubel and Wiesel. “Receptive fields of single neurons in the cat’s striate cortex”. By placing electrodes into the primary visual cortex of an anesthetized cat, they showed that simple and complex cells exist, and that visual processing begins with simple structures such as oriented edges. Nearly every architecture in this course inherits that idea.

1959 — Russell Kirsch. The world’s first digital image: an apparatus that turned photographs into grids of numbers. The subject was Kirsch’s infant son, a grainy 5 cm square, \(176 \times 176\) pixels, 30,976 in total. It now hangs in the Portland Art Museum.

1963 — Lawrence Roberts. His PhD thesis, “Machine Perception of Three-Dimensional Solids”, extracted 3D information about solid objects from 2D photographs of line drawings, and dealt with camera transformations, perspective effects, and the assumptions that make depth perception possible.

1966 — The MIT Summer Vision Project. Seymour Papert, with Gerald Sussman, set out to build a platform that would segment foreground from background and extract non-overlapping objects from real images. It was scoped as a summer project. It is often retold as the moment the field learned how hard the problem is.

1971–73 — Line labelling. Huffman and Clowes independently showed how to discern shape in a line drawing by labelling each line as convex, concave or occluded (Huffman, Impossible objects as nonsense sentences, 1971; Clowes, On seeing things, 1971).

Pictorial structures. Fischler and Elschlager: given a description of a visual object, find it in a photograph. Their formulation needed two things that recur constantly since, a descriptive scheme for the object and a metric for the goodness of a match.

Shape-based recognition. Binford’s generalized cylinders (1971), and Marr and Nishihara’s skeletons and cylinders (1978), tried to recognise objects by recovering their volumetric structure. MIT’s AI Lab began offering a “Machine Vision” course.

1982 — David Marr. Vision: A computational investigation into the human representation and processing of visual information established that vision is hierarchical: low-level algorithms detecting edges, curves and corners feed higher-level understanding. Marr’s representational framework has three stages:

a primal sketch, representing edges, bars and boundaries,
a 2½-D sketch, adding surfaces, depth and discontinuities,
a 3D model, organised hierarchically in surface and volumetric primitives.

1979–82 — Fukushima’s Neocognitron. A self-organizing network of simple and complex cells that recognised patterns regardless of position shifts, built directly on Hubel and Wiesel’s findings. It included convolutional layers with weight vectors, or filters. This is the original ConvNet, roughly thirty years before the deep learning era.

2. Towards algorithms and practice: low-level understanding#

1981 — Optical flow. Horn and Schunck estimated the direction and speed of motion across two frames by formulating flow as a global energy functional and minimising it.

1986 — The Canny edge detector. A multi-stage operator built on a computational theory of edge detection, using the calculus of variations to find the function optimising a stated criterion. Well defined, simple to implement, and still the default edge detector forty years later. You will build it in Week 2.

1987 — Recognition by components. Biederman’s bottom-up theory of object recognition: objects are assembled from geons, primitive 3D shapes such as cylinders and cones.

1988 — Snakes. Active contour models that delineate an object outline in a noisy 2D image, widely used for object segmentation and tracking.

3. What the history tells us#

Two threads run through all of it. The first is hierarchy: from Hubel and Wiesel’s simple and complex cells, through Marr’s primal sketch and 2½-D sketch, to the layers of a modern CNN, the recurring answer is that vision is built in stages from simple local measurements to global understanding. The second is the shift from designed to learned. For four decades the descriptive scheme and matching metric were specified by hand. Weeks 1 and 2 of this course cover that tradition at its best; from Week 3 onwards, the features are learned from data instead.

Key takeaways#

Year	Contribution	Why it matters
1959	Hubel and Wiesel, simple and complex cells	Vision starts from oriented edges; the biological basis for convolutional layers
1959	Kirsch’s first digital image	176 × 176 pixels; images become arrays of numbers
1963	Roberts, 3D solids from line drawings	The first attempt at recovering 3D from 2D
1966	MIT Summer Vision Project	The field’s first lesson in how hard the problem is
1971–73	Huffman and Clowes, line labelling	Convex, concave, occluded: structure from drawings
1978	Marr and Nishihara, skeletons and cylinders	Recognition by recovering volumetric structure
1982	Marr’s Vision	Vision is hierarchical: primal sketch, 2½-D sketch, 3D model
1979–82	Fukushima’s Neocognitron	The original ConvNet: convolution, simple and complex cells, shift invariance
1981	Horn and Schunck, optical flow	Motion as a global energy minimisation
1986	Canny edge detector	Still the standard; built in Week 2
1987	Biederman, geons	Objects as assemblies of 3D primitives
1988	Snakes, active contours	Energy-minimising outlines for segmentation

Exercises#

Trace one idea forward. Follow “simple and complex cells” from Hubel and Wiesel (1959) through the Neocognitron (1980) to a modern CNN. What is genuinely the same, and what changed?
Why did the Summer Vision Project fail? Using the reasons from Lecture 1, explain which specific assumption made “segment the objects” look like a summer’s work.
Marr’s three stages. Map each stage of Marr’s framework onto a layer or group of layers in a modern CNN. Where does the analogy hold, and where does it break?
Designed versus learned. Pictorial structures needed a descriptive scheme and a matching metric, both specified by hand. In a modern detector, what plays each of those two roles, and where does it come from?