KinetiqVision
← Back to curriculum

Module 6 — Video & motion

Optical flow & motion

Brightness constancy, Lucas–Kanade, Horn–Schunck, flow visualization, motion boundaries, and when flow fails.

~75 min read + exercises

Optical flow & motion

Before we begin

Optical flow estimates how each pixel moves between consecutive video frames — a 2D motion field (u,v)(u, v).

Learning objectives

  • State the brightness constancy assumption.
  • Contrast Lucas–Kanade (local) vs Horn–Schunck (global).
  • Know when flow fails (occlusion, lighting, lack of texture).
  • Read a color-wheel flow visualization.

Brightness constancy

I(x,y,t)≈I(x+u,y+v,t+1)I(x, y, t) \approx I(x + u, y + v, t + 1)

Linearize → optical flow constraint Ixu+Iyv+It=0I_x u + I_y v + I_t = 0.

One equation, two unknowns — aperture problem. Need a window (LK) or smoothness prior (HS).

Lucas–Kanade

Assume constant flow in a small patch. Stack equations for pixels in window → least squares for (u,v)(u, v).

Works on textured patches; fails on uniform regions.

Dense flow

Modern networks (RAFT, PWC-Net) predict dense flow end-to-end — state of the art on benchmarks.

Classical Farneback in OpenCV: cv2.calcOpticalFlowFarneback.

Motion boundaries

Different objects move differently — flow discontinuities at edges. Layered motion and segmentation-aware flow handle mixed scenes.

What's next

Lesson 2 — Object tracking & data association