Ch 20 / 30 Breadth-First and Depth-First Search 0/0 exercises Exercises ↓

AmouAI Hub/Courses/Data Structures & Algorithms/Chapter 20

Part 5 · Graphs · Chapter 20

Breadth-First and Depth-First Search

Two ways to explore a maze: flood it evenly, or follow one corridor to the end and back up. Same code, different container — and that one swap changes everything.

Reading
CLRS Ch. 22.2–22.3
Focus
Order of exploration
Cost
O(V + E)
Powers
Shortest paths · components

By the end of this chapter you can

  1. Implement BFS and DFS from one template, differing only in the container
  2. Prove that BFS finds shortest paths on unweighted graphs, and say why DFS does not
  3. Reconstruct a path from the parent pointers a traversal leaves behind
  4. Mark vertices as seen at the right moment, and explain what goes wrong otherwise
  5. Choose between BFS and DFS for a stated problem

1One algorithm, two containers

BFS and DFS are usually taught as two algorithms. They are one algorithm with a parameter, and seeing that makes both easier to remember.

Two ways to search a building

Depth-first: pick a corridor, follow it to the end, try every door, back up only when there is nothing left, then take the next corridor. One person with a piece of chalk.

Breadth-first: check every room on this floor, then every room on the next floor, then the next. A search party spreading out in rings.

Both visit every room. They differ only in which unexplored room you pick next — the newest one you found, or the oldest.

Here is the template. The frontier holds vertices discovered but not yet expanded. The only difference between the two algorithms is which end you take from.

one_template.py
from collections import deque

def search(graph, start, use_bfs=True):
    frontier = deque([start])
    seen = {start}
    order = []

    while frontier:
        vertex = frontier.popleft() if use_bfs else frontier.pop()
        #                  ↑ oldest              ↑ newest
        #        THIS IS THE ONLY DIFFERENCE
        order.append(vertex)

        for neighbour in graph[vertex]:
            if neighbour not in seen:
                seen.add(neighbour)          # mark on DISCOVERY, not on visit
                frontier.append(neighbour)

    return order
Queue or stack, and nothing else

A queue serves the oldest waiting vertex, so the search expands in rings of increasing distance: breadth-first.

A stack serves the newest, so the search plunges as deep as it can before backing up: depth-first.

Everything else in this chapter follows from that one choice. It is worth being able to state it in one sentence, because it is the kind of thing that makes an interview answer sound like understanding rather than recall.

2Mark on discovery, not on visit

A one-line detail that decides whether your traversal is O(V+E) or quadratic. It is worth its own section because everyone gets it wrong once.

There are two moments you could add a vertex to seen: when you first discover it and push it onto the frontier, or when you later pop it to expand it. The difference matters.

when_to_mark.py
# WRONG — mark when popped
while frontier:
    v = frontier.popleft()
    if v in seen:
        continue
    seen.add(v)
    for n in graph[v]:
        frontier.append(n)       # n may already be waiting, several times over

# In a dense graph a vertex can be pushed once per incoming edge. The
# frontier bloats to O(E) and the algorithm does far more work than it
# needs to — still correct, but slow and memory-hungry.


# RIGHT — mark on discovery
while frontier:
    v = frontier.popleft()
    for n in graph[v]:
        if n not in seen:
            seen.add(n)          # ← immediately, before it goes on the frontier
            frontier.append(n)

# Each vertex enters the frontier at most once. O(V) memory, O(V + E) time.
The exception: Dijkstra

Chapter 22's Dijkstra deliberately uses the wrong-looking version — it allows a vertex to sit in the priority queue several times and skips duplicates when popped.

That is not sloppiness. A vertex's best-known distance can improve after it has been pushed, so you genuinely need to reconsider it. In an unweighted BFS the first discovery is already optimal, so there is nothing to reconsider — which is exactly why marking on discovery is safe there and not there.

3BFS finds shortest paths

The single most useful property in this chapter, and the reason BFS is the default for maze and puzzle problems.

Because BFS expands vertices in order of discovery, and each expansion adds vertices exactly one edge further out, it visits all vertices at distance 1, then all at distance 2, then distance 3.

So the first time BFS reaches a vertex, it has reached it by a shortest path. It cannot do better later, because “later” means “further out”.

The guarantee, stated exactly

On an unweighted graph (or one where every edge costs the same), BFS from s finds a shortest path from s to every reachable vertex.

DFS does not. DFS finds a path, and it is frequently a terrible one — it will happily wander the entire graph to reach a neighbour.

The moment edges have different weights, BFS loses the guarantee too, and you need Dijkstra. Chapter 22.

shortest_path.py
from collections import deque

def shortest_path(graph, start, goal):
    """Fewest edges from start to goal, as a list of vertices."""
    if start == goal:
        return [start]

    parent = {start: None}          # also serves as the visited set
    q = deque([start])

    while q:
        v = q.popleft()
        for n in graph[v]:
            if n not in parent:
                parent[n] = v       # remember HOW we got here
                if n == goal:
                    return rebuild(parent, goal)
                q.append(n)
    return None                     # unreachable


def rebuild(parent, goal):
    """Walk the parent pointers backwards, then reverse."""
    path = []
    while goal is not None:
        path.append(goal)
        goal = parent[goal]
    return path[::-1]

Note the shape of that code, because it recurs constantly:

  • The parent dict does two jobs at once — it is the visited set and the record of how each vertex was reached.
  • No path is ever stored during the search. Only one pointer per vertex, reconstructed at the end.

If you need the distances rather than the path, the same traversal gives them:

distances.py
def distances(graph, start):
    dist = {start: 0}
    q = deque([start])
    while q:
        v = q.popleft()
        for n in graph[v]:
            if n not in dist:
                dist[n] = dist[v] + 1
                q.append(n)
    return dist            # anything missing is unreachable

Level-by-level BFS

When you need to know which round you are in — “how many moves so far?” — process a whole level at a time by recording the frontier size first. The same trick as Chapter 14.

levels.py
def levels(graph, start):
    seen = {start}
    frontier = [start]
    depth = 0
    while frontier:
        print(f"distance {depth}: {frontier}")
        nxt = []
        for v in frontier:                    # everything at THIS distance
            for n in graph[v]:
                if n not in seen:
                    seen.add(n)
                    nxt.append(n)
        frontier = nxt
        depth += 1

4DFS, and what it is for

DFS gives no distance guarantee. What it gives instead is structure — and that structure is what topological sort and cycle detection are built on.

DFS is most naturally written recursively, because the call stack is the frontier stack:

dfs.py
def dfs(graph, v, seen=None):
    if seen is None:
        seen = set()
    seen.add(v)
    process(v)                       # PRE-order: on the way down
    for n in graph[v]:
        if n not in seen:
            dfs(graph, n, seen)
    finish(v)                        # POST-order: on the way back up
    return seen

# The iterative version, for graphs deeper than ~1000:
def dfs_iterative(graph, start):
    stack = [start]
    seen = set()
    while stack:
        v = stack.pop()
        if v in seen:
            continue
        seen.add(v)
        process(v)
        for n in reversed(graph[v]):   # reversed → same order as recursion
            if n not in seen:
                stack.append(n)
The post-order moment is where the value is

The line marked finish(v) runs when every vertex reachable from v has been fully processed. That single fact powers:

  • Topological sort — finish times in reverse order are a valid ordering (Chapter 21)
  • Cycle detection — meeting a vertex that is started but not finished means a back edge, and a back edge means a cycle
  • Strongly connected components — Tarjan's and Kosaraju's algorithms are both bookkeeping on finish times
  • Bridges and articulation points — which edges, if cut, disconnect the graph

BFS has no equivalent moment. That is the trade: BFS knows about distance, DFS knows about structure.

Three colours

For cycle detection, a two-state visited set is not enough. You need to distinguish “finished” from “currently on the stack”:

three_colours.py
WHITE, GREY, BLACK = 0, 1, 2      # unseen / in progress / finished

def has_cycle(graph):
    colour = {v: WHITE for v in graph}

    def visit(v):
        colour[v] = GREY                    # on the current path
        for n in graph[v]:
            if colour[n] == GREY:
                return True                 # ← back edge: a cycle
            if colour[n] == WHITE and visit(n):
                return True
        colour[v] = BLACK                   # done, and off the path
        return False

    return any(colour[v] == WHITE and visit(v) for v in graph)

# GREY means A. Reaching a GREY vertex means
# you have looped back onto your own path.
#
# A BLACK vertex is fine to meet again — it just means two different paths
# reach the same finished region.

5Choosing, and the classic applications

A short decision rule, then the problems each one owns.

Use BFS whenUse DFS when
You need the shortest path (unweighted)You need to know if a path exists
The answer is likely near the startThe answer is likely deep
You need distances or levelsYou need topological order or cycle detection
The graph is very deep (avoids stack overflow)The graph is very wide (uses less memory)
Example: fewest moves in a puzzleExample: does this dependency graph have a cycle?

Memory is the practical trade. BFS holds an entire level at once, which on a wide graph is huge. DFS holds one root-to-leaf path, which on a deep graph is huge. Pick the one whose bad case your graph does not have.

Time (both)
O(V + E)
BFS space
O(width)
DFS space
O(depth)
Shortest path
BFS only
Cycle detection
DFS

Applications you should recognise on sight

ProblemWhichWhy
Fewest moves / shortest maze routeBFSUnweighted shortest path
Word ladder (COLD → WARM)BFSShortest path on an implicit graph
Count islands in a grideitherOne traversal per unvisited land cell
Flood fill / paint bucketeitherReach everything of the same colour
Detect a cycleDFSNeeds the grey/black distinction
Build order / topological sortDFS or KahnNeeds finish times, Ch. 21
Bipartite checkeither2-colour while traversing
Maze generationDFSLong winding corridors are the point
Web crawlerBFSStay near the seed pages
Solving SudokuDFSBacktracking is DFS on a state graph, Ch. 27
islands.py
# Counting islands — the pattern for every A problem
def count_islands(grid):
    if not grid:
        return 0
    rows, cols = len(grid), len(grid[0])
    seen = set()
    count = 0

    def flood(r, c):
        stack = [(r, c)]
        while stack:
            cr, cc = stack.pop()
            if (cr, cc) in seen:
                continue
            seen.add((cr, cc))
            for dr, dc in ((1,0), (-1,0), (0,1), (0,-1)):
                nr, nc = cr + dr, cc + dc
                if (0 <= nr < rows and 0 <= nc < cols
                        and grid[nr][nc] == "1" and (nr, nc) not in seen):
                    stack.append((nr, nc))

    for r in range(rows):
        for c in range(cols):
            if grid[r][c] == "1" and (r, c) not in seen:
                count += 1        # a new island
                flood(r, c)       # absorb all of it
    return count

# Same shape as counting connected components in Chapter 19 — because
# that is exactly what it is.
The habit worth forming

When you meet a new problem, ask: is this a shortest-path question or a reachability question?

Shortest → BFS. Reachability, ordering or structure → DFS. That one question resolves the choice almost every time, and it is faster than trying to remember a table.

What to carry forward

  • BFS and DFS are one algorithm. Queue → breadth-first, stack → depth-first, and nothing else differs.
  • Mark vertices seen on discovery, not when popped, or the frontier bloats to O(E). Dijkstra is the deliberate exception.
  • BFS finds shortest paths on unweighted graphs, because it reaches every vertex in distance order. DFS gives no such guarantee.
  • Store one parent pointer per vertex and rebuild the path at the end. Never store paths during the search.
  • DFS's value is the post-order moment — when a vertex is finished. Topological sort, cycle detection and SCCs are all built on it.
  • Cycle detection needs three colours: white unseen, grey on the current path, black finished. Meeting grey means a cycle.
  • Shortest → BFS. Reachability or structure → DFS. That question settles the choice.

>_Playground

One template, both algorithms, on the same graph. Then shortest paths, level-by-level BFS and island counting.

scratch.pypython not loaded
Real Python, running in your browser. Nothing is installed or uploaded.
Output appears here. The first run takes a few seconds while Python loads.

Exercises

Checked automatically the moment you submit. Work top to bottom — each one assumes the last. Your answers are saved in this browser.

All Warm-up Core Challenge Reset chapter

Chapter 21 — Topological Sort and Cycle Detection

You cannot take the advanced course before the intro course. Ordering tasks under dependencies — and detecting the impossible loop.

Continue →