Ch 21 / 30 Topological Sort and Cycle Detection 0/0 exercises Exercises ↓

AmouAI Hub/Courses/Data Structures & Algorithms/Chapter 21

Part 5 · Graphs · Chapter 21

Topological Sort and Cycle Detection

You cannot take the advanced course before the intro course. Ordering tasks under dependencies — and detecting the impossible loop where A waits on B waits on A.

Reading
CLRS Ch. 22.4
Focus
Order under constraints
Cost
O(V + E)
Needs
A DAG

By the end of this chapter you can

  1. Define a DAG and state exactly when a topological order exists
  2. Implement Kahn's algorithm with in-degrees, and the DFS finish-time version
  3. Detect a cycle as a by-product of the sort, and report which vertices are involved
  4. Find the longest path in a DAG, and say why it is easy there and hard elsewhere
  5. Recognise a topological-sort problem from its wording

1Order under constraints

Some things must happen before others. Given a pile of such rules, produce an order that respects all of them — or prove none exists.

Getting dressed

Socks before shoes. Underwear before trousers. Shirt before jacket. Trousers before belt.

Nobody specifies a full sequence — only pairwise constraints. Several valid orders exist (socks and shirt are independent), and you need any one of them.

Now add one rule: shoes before socks. Suddenly no order exists at all, and the useful output is not an ordering but an explanation.

A topological sort of a directed graph is an ordering of its vertices such that every edge points forwards. If A → B, then A comes before B in the output.

The existence condition

A topological order exists if and only if the graph is a DAG — a directed acyclic graph.

The reason is immediate: if A → B → C → A, then A must come before B, which must come before C, which must come before A. A cannot come before itself.

So topological sorting and cycle detection are the same computation. An algorithm that produces the order also proves there is no cycle; one that gets stuck has found one.

Two more facts worth having straight:

  • The order is usually not unique. Independent vertices can go in either order, so a DAG generally has many valid topological orders. Any is acceptable unless the problem says otherwise.
  • Undirected graphs have no topological order, because an undirected edge is a constraint in both directions at once.

2Kahn's algorithm

Count how many prerequisites each vertex still has. Repeatedly take one with none left. That is the whole algorithm.

The in-degree of a vertex is the number of edges pointing at it — the number of prerequisites it is still waiting on. A vertex with in-degree 0 is ready to go now.

kahn.py
from collections import deque

def topo_sort(graph):
    """graph: {vertex: [successors]}. Returns an order, or None if cyclic."""
    indeg = {v: 0 for v in graph}
    for v in graph:
        for n in graph[v]:
            indeg[n] += 1

    q = deque(v for v in graph if indeg[v] == 0)   # ready immediately
    order = []

    while q:
        v = q.popleft()
        order.append(v)
        for n in graph[v]:
            indeg[n] -= 1          # one prerequisite satisfied
            if indeg[n] == 0:
                q.append(n)        # it just became ready

    return order if len(order) == len(graph) else None
    #                                              ↑ stuck: a cycle
Why the length check detects a cycle

If the queue empties before every vertex has come out, the remaining vertices all still have in-degree above zero — each is waiting on another that has not been emitted.

Follow those “waiting on” arrows backwards from any of them: because the set is finite and each has a predecessor inside the set, you must eventually repeat a vertex. That repetition is a cycle.

So the vertices left with a positive in-degree are exactly the ones involved in, or blocked by, a cycle — which is a genuinely useful error message.

Getting a specific order

Swap the queue for a heap and you get the lexicographically smallest valid order — useful when the output must be deterministic.

lexicographic.py
import heapq

def topo_sort_lexicographic(graph):
    indeg = {v: 0 for v in graph}
    for v in graph:
        for n in graph[v]:
            indeg[n] += 1
    h = [v for v in graph if indeg[v] == 0]
    heapq.heapify(h)                      # ← the only change
    order = []
    while h:
        v = heapq.heappop(h)              # smallest ready vertex
        order.append(v)
        for n in graph[v]:
            indeg[n] -= 1
            if indeg[n] == 0:
                heapq.heappush(h, n)
    return order if len(order) == len(graph) else None

# O((V + E) log V) instead of O(V + E) — the price of determinism.
Kahn's algorithm
O(V + E)
DFS version
O(V + E)
Lexicographic
O((V+E) log V)
Space
O(V)
Cycle detection
free

3The DFS version

Chapter 20 said DFS's value is the moment a vertex is finished. This is the payoff.

When a DFS finishes a vertex, everything reachable from it has already been finished. So if you record vertices in finish order and then reverse the list, every vertex appears before everything it can reach — which is exactly a topological order.

topo_dfs.py
def topo_sort_dfs(graph):
    WHITE, GREY, BLACK = 0, 1, 2
    colour = {v: WHITE for v in graph}
    order = []

    def visit(v):
        colour[v] = GREY                     # on the current path
        for n in graph[v]:
            if colour[n] == GREY:
                raise ValueError(f"cycle involving {n}")
            if colour[n] == WHITE:
                visit(n)
        colour[v] = BLACK
        order.append(v)                      # ← FINISH time, not start time

    for v in graph:
        if colour[v] == WHITE:
            visit(v)

    return order[::-1]                       # reverse finish order
It must be finish time, and it must be reversed

Appending on arrival instead of on finish gives you a pre-order walk, which is not a topological order — a vertex can be visited before something that must precede it.

And forgetting the reversal gives you the order exactly backwards, which is a bug that passes every test on a symmetric example and fails on a real one.

Kahn (queue)DFS (finish times)
Detects cyclesYes — the length checkYes — the grey check
Names the cycleThe leftover verticesThe exact grey edge found
IterativeNaturallyNeeds an explicit stack for deep graphs
Lexicographic orderEasy — use a heapAwkward
Extends to longest pathYes, naturallyYes
Intuition“peel off what is ready”“finish deepest first”

Kahn's is the one to reach for by default: it is iterative, it handles deep graphs without a recursion limit, it produces a useful error, and it is easier to explain out loud.

4Longest path in a DAG

Finding the longest path in a general graph is NP-hard. In a DAG it is linear, and the topological order is why.

In a graph with cycles, “longest path” is meaningless — go round the cycle again and it is longer. Restrict to simple paths and the problem becomes NP-hard: no known polynomial algorithm.

In a DAG it is easy. Process the vertices in topological order and, by the time you reach a vertex, every path into it has already been computed.

longest_path.py
def longest_path(graph, weights=None):
    """Longest path in a DAG. weights: {(u, v): cost}, default 1 per edge."""
    order = topo_sort(graph)
    if order is None:
        raise ValueError("not a DAG — longest path is undefined")

    dist = {v: 0 for v in graph}
    parent = {v: None for v in graph}

    for v in order:                  # ← in topological order, so dist[v] is final
        for n in graph[v]:
            w = weights.get((v, n), 1) if weights else 1
            if dist[v] + w > dist[n]:
                dist[n] = dist[v] + w
                parent[n] = v

    end = max(dist, key=dist.get)
    path = []
    while end is not None:
        path.append(end)
        end = parent[end]
    return path[::-1], max(dist.values())
Why the topological order is the whole trick

When you process vertex v, every vertex with an edge into v comes earlier in the order, so all of them have already been processed. dist[v] is therefore final and can be used with confidence.

That is the same argument dynamic programming makes in Chapter 25, and it is not a coincidence: a topological order is a valid order for filling in a DP table. The subproblem dependency graph is a DAG, and topological order is the evaluation order.

The critical path

The longest path through a task DAG is the critical path: the minimum possible project duration, and the set of tasks where any delay delays everything. This is what project management software computes, and it is fifteen lines.

critical_path.py
tasks = {
    "design":   [("build", 5)],       # design takes 5 days, then build can start
    "build":    [("test", 10)],
    "test":     [("ship", 3)],
    "docs":     [("ship", 2)],
    "design2":  [("docs", 1)],
    "ship":     [],
}
# The longest path from any start to "ship" is the earliest possible
# ship date. Every task on it is critical; everything else has slack.

5Where this shows up

Topological sort is one of the most widely deployed algorithms in software, and it is almost always invisible.

SystemVerticesEdgesWhat a cycle means
make, Bazel, any build toolFiles / targets“depends on”Circular dependency — build fails
Package managers (pip, npm, apt)Packages“requires”Unresolvable install
Spreadsheet recalculationCells“formula references”The circular-reference error
Course planningCoursesPrerequisitesAn impossible degree
Task schedulers (Airflow, CI pipelines)Jobs“must run after”The pipeline never starts
CompilersModules“imports”Circular import
Symbol resolution / linkingSymbols“refers to”Unresolvable reference
Class initialisation (JVM, Python)Classes“inherits / uses”Initialisation deadlock
Git historyCommits“parent of”Impossible — commits are a DAG by construction

Notice the last row. Git is a DAG on purpose: a commit references its parents, never its children, which is precisely what makes history immutable and merges well-defined.

Recognising one

The wording to watch for

Reach for topological sort when you see:

  • before”, “after”, “prerequisite”, “depends on”, “must precede
  • “Is there a valid order?”
  • “Can all of these be completed?” — which is asking whether the graph is a DAG
  • “What is the minimum time to finish, given tasks can run in parallel?” — that is the longest path

Reporting the cycle, not just detecting it

“A circular dependency exists” is a bad error message. “auth → database → logging → auth” is a good one. Kahn's leftovers give you the set; DFS gives you the exact loop:

find_cycle.py
def find_cycle(graph):
    """Returns the actual cycle as a list, or None."""
    WHITE, GREY, BLACK = 0, 1, 2
    colour = {v: WHITE for v in graph}
    stack = []

    def visit(v):
        colour[v] = GREY
        stack.append(v)
        for n in graph[v]:
            if colour[n] == GREY:
                i = stack.index(n)
                return stack[i:] + [n]      # ← the loop, spelled out
            if colour[n] == WHITE:
                found = visit(n)
                if found:
                    return found
        colour[v] = BLACK
        stack.pop()
        return None

    for v in graph:
        if colour[v] == WHITE:
            found = visit(v)
            if found:
                return found
    return None

# The GREY vertex is on the current path, so the path from where it first
# appears in `stack` up to now IS the cycle. That is why grey exists.

What to carry forward

  • A topological order puts every vertex before everything it points to. It exists if and only if the graph is a DAG.
  • Topological sorting and cycle detection are the same computation: succeed and you have an order, get stuck and you have found a cycle.
  • Kahn's algorithm: count in-degrees, repeatedly take a vertex with none left. If fewer than V come out, the leftovers are in or behind a cycle.
  • The DFS version uses reverse finish order. Recording on arrival instead of on finish is wrong, and forgetting the reversal is worse.
  • Longest path is NP-hard in general and linear in a DAG, because the topological order guarantees every predecessor is already final.
  • A topological order is a valid DP evaluation order — the same argument Chapter 25 makes.
  • Report the actual cycle, not just its existence. The grey stack gives it to you for free.

>_Playground

Both algorithms on the same graph, then a cycle, then the critical path of a small project.

scratch.pypython not loaded
Real Python, running in your browser. Nothing is installed or uploaded.
Output appears here. The first run takes a few seconds while Python loads.

Exercises

Checked automatically the moment you submit. Work top to bottom — each one assumes the last. Your answers are saved in this browser.

All Warm-up Core Challenge Reset chapter

Chapter 22 — Shortest Paths: Dijkstra, Bellman-Ford, A*

What your maps app is doing. Dijkstra is BFS that respects distance; A* is Dijkstra with an intuition.

Continue →