AmouAI Hub/Courses/Data Structures & Algorithms/Chapter 21
Topological Sort and Cycle Detection
You cannot take the advanced course before the intro course. Ordering tasks under dependencies — and detecting the impossible loop where A waits on B waits on A.
By the end of this chapter you can
- Define a DAG and state exactly when a topological order exists
- Implement Kahn's algorithm with in-degrees, and the DFS finish-time version
- Detect a cycle as a by-product of the sort, and report which vertices are involved
- Find the longest path in a DAG, and say why it is easy there and hard elsewhere
- Recognise a topological-sort problem from its wording
1Order under constraints
Some things must happen before others. Given a pile of such rules, produce an order that respects all of them — or prove none exists.
Socks before shoes. Underwear before trousers. Shirt before jacket. Trousers before belt.
Nobody specifies a full sequence — only pairwise constraints. Several valid orders exist (socks and shirt are independent), and you need any one of them.
Now add one rule: shoes before socks. Suddenly no order exists at all, and the useful output is not an ordering but an explanation.
A topological sort of a directed graph is an ordering of its vertices such that
every edge points forwards. If A → B, then A comes before B in the output.
A topological order exists if and only if the graph is a DAG — a directed acyclic graph.
The reason is immediate: if A → B → C → A, then A must come before B,
which must come before C, which must come before A. A cannot come before itself.
So topological sorting and cycle detection are the same computation. An algorithm that produces the order also proves there is no cycle; one that gets stuck has found one.
Two more facts worth having straight:
- The order is usually not unique. Independent vertices can go in either order, so a DAG generally has many valid topological orders. Any is acceptable unless the problem says otherwise.
- Undirected graphs have no topological order, because an undirected edge is a constraint in both directions at once.
2Kahn's algorithm
Count how many prerequisites each vertex still has. Repeatedly take one with none left. That is the whole algorithm.
The in-degree of a vertex is the number of edges pointing at it — the number of prerequisites it is still waiting on. A vertex with in-degree 0 is ready to go now.
from collections import deque
def topo_sort(graph):
"""graph: {vertex: [successors]}. Returns an order, or None if cyclic."""
indeg = {v: 0 for v in graph}
for v in graph:
for n in graph[v]:
indeg[n] += 1
q = deque(v for v in graph if indeg[v] == 0) # ready immediately
order = []
while q:
v = q.popleft()
order.append(v)
for n in graph[v]:
indeg[n] -= 1 # one prerequisite satisfied
if indeg[n] == 0:
q.append(n) # it just became ready
return order if len(order) == len(graph) else None
# ↑ stuck: a cycleIf the queue empties before every vertex has come out, the remaining vertices all still have in-degree above zero — each is waiting on another that has not been emitted.
Follow those “waiting on” arrows backwards from any of them: because the set is finite and each has a predecessor inside the set, you must eventually repeat a vertex. That repetition is a cycle.
So the vertices left with a positive in-degree are exactly the ones involved in, or blocked by, a cycle — which is a genuinely useful error message.
Getting a specific order
Swap the queue for a heap and you get the lexicographically smallest valid order — useful when the output must be deterministic.
import heapq
def topo_sort_lexicographic(graph):
indeg = {v: 0 for v in graph}
for v in graph:
for n in graph[v]:
indeg[n] += 1
h = [v for v in graph if indeg[v] == 0]
heapq.heapify(h) # ← the only change
order = []
while h:
v = heapq.heappop(h) # smallest ready vertex
order.append(v)
for n in graph[v]:
indeg[n] -= 1
if indeg[n] == 0:
heapq.heappush(h, n)
return order if len(order) == len(graph) else None
# O((V + E) log V) instead of O(V + E) — the price of determinism.3The DFS version
Chapter 20 said DFS's value is the moment a vertex is finished. This is the payoff.
When a DFS finishes a vertex, everything reachable from it has already been finished. So if you record vertices in finish order and then reverse the list, every vertex appears before everything it can reach — which is exactly a topological order.
def topo_sort_dfs(graph):
WHITE, GREY, BLACK = 0, 1, 2
colour = {v: WHITE for v in graph}
order = []
def visit(v):
colour[v] = GREY # on the current path
for n in graph[v]:
if colour[n] == GREY:
raise ValueError(f"cycle involving {n}")
if colour[n] == WHITE:
visit(n)
colour[v] = BLACK
order.append(v) # ← FINISH time, not start time
for v in graph:
if colour[v] == WHITE:
visit(v)
return order[::-1] # reverse finish orderAppending on arrival instead of on finish gives you a pre-order walk, which is not a topological order — a vertex can be visited before something that must precede it.
And forgetting the reversal gives you the order exactly backwards, which is a bug that passes every test on a symmetric example and fails on a real one.
| Kahn (queue) | DFS (finish times) | |
|---|---|---|
| Detects cycles | Yes — the length check | Yes — the grey check |
| Names the cycle | The leftover vertices | The exact grey edge found |
| Iterative | Naturally | Needs an explicit stack for deep graphs |
| Lexicographic order | Easy — use a heap | Awkward |
| Extends to longest path | Yes, naturally | Yes |
| Intuition | “peel off what is ready” | “finish deepest first” |
Kahn's is the one to reach for by default: it is iterative, it handles deep graphs without a recursion limit, it produces a useful error, and it is easier to explain out loud.
4Longest path in a DAG
Finding the longest path in a general graph is NP-hard. In a DAG it is linear, and the topological order is why.
In a graph with cycles, “longest path” is meaningless — go round the cycle again and it is longer. Restrict to simple paths and the problem becomes NP-hard: no known polynomial algorithm.
In a DAG it is easy. Process the vertices in topological order and, by the time you reach a vertex, every path into it has already been computed.
def longest_path(graph, weights=None):
"""Longest path in a DAG. weights: {(u, v): cost}, default 1 per edge."""
order = topo_sort(graph)
if order is None:
raise ValueError("not a DAG — longest path is undefined")
dist = {v: 0 for v in graph}
parent = {v: None for v in graph}
for v in order: # ← in topological order, so dist[v] is final
for n in graph[v]:
w = weights.get((v, n), 1) if weights else 1
if dist[v] + w > dist[n]:
dist[n] = dist[v] + w
parent[n] = v
end = max(dist, key=dist.get)
path = []
while end is not None:
path.append(end)
end = parent[end]
return path[::-1], max(dist.values())When you process vertex v, every vertex with an edge into v comes
earlier in the order, so all of them have already been processed. dist[v] is
therefore final and can be used with confidence.
That is the same argument dynamic programming makes in Chapter 25, and it is not a coincidence: a topological order is a valid order for filling in a DP table. The subproblem dependency graph is a DAG, and topological order is the evaluation order.
The critical path
The longest path through a task DAG is the critical path: the minimum possible project duration, and the set of tasks where any delay delays everything. This is what project management software computes, and it is fifteen lines.
tasks = {
"design": [("build", 5)], # design takes 5 days, then build can start
"build": [("test", 10)],
"test": [("ship", 3)],
"docs": [("ship", 2)],
"design2": [("docs", 1)],
"ship": [],
}
# The longest path from any start to "ship" is the earliest possible
# ship date. Every task on it is critical; everything else has slack.5Where this shows up
Topological sort is one of the most widely deployed algorithms in software, and it is almost always invisible.
| System | Vertices | Edges | What a cycle means |
|---|---|---|---|
make, Bazel, any build tool | Files / targets | “depends on” | Circular dependency — build fails |
| Package managers (pip, npm, apt) | Packages | “requires” | Unresolvable install |
| Spreadsheet recalculation | Cells | “formula references” | The circular-reference error |
| Course planning | Courses | Prerequisites | An impossible degree |
| Task schedulers (Airflow, CI pipelines) | Jobs | “must run after” | The pipeline never starts |
| Compilers | Modules | “imports” | Circular import |
| Symbol resolution / linking | Symbols | “refers to” | Unresolvable reference |
| Class initialisation (JVM, Python) | Classes | “inherits / uses” | Initialisation deadlock |
| Git history | Commits | “parent of” | Impossible — commits are a DAG by construction |
Notice the last row. Git is a DAG on purpose: a commit references its parents, never its children, which is precisely what makes history immutable and merges well-defined.
Recognising one
Reach for topological sort when you see:
- “before”, “after”, “prerequisite”, “depends on”, “must precede”
- “Is there a valid order?”
- “Can all of these be completed?” — which is asking whether the graph is a DAG
- “What is the minimum time to finish, given tasks can run in parallel?” — that is the longest path
Reporting the cycle, not just detecting it
“A circular dependency exists” is a bad error message. “auth → database → logging → auth” is a good one. Kahn's leftovers give you the set; DFS gives you the exact loop:
def find_cycle(graph):
"""Returns the actual cycle as a list, or None."""
WHITE, GREY, BLACK = 0, 1, 2
colour = {v: WHITE for v in graph}
stack = []
def visit(v):
colour[v] = GREY
stack.append(v)
for n in graph[v]:
if colour[n] == GREY:
i = stack.index(n)
return stack[i:] + [n] # ← the loop, spelled out
if colour[n] == WHITE:
found = visit(n)
if found:
return found
colour[v] = BLACK
stack.pop()
return None
for v in graph:
if colour[v] == WHITE:
found = visit(v)
if found:
return found
return None
# The GREY vertex is on the current path, so the path from where it first
# appears in `stack` up to now IS the cycle. That is why grey exists.What to carry forward
- A topological order puts every vertex before everything it points to. It exists if and only if the graph is a DAG.
- Topological sorting and cycle detection are the same computation: succeed and you have an order, get stuck and you have found a cycle.
- Kahn's algorithm: count in-degrees, repeatedly take a vertex with none left. If fewer than V come out, the leftovers are in or behind a cycle.
- The DFS version uses reverse finish order. Recording on arrival instead of on finish is wrong, and forgetting the reversal is worse.
- Longest path is NP-hard in general and linear in a DAG, because the topological order guarantees every predecessor is already final.
- A topological order is a valid DP evaluation order — the same argument Chapter 25 makes.
- Report the actual cycle, not just its existence. The grey stack gives it to you for free.
>_Playground
Both algorithms on the same graph, then a cycle, then the critical path of a small project.
✓Exercises
Checked automatically the moment you submit. Work top to bottom — each one assumes the last. Your answers are saved in this browser.
Chapter 22 — Shortest Paths: Dijkstra, Bellman-Ford, A*
What your maps app is doing. Dijkstra is BFS that respects distance; A* is Dijkstra with an intuition.