Amortization & Union-Find
Amortized analysis asks: what is the average cost per operation over a sequence, even when individual operations vary wildly? Four methods — aggregate, accounting, charging, potential — each reveal the same truth from a different angle. Union-Find applies all of them to reach $O(\alpha(n))$ per operation: essentially constant, but provably not quite.
What Is Amortized Analysis?
Worst-case per-operation analysis is often too pessimistic. Table doubling costs $\Theta(n)$ when it fires — but it fires rarely. Amortized analysis spreads the cost of expensive operations over the cheap ones that precede them.
Method 1 — Aggregate Method
Compute the total cost of $k$ operations directly, then divide: $\hat{c} = \text{total} / k$.
Table doubling. Starting from size 1, after $n$ insertions the table has doubled $\lfloor\log_2 n\rfloor$ times. Total doubling cost: $1 + 2 + 4 + \cdots + 2^{\lfloor\log_2 n\rfloor} = \Theta(n)$. Plus $n$ unit-cost insertions. Total: $\Theta(n)$. Amortized per insertion: $\Theta(1)$.
Method 2 — Accounting Method
Prepay for future expensive operations by storing credits (coins) on data structure objects. An operation can store credits it doesn’t need now; a later expensive operation spends stored credits. The invariant: credit balance never goes negative.
Table doubling with accounting. Each insertion stores $c$ coins on the newly inserted element. When the table doubles from $m$ to $2m$: the $m/2$ elements inserted since the last doubling each contributed $c$ coins → total $cm/2$ coins available. Set $c = 2$: pays for $\Theta(m)$ doubling. Amortized cost per insertion: $O(1)$.
2-3 tree insertions. Goal: $O(\log n)$ amortized per insert, $O(0)$ per delete. Naively save a $\log n$ coin on insert and spend on delete — but the tree may grow between insert and delete, making the coin insufficient. Use the charging method instead.
Method 3 — Charging Method
Operations can charge cost retroactively to past operations. Each past operation can be charged at most once (or a bounded number of times). The amortized cost equals actual cost minus charges received from future operations plus charges paid to past operations.
Table doubling and halving. When doubling fires (table $\frac{1}{2}$ full $\to$ full), charge the $m/2$ insertions since last resize: each charged $O(1)$. When halving fires (table $\frac{1}{4}$ full $\to$ $\frac{1}{2}$ full), charge the $m/4$ deletions since last resize: each charged $O(1)$. Amortized cost per insert or delete: $O(1)$.
Free deletion in 2-3 trees. Each delete charges the insert that brought the tree to its current size $n$. Each insert is charged at most once (before the tree shrinks and returns to size $n$, another insert must happen). Amortized insert: $O(\log n)$, amortized delete: $O(0)$.
Method 4 — Potential Method
Define a potential function $\Phi: \text{DS state} \to \mathbb{R}_{\geq 0}$, representing stored energy in the data structure. The amortized cost of each operation is:
Telescoping: $\sum \hat{c}_i = \sum c_i + \Phi(\text{final}) - \Phi(\text{initial})$. If $\Phi(\text{final}) \geq \Phi(\text{initial}) = 0$, then $\sum c_i \leq \sum \hat{c}_i$. The potential method is the inverse of the accounting method: specifying $\Phi$ determines $\Delta\Phi$ per operation; specifying credits per operation determines $\Phi$.
Insert in 2-3 trees. Let $\Phi = \text{number of 3-nodes}$ (3-nodes are “bad”: they cause splits). A split converts one 3-node into two 2-nodes: $\Delta\Phi \leq 1 - (\text{splits})$. Amortized splits $=$ actual splits $+ \Delta\Phi \leq 1$. Each insert causes $O(1)$ amortized splits regardless of the worst-case $O(\log n)$ actual splits.
Insert and delete in 2-5 trees. In 2-3 trees, splits create 2-nodes which are bad for merges; a split-merge alternation has no amortised bound. Fix: use 2-5 trees. A split of a 5-node creates two 3-nodes (not 2-nodes). A merge of two 2-nodes creates a 3-node (not a 5-node). Neither operation creates bad nodes. Let $\Phi = \text{5-nodes} + \text{2-nodes}$. Both insert and delete have $O(1)$ amortized structural changes.
class DynamicArray:
"""
Dynamic array with table doubling and halving.
Amortized O(1) per append and delete (at end).
Demonstrates the potential method concretely:
Phi = 2*n - m (where n = size, m = capacity)
Phi >= 0 always (since n >= m/2 after any resize).
"""
def __init__(self):
self._data = [None] # underlying storage, capacity m=1
self._n = 0 # number of elements
def __len__(self): return self._n
def __getitem__(self, i): return self._data[i]
def append(self, val):
"""O(1) amortized. Actual cost: O(n) when doubling fires, O(1) otherwise."""
if self._n == len(self._data):
self._resize(2 * len(self._data)) # double: O(n), rare
self._data[self._n] = val
self._n += 1
# Potential: Phi = 2*n - m
# After append (no resize): delta_Phi = 2*(n+1)-m - (2*n-m) = 2
# Amortized = 1 (actual) + 2 (delta_Phi) = 3 = O(1)
# After append (with resize to 2m): delta_Phi = 2 - 2*m (big drop pays for resize)
def pop(self):
"""O(1) amortized. Halve when table is 1/4 full to avoid thrashing."""
if self._n == 0: raise IndexError
val = self._data[self._n - 1]
self._data[self._n - 1] = None
self._n -= 1
# Halve only at 1/4 full (not 1/2): avoids O(n) on alternating push/pop
if self._n > 0 and self._n == len(self._data) // 4:
self._resize(len(self._data) // 2)
return val
def _resize(self, new_cap):
"""Copy to new allocation. O(n)."""
new_data = [None] * new_cap
for i in range(self._n):
new_data[i] = self._data[i]
self._data = new_data
def capacity(self): return len(self._data)
def demo_amortized_cost():
"""Measure actual vs amortized cost of n appends."""
arr = DynamicArray()
total_resizes = 0
resize_costs = 0
n = 1024
for i in range(n):
old_cap = arr.capacity()
arr.append(i)
if arr.capacity() != old_cap: # resize fired
total_resizes += 1
resize_costs += old_cap # O(old_cap) copy cost
print(f"n={n}: {total_resizes} resizes, "
f"total resize cost={resize_costs} โ {resize_costs/n:.1f}n")
# Output: 10 resizes, total resize costโ2n (confirming O(1) amortized)// Dynamic array with table doubling and halving
// Amortized O(1) per push_back and pop_back
#include <vector>
#include <stdexcept>
using namespace std;
class DynamicArray {
vector<int> data; // underlying storage
int n = 0; // logical size
void resize(int new_cap) {
data.resize(new_cap); // O(n) copy
}
public:
DynamicArray() { data.resize(1); }
// O(1) amortized: actual O(n) when doubling fires
void push_back(int val) {
if (n == (int)data.size())
resize(2 * data.size()); // double
data[n++] = val;
// Potential Phi = 2n - m:
// amortized = 1 + delta_Phi = 1 + 2 = 3 = O(1) (no resize)
// amortized = n + delta_Phi โ n - 2(n-1) โ O(1) (with resize)
}
// O(1) amortized: halve at 1/4 to avoid thrashing
int pop_back() {
if (n == 0) throw underflow_error("empty");
int val = data[--n];
if (n > 0 && n == (int)data.size() / 4)
resize(data.size() / 2); // halve
return val;
}
int size() const { return n; }
int capacity() const { return data.size(); }
};Part II — Union-Find (Disjoint Set Union)
Maintain a collection of disjoint sets supporting three operations:
— FIND-SET($x$): return the representative of the set containing $x$.
— UNION($x$, $y$): merge the sets containing $x$ and $y$.
Applications: connected components in dynamic graphs, Kruskal’s MST algorithm, clustering, network connectivity.
Naive to Efficient: Four Representations
Linked list (naive): FIND-SET walks to head: $\Theta(n)$. UNION concatenates: $\Theta(n)$. Total for $n$ operations: $O(n^2)$.
Linked list + weighted union (smaller-into-larger): always merge the shorter list into the longer. Each element’s rep pointer moves only when its set at least doubles → at most $\log n$ times. Amortized UNION: $O(\log n)$. Total for $n$ ops: $O(n\log n)$.
Forest of trees + union by rank: represent each set as a rooted tree, rep = root. UNION attaches the shorter tree under the taller. Height stays $O(\log n)$.
Forest + union by rank + path compression: during FIND-SET, redirect all nodes on the path directly to the root. Future queries on those nodes cost $O(1)$. Combined: $O(\alpha(n))$ amortised per operation.
Path Compression — Potential Analysis
With path compression alone: define $\Phi = \sum_i \log \text{weight}[x_i]$ (sum of log-weights). Union increases root’s weight, increasing $\Phi$ by at most $\log n$. During FIND-SET, each step moves $c$’s subtree out of $p$’s subtree. If $\text{weight}[c] \geq \frac{1}{2}\text{weight}[p]$: potential drops by $\geq 1$, paying for the step. At most $\log n$ steps have $\text{weight}[c] < \frac{1}{2}\text{weight}[p]$. Total: $O(m\log n)$ for $m$ operations.
The Inverse Ackermann Function $\alpha(n)$
With both union by rank and path compression, the amortised bound drops to $O(\alpha(n))$ per operation — essentially constant for all practical purposes:
Values: $A_0(1)=2$, $A_1(1)=3$, $A_2(1)=7$, $A_3(1)=2047$, $A_4(1) > 2^{2047}$.
Inverse: $\alpha(n) = \min\{k : A_k(1) \geq n\}$. For any $n$ that will ever arise in practice, $\alpha(n) \leq 4$. Not constant in theory (Tarjan 1975 proved the $O(m\alpha(n))$ bound is tight), but constant in every practical sense.
Theorem (Tarjan 1975): $m$ operations on $n$ elements with union by rank and path compression run in $\Theta(m\alpha(n))$ total time.
class UnionFind:
"""
Disjoint Set Union (DSU) with:
- Union by rank: attach smaller-rank tree under larger-rank root
- Path compression: flatten tree on every FIND-SET
Amortized O(alpha(n)) per operation โ essentially O(1) in practice.
"""
def __init__(self, n):
self.parent = list(range(n)) # parent[x] = x means x is a root
self.rank = [0] * n # upper bound on height of subtree
self.n_components = n
def find(self, x):
"""
Find representative of x. Path compression: redirect all nodes
on the path directly to the root (two-pass or iterative).
"""
if self.parent[x] != x:
self.parent[x] = self.find(self.parent[x]) # path compression
return self.parent[x]
def union(self, x, y):
"""
Merge sets containing x and y. Union by rank: attach smaller
rank tree under larger rank root. Return True if merged, False if same set.
"""
rx, ry = self.find(x), self.find(y)
if rx == ry: return False # already in same set
# Union by rank: smaller rank attaches under larger rank
if self.rank[rx] < self.rank[ry]:
rx, ry = ry, rx # ensure rank[rx] >= rank[ry]
self.parent[ry] = rx # attach ry under rx
if self.rank[rx] == self.rank[ry]:
self.rank[rx] += 1 # only increase rank on tie
self.n_components -= 1
return True
def connected(self, x, y):
"""O(alpha(n)) amortized."""
return self.find(x) == self.find(y)
def component_count(self):
return self.n_components
def kruskal_mst(n, edges):
"""
Kruskal's MST using Union-Find.
edges: list of (weight, u, v). Returns MST edges.
O(E log E) โ dominated by sorting; Union-Find is O(E * alpha(V)).
"""
edges.sort() # sort by weight
uf = UnionFind(n)
mst = []
for w, u, v in edges:
if uf.union(u, v): # adds edge only if u,v not connected
mst.append((w, u, v))
if len(mst) == n - 1:
break # MST complete
return mst// Union-Find: union by rank + path compression
// Amortized O(alpha(n)) per operation
#include <vector>
#include <numeric>
#include <algorithm>
using namespace std;
struct UnionFind {
vector<int> parent, rank_;
int components;
UnionFind(int n) : parent(n), rank_(n, 0), components(n) {
iota(parent.begin(), parent.end(), 0); // parent[i] = i
}
// Path compression (iterative โ avoids stack overflow for large n)
int find(int x) {
while (parent[x] != x) {
parent[x] = parent[parent[x]]; // path halving (1-pass)
x = parent[x];
}
return x;
}
// Union by rank
bool unite(int x, int y) {
x = find(x); y = find(y);
if (x == y) return false;
if (rank_[x] < rank_[y]) swap(x, y);
parent[y] = x;
if (rank_[x] == rank_[y]) rank_[x]++;
components--;
return true;
}
bool connected(int x, int y) { return find(x) == find(y); }
};
// Kruskal's MST using Union-Find
struct Edge { int u, v, w; };
vector<Edge> kruskal(int n, vector<Edge> edges) {
sort(edges.begin(), edges.end(),
[](auto& a, auto& b){ return a.w < b.w; });
UnionFind uf(n);
vector<Edge> mst;
for (auto& e : edges) {
if (uf.unite(e.u, e.v)) {
mst.push_back(e);
if ((int)mst.size() == n-1) break;
}
}
return mst;
}class BinaryCounter:
"""
Binary counter demonstrating potential method for amortized analysis.
Phi = number of 1-bits.
Amortized cost per increment = O(1), even though actual cost can be O(k).
"""
def __init__(self, bits=16):
self.bits = [0] * bits
self.total_actual_cost = 0
def increment(self):
"""Flip trailing 1s to 0, then flip first 0 to 1."""
i, flips = 0, 0
while i < len(self.bits) and self.bits[i] == 1:
self.bits[i] = 0
i += 1
flips += 1
if i < len(self.bits):
self.bits[i] = 1
flips += 1
self.total_actual_cost += flips
return flips
def phi(self):
"""Potential = number of 1-bits."""
return sum(self.bits)
def verify_amortized(self, n_increments):
"""
Verify: total actual cost <= 2 * n_increments (O(1) amortized).
Amortized cost per increment = actual + delta_Phi = flips + (1 - flips) = 1 + ... = 2.
"""
counter = BinaryCounter()
for i in range(n_increments):
counter.increment()
ratio = counter.total_actual_cost / n_increments
print(f"n={n_increments}: total_cost={counter.total_actual_cost}, "
f"ratio={ratio:.3f} (should be < 2)")
# Verify for several n
for n in [100, 1000, 10000, 100000]:
BinaryCounter().verify_amortized(n)
def potential_method_demo():
"""
Demonstrate potential method for table doubling.
Phi = 2*n - capacity. Show that amortized cost per append = O(1).
"""
arr = DynamicArray()
phi = lambda: 2 * len(arr) - arr.capacity()
prev_phi = phi()
total_amortized = 0
for i in range(32):
prev_phi = phi()
arr.append(i)
new_phi = phi()
actual = 1 if arr.capacity() == (arr.capacity()) else arr.capacity() // 2 + 1
delta = new_phi - prev_phi
amortized = 1 + delta # always approximately 3 or less
total_amortized += abs(amortized)
print(f"Total amortized (32 appends) = {total_amortized}") # โ 96 = 3*32// Binary counter amortized analysis verification
#include <vector>
#include <numeric>
#include <iostream>
using namespace std;
struct BinaryCounter {
vector<int> bits;
long long total_cost = 0;
BinaryCounter(int k=32) : bits(k, 0) {}
int phi() { return count(bits.begin(), bits.end(), 1); }
// Returns actual flip count for this increment
int increment() {
int i = 0, flips = 0;
while (i < (int)bits.size() && bits[i] == 1) {
bits[i++] = 0; ++flips;
}
if (i < (int)bits.size()) { bits[i] = 1; ++flips; }
total_cost += flips;
return flips;
}
};
void verify_amortized(int n) {
BinaryCounter c;
for (int i = 0; i < n; ++i) c.increment();
cout << "n=" << n << " total=" << c.total_cost
<< " ratio=" << (double)c.total_cost/n
<< " (should be < 2)\n";
}
// verify_amortized(100000) -> ratio โ 1.99999During a trading session, TTT’s system needs to track which instruments are “connected” in a spread relationship (e.g., NIFTY and BankNifty move together, or calendar spreads). As new spread relationships are confirmed from market data, we need: (1) add a relationship (UNION), (2) check if two instruments are in the same cluster (FIND), (3) count distinct clusters.
instruments = ['NIFTY', 'BANKNIFTY', 'FINNIFTY', 'MIDCPNIFTY',
'SENSEX', 'BANKEX', 'RELIANCE', 'HDFC']
idx = {s: i for i, s in enumerate(instruments)}
uf = UnionFind(len(instruments))
# Add spread relationships discovered from market data
relationships = [
('NIFTY', 'BANKNIFTY'), # highly correlated
('BANKNIFTY', 'FINNIFTY'), # financial sector cluster
('SENSEX', 'BANKEX'), # BSE cluster
('RELIANCE', 'HDFC'), # bluechip pair
]
for a, b in relationships:
uf.union(idx[a], idx[b])
# Query: same cluster?
print(uf.connected(idx['NIFTY'], idx['FINNIFTY'])) # True โ chain
print(uf.connected(idx['NIFTY'], idx['SENSEX'])) # False โ different exchange
print(f"Distinct clusters: {uf.component_count()}") # 4
# Each union/find: O(alpha(8)) = O(1) time
# Total session: O(m * alpha(n)) for m operations on n instruments
scipy.cluster.hierarchy and networkx connected components both use Union-Find internally with union by rank and path compression. For TTT’s Kruskal-based correlation tree (building a minimum spanning tree of instruments weighted by $1 - |\rho_{ij}|$), the bottleneck is sorting $O(n^2)$ correlation pairs — the Union-Find operations themselves are negligible at $O(\alpha(n))$. The amortization insight matters for the dynamic version: as new ticks arrive and correlations update, UNION and FIND run in near-constant time, allowing real-time clustering of instruments.
For the binary counter with $\Phi = c \cdot (\text{number of 1-bits})$, what value of $c$ makes the amortized cost per increment exactly $O(1)$, and what is that amortized cost?
Incrementing flips $t$ trailing 1-bits to 0 and one 0-bit to 1: actual cost $= t + 1$. $\Delta\Phi = c \cdot (1 - t)$ (one new 1-bit gained, $t$ 1-bits lost). Amortized cost $= (t+1) + c(1-t) = t(1-c) + (1+c)$. For this to be $O(1)$ independent of $t$, we need $c = 1$: amortized $= 0 \cdot t + 2 = 2 = O(1)$. Any $c \geq 1$ also works (gives amortized $\leq 1 + c$), but $c=1$ is the tightest choice.
Why does table halving trigger at $n = m/4$ (quarter full) rather than $n = m/2$ (half full)?
Suppose we halve at $n = m/2$. After halving: capacity $= m/2$, size $= m/2$ — the table is immediately full. The very next push triggers a doubling back to $m$. Then a pop triggers a halving back to $m/2$. Each push/pop costs $\Theta(n)$ — $O(n)$ amortized per operation, breaking our guarantee.
Halving at $n = m/4$ instead: after halving, capacity $= m/2$, size $= m/4$ — the table is only half full. We need $m/4$ more pushes before doubling, or $m/4$ more pops before halving again. Either way, at least $\Omega(m)$ operations before the next resize — restoring the $O(1)$ amortized bound.
In Union-Find with only union by rank (no path compression), what is the worst-case height of a tree after $n$ MAKE-SET and $n-1$ UNION operations?
Key invariant: a tree of rank $k$ contains at least $2^k$ nodes. Proof by induction: a rank-0 tree has 1 node $\geq 2^0$. When two rank-$k$ trees merge to form rank-$(k+1)$: total nodes $\geq 2^k + 2^k = 2^{k+1}$. Since there are at most $n$ nodes total, the maximum rank is $\lfloor\log_2 n\rfloor$. Since rank $\geq$ height (rank is an upper bound on height), height $\leq \log_2 n$.
Without path compression, FIND-SET costs $O(\log n)$ per operation (must walk up the tree of height $\log n$). With path compression added, this drops to $O(\alpha(n))$ amortised.
Why do 2-3 trees fail to give $O(1)$ amortized structural changes for both insert and delete simultaneously, and how do 2-5 trees fix this?
In 2-3 trees: a split of a 3-node creates two 2-nodes. 2-nodes are bad because a future delete can merge two 2-nodes into a 3-node — which is bad for the next insert. In the worst case: insert (split 3-node to two 2-nodes) $\to$ delete (merge two 2-nodes to 3-node) $\to$ insert again ... each operation costs $O(\log n)$ actual structural changes, with potential never dropping enough to cover future costs.
In 2-5 trees: splitting a 5-node creates two 3-nodes (not 2-nodes). Merging two 2-nodes creates a 3-node (not a 5-node). 3-nodes are neither bad for splits (need 5-nodes for that) nor bad for merges (need 2-nodes for that). Potential $\Phi = \text{5-nodes} + \text{2-nodes}$: every structural change either doesn’t create bad nodes or decreases $\Phi$, giving $O(1)$ amortized structural changes per operation. The general rule: $(a,b)$-trees with $b > 2a$ have this property.
Work through independently. Q1–3 direct application. Q4–7 synthesis. Q8–10 careful argument.