Search Algorithms Time Complexity

Understanding the Mathematical Foundations

Analyze BFS, DFS, UCS, and A* complexity with step-by-step proofs

Introduction & Key Notation

Understanding the time complexity of search algorithms is crucial for choosing the right algorithm for different problems. Each algorithm's performance depends on specific characteristics of the search space.

Mathematical Notation
  • b = branching factor (max successors per node)
  • d = depth of shallowest goal node
  • m = maximum depth of search tree
  • C* = cost of optimal solution
  • ε = minimum step cost (ε > 0)
  • h(n) = heuristic estimate from node n to goal
BFS DFS UCS A*
Problem:

Prove that the time complexity of Breadth-First Search (BFS) is O(b^d), where b is the branching factor and d is the depth of the shallowest goal node.

In your proof, explain:

  • How many nodes are at each level of the search tree?
  • What is the total number of nodes expanded?
  • Why is the space complexity also O(b^d)?
✅ Solution

Step-by-step analysis:

At each level of the search tree:

  • Level 0: 1 node (root)
  • Level 1: b nodes
  • Level 2: b² nodes
  • ...
  • Level d: b^d nodes
Total nodes expanded = $1 + b + b^2 + \ldots + b^d \approx b^d$

The geometric series is dominated by its largest term $b^d$, so:

  • Time complexity: O(b^d) - must expand all nodes up to depth d
  • Space complexity: O(b^d) - must store entire frontier at depth d
Key insight: BFS explores level by level, so it must examine all nodes at depth d-1 before finding the goal at depth d.
Problem:

What is the time complexity of Depth-First Search (DFS)? Explain why it differs from BFS.

In your answer, address:

  • How does DFS exploration differ from BFS?
  • What is the worst-case scenario for DFS?
  • How does the space complexity compare to BFS?
✅ Solution

DFS Exploration Pattern:

DFS explores along one branch until maximum depth m, then backtracks.

Worst case: It expands nodes down to depth m before finding the goal.

Nodes generated = $1 + b + b^2 + \ldots + b^m \approx b^m$
DFS Complexity:
  • Time: O(b^m)
  • Space: O(b·m)
BFS Comparison:
  • Time: O(b^d)
  • Space: O(b^d)
Trade-offs:
  • DFS uses much less memory (linear vs exponential)
  • DFS may be slower if m >> d (goal is shallow but tree is deep)
  • DFS may be faster if goal is deep and appears early in search
Problem:

What is the time complexity of Uniform-Cost Search (UCS)? How does it relate to the cost of the optimal solution?

Consider:

  • How does UCS decide which nodes to expand?
  • What nodes will UCS expand before finding the optimal solution?
  • How do step costs affect the complexity?
✅ Solution

UCS Expansion Strategy:

UCS expands nodes in order of path cost g(n).

Worst case: UCS expands all nodes with path cost ≤ C* (cost of optimal solution).

Key insight: If minimum step cost is ε, then maximum depth ≈ C*/ε.

Time Complexity = $O(b^{1 + C^*/\varepsilon})$
Explanation:
  • UCS must expand all nodes with cost < C*
  • Number of such nodes depends on C*/ε (cost layers)
  • Each "cost layer" can have up to b^d nodes
  • Total cost layers ≈ C*/ε
Special Case:

When all step costs = 1, then C*/ε = d, and UCS behaves like BFS with O(b^d).

Problem:

What is the time complexity of A* Search? How does the quality of the heuristic function affect performance?

Analyze:

  • How does A* decide which nodes to expand using f(n) = g(n) + h(n)?
  • What happens with a perfect heuristic vs. a poor heuristic?
  • What are the best-case and worst-case scenarios?
✅ Solution

A* Expansion Strategy:

A* expands nodes in order of f(n) = g(n) + h(n).

Key principle: If h(n) is admissible and consistent, A* is optimal.

Heuristic Quality Time Complexity Explanation
Perfect heuristic
h(n) = true cost		 O(d) Expands only optimal path
Good heuristic
close to true cost		 O(b^δd)
where δ < 1 		Expands subset of nodes
Poor heuristic
h = 0 (like UCS)		 O(b^{C*/ε}) Expands as many nodes as UCS
A* Time Complexity Range: $O(d)$ to $O(b^{C^*/\varepsilon})$
Key insight: A good heuristic can dramatically reduce the search space by guiding the search toward the goal, making A* much more efficient than uninformed search algorithms.
Problem:

Compare the time complexities of BFS, DFS, UCS, and A* in different scenarios. When would you choose each algorithm based on complexity considerations?

Consider these scenarios:

Scenario 1:

Unweighted graph, shallow goal (d << m)

Scenario 2:

Weighted graph, good heuristic available

Scenario 3:

Memory-constrained environment

Scenario 4:

Need guaranteed optimal solution

✅ Solution
Complexity Summary:
Algorithm Time Complexity Space Complexity Optimal?
BFS O(b^d) O(b^d) Yes (unweighted)
DFS O(b^m) O(bm) No
UCS O(b^{C*/ε}) O(b^{C*/ε}) Yes (weighted)
A* O(b^{εd}) to O(b^{C*/ε}) O(b^{εd}) to O(b^{C*/ε}) Yes (with admissible h)
Scenario Analysis:
Scenario 1: Unweighted, shallow goal

Choose BFS: O(b^d) is much better than DFS's O(b^m)

BFS guarantees shortest path and is most efficient when goal is shallow
Scenario 3: Memory-constrained

Choose DFS: O(bm) space vs exponential space for others

Accept potential time penalty for dramatic space savings
Scenario 2: Weighted, good heuristic

Choose A*: Can achieve near O(d) performance

Much more efficient than UCS's O(b^{C*/ε})
Scenario 4: Need optimal solution

Unweighted: BFS - O(b^d)

Weighted, no heuristic: UCS - O(b^{C*/ε})

Weighted, with heuristic: A* - potentially much faster

Key Takeaway: Algorithm choice depends on balancing optimality requirements, available memory, and heuristic information quality.

Summary & Key Takeaways

Core Insights:
  • BFS: Best for unweighted graphs with shallow goals - O(b^d)
  • DFS: Memory-efficient but potentially slow - O(b^m) time, O(bm) space
  • UCS: Optimal for weighted graphs - O(b^{C*/ε})
  • A*: Best with good heuristics - O(d) to O(b^{C*/ε})

Ready for More Practice?

Try implementing these algorithms and measuring their performance

Go to Search Lab Back to Lecture 3