Lecture 7 — Simulated Annealing and Probabilistic Local Search

Escaping local optima, temperature schedules, theory intuition

1 Learning Objectives

By the end of this lecture, you should be able to:

understand why probabilistic acceptance helps escape local optima
explain the core mechanism of simulated annealing
design temperature schedules and acceptance rules
analyze exploration–exploitation trade-offs in probabilistic local search
recognize practical strengths and limitations of simulated annealing

2 Lecture Roadmap

This lecture proceeds in seven blocks:

motivation for probabilistic local search
simulated annealing acceptance mechanism
temperature control and cooling schedules
escaping local optima and phase behavior
neighborhood sampling and scalability
practical tuning, diagnostics, and pitfalls
SA as a component in hybrid optimization systems

3 From Greedy Descent to Probabilistic Search

Pure descent methods such as hill climbing follow a strictly greedy improvement rule: only moves that improve the objective value are accepted. While this guarantees rapid convergence, it also implies inevitable termination at a local optimum, regardless of how poor that solution may be in a global sense.

To address this fundamental limitation, probabilistic local search relaxes the strict descent requirement:

Occasionally accept worse solutions in a controlled manner in order to escape local optima.

By allowing non-improving moves with a certain probability, probabilistic search methods introduce controlled randomness into the search process. This randomness enables the algorithm to move out of locally optimal regions, explore alternative areas of the solution space, and potentially discover better solutions later on.

Crucially, probabilistic local search does not abandon the goal of improvement. Instead, it balances short-term deterioration against long-term progress, maintaining a bias toward better solutions while avoiding premature convergence.

Simulated annealing (SA) is the most classical, well-studied, and widely applied algorithm built on this idea. It extends local search with a principled probabilistic acceptance mechanism that systematically governs when and how worse solutions may be accepted.

4 Intuition from Physics

Simulated annealing is inspired by the annealing process in statistical physics, which is used to obtain low-energy, stable crystalline structures in materials.

The physical process can be summarized as follows:

a material is heated to a high temperature,
atoms gain energy and move freely, exploring many configurations,
the material is cooled slowly and carefully, and
atoms gradually settle into a low-energy, highly ordered structure.

This physical intuition maps naturally to optimization:

candidate solutions correspond to physical states of the system,
the objective value corresponds to the system’s energy, and
temperature controls the amount of randomness in the system and the likelihood of accepting higher-energy states.

At high temperatures, the system explores broadly, accepting many configurations regardless of quality. As the temperature decreases, randomness is reduced, and the system increasingly favors lower-energy states. This gradual cooling motivates a smooth transition from exploration to exploitation, which is precisely the behavior desired in optimization algorithms.

The strength of simulated annealing lies in this analogy: it provides a simple, intuitive, and theoretically grounded mechanism for escaping local optima while guiding the search toward high-quality solutions over time.

5 Simulated Annealing: Core Idea

Simulated annealing extends classical local search by introducing a probabilistic acceptance mechanism that allows the algorithm to move beyond the limitations of greedy descent. Instead of requiring strict improvement at every step, simulated annealing permits controlled acceptance of non-improving moves.

At each iteration of the algorithm:

an improving move is always accepted, ensuring that progress toward better solutions is never blocked,
a worsening move may be accepted with a probability that depends on
- the magnitude of deterioration in the objective value, and
- the current temperature, which regulates the level of randomness.

This mechanism allows the search to temporarily move to worse solutions in order to escape local optima, traverse plateaus, and explore new regions of the solution space. Over time, as randomness is reduced, the algorithm increasingly behaves like a greedy local search, refining high-quality solutions.

6 Acceptance Probability

Let:

\(s\) be the current solution,
\(s'\) be a neighboring solution of \(s\), and
\(\Delta = f(s') - f(s)\) for a minimization problem.

The acceptance rule in simulated annealing is defined as follows:

if \(\Delta \le 0\), accept \(s'\) unconditionally,
if \(\Delta > 0\), accept \(s'\) with probability \[ p = \exp\left(-\frac{\Delta}{T}\right), \]

where \(T > 0\) denotes the temperature parameter.

6.1 Interpretation of the Acceptance Rule

The acceptance probability has several important implications:

Small deteriorations (small \(\Delta\)) are relatively likely to be accepted,
Large deteriorations are accepted with very low probability,
Higher temperatures increase the probability of accepting worse solutions, promoting exploration,
Lower temperatures reduce randomness, causing the algorithm to behave increasingly like greedy descent.

This probabilistic rule provides a smooth and principled way to transition from exploratory behavior in early iterations to exploitative refinement in later stages, which is the defining strength of simulated annealing.

7 Generic Simulated Annealing Algorithm

A generic simulated annealing algorithm can be described by the following steps:

Generate an initial feasible solution \(s\)
- typically produced by a constructive heuristic or random initialization.
Choose an initial temperature \(T_0\)
- high enough to allow frequent acceptance of worsening moves at the start.
Repeat
- Generate a neighbor \(s' \in \mathcal{N}(s)\) using a predefined neighborhood operator,
- Compute the objective difference \[ \Delta = f(s') - f(s), \]
- Accept or reject the move according to the acceptance rule:
  - accept unconditionally if \(\Delta \le 0\),
  - otherwise accept with probability \(\exp(-\Delta/T)\),
- Update the temperature according to the chosen cooling schedule.
Stop when a termination criterion is met
- such as reaching a minimum temperature, exceeding a time limit, or observing no improvement for a fixed number of iterations.

At the end of the algorithm, return the best solution encountered during the entire search, not necessarily the final solution. This practice improves robustness, as simulated annealing may temporarily move to worse solutions during exploration.

7.1 Implementation-Oriented SA Skeleton

An implementation-friendly version of SA is:

Input: initial solution s, temperature T, cooling schedule, iteration budget
best <- s
while stopping criterion not met:
  sample neighbor s' from N(s)
  Delta <- f(s') - f(s)   # minimization
  if Delta <= 0:
    s <- s'
  else:
    accept with probability exp(-Delta / T)
    if accepted: s <- s'
  if f(s) < f(best): best <- s
  update temperature T
return best

This form makes three control points explicit: neighbor sampling, acceptance test, and temperature update.

7.2 Practical Remarks

Simulated annealing typically evaluates one neighbor per iteration, making it scalable to very large neighborhoods.
Neighborhood generation is often randomized to ensure sufficient exploration.
The performance of the algorithm is highly sensitive to the choice of temperature parameters and cooling schedule.

This generic framework can be adapted to a wide range of optimization problems by customizing the solution representation, neighborhood operators, and cooling strategy.

7.3 Numerical and Implementation Notes

Practical implementations should account for the following:

guard against numerical underflow when \(T\) is very small and \(\Delta/T\) is large,
avoid letting temperature reach exactly zero before termination,
seed random number generators for reproducible experiments,
track both current and best-so-far objectives during the run.

These details do not change the algorithm conceptually but strongly affect reproducibility and reliability.

8 Temperature and Cooling Schedules

8.1 Role of Temperature

The temperature parameter is the central control variable in simulated annealing. It regulates the level of randomness in the search process and directly determines the balance between exploration and exploitation:

High temperature → strong exploration
- many worsening moves are accepted,
- the search explores diverse regions of the solution space.
Low temperature → greedy exploitation
- worsening moves are rarely accepted,
- the algorithm behaves similarly to hill climbing.

At very high temperatures, simulated annealing approximates a random walk, largely ignoring the objective value. At very low temperatures, it effectively reduces to a deterministic local search method.

The gradual reduction of temperature is what allows simulated annealing to transition smoothly from global exploration to local refinement.

8.2 Cooling Schedules

A cooling schedule specifies how the temperature is reduced over time. The choice of cooling schedule strongly influences convergence speed, solution quality, and robustness.

8.2.1 Geometric Cooling (Most Common in Practice)

\[ T_{k+1} = \alpha T_k, \quad 0 < \alpha < 1 \]

simple to implement and widely used,
provides a smooth and controllable temperature decrease,
typical values: \(\alpha \in [0.90, 0.99]\).

Geometric cooling offers a good compromise between exploration and runtime efficiency, which explains its popularity in practical applications.

8.2.2 Linear Cooling

\[ T_{k+1} = T_k - \beta \]

conceptually simple and easy to interpret,
temperature decreases at a constant rate.

However, linear cooling is highly sensitive to the choice of \(\beta\). If \(\beta\) is too large, the algorithm cools too quickly and behaves greedily. If too small, convergence becomes excessively slow.

8.2.3 Logarithmic Cooling (Theoretical Schedule)

\[ T_k = \frac{c}{\log(k+1)} \]

guarantees convergence to a global optimum under idealized assumptions,
provides strong theoretical justification for simulated annealing.

In practice, logarithmic cooling is rarely used because it cools extremely slowly, requiring an impractically large number of iterations to reach low temperatures.

8.3 Practical Considerations

faster cooling schedules reduce runtime but increase the risk of premature convergence,
slower cooling improves solution quality but may be computationally expensive,
effective simulated annealing implementations often rely on empirical tuning.

Selecting a cooling schedule is therefore a balance between theoretical guarantees and practical performance, guided largely by problem structure and computational constraints.

8.4 Choosing the Initial Temperature

In practice, a useful calibration goal is to start with a moderate-to-high acceptance rate for worsening moves (often around 0.6 to 0.9 on sampled deltas).

A common workflow:

sample a batch of random neighbor moves from initial solutions
estimate typical positive deterioration values \(\Delta>0\)
choose \(T_0\) so that \(\exp(-\Delta/T_0)\) gives the desired early acceptance behavior

This data-driven initialization is usually more reliable than selecting \(T_0\) arbitrarily.

8.5 Cooling-Rate Trade-Off

The geometric factor \(\alpha\) in \(T_{k+1}=\alpha T_k\) controls search behavior strongly:

smaller \(\alpha\) (faster cooling) gives shorter runtime but higher risk of premature greediness,
larger \(\alpha\) (slower cooling) improves exploration and often quality, but increases computation.

In practice, tuning \(\alpha\) is equivalent to tuning the exploration budget.

9 Escaping Local Optima

Simulated annealing escapes local optima through a principled relaxation of greedy decision making. In contrast to hill climbing, which halts as soon as no improving move exists, simulated annealing continues the search by:

allowing uphill (worsening) moves early in the search, enabling the algorithm to cross barriers surrounding local optima,
gradually reducing randomness as the temperature decreases, limiting acceptance of poor moves over time, and
transitioning smoothly from exploration to exploitation, rather than switching abruptly.

This controlled deterioration mechanism allows the algorithm to move out of locally optimal regions, traverse plateaus, and explore distant areas of the solution space before committing to refinement. As a result, simulated annealing significantly reduces the risk of premature convergence while still achieving strong local improvement in later stages.

10 Exploration–Exploitation Trade-Off in Simulated Annealing

The behavior of simulated annealing can be naturally divided into phases governed by temperature.

10.1 Early Phase: Exploration

temperature is high,
many worsening moves are accepted,
the search explores the solution space broadly,
dependence on the initial solution is reduced.

In this phase, the algorithm prioritizes diversification and structural exploration over solution quality.

10.2 Late Phase: Exploitation

temperature is low,
worsening moves are rarely accepted,
the algorithm behaves similarly to hill climbing,
strong intensification around high-quality solutions occurs.

In this phase, the focus shifts to fine-grained improvement and convergence.

Designing an effective simulated annealing algorithm requires carefully balancing these phases. Excessive exploration leads to slow convergence, while premature exploitation increases the risk of getting trapped in poor local optima.

11 Neighborhood Design in Simulated Annealing

Simulated annealing uses the same neighborhood concepts introduced in local search:

binary flip moves,
swap or insertion moves in permutations,
reassignment moves in allocation problems.

However, simulated annealing differs in how neighborhoods are explored:

neighbors are sampled randomly, rather than exhaustively evaluated,
typically one neighbor is evaluated per iteration,
full neighborhood enumeration is avoided.

This sampling-based approach makes simulated annealing highly scalable, even when neighborhoods are extremely large or complex. The algorithm relies on probabilistic acceptance rather than exhaustive comparison to guide the search.

12 Theoretical Intuition

Under idealized assumptions, simulated annealing has strong theoretical properties. In particular, if:

runtime is infinite,
a logarithmic cooling schedule is used, and
the neighborhood graph is fully connected,

then simulated annealing converges to a global optimum with probability one.

Although these assumptions are never satisfied in practice, the theory provides important intuition:

slower cooling increases solution quality,
overly greedy behavior limits exploration, and
randomness must be reduced gradually rather than abruptly.

These insights guide practical parameter tuning, even when theoretical guarantees cannot be achieved.

13 Diagnostics and Tuning in Practice

To tune SA effectively, monitor:

acceptance ratio of worsening moves over time,
best-objective trajectory versus iteration/time,
stagnation periods (long intervals without best-solution updates),
variability across random seeds.

If acceptance drops too quickly, cooling is likely too aggressive. If acceptance remains high late in the run, cooling is likely too slow.

14 Practical Strengths

Simulated annealing remains popular in practice because it:

is conceptually simple and easy to implement,
applies naturally to both discrete and continuous problems,
requires little problem-specific customization,
effectively escapes local optima and plateaus.

It has been successfully applied in scheduling, layout optimization, routing, network design, and many other combinatorial optimization problems.

15 Practical Limitations

Despite its strengths, simulated annealing has several limitations:

performance is sensitive to temperature parameters and cooling schedules,
overly conservative cooling leads to slow convergence,
no guarantee of optimality exists in finite time,
difficult landscapes may still cause stagnation or slow progress.

For these reasons, simulated annealing is often embedded within hybrid algorithms, where it is combined with other heuristics, local search strategies, or population-based methods to improve robustness and performance.

15.1 Common Failure Modes and Fixes

Failure mode	Typical cause	Practical fix
behaves like hill climbing too early	initial temperature too low or cooling too fast	increase \(T_0\) and/or use slower cooling
random-walk behavior too long	temperature remains too high	speed up cooling or tighten iteration budget per temperature
high run-to-run variance	insufficient runtime or unstable parameterization	use multi-seed evaluation and robust parameter ranges
poor final refinement	cooling stops too early	extend low-temperature phase or add final local search pass

16 Simulated Annealing in Hybrid and Modern Algorithms

In contemporary soft computing and optimization practice, simulated annealing is rarely viewed as an isolated technique. Instead, it is commonly used as a building block within more powerful hybrid and composite algorithms.

Simulated annealing is frequently employed as:

a standalone optimizer
- particularly for problems where simplicity, flexibility, and robustness are desired,
- useful when problem structure is not well understood or rapidly changes.
a diversification mechanism within larger frameworks
- used to escape local optima generated by greedy or deterministic methods,
- injected periodically to perturb the search and explore new regions.
a post-processing or refinement phase
- applied after another algorithm produces a good solution,
- refines solutions by controlled exploration around high-quality regions.

16.1 Common Hybridizations

Simulated annealing is often combined with other heuristic and metaheuristic approaches to leverage complementary strengths:

Hill climbing
- hill climbing provides fast intensification,
- simulated annealing introduces probabilistic moves to escape local optima.
Tabu search
- tabu mechanisms prevent cycling and exploit memory,
- simulated annealing adds controlled randomness and diversification.
Genetic algorithms
- genetic operators provide population-based exploration,
- simulated annealing or local search improves individual solutions
- such hybrids are often referred to as memetic algorithms.
Large neighborhood search
- simulated annealing controls acceptance of large, disruptive moves,
- allows effective exploration of complex neighborhoods.

16.2 Practical Perspective

From a practical standpoint, simulated annealing is best understood not as a competitor to other methods, but as a flexible probabilistic search component. Its acceptance mechanism and temperature control can be adapted to guide search behavior at different stages of an algorithm, making it a versatile tool in modern hybrid optimization systems.

16.3 Key Takeaways

simulated annealing relaxes greedy acceptance rules
probabilistic acceptance enables escape from local optima
temperature controls randomness and exploration
cooling schedules determine convergence behavior
SA bridges local search and global exploration

17 Mini Exercises

Simulated annealing vs hill climbing Implement a simple simulated annealing algorithm for a binary optimization problem (for example, a knapsack or selection problem).
- Compare its performance with hill climbing in terms of solution quality and runtime.
- Analyze how often simulated annealing escapes local optima that trap hill climbing.
Effect of cooling schedules Experiment with different cooling schedules (geometric, linear, and aggressive fast cooling).
- Observe how the choice of schedule affects convergence speed.
- Compare the final solution quality obtained under different schedules.
Behavior at extreme temperatures Explain the behavior of simulated annealing when:
- the temperature is extremely high, and
- the temperature is extremely low. Relate these behaviors to random search and greedy descent.
Theory versus practice Discuss why logarithmic cooling schedules are theoretically appealing.
- Explain the assumptions under which global convergence is guaranteed.
- Analyze why these schedules are impractical for real-world optimization problems.
Acceptance-probability calculation For a minimization problem, suppose a move has deterioration \(\Delta=7\).
- Compute the acceptance probability at temperatures \(T=20\), \(T=5\), and \(T=1\).
- Interpret how the probabilities reflect exploration versus exploitation.
- Explain what this implies for schedule design over time.

18 References for This Chapter

Kirkpatrick, S., Gelatt, C. D., & Vecchi, M. P. (1983). Optimization by simulated annealing. Science, 220(4598), 671-680. DOI: 10.1126/science.220.4598.671
Cerny, V. (1985). Thermodynamical approach to the traveling salesman problem: An efficient simulation algorithm. Journal of Optimization Theory and Applications, 45(1), 41-51. DOI: 10.1007/BF00940812
Geman, S., & Geman, D. (1984). Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6(6), 721-741. DOI: 10.1109/TPAMI.1984.4767596
Hajek, B. (1988). Cooling schedules for optimal annealing. Mathematics of Operations Research, 13(2), 311-329. DOI: 10.1287/moor.13.2.311
Johnson, D. S., Aragon, C. R., McGeoch, L. A., & Schevon, C. (1989). Optimization by simulated annealing: An experimental evaluation; part I, graph partitioning. Operations Research, 37(6), 865-892. DOI: 10.1287/opre.37.6.865
Johnson, D. S., Aragon, C. R., McGeoch, L. A., & Schevon, C. (1991). Optimization by simulated annealing: An experimental evaluation; part II, graph coloring and number partitioning. Operations Research, 39(3), 378-406. DOI: 10.1287/opre.39.3.378
Aarts, E., & Korst, J. (1988). Simulated Annealing and Boltzmann Machines. Wiley. URL: https://onlinelibrary.wiley.com/doi/book/10.1002/9780470172209