Lecture 8 — Tabu Search and Memory-Based Heuristics

Short-term and long-term memory, tabu lists, aspiration

1 Learning Objectives

By the end of this lecture, you should be able to:

  • understand the motivation for memory-based search
  • explain the core components of tabu search
  • distinguish short-term and long-term memory mechanisms
  • design tabu lists and aspiration criteria
  • analyze intensification and diversification in tabu search

2 Lecture Roadmap

This lecture proceeds in eight blocks:

  1. motivation for memory-based search
  2. tabu search core loop and admissible move logic
  3. short-term memory, tabu lists, and tenure effects
  4. aspiration criteria and override logic
  5. move-selection strategy and computational implications
  6. long-term memory for diversification
  7. tuning, diagnostics, and practical failure modes
  8. comparison and hybridization with simulated annealing

5 Generic Tabu Search Framework

A generic tabu search algorithm can be described as a structured extension of local search, augmented with memory-based control mechanisms. The high-level procedure is as follows:

  1. Generate an initial feasible solution \(s\)

    • typically obtained via a constructive heuristic or simple local search,
    • feasibility is maintained throughout the search.
  2. Initialize tabu memory structures

    • create and empty tabu lists or attribute memories,
    • set parameters such as tabu tenure and aspiration rules.
  3. Repeat

    • Generate the neighborhood \(\mathcal{N}(s)\) of the current solution,
    • Identify tabu moves or attributes and temporarily forbid them,
    • Apply aspiration criteria to allow tabu moves that meet predefined quality thresholds,
    • Select the best admissible move, even if it is non-improving,
    • Update the current solution by applying the selected move,
    • Update tabu memory, recording attributes of the executed move and expiring old tabu entries.
  4. Stop when a termination criterion is met

    • such as a maximum number of iterations,
    • a time limit, or
    • lack of improvement over a specified number of iterations.

At the end of the algorithm, return the best solution encountered during the entire search, not necessarily the final solution. This is important because tabu search may temporarily move to worse solutions as part of its exploration strategy.


5.1 Implementation-Oriented Tabu Search Skeleton

Input: initial solution s, tabu tenure tau, stopping criterion
best <- s
tabu_memory <- empty
while not stop:
  generate candidate moves from N(s)
  filter tabu moves unless aspiration is satisfied
  choose best admissible move m*
  apply m* to obtain s
  update tabu_memory with attributes of m*
  expire entries older than tau
  if f(s) better than f(best): best <- s
return best

This structure makes the main control points explicit: candidate generation, admissibility filtering, move selection, and memory updates.


5.2 Key Observations

  • Tabu search always selects the best admissible move, ensuring strong intensification at each iteration.
  • The use of tabu restrictions prevents short-term cycling while encouraging exploration of new regions.
  • Allowing non-improving moves enables escape from local optima without relying on randomness.

This framework is highly adaptable: the definition of neighborhoods, tabu attributes, tenure lengths, and aspiration criteria can be customized to suit a wide variety of optimization problems, making tabu search one of the most flexible and powerful metaheuristic paradigms.


7 Short-Term Memory and Tabu Lists

7.1 Tabu Lists

The most recognizable and widely used component of tabu search is the tabu list, which implements short-term memory.

A tabu list:

  • records information about recent moves or solution attributes,
  • temporarily forbids these items by declaring them tabu, and
  • automatically removes tabu status after a predefined number of iterations, known as the tabu tenure.

The primary purpose of the tabu list is to prevent cycling, that is, to avoid the search repeatedly undoing recent moves or oscillating among a small set of solutions. By enforcing temporary prohibitions, tabu search is forced to explore new configurations even when the best local move would revert a recent decision.


7.2 What Can Be Tabu?

The specific content of the tabu list depends on the problem structure and the chosen representation. Common options include:

  • Moves

    • forbidding the reversal of a recent move,
    • for example, preventing a binary variable that was just flipped from being flipped back immediately.
  • Attributes

    • forbidding certain solution characteristics rather than entire moves,
    • for example, preventing an item from being assigned to a recently used position or facility.
  • Solutions

    • forbidding complete solutions that were recently visited,
    • rarely used in large-scale problems due to excessive memory requirements.

In practice, attribute-based tabu restrictions are the most common and scalable approach. They generalize well, require limited memory, and effectively prevent short-term cycling without overly constraining the search.


7.3 Tabu Tenure

The tabu tenure specifies how long a move or attribute remains tabu after it is applied.

  • Short tenure

    • provides weak protection against cycling,
    • may allow the search to revert too quickly to previously visited solutions.
  • Long tenure

    • enforces stronger diversification,
    • but risks forbidding useful or improving moves for too long.

Tabu tenure can be implemented as:

  • fixed, using a constant number of iterations, or
  • adaptive, where the tenure changes dynamically based on search behavior, progress, or problem size.

Selecting an appropriate tabu tenure is critical for performance. Too small a value reduces the effectiveness of tabu search, while too large a value may overly restrict the search and slow convergence. Balancing tabu tenure is therefore a key design decision in practical tabu search implementations.


7.4 Tuning Tabu Tenure in Practice

A practical rule is to relate tenure to neighborhood size and problem dimension, then tune empirically.

Common behaviors:

  • tenure too small:

    • frequent backtracking and short cycles,
    • weak diversification.
  • tenure too large:

    • excessive move prohibition,
    • slow local refinement and possible stagnation.

Practical workflow:

  1. start with a moderate fixed tenure
  2. monitor cycling and stagnation indicators
  3. increase tenure when cycling dominates
  4. decrease tenure when progress becomes too constrained

Many robust implementations use adaptive tenure windows rather than a single fixed value.


8 Aspiration Criteria

In tabu search, tabu restrictions are intentionally temporary and conditional, not absolute. An aspiration criterion provides a mechanism to override tabu status when enforcing the restriction would be counterproductive.

The most common and widely used aspiration rule is:

A tabu move is allowed if it produces a solution that is better than any solution seen so far.

This rule ensures that tabu restrictions never block significant progress toward high-quality solutions. If a move leads to a new global best solution, it should be accepted regardless of its tabu status.


8.1 Purpose of Aspiration Criteria

Aspiration criteria serve several important roles:

  • Preventing over-restriction

    • ensures that tabu lists do not prohibit genuinely beneficial moves,
    • avoids stagnation caused by overly aggressive tabu tenure.
  • Preserving intensification

    • allows the search to exploit promising regions fully,
    • ensures that the best solutions remain accessible.
  • Balancing control and flexibility

    • tabu restrictions guide the search,
    • aspiration criteria restore flexibility when needed.

8.2 Alternative Aspiration Rules

While global-best aspiration is the most common, other aspiration criteria may also be used:

  • allow a tabu move if it improves upon the current solution by a specified margin,
  • allow tabu moves after a certain number of iterations without improvement,
  • allow tabu moves that satisfy problem-specific quality or feasibility thresholds.

These variants provide additional control and can be tailored to specific problem structures.


8.3 Practical Insight

Aspiration criteria are essential for making tabu search robust. Without them, tabu restrictions could inadvertently prevent the algorithm from reaching high-quality solutions. With well-designed aspiration rules, tabu search maintains both discipline and adaptability, ensuring steady progress throughout the search.


11 Long-Term Memory and Diversification

11.1 Motivation

While short-term memory in tabu search is effective at preventing immediate cycling and supporting local intensification, it does not by itself ensure that the search explores different regions of the solution space. Over time, the search may still become confined to a limited area, repeatedly visiting structurally similar solutions.

Long-term memory is introduced to counteract this tendency by explicitly encouraging diversification. Its role is strategic rather than tactical: instead of controlling the next few moves, it influences the overall trajectory of the search.


11.2 Role of Long-Term Memory

Long-term memory mechanisms are designed to:

  • identify regions of the solution space that have been overexplored,
  • detect regions that have been rarely or never visited,
  • bias future search decisions toward underexplored structures, and
  • maintain a balance between exploration and exploitation over long time horizons.

Unlike short-term memory, which is typically iteration-based, long-term memory accumulates information over many iterations.


11.3 Forms of Long-Term Memory

Several forms of long-term memory are commonly used in tabu search.

11.3.1 Frequency-Based Memory

Frequency-based memory records how often certain solution attributes or moves appear during the search.

Typical attributes include:

  • assignment of an item to a location,
  • activation of a facility or resource,
  • selection of a particular edge or arc in routing problems.

Diversification is encouraged by:

  • penalizing frequently used attributes in the objective function, or
  • favoring moves that introduce rarely used attributes.

This approach systematically pushes the search away from familiar patterns and toward novel configurations.


11.3.2 Elite Solution Memory

Elite solution memory stores a collection of the best solutions found so far.

These elite solutions are used to:

  • intensify the search by restarting from high-quality solutions,
  • recombine features of good solutions, or
  • guide diversification by comparing current solutions against elite structures.

Elite memory provides a long-term reference that preserves valuable information discovered during the search.


11.3.3 Strategic Oscillation

Strategic oscillation deliberately allows the search to move between:

  • feasible solutions, and
  • controlled infeasible solutions.

By temporarily relaxing constraints, the search can:

  • cross infeasible regions that separate distant feasible basins,
  • escape from regions blocked by strict feasibility requirements, and
  • discover high-quality feasible solutions unreachable by purely feasible moves.

Strategic oscillation is particularly effective in constrained optimization problems.


11.4 Diversification Strategies in Practice

Long-term memory supports diversification through several practical strategies:

  • Penalty adjustment

    • dynamically increase penalties for frequently used attributes,
    • reduce penalties for rare or unused attributes.
  • Guided restarts

    • restart the search from solutions constructed using rarely used components,
    • avoid purely random restarts.
  • Adaptive tabu tenure

    • increase tabu tenure when cycling or stagnation is detected,
    • decrease tenure during intensification phases.

These strategies ensure that diversification is guided and informed, rather than random.


11.5 Interaction with Intensification

Effective tabu search alternates between:

  • intensification, driven mainly by short-term memory and elite solutions, and
  • diversification, driven by long-term memory mechanisms.

Long-term memory ensures that intensification does not become excessive and that the search periodically explores new regions where better solutions may exist.


11.6 Summary

Long-term memory is essential for making tabu search a truly global optimization method. It:

  • prevents long-term stagnation,
  • promotes systematic exploration of the solution space,
  • complements short-term memory and intensification, and
  • enables tabu search to scale to large and complex optimization problems.

By integrating short-term and long-term memory, tabu search achieves a powerful balance between depth and breadth in the search process.


12 Tabu Search vs Simulated Annealing

Tabu search and simulated annealing are two classical metaheuristics designed to overcome the limitations of greedy local search. While both allow non-improving moves and aim to escape local optima, they rely on fundamentally different mechanisms.

Aspect Tabu Search Simulated Annealing
Acceptance of worse moves Deterministic (best admissible move) Probabilistic (temperature-based)
Use of memory Explicit and structured None (memoryless)
Control of cycling Tabu lists and tenure Randomness
Exploration mechanism Memory-guided diversification Stochastic acceptance
Exploitation mechanism Intensification via best moves Gradual cooling
Parameter sensitivity Moderate (tabu tenure, memory rules) High (initial temperature, cooling rate)
Typical behavior Strategic exploration Stochastic exploration
Reproducibility High (often deterministic) Lower (randomized runs differ)

12.1 Key Conceptual Differences

Tabu search uses deterministic decision-making guided by memory. At each iteration, it selects the best admissible move, even if that move worsens the objective value. Memory structures prevent cycling and steer the search toward unexplored regions in a controlled manner.

Simulated annealing, in contrast, relies on randomized acceptance. Worse moves are accepted with a probability that decreases over time, controlled by the temperature schedule. There is no explicit memory of past decisions, and diversification emerges implicitly through randomness.


12.2 Strengths in Practice

  • Tabu search is particularly effective when:

    • neighborhoods are well structured,
    • cycling is a major concern,
    • deterministic behavior and reproducibility are desired.
  • Simulated annealing is particularly effective when:

    • the landscape contains many deep local optima,
    • a simple implementation is preferred,
    • problem-specific memory structures are difficult to design.

12.3 Complementarity

In modern soft computing practice, tabu search and simulated annealing are often combined rather than compared. For example:

  • simulated annealing can be used to diversify the search before applying tabu search for intensification,
  • tabu search can be embedded inside a probabilistic framework, or
  • both can be used as components of larger hybrid or matheuristic algorithms.

Together, they illustrate two complementary philosophies of escaping local optima: memory-driven strategic control versus probability-driven stochastic exploration.


14 Practical Limitations

Despite its effectiveness, tabu search is not without challenges:

  • Design complexity Choosing appropriate tabu attributes (moves, variables, assignments) requires problem-specific insight.

  • Parameter tuning Tabu tenure, memory length, and aspiration rules strongly influence performance and often require experimentation.

  • Increased algorithmic complexity Long-term memory, diversification strategies, and adaptive tenure mechanisms add implementation overhead.

  • Scalability of memory structures Poorly designed memory schemes can increase computational cost or unintentionally restrict the search.

Because of these factors, tabu search is rarely used as a completely standalone method in large systems.


14.1 Diagnostics and Failure Modes

Useful runtime diagnostics:

  • repeated-solution frequency (cycling signal),
  • best-objective improvement rate,
  • proportion of moves blocked by tabu restrictions,
  • distribution of attribute frequencies (diversification signal).

Common failure modes and fixes:

Failure mode Typical cause Practical fix
persistent short cycling tenure too short increase tenure or strengthen attribute-based tabu
search over-constrained tenure too long or overly broad tabu attributes relax tenure or narrow tabu definition
weak diversification short-term memory only add long-term frequency penalties or guided restarts
high runtime per iteration full neighborhood evaluation too expensive introduce candidate lists / partial evaluation

15 Role of Tabu Search in Modern Soft Computing

Tabu search occupies a central position in the evolution of soft computing methods:

  • it established memory-based control as a core optimization principle,
  • it inspired later metaheuristics that incorporate history and learning,
  • it demonstrated that deterministic strategies can compete with stochastic methods, and
  • it serves as a powerful intensification engine within hybrid frameworks.

In modern practice, tabu search is frequently embedded within larger algorithms, combined with local search, evolutionary methods, or exact solvers. Understanding tabu search provides deep insight into how strategic memory and controlled exploration can dramatically improve heuristic optimization performance.


15.1 Key Takeaways

  • tabu search enhances local search with memory
  • tabu lists prevent cycling and encourage exploration
  • aspiration criteria override overly restrictive tabu rules
  • short-term memory intensifies search
  • long-term memory enables diversification

16 Mini Exercises

  1. Tabu list design Consider a binary optimization problem with decision variables \(x \in \{0,1\}^n\).

    • Define an appropriate tabu list (move-based or attribute-based).
    • Explain which recent moves or variable changes should be declared tabu and why this prevents cycling.
  2. Effect of tabu tenure

    • Describe how a short tabu tenure versus a long tabu tenure affects the search trajectory.
    • Discuss the impact of tabu tenure on intensification, diversification, and convergence speed.
  3. Comparison with simulated annealing

    • Compare tabu search and simulated annealing in terms of how they escape local optima.
    • Focus on the roles of memory, determinism, and randomness in their exploration mechanisms.
  4. Aspiration criteria design

    • Propose an aspiration criterion for a routing or scheduling problem.
    • Explain under what conditions a tabu move should be allowed and how this criterion balances flexibility and control.
  5. Candidate-list design and trade-off

    • Propose a candidate-list strategy for a large neighborhood routing or scheduling problem.
    • Explain how candidate-list size affects runtime and solution quality.
    • Suggest one adaptive rule for changing candidate-list size during search.

17 References for This Chapter

  1. Glover, F. (1989). Tabu Search-Part I. ORSA Journal on Computing, 1(3), 190-206. DOI: 10.1287/ijoc.1.3.190
  2. Glover, F. (1990). Tabu Search-Part II. ORSA Journal on Computing, 2(1), 4-32. DOI: 10.1287/ijoc.2.1.4
  3. Glover, F., & Laguna, M. (1997). Tabu Search. Springer. DOI: 10.1007/978-1-4615-6089-0
  4. Taillard, E. D. (1991). Robust taboo search for the quadratic assignment problem. Parallel Computing, 17(4-5), 443-455. DOI: 10.1016/S0167-8191(05)80147-4
  5. Hertz, A., & de Werra, D. (1987). Using tabu search techniques for graph coloring. Computing, 39(4), 345-351. DOI: 10.1007/BF02239976
  6. Battiti, R., & Tecchiolli, G. (1995). Training neural nets with the reactive tabu search. IEEE Transactions on Neural Networks, 6(5), 1185-1200. DOI: 10.1109/72.410361
  7. Gendreau, M., & Potvin, J.-Y. (Eds.). (2019). Handbook of Metaheuristics (3rd ed.). Springer. DOI: 10.1007/978-3-319-91086-4
Back to top