Appendix B. Benchmarks and Datasets
This appendix curates benchmarks and datasets that enable fair, reproducible evaluation of optimization algorithms across continuous, combinatorial, geospatial, and machine-learning contexts. It explains why standardized test functions and problem instances matter, outlines typical properties used to stress algorithms (multimodality, convexity, separability, differentiability), and points to practical ways to obtain or synthesize data for experimentation. The material is geared toward helping practitioners select representative tasks, generate instances, and measure progress consistently.
For continuous optimization, it highlights canonical test functions used to probe solver behavior. Examples include Ackley (flat plateaus with a central basin), Bohachevsky (smooth, convex, separable), Bukin (rugged ridge with many local minima, non-differentiable, favoring derivative-free search), Gramacy & Lee (1-D with mixed local/global structure), and Griewank (many widespread local minima). By contrasting characteristics—convex versus non-convex, separable versus non-separable, multi- versus unimodal—the appendix motivates when gradient-based methods suffice and when black-box metaheuristics are preferable. It also notes that implementations can be coded from scratch or accessed through common Python libraries such as DEAP, pymoo, and PySwarms.
For combinatorial optimization, the appendix compiles widely used benchmark families—TSP, VRP, job-shop scheduling, assembly-line balancing, quadratic assignment, knapsack, set covering, and bin packing—so algorithms can be compared on standard instances. It then turns to geospatial data, describing spatial attributes and variables relevant to real-world planning (locations, infrastructure, demographics, mobility, environment, and emergency response) and illustrating how open sources and APIs such as OpenStreetMap, Overpass, and elevation services can be tapped for optimization inputs. Finally, for machine-learning-driven and neural combinatorial optimization, it notes that “datasets” are often generated problem instances, with examples including convex-hull point sets, TSP instances for sequence-model training, and large mobility datasets like taxi trip records; accompanying examples show how to prepare these for training and evaluation.
B.5 Data Folder
The data folder included in the GitHub repo of the book includes the following sample data:
- ALBP: dataset for assembly line balancing problem covered in Chapter 6.
- AdministrativeBoundaries: administrative boundaries of different regions of interest around the world in geojason format that can be used for map visualization.
- BikeShare: Bike Share Toronto (TBS) ridership data that contains anonymized trip data, including: trip start day and time, trip end day and time, trip duration, trip start station, trip end station and user type (see Listing B.4).
- CanadaTraffic: data from Statistics Canada’s dataset for Motor Vehicle Collisions in 2018 such as fatalities per 100,000 population, fatalities per Billion vehicles-kilometres, injuries per Billion vehicles-kilometres, fatalities per 100,000 licensed drivers and injuries per 100,000 licensed drivers (see Listing A.2).
- OntarioHealth: health region of Ontario, Canada (see Listing A.2).
- Police: Toronto Police Service Public Safety data (see Listing A.2).
- PoliticalDistricting: Political districting data used in Chapter 8.
- PtrNets: convex hull and tsp data used by pointer networks (Chapter 11).
- TSP: travelling salesman problem instances.
FAQ
What are optimization test functions and why are they used?
Optimization test functions (benchmark functions) are mathematical functions used to evaluate and compare optimization algorithms. They expose algorithm strengths and weaknesses under known properties (e.g., convexity, separability, differentiability, multimodality). Common examples include Ackley (flat outer region with a central hole), Bohachevsky (unimodal, convex, separable), Bukin (many local minima, non-differentiable), Gramacy & Lee (1D with multiple local minima), and Griewank (many widespread local minima in 1D–3D).How do properties like separability, convexity, and differentiability affect solver choice?
- Convex + differentiable: gradient-based methods typically perform well (e.g., Bohachevsky).- Non-differentiable or black-box: prefer derivative-free methods (e.g., simulated annealing for Bukin).
- Separable: variables can be optimized independently, simplifying the search.
- Multimodal: algorithms need mechanisms to escape local minima (e.g., population-based or annealing).
When should I use derivative-free (black-box) solvers?
Use derivative-free solvers when the objective is non-differentiable, noisy, or gradients are unavailable/expensive. For instance, the Bukin function is continuous but non-differentiable and multimodal, motivating derivative-free methods such as simulated annealing.Where can I find code examples for optimization test functions?
See Listing B.1 for implementations and references to Python frameworks: DEAP, pymoo, and PySwarms. Link: https://github.com/Optimization-Algorithms-Book/Code-Listings/blob/main/Appendix%20B/Listing%20B.1_Optimization_test_functions.ipynbWhat benchmark datasets are available for the Traveling Salesman Problem (TSP) and Vehicle Routing Problem (VRP)?
- TSP: https://github.com/coin-or/jorlib/tree/master/jorlib-core/src/test/resources/tspLib/tsp- VRP: https://github.com/coin-or/jorlib/tree/master/jorlib-core/src/test/resources/tspLib/vrp and http://neumann.hec.ca/chairedistributique/data/
Where can I get benchmark instances for JSS, ALBP, QAP, Knapsack, SCP, and Bin Packing?
- Job Shop Scheduling (JSS): http://people.brunel.ac.uk/~mastjjb/jeb/orlib/files/jobshop1.txt and http://people.brunel.ac.uk/~mastjjb/jeb/orlib/files/jobshop2.txt- Assembly Line Balancing (ALBP): https://assembly-line-balancing.de/
- Quadratic Assignment (QAP): http://mistic.heig-vd.ch/taillard/problemes.dir/qap.dir/qap.html
- Knapsack: http://people.brunel.ac.uk/~mastjjb/jeb/orlib/mknapinfo.html
- Set Covering (SCP): http://people.brunel.ac.uk/~mastjjb/jeb/orlib/scpinfo.html
- Bin Packing: http://people.brunel.ac.uk/~mastjjb/jeb/orlib/binpackinfo.html
Optimization Algorithms ebook for free