A Genetic Programming Hyper-Heuristic for the Multidimensional Knapsack Problem

—Hyper-heuristics are a class of high-level search techniques which operate on a search space of heuristics rather than directly on a search space of solutions. Early hyper-heuristics focussed on selecting and applying a low-level heuristic at each stage of a search. Recent trends in hyper-heuristic research have led to a number of approaches being developed to automatically generate new heuristics from a set of heuristic components. This work investigates the suitability of using genetic programming as a hyper-heuristic methodology to generate constructive heuristics to solve the multidimensional 0-1 knapsack problem. A population of heuristics to rank knapsack items are trained on a subset of test problems and then applied to unseen instances. The results over a set of standard benchmarks show that genetic programming can be used to generate constructive heuristics which yield human-competitive results.


I. INTRODUCTION
Many optimisation problems create a search space which is too large to enumerate and exhaustively search for an optimal solution.A large number of heuristics and meta-heuristics have been successfully applied to such NP-hard problems.A weakness of these approaches is the need to manually adapt the method used in order to solve different problem domains or classes of problem.Hyper-heuristics are an emerging class of high-level search techniques designed to automate the heuristic design process and raise the level of generality at which search methods operate [1].Hyper-heuristics operate on a search space of heuristics unlike traditional computational search methods which operate directly on a search space of solutions.Hyper-heuristic research is driven by the desire to provide more general search methods which select and create methods for solving problems rather than searching directly for solutions.
Hyper-heuristics can be broadly split into two main categories, those methodologies which select a low-level heuristic to apply at a given point in the search and those methodologies which create new heuristics from a set of low-level components [2].Here we will be concerned with the latter of these two categories.A motivation for this area of research is that a heuristic can be automatically specialised to a given class of problems, with certain characteristics.It is often prohibitively expensive to manually tune a heuristic methodology for each new class of problem instances, but doing so would produce better results.If we can research methods to successfully automate that design process, then much better results can potentially be obtained, with no extra human effort.Genetic programming is a standard evolutionary computation technique which has been successfully employed to automatically generate heuristics for a number of NP-hard [3] combinatorial optimisation problems [4], [5], [6], [7], [8], [9], [10].The multidimensional 0-1 knapsack problem is a standard NP-hard problem which is derived from real-world applications such as capital budgeting [11] and project selection [12].
In this paper, we investigate the suitability of genetic programming to evolve reusable constructive heuristics for the multidimensional 0-1 knapsack problem.We compare the performance of the automatically generated heuristics to human-designed constructive heuristics and meta-heuristics from the literature over a set of standard benchmark instances.
Section II provides an overview of hyper-heuristics in general and genetic programming hyper-heuristic in particular.Then the multidimensional 0-1 knapsack problem is described in section III.Section IV provides the experimental design and settings of genetic programming for solving the problem which is followed by section V discussing the results of the computational experiments.Finally, section VI concludes our work.

II. HYPER-HEURISTICS
Denzinger et al. [13] first used the term 'hyper-heuristic' to describe a system which selects and combines a number of artificial intelligence methods.Although this was when the term 'hyper-heuristic' originated, the idea of operating on a search space of heuristics can be traced back to the early 1960's.Fisher and Thompson [14] showed that combining rules for job shop scheduling could yield better results than taking any single individual rule.Cowling et al. [15] introduced the term in the field of combinatorial optimisation and defined hyper-heuristics as 'heuristics to choose heuristics'.A more recent definition was provided by Burke et al. [2], [16] to include hyper-heuristics which generate new heuristics from components of existing heuristics: 'A hyper-heuristic is a search method or learning mechanism for selecting or generating heuristics to solve computational search problems.'

A. A classification of hyper-heuristic approaches
Burke et al. [2] outline two main categories of hyperheuristics; heuristic selection methodologies and heuristic generation methodologies.Heuristic selection methodologies select a low-level heuristic to apply at a given point in the search space.Heuristic generation methodologies automatically generate new heuristics from a set of low-level components or building blocks.In either case, the set of low-level heuristics being selected or generated can either be further split to distinguish between those which construct solutions from scratch (constructive) and those which modify an existing solution (perturbative) (see [17] for more).As well as the nature of the search space, hyper-heuristics can learn from feedback concerning heuristic performance throughout the search process.Hyper-heuristics which utilise online learning continuously adapt throughout the search process based on the feedback they receive.Hyper-heuristics using offline learning train a hyper-heuristic on a subset of instances before being applied to a larger set of unseen instances.

B. Genetic programming as a hyper-heuristic
Genetic programming is one of the more recently developed classes of evolutionary algorithms proposed by Koza [18].Unlike traditional forms of evolutionary computation, populations of computer programs usually expressed as tree structures are evolved.Rather than producing fixed-length encoded representations of candidate solutions to a given problem, the evolved program itself, when executed, is the solution.Burke et al. [19] outline the suitability of genetic programming as a hyper-heuristic to generate new heuristics and survey previous work attempting to create heuristics using genetic programming.One of the advantages highlighted is that genetic programming relies on expert knowledge to define its terminal and function sets.As human expert knowledge is necessary, domain specific information can be incorporated into the fundamental components of the system.A second advantage is that other methods (such as genetic algorithms) may restrict the length of an encoded solution in order to facilitate simple genetic operators, genetic programming trees have variable length representation.This can be useful if the best length encoding for heuristic representation is not known.Finally, genetic programming can be used to evolve trees as executable programs allowing low-level heuristics to be generated directly.
Genetic programming has successfully been used to evolve new constructive heuristics comparable to human designed heuristics for a number of problem domains.Burke et al. [4], [5] showed that stand-alone heuristics generated using genetic programming could outperform the human designed 'bestfit' heuristic from the literature on unseen instances of the same class of one dimensional bin packing problems.This work was extended to three dimensional bin packing by Allen at al. [20] and generalised by Burke et al. [6] to include one, two and three dimensional bin packing problems, again obtaining human competitive results.A similar method was presented by Burke et al. [21] for two dimensional strip packing problems.Bader-El-Din and Poli [8] used genetic programming to quickly generate 'disposable' heuristics to solve the satisfiability problem.Again, this work generated heuristics comparable to those which were human designed.However, only a limited search space of heuristics was covered.Kumar et al. [22] used genetic programming as a hyperheuristic to evolve heuristics for the biobjective 0-1 knapsack problem.This system successfully created 'reusable' heuristics able to produce a set of Pareto-optimal solutions.The Pareto fronts generated using this approach are indistinguishable from those obtained using the human-desinged profit-to-weight ratio heuristic.Hauptman et al. [10] employ genetic programming to generate solvers for two common puzzles including the NP-Complete Freecell.Genetic programming has also been used as a hyper-heuristic by Keller and Poli [23] for the travelling salesman problem, by Fukunaga [9] to generate local search heuristics for satisfiability and by Geiger et al. [7] to create dispatching rules for the job shop problem.At a higher level of abstraction, Hyde et al. [24] evolve the acceptance criteria component of a selection hyper-heuristic.The evolved acceptance criteria performed well when compared to standard acceptance criteria from the literature on instances of both bin packing and MAX-SAT.

III. THE MULTIDIMENSIONAL 0-1 KNAPSACK PROBLEM
The multidimensional 0-1 knapsack problem is an NPhard [3] combinatorial optimisation problem whereby the objective is to select a subset of items which maximise profit whilst conforming to a number of constraints.Each item consumes a certain amount of resources in each of the knapsacks dimensions and the capacity of each of the knapsacks dimensions must be respected.More formally, a multidimensional 0-1 knapsack problem with n items and m dimensions can be defined as: with where p j is the profit for selecting item j, x 1 ,...,x n is a set of decision variables indicating whether or not object j is selected, b i is the capacity of each dimension i and a ij is the resource consumption of item j in dimension i.
Senju and Toyoda [25] proposed a heuristic which starts with all variables x 1 ,...,x n set to 1 and successively sets variables to 0 in order of increasing utility value until a feasible solution is found.Magazine and Oguz [26] presented a heuristic algorithm which combined the method of Senju and Toyoda [25] with the Generalised Lagrange Multiplier approach of Everett [27] to fix certain variables.This work was improved by Volgenant and Zoon [28].More recently, a variety of exact and meta-heuristic methods have also been proposed in the literature to solve the multidimensional 0-1 knapsack problem including simulated annealing [29], [30], neural networks [31], genetic algorithms [32], memetic algorithms [33], [34], selection hyper-heuristics [35] and particle swarm optimisation [36] and core-based and tree search algorithms [37], [38], [39].Fréville [40] provides a more complete survey of the multidimensional 0-1 knapsack problem literature.

IV. A GENETIC PROGRAMMING HYPER-HEURISTIC FOR THE MULTIDIMENSIONAL KNAPSACK PROBLEM
A number of papers in the literature [25], [26], [33], [41], [42], [43], [44] make use of an add and (or) drop phase to either construct, improve or repair solutions to the multidimensional 0-1 knapsack problem.These techniques more often than not use a utility-weight value to score and sort the objects in order of their relative efficiency.Here we use genetic programming to evolve a population of heuristics which assign a score to each potential knapsack item.Each heuristic is evaluated by attempting to add the items to the knapsack in descending order determined by these scores (highest rank first) over a set of 'training' instances.

A. Genetic programming function and terminal sets
The first four rows of Table I show the function set of the genetic programming runs.The arithmetic operators add, subtract, multiply and protected divide are chosen to be included in the function set.In addition to standard add, subtract and multiply operators we use 'protected divide' instead of the traditional divide function.As there is always a possibility that the denominator could be zero, protected divide replaces zero with 0.001.The rest of Table I shows the terminal set.The avgDif f for item j is calculated as: Depending on the number of dimensions in the set of instances currently being considered there is a set of m conDim terminals for each of conDim j,1 ... conDim j,m .

B. Experimental design
The ORLib instances introduced by Chu and Beasley [33] are used to test the genetic programming hyper-heuristic.ORLib contains 270 instances with n ∈ {100, 250, 500} variables, m ∈ {5, 10, 30} dimensions and tightness ratio ∈ {0.25, 0.50, 0.75}.As optimal solutions are not known for these instances the %-gap is used to measure performance.The %-gap is the distance from the upper bound provided by the solution to the related LP-relaxed problem calculated as: 100 * LP opt−SolutionF ound LP opt (5) For each set of 10 instances we use 5 to 'train' the hyperheuristic before applying the best evolved heuristic to the further 5 as yet unseen 'test' instances.The fitness of an individual in the GP population is measured as the sum of the profit obtained on the 5 training instances.The next generation of the genetic programming run is then populated using the best performing heuristics.Table II shows the parameters used in the genetic programming runs.The mutation operator uses the 'grow' method described by Koza [18], with a set depth of five.The crossover operator produces two individuals with a maximum depth of 17.Each experiment was repeated 5 times for each set of instances.All experiments were carried out on an Intel i7 2 GHz CPU with 6 GB memory using the genetic programming implementation of the ECJ (Evolutionary Computation in Java) package.

V. EXPERIMENTAL RESULTS
Table III shows the average results in terms of %-gap for the best evolved heuristic of 5 runs for each set of instances in ORLib.Each set of instances consists of 5 'training' instances and 5 'test' instances and is labelled as ORmxn with m ∈ {5, 10, 30} dimensions and n ∈ {100, 250, 500} variables.Each mxn combination also varies with tightness ratio ∈ {0.25, 0.50, 0.75}.From this table we note that a better average percentage gap is obtained when tightness ratio increases.
Table IV shows the average %-gap for a number of techniques from the literature over all 270 instances in the ORLib benchmark set.Our approach generates heuristics which can outperform previous human-designed constructive heuristics.The heuristics generated by our genetic programming hyperheuristic achieve an average %-gap of 3.04, this is lower than the human-designed constructive heuristic methods proposed by Akc ¸ay et al. [45], Volgenant and Zoon [28] and Magazine and Oguz [26].They can also outperform the primitive MIP applied by Chu and Beasley [33].Many of the better performing techniques in the literature, such as the MA of Chu and Beasley [33] and the selection hyper-heuristic of Drake et al. [35], make use of a repair operator based on solutions to the LP-relaxed version of the multidimensional 0-1 knapsack problem.Once a reusable constructive heuristic has been evolved it takes considerably less computational effort to find a solution than calculating the LP-relaxed solutions.In this case, the LP-relaxed solutions must be calculated for every new instance encountered.Approaches using LP-relaxed solutions also then require the extra effort of applying a metaheuristic or other technique afterwards.Figure 1 shows the fitness value of the best performing heuristic at each generation of a sample GP run on the OR5x100 set.This plot shows a steady improvement in solution quality with the bestof-run heuristic not found until the latter stages of the run.In this work we have shown that genetic programming can be used as a hyper-heuristic to generate reusable constructive heuristics for the multidimensional 0-1 knapsack problem.Our method is classified as a hyper-heuristic approach as it operates on a search space of heuristics rather than a search space of solutions.To the authors knowledge, this is the first time in literature a GP hyper-heuristic has been used to solve the multidimensional 0-1 knapsack problem.This method has shown that automatically generated heuristics can be competitive with human-designed heuristics from the literature.Many methods make use of an add and (or) drop phase to either construct, improve or repair solutions.As future work, the rankings derived from the evolved heuristics can be used to define the order in which items are considered to be added or dropped.Many of the best results in the literature rely on knowledge gained from the LP-relaxed version of the multidimensional 0-1 knapsack problem.We intend to incorporate the optimal LP-relaxed results as part of a more comprehensive function set to attempt to improve the heuristics generated.

Fig. 1 .
Fig. 1.Best solution found at each generation of a sample run on the on the 5 training instances of the OR5x100 set