An Adaptive Memetic P System to Solve the 0/1 Knapsack Problem

Memetic Algorithms are traditionally composed of an evolutionary framework and one or more local search elements. However, modern generation Memetic Algorithms do not necessarily follow a pre-established scheme and are hybrid structures of various types. By following these modern trends, the present paper proposes an original and unconventional adaptive memetic structure generated by the hybridisation of a set of theoretical computational models, namely P Systems, and an evolutionary algorithm employing adaptation rules and moving operators inspired by Evolution Strategies. The resulting memetic algorithm, namely Adaptive Optimisation Spiking Neural P System (AOSNPS), is a tailored algorithm to solve optimisation problems with binary encoding.More specifically AOSNPS is composed of a family of parallel spiking neural P systems, each of them generating a binary vector representing a candidate solution on the basis of internal probability parameters and an adaptive Evolutionary Guider Algorithm that evolves the probabilities encoded in each P system. Numerical result shows that the proposed approach is effective to solve the 0/1 knapsack problem and outperforms various algorithms proposed in the literature to solve the same class of problems.


I. INTRODUCTION
In their early implementations, Memetic Algorithms (MAs) were identified with an evolutionary algorithm employing a local search [1], [2]. Subsequently, MAs no longer referred to a specific algorithmic structure but progressively became a strategy about the design of algorithms by using multiple and diverse search operators, see [3].
This fact has been emphasised in the review paper [4] which categorises MAs into three major groups • Simple Hybrids: this category includes hybrid algorithms generated by two or more algorithms (usually one global and at least one local) combined together in a synergistic manner, see [1]. Some hybridisations belonging to this category can be very efficient to address specific domains, see e.g. [5]- [8].
Thus, simple and adaptive hybrids tend to have a prearranged structure composed of an evolutionary framework and one or more local searches (possibly adaptively activated) that enhance upon the performance of one solution [19]. Conversely, modern MAs are thought as algorithms resulting from any hybridisation as long as they are capable to efficiently address a class of problems.
By following this consideration, this paper proposes an MA based on an unconventional hybridisation to solve binary problems and focuses on the 0/1 knapsack problem. More specifically, the proposed algorithm makes use of an adaptive evolutionary framework which includes an adaptive variation operator inspired by Evolutionary Strategy (ES) and another mutation operator that re-samples the points on the basis of the diversity and historical data. This evolutionary framework, namely Evolutionary Guider Algorithm, does not vary the solutions of the knapsack problem but varies the probabilities of the neurons of a family of Spiking Neural P Systems (SNPSs) [20], [21]. A SNPS is a general purpose computational structure belonging to the field of Membrane Computing, see [22]- [24]. In the context of the proposed MA, the SNPSs arranged in a family are used as number generators to generate strings of binary numbers (the candidate solutions of the 0/1 knapsack problem). Hence, the Evolutionary Guider Algorithm modifies and detects the parameters that control the generation of the candidate solutions.
The resulting algorithm namely Adaptive Optimisation Spiking Neural P System (AOSNPS) enhances a previous simpler algorithm based on a family of P systems proposed in [25].
The remainder of this article is organised in the following way. Section II briefly introduces the 0/1 knapsack problem, provides the basic definition of P systems (Subsections II-A) and describes the algorithm in [25] (Subsections II-B and II-C). Section III describes the proposed AOSNPS. Section IV displays the numerical results of the proposed approach in different scenarios and against multiple algorithms used to solve this problem. Finally, Section V provides the conclusions of this study.

II. BACKGROUND: 0/1 KNAPSACK PROBLEM AND OPTIMISATION BY P SYSTEMS
In this paper we aim at solving the 0/1 Knapsack Problem which is formulated as follows. Given a group of items, each item with its own weight and price, and a knapsack of limited capacity, the problem consists of selecting the items to make the total price of the knapsack as high as possible without violating its maximum capacity. If we indicate with m the total number of items available and we label each item with a number j = 1 . . . m, we may represent the selection of the items as a vector of binary numbers.
Then, the 0/1 Knapsack problem can be expressed as the maximisation of the function where p j and ω j are price and weight of the j th item, respectively and C is the capacity of the knapsack. The parameters of the problem have been set in following way. The weights ω j have been sampled from the interval [1, Ω], with Ω = 50, p j = ω j + 1 2 Ω and the average knapsack capacity C is applied: In the following subsections, we briefly summarise the relevant theory about P systems and present the Optimisation Spiking Neural P System (OSNPS) introduced in [25] to solve the 0/1 knapsack problem.

A. Spiking Neural P System
The SNPS [20], [26] is an automaton represented by the tuple Π = (O, σ 1 , · · · , σ m , syn, σ out ) where 1) O = {a} is the singleton alphabet, a is called spike 2) σ 1 , · · · , σ m are neurons, where each neuron σ i is a pair if the neuron σ i has an initial numbers of spikes n i = 5, it contains the work aaaaa = a 5 (b) R i set of the two following rules: (i) spiking rule: E/a c → a; d where E is a regular expression over O, and c ≥ 1, d ≥ 0. The spiking rule means that a neuron containing a word a ni may lose c < n i spikes after a delay of d steps (ii) forgetting rule: a s → λ, for some s ≥ 1, with the restriction that for each rule E/a c → a; d of type (i) from R i , we have a s / ∈ L (E). The forgetting rule means that a neuron can lose all its s spikes as long as a s does not belong to the language L (E) identified by the regular expression E.
3) the connections between neurons are described by the synapses syn where syn ⊆ {σ 1 , . . . σ m } × {σ 1 , . . . σ m } with (σ i , σ i ) / ∈ syn for i = 1, . . . , m. The synapses govern the propagation of the spikes, if the neuron σ i spikes and (σ i , σ j ) ∈ syn then the same spiking action is repeated on the neuron σ j 4) the output neuron σ out ∈ {σ 1 , . . . σ m }. The distinguishing feature of SNPS is that an input causes a nondeterministic spike train in the output, see [27]. If the output neuron spikes, then we code this behaviour as a 1 and otherwise we code it as a 0. Hence, the spike train can be represented by a sequence of ones and zeros. Details about the architecture of SNPS are described and discussed in detail in [20], [25], [28].

B. Extended Spiking Neural P System
In [25], in order to exploit the potentials of SNPSs for addressing an optimisation problem, the SNPS has been extended and slightly modified. The resulting implementation of P system, namely Extended Spiking Neural P System (ESNPS) is a tuple of the type Further comments on the neurons are (a) in each neuron σ i , there is only 1 initial spike (n i = 1, ∀i) (b) R i = {r 1 i , r 2 i } is a set of rules, spiking r 1 i : a → a and forgetting r 2 i : a → λ, respectively (c) P i = {p 1 i , p 2 i } is a finite set of probabilities, where p 1 i and p 2 i are the selection probabilities of rules r 1 i and r 2 i , respectively, and satisfy p 1 , work as a step by step supplier of spikes to neurons the output is a spike train formed by concatenating the outputs of σ 1 , σ 2 , . . . , σ m .
The structure of an ESNPS is shown in Fig. 1.

C. Optimisation Spiking Neural P System
The OSNPS is composed of multiple parallel ESNPSs and a Guider Algorithm, that supervises the family of ESNPSs, as shown in Fig. 2. Each ESNPS i , with 1 ≤ i ≤ H, has the structure shown in Fig. 1. The Guider Algorithm in Fig. 2 is used to adjust the probabilities P i of each ESNPS i . The input of the Guider Algorithm is a spike train T s , that is a set of H binary strings/candidate solutions of length m (a binary matrix of H × m elements) produced by each ESNPS i . The output of the Guider Algorithm is the rule probability matrix P R = [p 1 ij ] H×m , which is composed of the spiking rule probabilities of H ESNPSs, i.e., The probabilities are fed back to the ESNPSs which are updated accordingly.
Algorithm 1 provides the details of the Guider Algorithm used in OSNPS [25]. The basic idea is that one current best solution (binary vector) is stored in memory and two candidate solutions x k1 and x k2 are randomly selected, with a certain probability p a j , and compared on the basis of a fitness (objective function) f that measures the value/profit of the knapsack. At each objective function call, in order to handle the case when the total weight of all selected items exceeds the capacity C, we implemented the random chromosome repair technique suggested in [29]- [31].
If x k1 and x k2 are generated, for each design variable x j , the current solution is perturbed (x inherits the design variable of the winning candidate solution). Then, the probability associated with the most successful solution is increased by a constant learning rate Δ. Conversely, the probability associated with the candidate solution displaying a lower performance is reduced. If x k1 and x k2 are not generated, the binary variable of x j is directly used to affect the probability, see lines 19-24 of Algorithm 1. More details on OSNPS can be found in [25].

III. ADAPTIVE OPTIMISATION SPIKING NEURAL P SYSTEM
The proposed AOSNPS employs the same structure of OS-NPS depicted in Fig. 2 but hybridises the theoretical computer science logic of ESNPSs with that of Evolutionary Algorithms (EAs) [32] to manipulate the probability matrix P R . More specifically, the Guider Algorithm is an EA embedding two elements inspired by Evolution Strategies (ES) [33], [34] within it: 1) an adaptive learning rate Δ; 2) an adaptive mutation. The novel mechanisms are described in Subsections III-A and III-B, respectively. The new Guider Algorithm embedding these two elements is outlined in Subsection III-C.

A. Adaptive Learning Rate
The learning rate Δ is the step size of probability adjustment for the elements of the probability matrix P R : In [25], Δ is a random number between 0.005 and 0.02 set at the beginning of the optimisation and kept constant throughout the OSNPS execution.  while (j ≤ m) do 6: if rand() < p a j then 7: x j = x k1 10: else 11: end if 13: if (x j == 1) then 14: else 16: else 19: if (x j == 1) then 20: end if 25: if (p 1 ij > 1) then 26: else 28: if (p 1 ij < 0) then 29: has a low probability of execution since p 1 i + p 2 i = 1. If the output neuron spikes, a 1 is written in the candidate solution, otherwise a 0 is written. Thus, if we want to get 1, p 1 i should be large (ideally p 1 i = 1 ) and if we want to get 0, p 1 i should be small (ideally p 1 i = 0). This paper proposes an adaptive probability adjustment step size for each neuron. At each time unit, the adaptive updating rule of probability is where Δ a ij (a stands for "adaptive") is the step size and is defined as Δ a ij is designed to take the middle point of the distance between the current probability p 1 ij and the ideal probability. The ideal probability P b = {0, 1} is the lower or upper bound of the probability of p 1 ij (the pedex b stands for "bound"). For P b = 1 the update rule is Compared with the learning rate Δ defined in OSNPS, the adaptive learning rate Δ a ij proposed in this paper presents the following advantages.
• Δ a ij changes for each neuron at each time unit during the algorithm execution by following the learning needs. If the distance between the current probability p 1 ij and the ideal probability P b is big, the learning rate Δ a ij is big. On the other hand, if the distance between the current probability p 1 ij to the ideal probability P b is small, the learning rate is also small. • The adaptive learning rate allows a quick achievement of the desired probability. For example, if we want to get 1 from a neuron, from an initial probability p 1 ij = 0.1 the proposed adaptive learning rate Δ a ij enables to reach 0.9 < p 1 ij < 1 after four steps whereas with a constant Δ over 40 steps are needed. • The probability of p 1 ij does not overflow. In OSNPS, it may result p 1 ij > 0 or p 1 ij < 0. In this case, an extra mechanism is required. The use of the proposed adaptive learning rate ensures that the probability overflow never occurs.

B. Adaptive Mutation
The proposed AOSNPS makes also use of an adaptive mutation strategy. Two dynamic parameters P m1 and P m2 are defined to characterise and monitor the evolutionary state of AOSNPS. The parameter P m1 is the first mutation probability which varies between 0 and 1 and is defined as follows: where G bf (gen) is the best fitness value ever computed at the generation time of the execution and G bf (0) is the best fitness at the initialization. The parameter N max determines a stopping criterion: if the global best fitness does not improve for consecutive N max generations the algorithm stops. The parameter P m2 is the second mutation probability and is defined as follows: where DP a (gen) is an aggregated metric representing the diversity (of probabilities) at the current generation gen, see [11], [12]. This value is calculated as: The value DP a (0) is the average probability at the initialization. The triggering rule for adaptive mutation is defined as where rand 1 () and rand 2 () are two random numbers in the range [0 1]. The logic of the mutation probabilities P m1 and P m2 is summarised in the following points.
• If a new current best solution has been detected at the current generation (G bf (gen) > G bf (gen − 1)), then the mutation is not triggered (P m1 = 0). Conversely, if the current best solution has not been updated (G bf (gen) = G bf (gen − 1)), the probability of triggering the mutation is increased (P m1 = P m1 + 1 Nmax ). • If the diversity DP a (gen) is larger than that at initialization DP a (0)), the probability for the mutation to be triggered is low. • If the current best solution is not updated for many generations ((G bf ) remains the same), and the diversity DP a (gen) of the probability matrix P R is low, the mutation is performed with a high probability (P m1 is large, P m2 is small). When triggered the mutation of the matrix P R is performed in the following way: where rand 3 () and rand() are two random numbers in the range [0 1]. The mutation probability P m j is a parameter sampled in the range [0 0.1] at the beginning of the Guider Algorithm's execution. R bestf it ∈ [1 H] is the row coordinate of the best candidate solution found at the current generation. This means that the best candidate solution, in an elitist fashion, does not participate to the mutation.

C. The Evolutionary Guider of Adaptive Optimisation Spiking Neural P System
Based on the adaptive learning rate and the adaptive mutation rule, the Guider Algorithm is an EA operating on the learning probabilities to support the generation of successful solution for the 0/1 knapsack problem.
One current best solution x is stored in memory and modified by crossover with other candidate solutions. The candidate solutions are generated by the parallel ESNPSs (one solution by each ESNPS j ) and processed in pairs by the Evolutionary Guider Algorithm that evaluates and compare their fitness values. The evaluation of these candidate solutions are used to modify the probability matrix P R and hence generate solutions with a higher performance. Like for OSNPS, the constraint is handled by means of the random chromosome repair technique, see [29]- [31].
The evolution of the P R matrix is handled by two mechanisms: the update of the adaptive learning rate which happens every time two candidate solutions are sampled and described in Subsection III-A, the re-initialisation of the neuron probability which is triggered only by the satisfaction of the conditions described in Subsection III-B. Hence, we may consider AOSNPS as a co-evolutionary memetic algorithm, see [8], [35], [36], where the candidate solutions of the 0/1 knapsack problem evolve with the probabilities of the neurons in each of the P systems. Each P system can be seen as an a local search algorithm (each of them depends on different probabilities). In this unconventional memetic algorithm, the probabilities are directly manipulated by an adaptive EA, the Evolutionary Guider Algorithm, while the fitness refers only to the solutions generated by the P systems. The latter have the role of searching the decision space on the basis of the probabilities determined by the Evolutionary Guider Algorithms.
The pseudocode of the proposed Evolutionary Guider is shown in Algorithm 2.

IV. NUMERICAL RESULTS
In order to validate the proposed AOSNPS, we tested it against the following algorithms designed/used in the literature to solve the 0/1 kanpsack problem • Genetic Quantum Algorithm (GQA) [37] • Novel Quantum Evolutionary Algorithm (NQEA) [38] • Optimisation Spiking Neural P System (OSNPS) [25] For each algorithm we used the recommended parameter setting used in the original paper. The proposed AOSNPS has been run with H = 50 ESNPS with 1002 neurons each. Regarding the Evolutionary Guider Algorithm, the learning probability p a j (j = 1, . . . , m) uses the same value used in OSNPS (a random number in the range [0.05, 0.2]), the mutation probabilities p m j = 0.01 (j = 1, . . . , m) and N max = 500. The 0/1 knapsack problem has been run in different scenarios (problem instances), i.e. m = 1000, 2000, 3000, 4000, 5000, 6000, 7000, 8000, 9000, 10000. Each algorithm in this paper has been run for 30 independent  while (j ≤ m) do 8: if (rand () < p a j ) then 9: k1, k2 = ceil(rand * H), k1 = k2 = i and correct x k 1 and x k 2 if violate the constraint 10: end if 53: end while Ensure: Rule probability matrix PR runs. Each run has been continued for 400000 function calls, that 8000 generations for the AOSNPS. Table I displays the numerical results in terms of mean value μ ± standard deviation σ of the knapsack cost f (x). The best results for each problem instance are highlighted in bold.
Furthermore, the statistical significance of the results has been enhanced by the application of the Wilcoxon rank sum test, see [39]. A "+" indicates that AOSNPS significantly outperforms competitor, a "-" indicates that the competitor significantly outperforms AOSNPS, and a "=" indicates that there is no significant difference in performance.
Numerical results in Table I show that the proposed memetic implementation is very efficient to address the 0/1 knapsack problem since AOSNPS achieves the best results in nine cases out of the ten considered. In only one scenario (with the smallest number of items m = 1000) NQEA achieves slightly better results than AOSNSP. The effectiveness of the Evolutionary Guider appears from the direct comparison against OSNPS: AOSNPS systematically outperforms OSNPS.
In order to further strengthen the statistical analysis of the presented results, we performed the Holm-Bonferroni [40] procedure for the four algorithms (N A = 4) and ten problem instances (N p = 10). The rank R k for k = 1, . . . , N A by assigning, for each problem instance has been calculated. For each problem instance, a score N A is assigned to the best algorithm, a score N A − 1 to the second best, . . ., 1 to the worst algorithm. The ranks R k are the scores averaged over all the problem instances. Let us indicate with R 0 the ranking of AOSNPS. For the remaining N A − 1 algorithm the score z k is calculated as the values z k have been calculated as: .
By means of the z k values, the corresponding cumulative normal distribution values p k have been calculated, see [41]: These p k values have then been compared with the corresponding δ/k where δ is the level of confidence, set to 0.05 in this case.
These p k values have then been compared with the corresponding δ/k where δ is the level of confidence, set to δ = 0.05 in this case. Table II displays the ranks, z k values, p k values, and corresponding δ/k obtained. Moreover, it is indicated whether the null-hypothesis (that the two algorithms have indistinguishable performances) is "Rejected", i.e. the algorithms have statistically different performance, or "Accepted" if the distribution of values can be considered the same (there is no outperformance).
The Holm-Bonferroni procedure in Table II shows that AOSNPS achieves the best performance over all the algorithms taken into account. It can be observed that AOSNPS achieves the best ranking and significantly outperforms GQA and OSNPS. In order to emphasise the advantages of the proposed memetic approach over the version [25], we present in Fig.  3 the performance trend of OSNPS and AOSNPS. The trends clearly show the superiority of the newly proposed AOSNPS throughout the entire evolution. Finally, in order to illustrate how the concept of convergence can be exported to the context of P systems, we plotted the average Hamming distance D hm calculated over all binary strings produced by the H ESNPSs at each generation gen: where, x ik and x jk are the kth bits in the ith and jth binary solutions, respectively; m is the number of bits in a binary solution; H is the number of individuals (number of ESNPSs); the symbol ⊕ represents the OR operator.
It can be noticed that a large value of D hm indicates a high variety between each pair of (binary) candidate solutions in a population. The plot in Fig. 4 shows that AOSNPS achieves and maintains much higher population diversity values than OSNPS. Higher diversity values appear to be beneficial to overcome suboptimal basins of attraction and enhance upon the current best solution.

V. CONCLUSION
This paper proposes a novel MA generated by the combination multiple spiking neural P systems and an adaptive Evolutionary Guider Algorithm for solving the 0/1 knapsack problem. Each neural P systems generates a (binary) candidate solution on the basis of the probabilities encoded in its neurons while the Evolutionary Guider Algorithm supervises the search by optimising the functioning of the spiking neural P systems.
The resulting algorithm, Adaptive Optimisation Spiking Neural P Systems (AOSNPS), has been thoroughly tested over multiple problem instances and against various algorithms used to solve the 0/1 knapsack problems. Numerical results show that this unconventional approach is indeed powerful and is able to outperform its competitors for most of the problems under consideration. Furthermore, a direct comparison with another algorithm based on a similar structure shows the superiority of AOSNPS in terms of both solution detected and population diversity.
A further important contribution of this study is the novel memetic logic that successfully combines theoretical computer science and metaheuristic optimisation.