International Journal of Computational Intelligence Systems an Optimization Algorithm Based on Binary Difference and Gravitational Evolution an Optimization Algorithm Based on Binary Difference and Gravitational Evolution

This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. The publisher does not give any warranty express or implied or make any representation that the contents will be complete or accurate or up to date. The accuracy of any instructions, formulae, and drug doses should be independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims, proceedings, demand, or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of this material. Abstract Universal gravitation is a natural phenomenon. Inspired by Newton's universal gravitation model and based on binary differences strategy, we propose an algorithm for global optimization problems, which is called the binary difference gravitational evolution (BDGE) algorithm. BDGE is a population-based algorithm, and the population is composed of particles. Each particle is treated as a virtual object with two attributes of position and quality. Some of the best objects in the population compose the reference-group and the rest objects compose the floating-group. The BDGE algorithm could find the global optimum solutions through two critical operations: the self-update of reference-group and the interactive-update process between the reference-group and floating-group utilizing the gravitational evolution method. The parameters of BDGE are set by a trial-and-error process and the BDGE is proved that it can converge to the global optimal solution with probability 1. Benchmark functions are used to evaluate the performance of BDGE and to compare it with classic Differential Evolution. The simulation results illustrate the encouraging performance of the BDGE algorithm with regards to computing speed and accuracy.


Introduction
Evolutionary algorithms are a series of problem-solving methods that based on simulation of the natural evolution system, and its development can be traced back to 1950s.Compared with the classic optimization methods, an evolutionary algorithm has many advantages.For example, it is unconstrained by the search space limitations; it is unconstrained by the function types; and function gradient information is not essential, etc. Evolutionary algorithms have been widely used for optimization problem.Biological evolution theory inspired the emergence and development of bionics, which motivated many types of evolutionary algorithms, such as Genetic Algorithm [1][2], Simulated Annealing [3], Immune Algorithm [4][5][6], Particle Swarm Optimization [7][8][9], Ant Colony Optimization [10][11] and Bacterial Foraging Optimization [12], etc.All these contributions have gained great achievements on the field of optimization.
Chris and Tsang firstly proposed the concept and frame of GELS (Gravitational Emulation Local Search) [13] in 1995, which was then further developed by Webster [14].Balachandar and Kannan [15] proposed RGES (Randomized Gravitational Emulation Search) algorithm in 2007, which overcame some weak points of GELS, such as a relatively slow convergence rate and low quality of solutions.The three gravitational emulation search methods above were initially applied to solve combinatorial optimization problems such as Traveling Salesman Problem (TSP).Gravitational Optimization) [16], which was based on Einstein's theory and Newton's law of gravitation.The SGO simulated the process that a number of planets shift in the space to search for the planet with the most massive.The geometric transformation of the space generates a force to make the planets shifting faster or slower.In 2007, Chuang and Jiang proposed an algorithm named IRO (Integrated Radiation Optimization) [17], based on the phenomenon that the movements of one planet in the gravitational field were compositively effected by the sum of all the other planets' gravitational radiation forces.Rashedi proposed the GSA (Gravitational Search Algorithm) in 2007 and it was continuously improved thereafter [18][19][20].In GSA, a particle's total gravitation was a sum of all other particles' gravitations with random weights, and the total gravitation generated the acceleration to move.GPSO (Particle Swarm Optimization based on Gravitation) [21] was proposed by Kang and Wang et.al in 2007, which introduced an acceleration into Particle Swarm Optimization.All the four algorithms above were methods based on the position and displacement, and inspired by gravitation, each of which has different update formulas, respectively.
In this paper, the proposed algorithm Binary-Difference Gravitational Evolution (BDGE) shares the same point with the algorithms above, that is, all these algorithms utilize the concept of gravitation.However, BDGE does not simulate the physical movement processes as above, but via a clustering process based on elitist strategy.In BDGE, objects in the population are clustered into two different groups: the referenceobject group and the floating-object group.Objects of different groups are updated independently or cooperatively, and the binary difference [25] strategy is adopted to update objects in the updating processes independently or cooperatively, i.e. the self-update of reference-group and the interactive-update group and interacting process between the two groups.This paper is organized as follows.Section 2 presents both the details and ensemble of BDGE.Experimental study is given in Section 3, where benchmark functions are used to evaluate the performance.Section 4 gives an analysis on the parameters, and the convergence of algorithm is analyzed in Section 5, followed by the conclusion in Section 6.

Binary Difference Strategy
The Differential Evolution (DE) [22][23] algorithm emerged as a very competitive form of evolutionary algorithm more than a decade ago, and was first proposed by R. Storn and K. Price in 1995, which was a simple and efficient heuristic algorithm for global optimization over continuous spaces and its feature is a mutation with a differential strategy.The selection of differential strategies would make significant influence on the performance of the algorithm.In consideration of diversity, there are at least three individuals involved in mutation in DE [23][24].To simplify the mutation operation, the binary difference strategy was introduced, which is similar to the traditional but with only two individuals involved.As discussed in [25], binary difference using a sorted population could improve the performance of DE, especially in the aspect of convergence speed for low-dimensional problems.
Because the objects of the reference-object group in BDGE are sorted by their qualities, binary difference is appropriate to be introduced into BDGE.
In binary difference, two objects are selected as candidates, of which the one with better fitness value is selected as the central object, and the objects with relative worse fitness values are involved in mutation.The process of producing new objects is as follows.A central object is temporarily fixed and the worse-fitness objects are updated one after another according to the central object.The new objects are more likely to be close to the central object.
The binary difference strategy not only is simpler, but also takes advantages of the gravitation clustering.The population is divided into two groups by gravitation clustering, and the binary difference is used for the interactive update of the two groups.The dimensional update of an object with binary difference strategy is as follows: where () Xi and () Xjare two objects in a population, and () X new represents the newly produced individual.
The subscript kk means dimension kk, csign is a random symbol, 1 c is a constant, and (0,1) U is a random value uniformly distributed in [0,1]

Gravitational Grouping Mode
Gravitation is an attractive force between two objects, which exists between any two objects with masses.The value of gravitation is in proportion to the mass of either object, while in inverse proportion to the distance between them.Calculation of gravitation can be described as follows: where 6.672 G  is the gravitational constant, 1 m and 2 m are qualities of two objects, and r represents the distance between two objects.
Without loss of generality, we consider only the maximization problem as optimization problem considered in this paper: max ( ) ii l u i n  are the lower and upper bound.() f  is the objective function.We treat a particle in the search space S as an object, using gravitation for optimization.For any two objects A is simply described as: where 1,2 r means distance measurement, and 0 K is a small-valued constant to ensure denominator unequal to zero in Formula (2).() h  is a one-dimensional scale transformation function satisfied ( , ) x     , ( ) 0 hx  , which strictly increase monotonically.For convenience, () h  is set as the absolute value here, in accordance with the concept that quality is a nonnegative value.( )  represents the quality of an object, which is scale transformed from objective function.Formula ( 2) is essentially equal to Formula (1), though the form is changed.In this paper, we calculate the gravitation measurements, but not the true gravitation values.
For convenience, the fitness of an object we mentioned in this paper indicates the quality, which is a scale transformation of the objective function.

Binary Difference Gravitational Evolution Summary
The population denoted by RN is assemble of all objects, which are clustered into two groups, i.e. the reference-object group R , and the floating-object group () j mN represents the quality of the floating-object j N .
( , ) U a b is a random value uniformly distributed in interval [ , ] ab .
Binary Difference Gravitational Evolution algorithm can be summarized as follows: Step1 The reference-object group R , and the floating-object group N are randomly generated: where 1 n and 2 n are two integer numbers, and represent the group sizes of two groups, respectively.
Step2 If the halt conditions are satisfied, then halt.
Otherwise resort the population RN , which should satisfy the following rules: Step3 Binary difference strategy is applied to any two reference-objects and a new object /1 (1 ) Published by Atlantis Press Copyright: the authors 485 Downloaded by [Nanyang Technological University] at 22:58 24 June 2012 (0,1) 0.5 , 1 1, (0,1) 0.5 , and then resort the reference-object group R , which should satisfy the following rules: Then a bidirectional selection runs: (1) For every floating-object Step5 There are two situations for update of floating-objects based on step4.
Then go to Step2.

Sorting Operation
The population is divided into the reference-object group and the floating-object group in BDGE, where the reference-objects have the better fitness values, thus they are objects with bigger qualities.The referenceobjects are arranged according to their qualities in the group.

Reference-objects
Ordering Rules Fig. 1 Ordering of Reference-object Group The sorting operation is applied to the referenceobject group only, and the rules of sorting is illustrated in Figure 1, where () m  represent quality of an object, and objects are sorted in a proper sequence by their values of qualities.The other part of population, i.e. the floating-object group, does not need to be sorted, and the only requirement we need to ensure is that any floating-object's fitness value should never be better than the worst reference-object's.

Published by Atlantis Press
Copyright: the authors 486 Downloaded by [Nanyang Technological University] at 22:58 24 June 2012 Binary Difference Gravitational Evolution

Method of Update
There are two types of update methods in BDGE, i.e. the self-update of reference-group and the interactiveupdate process between the reference-group and floating-group.
Fig. 2 An Example of Reference-objects Self-update As it is showed in Figure 2, the self-update process of the reference-objects occurred after objects are sorted.Reference-objects ① and ② , ① and ③ , ② and ③ …are used to produce new objects one pair after another by binary difference.Thus, any two referenceobjects are selected in the process above.The rule of new objects' selection is that we select the superior with bigger quality and eliminate the inferior.The purpose of a reference-object is to keep a high level on quality as well as fitness, which to a certain degree ignores the diversity of the reference-group but speed up the convergence rate only.

Fig. 3 An Example of Gravitation Clustering
After the self-update process of reference-group, an interactive-update of reference-objects and floatingobjects are performed.Figure 3 shows an example of gravitation clustering, the essential of which is the interactive-update.Compared with reference-objects ② and ③, floating-object ① has a bigger gravitation measurement with reference-object ①, so floatingobject ① selects reference-object ①, the lines of reference-object ① and floating-object ① shows their relationship.The similar progress occurs on other objects.For the reference-object ①, there has four floating-objects (①, ③, ④ and ⑧) selected, which floating-object ③ (with a dot) has the minimum gravitation measurement with reference-object ①, then floating-object ③ would be eliminated from population.Similarly, other reference-objects eliminate floatingobjects as above to finish the interactive-update process between the reference-group and floating-group.
The positions of eliminated floating-objects are regenerated in a random way as initialization.The regeneration is called forced-update, which means being replaced and not participate in generating the next generations.The forced-update step of interactiveupdate contributes a lot in the goal of keeping population diversity.

Physical Meanings
As showed in both Formula (1) and ( 2), if two objects have a large value of gravitation, that may be caused by either a small distance in the denominator or big quality of each object in the numerator.The second case is what we expected, while the former may lead to a trap of local optimum.To solve this problem, we introduce a threshold parameter  , the value of which will be discussed in another paragraph.

Advantages of Gravitation Clustering:
Fig. 4 Searching Method Figure 4 shows an example of one dimension, as shown reference -objects 1 R , 2 R , and floating-objects 1 N , 2 N , according to Formula (2), 1 F and 2 F are large (when i F is small, the relationship of i R and i N is ignored), where 1 F is caused by big qualities of 1 R , and it is good for exploring on new region, which increases diversity.On the contrary, 2 F may be caused by the small distance 2 r .This is good for exploiting in the current region, which increases accuracy, but may cause a risk of being trapped in local optimums.

We choose
. The halting condition is set the same as that in DE.

Benchmark Functions
There are 9 benchmark functions presented in table 1, which are used to evaluate the performance of BDGE, and to compare it with the classic Differential Evolution.

Simulation Results
We run each algorithm 50 times independently on each benchmark function above.The dimension of problems is set to be 10.Successful run means the difference between the obtained solution and the true value to be less than 6 10  before the number of function evaluations reaches the maximum value, which is set to be 2,000,000.As shown in Table 2, where Best /Worst mean the best and worst solutions in the testing based on the average of 50 times of independent runs; Std and FEs is short for standard deviation and function evaluations.According to the data in Table 2, Differential Evolution cannot always find the optimal value of f 7 , for the worst solution in the 50 runs is 0.9949, dissatisfying the precision.We define when the obtained solution which has the precision less then  2 shows BDGE has a better performance than DE due to a smaller number of function evaluations, of which a maximal disparity was that DE got a more than 10-time function evaluations than BDGE, that means the latter is 10 times faster than the former.
Figure 5 shows the comparison between DE and BDGE, which illustrate the fitness values decreased as iteration increasing for optimizing the functions.As can be observed clearly from the figures, BDGE reached the global optimum requiring less number of iteration than DE.During the optimizing process, the blue solid line declines faster than the red dotted line, which illustrates that the BDGE converges faster than DE does.

The group sizes 1 n and 2 n
The values of group size 1 n and 2 n may influence the performance of BDGE a lot.Here we discuss these two parameters, using Schaffer 1 Problem (SF1) as an example, which was a classical deceptive problem.
We optimize respectively.For each pair of 1 n and 2 n , 50 runs are conducted independently, and a total of 20 20  times of runs are conducted for the two problems.Figure 6 shows the influences on optimization It can be seen from Figure 6 that for a 2-dimension problem, when 12 nn  , it has a higher success rate, while a lower rate when 12 nn  .From Figure 6(b), when 12 nn  , it has a lower number of average function evaluation, while an opposite result when 12 nn  .As an observation from Figure 6, a selection of 12 nn  is good for the 2-dimension Schaffer Problem.The same observation can be obtained from Figure 7 for the 5dimension problem.
Intuitively according to the algorithm procedure, when 12 nn  , the reference-objects' update takes high percentage of the whole update progress, which leads a fast convergence.However, it will lose diversity especially when dealing with deceptive problem.When 12 nn  , this is another status that the diversity is contented enough but with a little slow convergence.BDGE is an algorithm that can balance between a high convergence rate and a high diversity.
The floating-object group size could be arbitrarily big but a too large group size is time consuming.The conclusion is 1 n should be appropriate small, 2 n could be slightly larger than 1 n , and their relationship should , , 100 100 ( , , , ) 0 Here we separately test the problem with a dimension of 10, 20, 30 and 100, respectively.The parameters are set as follows:  As shown in Table 3, with the value of  decreasing, the standard deviation Std and the average number of fitness function evaluations FEs decrease as well, which means a better performance in optimization.From the statistic results shown in Table 3, it can be observed that in general, a setting of As for low-dimensional problems, a setting of  nearby the precision value leads to nice performances, while for a high-dimensional problem, the smaller  is set, the better result would be got.For example, for Shifted Sphere Function with 100-dimension, when

Analysis of Algorithm Convergence
BDGE is a type of randomized optimization algorithms.The conditions of proving the convergence of a randomized algorithm were firstly proposed by Solis and Wets [27].They have given the theorems to prove whether an algorithm has converged to the global optimal with probability 1, which can be summarized as follows: where D is a function to generate potential solutions,  is a random vector generated from the probability space (R , , ) n k B  , and f is the objective function.S , which is the subspace of R n , represents the constraint space of the problem.k  is probability measurement on B , which is the  domain of R n subset.Hypothesis 2 [27] if A is a Borel subset of S , satisfies ( ) 0 where R  represents the set of the global optimal solutions.
According to the theorem, if both Hypothesis 1 and Hypothesis 2 are satisfied simultaneously for BDGE, it can be confirmed that the proposed BDGE algorithm converges to the global optimal solution with probability 1.The convergence proof of BDGE is given as follows: In the BDGE algorithm, the return value before the th t iteration is the function value of ( 1) It can be inferred Hypothesis 1 is satisfied for BDGE.For Hypothesis 2, all that is needed is to prove the S - sized sample space contains S , thus, Suppose there are N iterations in the search, and the range of the th i iteration is i S , which is the support set as well.Therefore, the union space of the population (a set of individuals) is In conclusion, BDGE converges to the global optimal solution with probability 1, according to the theorem.

Conclusion
In this paper, the BDGE algorithm based on two critical models, i.e. binary difference and gravitational evolution for global optimization, was proposed.The proposed algorithm BDGE was compared with the Differential Evolution by testing both algorithms on benchmark functions.Simulation results show that the BDGE can explore the solution space more effectively than DE to obtain the global solution, and the BDGE requires a much smaller size population than DE does.The parameters of BDGE are studied and set by trialand-error, and the convergence analysis was also conducted to show BDGE can converge to the global optimal solution with probability 1. Certainly, there is still room to further improve BDGE.For instance, the number of parameters should be reduced to simplify the algorithm, and the clustering and gravitational models should be studied to solve high-dimensional problems, which are our future research work.

4 10 
as a successful run.There are 46 successful runs in 50 total runs while Published by Atlantis Press Copyright: the authors 488 Downloaded by [Nanyang Technological University] at 22:58 24 June 2012 Binary Difference Gravitational Evolution optimizing f 7 utilizing DE, while for other functions in this test, the successful runs are all 50, i.e.DE can solve the rest problems and the proposed BDGE can optimize all the functions with relative smaller Std and FEs values, which means a higher precision and fast convergence speed, Table

and 2 n
of fitness function evaluations overstepped the Max-FEs, which was set to be 150,000 for 2-dimension problem and 2,000,000 for 5-dimension problem, respectively.Other parameters of BDGE are set as follows, are experimentally set to be 1, 2, ..., 20,Published by Atlantis PressCopyright: the authors 489Downloaded by [Nanyang Technological University] at 22:58 24 June 2012 J. Li et al.
and Figure.7 shows that of 5-dimension.According to the halt condition, we adopt success as operation fitness function exceeds Max-FEs.

Fig. 6 Fig. 7 The 4 . 2 .
Fig. 6 The Effect of Alternative Group Sizes on Optimization Schaffer 1 Problem (SF1) of 2dimension (a) Result of Success Rate, (b) Result of Average Function Evaluations most problems, which can be used as a common setting.

M
represents the support set of the th i individual's sample space in th t iteration.

.
The range of an individual is adjustable, and when range covers the boundary of the solution space, though there are only a few individuals, . Binary difference uses a N .Followings are the symbols used in the BDGE descriptions.() i XR represents the position of the th i , () jk XNrepresents its th k dimension and

Table . 2
Comparison of results on 50-time independent random testing with benchmark functions

Table . 3
The result of benchmark testing on Shifted Sphere Function with different values of  Downloaded by [Nanyang Technological University] at 22:58 24 June 2012 J. Li et al. the objective function.The function D of Hypothesis 1 is defined as: ) i f x t  , and ( ( )) i f x t represents the function value of the th t iteration value () i xt, where () fx represents Published by Atlantis Press Copyright: the authors 491