## Function optimization with Genetic Algorithm by using GPdotNET

Content

1. Introduction
2. Analytic function optimization module in GPdotNET
3. Examples of function optimizations
4. C# Implementation behind GPdotNET Optimization module

Introduction

GPdotNET is artificial intelligence tool for applying Genetic Programming and Genetic Algorithm in modeling and optimization of various engineering problems. It is .NET (Mono) application written in C# programming language which can run on both Windows and Linux based OS, or any OS which can run Mono framework. On the other hand GPdotNET is very easy to use. Even if you have no deep knowledge of GP and GA, you can apply those methods in finding solution. The project can be used in modeling any kind of engineering process, which can be described with discrete data. It is also be used in education during teaching students about evolutionary methods, mainly GP and GA. GPdotNET is open source project hosted at http://gpdotnet.codeplex.com

With releasing of  GPdotNET v2 it is also possible to find optimum for any analytic function regardless of independent variables. For example you can find optimum value for an analytically defined function with 2, 5, 10 or 100 independent variables. By using classic methods, function optimization of 3 or more independent variables is very difficult and sometimes impossible. It is also very hard to find optimum value for functions which are relatively complex regardless of number of independent variables.
Because GPdotNET is based on Genetic Algorithm we can find approximated optimum value of any function regardless of the limitation of number of independent variables, or complex definition. This blog post is going to give you a detailed and full description how to use GPdotNET to optimize function. Blog post will also cover C# implementation behind optimization proce by showing representation of Chromosome with real number, as well as Fitness calculation which is based on Genetic Programming tree expression. In this blog post it will also be presented several real world problem of optimization which will be solved with GPdotNET.

# Analitic Function Optimization Module in GPdotNET

When GPdotNET is opened you can choose several predefined and calucalted models from various domain problems, as weel as creating New model among other options. By choosing New model new dialog box appears like picture below.

By choosing Optimization of Analytic Function (see pic above) and pressing OK button, GPdotNET prepares model for optimization and opens 3 tab pages:

1. Analytic function,
2. Settings and
3. Optimize Model.

## Analytic function

By using “Analytic function” tab you can define expression of a function. More information about how to define mathematics expression of analytic function can be found on this blog post.

By using “Analytic definition tool” at the bottom of the page, it is possible to define analytic expression. Expression tree builder generates function in Genetic Programming Expression tree, because GPdotNET fully implements both methods. Sharing features of Genetic programming  in Optimization based Genetic Algorithm is unique and it is implement only in GPdotNET.

When the process of defining function is finished, press Finish button in order to proceed with further actions. Finish button action apply all changes with Optimization Model Tab. So if you have made some changed in function definition, by pressing Finish button changes will be send to optimization tab.
Defining expression of function is relatively simple, but it is still not natural way for defining function, and will be changed in the future. For example on picture 2, you can see Expression tree which represent:

$f(x,y)=y sin{4x}+1.1 x sin{2y}$.

## Setting GA parameters

The second step in optimization is setting Genetic Algorithm parameter which will be used in optimization process. Open the Setting tab and set the main GA parameters, see pic. 3.

To successfully applied GA in the Optimization, it is necessary to define:

1.  population size,
2. probabilities of genetic operators and
3. selection methods.

These parameters are general for all GA and GP models. More information about parameters you can find at http://bhrnjica.net/gpdotnet.

## Optimize model (running optimization)

When GA parameters are defined, we can start with optimization by selecting Optimization model tab. Before run, we have to define constrains for each independent variables. This is only limitation we have to define i  order to start optimization. The picture below shows how to define constrains in 3 steps:

1.  select row by left mouse click,
2. enter min and max value in text boxes
3. Press update button.

Perform these 3 actions for each independent variable defined in the function.

When the process of defining constrains is finished, it is time to run the calculation by pressing Optimize button, from the main toolbar(green button). During optimization process GPdotNET is presenting nice animation of fitness values, as well as showing current best optimal value. The picture above shows the result of optimization process with GPdotNET. It can be seen that the optimal value for this sample is $f_{opt}(9.96)=-100.22$.

# Examples of function optimization

In this topic we are going to calculate optimal value for some functions by using GPdotNET. Zo be prove that the optimal value is correct or very close to correct value we will use Wolfram Alpha or other method.

### Function: x sin(4x)+1.1 x sin(2y)

GP Expression tree looks like the following picture (left size):

Optimal value is found (right above picture) for 0.054 min, at 363 generation of total of 500 generation. Optimal value is f(8.66,9.03)=-18.59.

Here is Wolfram Alpha calculation of the same function. http://www.wolframalpha.com/input/?i=min+x*sin%284*x%29%2B+1.1+*y*+sin%282+*y%29%2C+0%3Cx%3C10%2C0%3Cy%3C10

### Function:  (x^2+x)cos(x),  -10≤x≤10

GP expression tree looks like the following picture (left size):

Optimal value is found for 0.125 min, at 10 generation of total of 500 generation. Optimal value is F(9.62)=-100.22.

Here is Wolfram Alpha calculation of the same function. http://www.wolframalpha.com/input/?i=minimum+%28x%5E2%2Bx%29*cos%28x%29+over+%5B-10%2C10%5D

### Easom’s function fEaso(x1,x2)=-cos(x1)•cos(x2)•exp(-((x1-pi)^2+(x2-pi)^2)), -100<=x(i)<=100, i=1:2.

GP expression tree looks like the following picture (left size):

Optimal value is found for 0.061 min, at 477 generation of total of 500 generation. Optimal value is F(9.62)=-1, for x=y=3.14.

Function can be seen at this MatLab link.

# C# Implementation behind GPdotNET Optimization module

GPdotNET Optimization module is just a part which is incorporated in to GPdotNET Engine. Specific implementation for this module is Chromosome implementation, as well as Fitness function. Chromosome implementation is based on  floating point value instead of classic binary representation. Such a Chromosome contains array of floating point values and each element array represent function independent variable. If the function contains two independent variables (x,y) chromosome implementation will contains array with two floating points. Constrains of chromosome values represent constrains we defined during settings of the optimization process. The following source code listing shows implementation of GANumChrosomome class in GPdotNET:

```public class GANumChromosome: IChromosome
{
private double[] val = null;
private float fitness = float.MinValue;
//... rest of implementation
}
```

When the chromosome is generated array elements get values randomly generated between min and max value defined by function definition. Here is a source code of Generate method.

```///
/// Generate values for each represented variable
///
public void Generate(int param = 0)
{
if(val==null)
val = new double[functionSet.GetNumVariables()];
else if (val.Length != functionSet.GetNumVariables())
val = new double[functionSet.GetNumVariables()];

for (int i = 0; i < functionSet.GetNumVariables(); i++)
val[i] = Globals.radn.NextDouble(functionSet.GetTerminalMinValue(i), functionSet.GetTerminalMaxValue(i));

}
```

Mutation is accomplish when randomly chosen array element randomly change itc value. Here is a listing:

```///
///  Select array element randomly and randomly change itc value
///
public void Mutate()
{
//randomly select array element
int crossoverPoint = Globals.radn.Next(functionSet.GetNumVariables());
//randomly generate value for the selected element
val[crossoverPoint] = Globals.radn.NextDouble(functionSet.GetTerminalMinValue(crossoverPoint), functionSet.GetTerminalMaxValue(crossoverPoint));
}
```

Crossover is little bit complicated. It is implemented based on Book Practical Genetic Algoritms see pages 56,57,48,59. Here is an implementation:

```///
/// Generate number between 0-1.
/// For each element array of two chromosomes exchange value based on random number.
///
///
public void Crossover(IChromosome ch2)
{
GANumChromosome p = (GANumChromosome)ch2;
int crossoverPoint = Globals.radn.Next(functionSet.GetNumVariables());
double beta;
for (int i = crossoverPoint; i < functionSet.GetNumVariables(); i++)
{
beta = Globals.radn.NextDouble();
val[i] = val[i] - beta * (val[i] - p.val[i]);
p.val[i] = p.val[i] + beta * (val[i] - p.val[i]);
}
}
```

Fitness function for Optimization is straightforward, it evaluates each chromosome against tree expression. For minimum the better chromosome is lower value. For maximum better chromosome is the chromosome with higher fitness value. Here is a implementation of Optimizatio Fitness function:

```///
/// Evaluates function agains terminals
///
///
///
///
public float Evaluate(IChromosome chromosome, IFunctionSet functionSet)
{
GANumChromosome ch = chromosome as GANumChromosome;
if (ch == null)
return 0;
else
{
//prepare terminals
var term = Globals.gpterminals.SingleTrainingData;
for (int i = 0; i < ch.val.Length; i++)
term[i] = ch.val[i];

var y = functionSet.Evaluate(_funToOptimize, -1);

if (double.IsNaN(y) || double.IsInfinity(y))
y = float.NaN;

//Save output in to output variable
term[term.Length - 1] = y;

if (IsMinimize)
y *= -1;

return (float)y;
}
}
```

# Summary

We have seen that Function optimization module within GPdotNET is powerful optimization tool. It can find pretty close solution for very complex functions regardless of number of independent variables. Optimization module use Genetic Algorithm method with floating point value chromosome representation described in several books about GA. It is fast, simple and can be used in education as well as in solving real problems. More info about GPdotNET can be found at http://bhrnjica.net/gpdotnet.

## MSNetwork 3. po redu Microsoft konferencija u BiH

Banja Vrućica 3. i 4. aprila 2013 g.

Sada već tradicionalno po treći put se održava Bosanskohercegovačka Microsoft konferencija:  MSNetwork 3. Mjesto održavanja ovaj put je Banja Vrućica zdravstveno turistički centar koji se nalazi u blizini Teslica, gradića na putu izmedju Banjaluke i Doboja.

Kao i prvi put kad se održavala u Banjaluci prije 3 godine, i ovaj put konferencija će ponuditi najbolje teme, predavače i cijelu konferenciju učiniti nezaboravnom bas onako kako je to bilo i ranije. Naravno, svaki put ljudi iz Microsofta BiH se potrude da ona bude bolja od prethodne pa i ovaj put ne sumnjam u to. Ovaj put rekordan broj predavača kao i predavanja. Prva konferencija je krenula sa 3 tracka, da bi prošle godine bio i MSC track na kojem predavanja daju ljudi iz MS Communitya, da bi ove godine bio i EDU track, posvećen nekim stručnim  temama.

Na ovoj konferenciji se se naći zaista za svakog ponešto. Gotovo svi poznati  predavači iz Makedonije, Srbije, BiH, Hrvatske i Slovenije, posebno predavači iz Njemačke i drugih evropskih zemalja,  naći će se 3 i 4 aprila u Banja Vrućici. Ukupno 60 predavača govorit će na konferencij što konferenciji čini vrlo atraktivnom, kvalitetnom i zanimljivom.

Koristim ovu priliku da na MSNetwork najavim svoje predavanje. Predavati na MSNetwork konferenciji zaista me čini sretnim i zahvaljujem se organizatorima što su ovo predavanje  uključili u zvanični dio konferencije.

Naziv predavanja, level i kratki opis pročitajte u narednom tekstu.

Paralelno i asinhrono programiranje – izazov za svakog programera (4.april. 2013, 11:30 dvorana Bosna)

Level: 300

Opis predavanja:

Multi-core procesori su realnost, proizvođači ih danas ugrađuju i u grafičke kartice, mobilne telefone pa čak i u veš-mašine. Direktna posljedica razvoja multi-core procesora je prestanak razvoja single-core procesora čiji takt već odavno stoji na magičnoj brojci oko 3 GHz. Kako proizvođači ovom tehnologijom ne mogu povećati takt počeli sa proizvodnjom multi-core procesora, ili višejezgrenih procesora u jednom hardverskom dijelu, što je dalo dodatni vjetar u leđa razvoju procesora. Danas se kućni računari kupuju sa 4 ili 8 jezgri, serveri i do 128 jezgri. Realno se pitanje postavlja: da li softver koji je razvijan nekoliko godina unazad odgovara takvom hardveru? Da li hardver na multi-core procesorima ima smisla vrtiti dosadašnja softverska rješenja? Moguće se upitati i to da li energija koju troši ovakav hardver odgovara korištenju softvera? Imate više od 1 procesora na PC-u, ali ne primjećujete da vaš softver radi brže? Još uvijek koristite klasu Thread ili BackgroundWorker ili Callback funkciju kako bi korisnika zavaravali dok se vaši podaci učitavaju u pozadini? Željeli bi programirati višenitne aplikacije, a da ne formirate niti? Ovo su samo neka od pitanja, čije odgovore daje paralelno i asinhrono programiranje u .NETu. Paralelno i asinhrono programiranje predstavlja novu paradigmu i izazove za moderne programere koji žele iskorištavati sve resurse PC-a, a ne samo jednu jezgru, koji žele programirati višenitno, a da ne formiraju niti, koji žele koristiti nova proširenja koja su sastavni dijelovi .NET 4.5 i C# 5.0.

Pored zvaničnog opisa ovdje bih dodao da će ovo predavanje obilovati realnim primjerima:

1. primjer paralelizacije riješavanje sistema linearnih jednačina sa više od 1000 nepoznatih.

2. primjer asinhronog procesuiranja zahtjeva na ASP.NET  web stranici.

3. procedura konverzije sekvencijalnog koda u asinhroni, praktična iskustva.

4. nekoliko jednostavnih primjera demonstracije Data Race, Thread-Safety, PLINQ,ThreadLocalState i sl.

Nadam se da će predavanje biti zanimljivo, a pogotovu za one koji žele više posmatrati Visual Studio od PowerPointa.

Vidimo se na konferenciji.

## Implementation Traveling Salesman Problem (TSP) with GPdotNET

After only 2 months of releasing GPdotNET v2.0 the new update is coming. Architecture of GPdotNET v2 allows implementation of wide range of GA & GP based problems. For this first update, it is implementation of Traveling Salesman Problem (TSP). Short description of this problem is to find the shortest path by visiting all cities only once. More info can be found here.  This problem is implemented just for several hours, because all important phases are already implemented.

The main implementations for this problem were:

1. TSPChromosome – class for chromosome representation
2. TSP Fitness – implementation
3. Other parts like selection, simulation of the result are mostly implemented.

## TSPChromosome class implementation

Instead of binary value, TSP chromosome must be array of integers by representing path around cities. The main problem with this chromosome type is genetic operations, because you cannot implement it easily.

For example standard Crossover in GA takes random point and split parents in two parts, see picture below. One part goes to first offspring, the second one goes to second offspring. The same action is performed for the second parent. The picture below shows standard crossover in GA.

Unfortunately this operation cannot be applied to TSP, because every city can be visited only once. But how can we perform crossover at all? There are several methods can be found on internet. The method applied in GPdotNET is the following.

Choose different random points for Parent 1 and Parent 2.

Offspring 1 is created based on the left part of Parent 1, and sub permutation of the rest of cities.

Offspring 2 is created based on the right part of Parent 2 and sub permutation of rest of the cities.

Picture below show crossover used in GPdotNET.

The problem with this crossover is that offsprings are not created based on genetic material from both parents.

Implementation of TSPChromosome crossover method is presented below:

```public void Crossover(IChromosome ch2)
{
var ch22 = ch2 as GATSPChromosome;
if (ch22 == null)
throw new Exception("Chromosome cannot be null!");

Order(true);
ch22.Order(false);

int randInd1 = Globals.radn.Next(length);
int randInd2 = Globals.radn.Next(length);

for (int i = 0; i < randInd1; i++)
{
int rand1 = Globals.radn.Next(randInd1);
int rand2 = Globals.radn.Next(randInd1);

var temp = value[randInd1];
value[randInd1] = value[randInd2];
value[randInd2] = temp;
}

for (int i = randInd2; i < length; i++)
{
int rand1 = Globals.radn.Next(randInd2,length);
int rand2 = Globals.radn.Next(randInd2,length);
var temp = ch22.Value[randInd1];
ch22.Value[randInd1] = ch22.Value[randInd2];
ch22.Value[randInd2] = temp;
}
}
```

Mutation operation for this chromosome can be implemented so that we randomly choose two cities and swap their positions in the path. Implementation for this operation is presented in code below:

``` public void Mutate() {     int randInd1 = Globals.radn.Next(length);     int randInd2 = Globals.radn.Next(length);     if(randInd1>randInd2)
{
int temp= randInd2;
randInd2=randInd1;
randInd1=temp;
}
int j = 0;
for (int i = randInd1; i <= randInd2; i++,j++)
{
if (i > (randInd2 - j))
break;
var temp1 = value[i];
var temp2 = value[randInd2-j];
value[i] = temp2;
value[randInd2 - j] = temp1;
}
}
```

Fitness of TSP

Fitness function is relatively simple. It calculates total path based on Chromosome array of cities. Fitness will be better if the path is shorter.

```public float Evaluate(IChromosome chromosome, IFunctionSet functionSet)
{
var ch = chromosome as GATSPChromosome;
if (ch == null)
return 0;
else
{
double y = 0, fitness;
double[] p1 = null;
double[] p2 = null;
int rCount = ch.Length;
for (int i = 0; i < rCount; i++)
{
p1 = Globals.GetTerminalRow(ch.Value[i]);
if (i + 1 == rCount)
p2 = Globals.GetTerminalRow(ch.Value[0]);
else
p2 = Globals.GetTerminalRow(ch.Value[i+1]);

// calculate distance betwee two points and make the sum
y += Math.Sqrt((p2[0] - p1[0]) * (p2[0] - p1[0]) + (p2[1] - p1[1]) * (p2[1] - p1[1]));

// check for correct numeric value
if (double.IsNaN(y) || double.IsInfinity(y))
return float.NaN;
}
//with this value we always search for maximum value of fitness
fitness = ((1.0 / (1.0 + y)) * 1000.0);

return (float)fitness;
}
}
```

## Loading City Map

TSP Solver needs City data of points in (X;Y) format. It must be loaded by CSV file format, the same as GPdotNET uses for loding data for modelling and optimization. The following data represent corect format:

CityMap data format:

## How to run TSP Solver on GPdotNET

Running TSP solver with GPdotNET is relatively easy. The procedure follows global convention for creating new model or solver.

1. Choose New.. from mainpanel.

2.  Choose Traveling Salesman Problem, and click OK button.

3. Open Load Data tab, (if it not opened), press Load Cities Map button, choose CityMap.CSV, and press OK. Data is presented on picture below.

4. Sett GA parameters on Setting Tab panel:

5. Open Simulation Tab panel. You can see graphically representation of loaded map, and set of simulation parameters.

6. Press Run ribon button, and Solver will start. During simulation you can see that the path length is becoming shorter and shorter

7. As final result, Solver find the path which is the shortest or very close to shorter one.

8. Exit and Save your solution.

We have seen how easy is to implement such a problem, base on existing foundation of the GPdotNET. More problems based on GA/GP is cumming in the GPdotNET.  Application is posted on codeplex.com, as GPdotNET v2.1 beta 1.

## Matrakci Calculator for Android

Matrakci Calculator on  Googel Play.

After Windows Phone version of Matrakci Calculator, I wanted to make Andorid version too but I dont know much about Java and Eclipse. So I decided to implement JQueryMobile application, which sombody needs to merge in to Eclipse.

After some time the result of work can be seen on picture below. I have used canvas for lattice drawing  and  used different fill colors to differentiate which numbers needs to be summed. I liked this version, because I have learned more about HTML5 and JavaScript programming as well. So my future application will be in HTML5 as well.

You can download Matrakci Calcualtor from this link at Google Play.

I want to thank my fried Nezir Zahirovic which he helped me in implementation of this app.

If you want to play online this calculator right now, check this link. It is described all you want to know about Matrakci and Lattice multiplication as well.