Blog Archives

GPdotNET 4.0 first look: Classification with Neural Networks


After some time of implementation and testing, the new version of GPdotNET is out. Go to codeplex page and download it. This is huge step in development of this project, because it contains completely new module based on Artificial Neural Network and other optimization methods e.g. Particle Swarm Optimization. gpdotnetv4_firstlook1

Almost all aspects of the architecture are changed, or will be changed when this version would be released.

The new Experiment class will replace existing classes for handling experimental data in GA based models, which is not implemented yet. Also New start page will contain more pre-calculated examples.

For this beta here are the new features:

  1. New Start Page will be extended with new examples of Neural Nets : – binary classification, – multiclass classification and – regressions examples.
  2. Improved module for loading experimental data, which now supports nonnumeric data like categorical or binary data.
  3. Depending of the output column of loaded experimental data different learning algorithm is selected. For example if the column is of categorical type, GPdotNET selects Neural Net algorithm with Cross-Entropy and Particle Swarm optimization learning algorithm. If the output column is numerical, GPdotNET selects the Neural Nets with Backpropagation learning algorithm. Currently only two learning algorithms are implemented. More will be implemented in the next beta release.

Classification problem with GPdotNET

This topic will give quick tutorial how to model classification problem with GPdotNET by using Neural Nets. For this tutorial you need some experimental data which you can download from this location.

  • Open GPdotNET choose New Command
  • New Dialog pops up, check Artificial Neural Nets, and Press Ok Button.

gpdotnetv4_firstlook2

After Solver Type selection, GPdotNET Creates “Load Experiment” Page in which you can load experimental data, and define the percentage of data for testing the model. gpdotnetv4_firstlook3

  • Press Load Data button.
  • New Popup dialog appears.
  • Press File button select the file you previously downloaded.
  • Check Semicolon and First Row Header check buttons.

Note: When the data is analyzed correctly you should see vertical line “|” between columns. Otherwise data will not be loaded correctly.

  • When the experimental data is formatted correctly, press the “Import Data” button.

The next step is preparing columns for modeling. gpdotnetv4_firstlook4

For each columns we have to set:

  • a) proper type of the column (numeric, categorical or binary),
  • b) type of the parameter (input, output or ignore)
  • c) normalization method (MinMax,Gauss or Custom normalization of the column values ).

To change Column Type, double click on the cell which shows the column type, combo box appears with list of available types.

gpdotnetv4_firstlook5

  • To change Parameter Type, double click on the cell which shows the param type, combobox appears with list of available parameter types.
  • gpdotnetv4_firstlook6
  • To change Normalization Type, double click on the cell which shows MinMax value, combobox appears with list of available normalization types.

gpdotnetv4_firstlook7

Note: you can set only one Output column. Put Parameter Type to Ignore if you want to skip column from modelling.

Now we have experimental data, and we can start modelling process. Before that we need to choose how much data will be treated for testing. Enter 10% for testing the data and press Start Modelling button.

Now we have more Pages:

1. Settings page for setting the parameters of the Neural Nets

2. Run page for simulation of searching solution

3. Prediction page which you can see how solution is good against testing data.

Settings Page

gpdotnetv4_firstlook8

As you can see in Settings Page you can set various parameters for Neural Nets and Particle Swarm Optimization. Try to train the model with different parameters values. For this tutorial you can leave parameters as are.

Modeling Page

Modeling page contains two diagrams. The first diagram shows errors with respect of the iteration number which is very useful for monitoring the searching process. The second diagram which is below the previous shows current best solution (blue line) in comparison with the experimental data (red line). Also on the left side of the page, you can see several iteration number, error, and other information about searching process. gpdotnetv4_firstlook9

Prediction Page

Prediction page shows how current best model predict data. Predict page contains tabular and graphical representation of predicted data, which is compared with data for testing. gpdotnetv4_firstlook10

Features don’t work in this BETA

1. Exporting Neural Network model

2. Saving to gpa file Neural Network Models

Brood Recombination new feature in GPdotNET


As currently implemented GPdotNET has classic crossover implementation without any intelligent way to exchange genetic materials. In most time classic crossover operation is destructive operation wasting lot of good genetic materials. By including brood recombination crossover can be slightly improved.

Brood recombination simple repeats crossover operation several time on the same parents, with different crossover points. After fitness evaluation of offspring, the best two child are kept and others are discarded. On that way there is a better chance to get better child than with classic crossover. The picture below graphically describes brood recombination.

brood_recombination

Brood Size – new GPdotNET parameter

The first feature which will be implemented is manually setting the Brood Size of crossover. By adding Brood Recombination, we will increase possibility that two chromosomes will exchange the best genetic material they have.

brood_recombination_parameter

The next feature will be brood size which will be generating dynamically and will be dependent of the generation number.

Citations of GPdotNET


gpdotnetv3beta2

Recently I have googled about GPdotNET to find out how people use GPdotNET. I was surprised that there are plenty of sites which are published GPdotNET as freeware software. I have also found several scientific papers which citated GPdotNT as well. Some other people used it as elegant example in their  lessons. Some students used GPdotNET in seminars and diploma works, master and phd thesis.

All in all I was very excited about it. So lets list some interested web sites and scientific paper which mentioned GPdotNET.

1.  Softpedia http://www.softpedia.com/get/Science-CAD/GPdotNET.shtml

2. StackOverflow : http://stackoverflow.com/questions/14761910/genetic-evolution-of-source-code/14762077#14762077

3.  Genetic Programming article:  http://www.answers.com/topic/genetic-programming

4.  Teaching lessons from  Faculty of Informatics, Burapha University, Thailand:  Evolutionary Algorithms Applied to Finance

5.  Karlsruhe Institute of Technology Paper Work: Evolutionary Algorithms.

6. Scientific paper, ACS Vol.14:  Use of Learning Methods to Improve Kinematic Models

7. Scientific paper, JPE Vol.15: Modeling of Discharge Energy in Electrical Discharge Machining by the use of Genetic Programming

8. Scientific paper, Thermophysics 2012: Evolution algorithms as toolfor optimization of water vapour transport properties of cellular concrete.

9. Scientific paper,  : Application Of Neuro Fuzzy Systems and Genetic Programming for Modelling Surface Roughness in Electrical Discharge Machining

10. Scientific paper, : Aspects regarding the use of Genetic Programming in modelling the cutting process

11. Scientific paper, IJMECH Vol.3, : Predicting Of Machining Quality In Electric Discharge Machining Using Intelligent Optimization Techniques

12. Scientific paper, SLAAI Jornal: Framework for Discovery of Data Models Using Genetic Programming

13. Book By AKERKAR, RAJENDRA, Introduction of Artifical Intelligence.

GPdotNET v3.0 is out


GPdotNET3.0Beta1

I am very proud to announce GPdotNET v3.0, the new version which brings 3 new solvers based on Genetic Algorithm and several new features. In previous posts I was  writing about those new features:

1. TSP and vector based chromosome implementation in GPdotNET

2. Exporting GP Model in to Wolfram Mathematica

3. Matrix based chromosome implementation

4. Introduction to Assignment and transportation problems based on Genetic Algorithm

The source code and Click Once installation are uploaded in codeplex site, so go and grab the new version. hppt://gpdotnet.codeplex.com

I am asking for feedback and bug reports. If you find something strange post a comment here on anywhere in blog site or codeplex site. I will try to answer as quickly as possible.

Here are screenshots with new Assignment and Transportation solvers in GPdotNET:

GPdotNET3.0Beta1sl3
Loaded  training data of Assignment problem

GPdotNET3.0Beta1sl4

Simulation Tab page with optimal result of Assignment problem

GPdotNET3.0Beta1sl5

Loaded  training data of Transportation problem

GPdotNET3.0Beta1sl6
Simulation Tab page with optimal result of Transportation problem