Monthly Archives: February 2015

GPdotNET 4.0 first look: Classification with Neural Networks

Posted on by
2

After some time of implementation and testing, the new version of is out. Go to codeplex page and it. This is huge step in development of this project, because it contains completely new module based on Artificial Neural Network and other optimization methods e.g. Particle Swarm Optimization.

Almost all aspects of the architecture are changed, or will be changed when this version would be released.

The new Experiment class will replace existing classes for handling experimental data in GA based models, which is not implemented yet. Also New start page will contain more pre-calculated examples.

For this beta here are the new features:

  1. New Start Page will be extended with new examples of Neural Nets : – binary classification, – multiclass classification and – regressions examples.
  2. Improved module for loading experimental data, which now supports nonnumeric data like categorical or binary data.
  3. Depending of the output column of loaded experimental data different learning algorithm is selected. For example if the column is of categorical type, GPdotNET selects Neural Net algorithm with Cross-Entropy and Particle Swarm optimization learning algorithm. If the output column is numerical, GPdotNET selects the Neural Nets with Backpropagation learning algorithm. Currently only two learning algorithms are implemented. More will be implemented in the next beta release.

Classification problem with GPdotNET

This topic will give quick tutorial how to model classification problem with GPdotNET by using Neural Nets. For this tutorial you need some experimental data which you can download from this .

  • Open GPdotNET choose New Command
  • New Dialog pops up, check Artificial Neural Nets, and Press Ok Button.

After Solver Type selection, GPdotNET Creates “Load Experiment” Page in which you can load experimental data, and define the percentage of data for testing the model.

  • Press Load Data button.
  • New Popup dialog appears.
  • Press File button select the file you previously downloaded.
  • Check Semicolon and First Row Header check buttons.

Note: When the data is analyzed correctly you should see vertical line “|” between columns. Otherwise data will not be loaded correctly.

  • When the experimental data is formatted correctly, press the “Import Data” button.

The next step is preparing columns for modeling.

For each columns we have to set:

  • a) proper type of the column (numeric, categorical or binary),
  • b) type of the parameter (input, output or ignore)
  • c) normalization method (MinMax,Gauss or Custom normalization of the column values ).

To change Column Type, double click on the cell which shows the column type, combo box appears with list of available types.

  • To change Parameter Type, double click on the cell which shows the param type, combobox appears with list of available parameter types.
  • To change Normalization Type, double click on the cell which shows MinMax value, combobox appears with list of available normalization types.

Note: you can set only one Output column. Put Parameter Type to Ignore if you want to skip column from modelling.

Now we have experimental data, and we can start modelling process. Before that we need to choose how much data will be treated for testing. Enter 10% for testing the data and press Start Modelling button.

Now we have more Pages:

1. Settings page for setting the parameters of the Neural Nets

2. Run page for simulation of searching solution

3. Prediction page which you can see how solution is good against testing data.

Settings Page

As you can see in Settings Page you can set various parameters for Neural Nets and Particle Swarm Optimization. Try to train the model with different parameters values. For this tutorial you can leave parameters as are.

Modeling Page

Modeling page contains two diagrams. The first diagram shows errors with respect of the iteration number which is very useful for monitoring the searching process. The second diagram which is below the previous shows current best solution (blue line) in comparison with the experimental data (red line). Also on the left side of the page, you can see several iteration number, error, and other information about searching process.

Prediction Page

Prediction page shows how current best model predict data. Predict page contains tabular and graphical representation of predicted data, which is compared with data for testing.

Features don’t work in this BETA

1. Exporting Neural Network model

2. Saving to gpa file Neural Network Models

GPdotNET v4.0- Introduction

Posted on by
4

This is the first post in series of posts that describe upcomming version of v4.0. Besides lot of improvements of the current version, the main part isimplementation of neural networks and set of other optimization methods of machine learning.

1. Modelling with GPdotNET v4.0

GPdotNET Logo

GPdotNET is C#, open source artificial intelligence tool for applying Genetic Algorithm and Artificial Neural Networks in modeling, prediction, optimization and pattern recognitions. With GPdotNET you can solve various engineering problems from classic regression and approximation to linear programming transportation and location problems and other machine learning based problems. By providing the learning algorithms GPdotNET uses a data of the research or experimental measures to learn about the problem. The results of learning algorithms are analytical models which can describe or predict the state of the problem, or can recognize the pattern. GPdotNET is very easy to use, even if you have no deep knowledge of GA, GP or ANN, you can apply those methods in finding solutions. The project can be used in modeling any kind of engineering process, which can be described with discrete data, as well as in education during teaching students about evolutionary methods, mainly GP and GA, as well as machine learning mainly Artificial Neural Networks.
The typical process of modelling with GPdotNET can be described in 5 steps.

  1. Choosing the type of the Solver: The first step is choosing the type of the solver. Which solver you will use depends on your intention what you want to do. For example if you want to make model for your experimental measurement you have several options which depend of your experimental data and the method you want to use. In GPdotNET you can use Genetic Programming or Neural Nets for modelling and prediction experimental data. But this is not strictly separate as may look on the flowchart below. That means that you can user Neural Networks for prediction, but training algorithm can be based on  Genetic Algorithm or Particle Swarm Optimization or Back Propagation algorithm.
  2. Loading your experimental data: GPdotNET uses powerful tool for importing your experimental data regardless of the type of data. You can import your numerical, binary or classification data. GPdotNET can automatically define classes, or format numerical data with floating or comma separated decimal values. More info can be find in Section 2.
  3. Setting Learning Parameters. After data is loaded and prepared successfully, you have to set parameters for the selected method. GPdotNET providers various parameters for each method, so you can set parameters which can provides and generates best output model.
  4. Searching for the solution: GPdotNET provides visualization of the searching solution so you can visually monitor how GPdotNET finds better solution as increasing the iteration number. If you provide data for testing calculated model, you can also see simulation of prediction.
  5. Saving and exporting the results: GPdotNET provides several options you can choose while exporting your solution. You can export your solution in Excel or text file, as well as in Wolfram Mathematica or R programming languages.

As would be seen, working in GPdotNET follows the same procedures regardless of the problem type. That means you have the same set of steps when modelling with Genetic Programming or Neural Networks. In fact GPdotNET contains the same set of input dialogs when you try to solve Traveling Salesman Problem with Genetic Algorithm or if you try to solve handwriting recognition by using Backpropagation Neural Networks. All learning algorithms within GPdotNET share the same UI.

The picture below shows the flowchart of the modelling in GPdotNET. The five steps described previously are depicted in the graphical forms surrounded with Start and Stop elements.

Modeling in GPdotNET v4.0

Besides parameters specific to learning algorithm, GPdotNET provides set of parameters which control the way of how iteration process should terminates as well as how iteration process should be processed by means of parallelization to use the multicore processors. During the problem searching GPdotNET records the history, so you can see when the best solution is found, how much time pass since last iteration process started, or how much time is remain to finish currently running iteration process.
Due to the fact that GP is the method which requires lot of processing time, GPdotNET provides parallelization, which speed up the process of searching. Enabling or disabling the parallelization processing is just a click of the button.

1.1 GPdotNET Open source project

From developer point of view GPdotNET is .NET (Mono) application written in C# programming language which can run both on Windows and Linux based OS, or any OS which supports Mono framework. Project started in 2006 within postgraduate study for modeling and optimization with evolutionary algorithms. As open source project, GPdotNET is first published on November 5 2009 on codeplex.com. The project is licensed under GNU Library General Public License (LGPL). For information about license and other kind of copyright please see . The project is hosted at . Main place for all news, documentation and code changes is my blog site at .

1.2 How to citate GPdotNET

GPdotNET is used from all around the world, in scientific papers, journals, books, for diploma works, master thesis or Dissertations. It is free to use GPdotNET with proper citation. So if you want to use the GPdotNET you need the right way to citate the tool.

Use this citation example in your paper, book etc.:

[1] B. I. Hrnjica, GPdotNET V4.0- artificial intelligence tool [Computer program], , accessed {date}.

Or

[1] Bahrudin I. Hrnjica, GPdotNET V4.0 – artificial intelligence tool [Computer program], , accessed {date}.