# Linear Regression with CNTK and C#

CNTK is Microsoft’s deep learning tool for training very large and complex neural network models. However, you can use CNTK for various other purposes. In some of the previous posts we have seen how to use CNTK to perform matrix multiplication, in order to calculate descriptive statistics parameters on data set.
In this blog post we are going to implement simple linear regression model, LR. The model contains only one neuron. The model also contains bias parameters, so in total the linear regression has only two parameters: w and b.
The image below shows LR model:

The reason why we use the CNTK to solve such a simple task is very straightforward. Learning on simple models like this one, we can see how the CNTK library works, and see some of not-so-trivial actions in CNTK.
The model shown above can be easily extend to logistic regression model, by adding activation function. Besides the linear regression which represent the neural network configuration without activation function, the Logistic Regression is the simplest neural network configuration which includes activation function.

The following image shows logistic regression model:
In case you want to see more info about how to create Logistic Regression with CNTK, you can see this official demo example.
Now that we made some introduction to the neural network models, we can start by defining the data set. Assume we have simple data set which represent the simple linear function $y=2x+1$. The generated data set is shown in the following table:

We already know that the linear regression parameters for presented data set are: $b_0=1$ and $b_1=2$, so we want to engage the CNTK library in order to get those values, or at least parameter values which are very close to them.

All task about how the develop LR model by using CNTK can be described in several steps:

Step 1: Create C# Console application in Visual Studio, change the current architecture to $x64$, and add the latest “CNTK.GPU “ NuGet package in the solution. The following image shows those action performed in Visual Studio.

Step 2: Start writing code by adding two variables: $X$ – feature, and label $Y$. Once the variables are defined, start with defining the training data set by creating batch. The following code snippet shows how to create variables and batch, as well as how to start writing CNTK based C# code.

First we need to add some using statements, and define the device where computation will be happen. Usually, we can defined CPU or GPU in case the machine contains NVIDIA compatible graphics card. So the demo starts with the following cod snippet:

```using System;
using System.Linq;
using System.Collections.Generic;
using CNTK;
namespace LR_CNTK_Demo
{
class Program
{
static void Main(string[] args)
{
//Step 1: Create some Demo helpers
Console.Title = "Linear Regression with CNTK!";
Console.WriteLine("#### Linear Regression with CNTK! ####");
Console.WriteLine("");
//define device
var device = DeviceDescriptor.UseDefaultDevice();
```

Now define two variables, and data set presented in the previous table:

```//Step 2: define values, and variables
Variable x = Variable.InputVariable(new int[] { 1 }, DataType.Float, "input");
Variable y = Variable.InputVariable(new int[] { 1 }, DataType.Float, "output");

//Step 2: define training data set from table above
var xValues = Value.CreateBatch(new NDShape(1, 1), new float[] { 1f, 2f, 3f, 4f, 5f }, device);
var yValues = Value.CreateBatch(new NDShape(1, 1), new float[] { 3f, 5f, 7f, 9f, 11f }, device);
```

Step 3: Create linear regression network model, by passing input variable and device for computation. As we already discussed, the model consists of one neuron and one bias parameter. The following method implements LR network model:

```private static Function createLRModel(Variable x, DeviceDescriptor device)
{
//initializer for parameters
var initV = CNTKLib.GlorotUniformInitializer(1.0, 1, 0, 1);

//bias
var b = new Parameter(new NDShape(1,1), DataType.Float, initV, device, "b"); ;

//weights
var W = new Parameter(new NDShape(2, 1), DataType.Float, initV, device, "w");

//matrix product
var Wx = CNTKLib.Times(W, x, "wx");

//layer
var l = CNTKLib.Plus(b, Wx, "wx_b");

return l;
}
```

First, we create initializer, which will initialize startup values of network parameters. Then we defined bias and weight parameters, and join them in form of linear model “$wx+b$”, and returned as Function type. The createModel function is called in the main method. Once the model is created, we can exam it, and prove there are only two parameters in the model. The following code create the Linear Regression model, and print model parameters:

```//Step 3: create linear regression model
var lr = createLRModel(x, device);
//Network model contains only two parameters b and w, so we query
//the model in order to get parameter values
var paramValues = lr.Inputs.Where(z => z.IsParameter).ToList();
var totalParameters = paramValues.Sum(c => c.Shape.TotalSize);
Console.WriteLine(\$"LRM has {totalParameters} params, {paramValues[0].Name} and {paramValues[1].Name}.");
```

In the previous code, we have seen how to extract parameters from the model. Once we have parameters, we can change its values, or just print those values for the further analysis.

Step 4: Create Trainer, which will be used to train network parameters w and b. The following code snippet shows implementation of Trainer method.

```public Trainer createTrainer(Function network, Variable target)
{
//learning rate
var lrate = 0.082;
var lr = new TrainingParameterScheduleDouble(lrate);
//network parameters
var zParams = new ParameterVector(network.Parameters().ToList());

//create loss and eval
Function loss = CNTKLib.SquaredError(network, target);
Function eval = CNTKLib.SquaredError(network, target);

//learners
//
var llr = new List();
var msgd = Learner.SGDLearner(network.Parameters(), lr);

//trainer
var trainer = Trainer.CreateTrainer(network, loss, eval, llr);
//
return trainer;
}
```

First we defined learning rate the main neural network parameter. Then we create Loss and Evaluation functions. With those parameters we can create SGD learner. Once the SGD learner object is instantiated, the trainer is created by calling CreateTrainer static CNTK method, and passed it further as function return. The method createTrainer is called in the main method:

```//Step 4: create trainer
var trainer = createTrainer(lr, y);
```

Step 5: Training process: Once the variables, data set, network model and trainer are defined, the training process can be started.

```//Ştep 5: training
for (int i = 1; i <= 200; i++)
{
var d = new Dictionary();
//
trainer.TrainMinibatch(d, true, device);
//
var loss = trainer.PreviousMinibatchLossAverage();
var eval = trainer.PreviousMinibatchEvaluationAverage();
//
if (i % 20 == 0)
Console.WriteLine(\$"It={i}, Loss={loss}, Eval={eval}");

if(i==200)
{
//print weights
var b0_name = paramValues[0].Name;
var b0 = new Value(paramValues[0].GetValue()).GetDenseData(paramValues[0]);
var b1_name = paramValues[1].Name;
var b1 = new Value(paramValues[1].GetValue()).GetDenseData(paramValues[1]);
Console.WriteLine(\$" ");
Console.WriteLine(\$"Training process finished with the following regression parameters:");
Console.WriteLine(\$"b={b0[0][0]}, w={b1[0][0]}");
Console.WriteLine(\$" ");
}
}
}
```

As can be seen, in just 200 iterations, regression parameters got the values we almost expected $b_0=0.995$, and $w=2.005$. Since the training process is different than classic regression parameter determination, we cannot get exact values. In order to estimate regression parameters, the neural network uses iteration methods called Stochastic Gradient Decadent, SGD. On the other hand, classic regression uses regression analysis procedures by minimizing the least square error, and solve system equations where unknowns are b and w.
Once we implement all code above, we can start LR demo by pressing F5. Similar output window should be shown:

Hope this blog post can provide enough information to start with CNTK C# and Machine Learning. Source code for this blog post can be downloaded here.

# Data Preparation Tool for Machine Learning

Regardless of machine learning library you use, the data preparation is the first and one of the most important step in developing predictive models. It is very often case that the data supposed to be used for the training is dirty with lot of unnecessary columns, full of missing values, un-formatted numbers etc. Before training the data must be cleaned and properly defined in order to get good model. This is known as data preparation. The data preparation consist of cleaning the data, defining features and labels, deriving the new features from the existing data, handling missing values, scaling the data etc.  It can be concluded that the total time we spend in ML modelling,the most of it is related to data preparation.

In this blog post I am going to present the simple tool which can significantly reduce the preparation time for ML. The tool simply loads the data in to GUI, and then the user can define all necessary information. Once the data is prepared user can store the data it to files which can be then directly imported into ML algorithm such as CNTK.

The following image shows the ML Data Preparation Tool main window.

From the image above, the data preparation can be achieved in several steps.

1. Load dirty data into ML Prep Tool, by pressing Import Data button
2. Transform the data by providing the flowing:
1. Type – each column can be:
1. Numeric – which holds continuous numeric values,
2. Binary – which indicates two class categorical data,
3. Category – which indicates categorical data with more than two classes,
4. String – which indicate the column will not be part of training and testing data set,
2. Encoding – in case of Binary and Category column type, the encoding must be defined. The flowing encoding is supported:
1. Binary Encoding with (0,1) – first binary values will be 0, and second binary values will be 1.
2. Binary encoding with (-1,1) – first binary values will be -1, and second binary values will be 1.
3. Category Level- which each class treats as numeric value. In case of 3 categories(R,G, B), encoding will be (0,1,2)
4. Category 1:N- implements One-Hot vector with N columns. In case of 3 categories(R,G, B), encoding will be R =  (1,0,0),G =  (0,1,0), B =  (0,0,1).
5. Category 1:N-1(0) – implements dummy coding with N-1 columns. In case of 3 categories(R, G, B), encoding will be R =  (1,0),G =  (0,1), B =  (0,0).
6. Category 1:N-1(-1) – implements dummy coding with N-1 columns. In case of 3 categories(R, G, B), encoding will be R =  (1,0),G =  (0,1), B =  (-1,-1).
3. Variable – defines features and label. Only one label, and at least one features can be defined. Also the column can be defined as Ignore variable, which will skip that column.  The following options are sported:
1. Input – which identifies the column as feature or predictor,
2. Output – which identifies the column as label or model output.
4. Scaling – defines column scaling. Two scaling options are supported:
1. MinMax,
2. Gauss Standardization,
5. Missing Values – defines the replacement for the missing value withing the column. There are several options related to numeric and two options (Random and Mode ) for categorical type.
3. Define the testing data set size by providing information of row numbers or percent.
4. Define export options
5. Press Export Button.

As can be seen this is straightforward workflow of data preparation.

Besides the general export options which can be achieved by selecting different delimiter options, you can export data set in to CNTK format, which is very handy if you play with CNTK.

After data transformations, the user need to check CNTK format int the export options and press Export in order to get CNTK training and testing files, which can be directly used in the code without any modifications.

Some of examples will be provided int he next blog post.

The project is hosted at GitHub, where the source code can be freely downloaded and used at this location: https://github.com/bhrnjica/MLDataPreparationTool .

In case you want only binaries, the release of version v1.0 is published here: https://github.com/bhrnjica/MLDataPreparationTool/releases/tag/v1.0

# Using CNTK and C# to train Mario to drive Kart

## Introduction

In this blog post I am going to explain one of possible way how to implement Deep Learning ML to play video game. For this purpose I used the following:

1.  N64 Nintendo emulator which can be found here,
2. Mario Kart 64 ROM, which can be found on internet as well,
3. CNTK – Microsoft Cognitive Toolkit
4. .NET Framework and C#

The idea behind this machine learning project is to capture images together with action, while you play Mario Kart game. Then captured images are transformed into features of training data set, and action keys into label hot vectors respectively.  Since we need to capture images, the emulator should be positioned at fixed location and size during playing the game, as well as during testing algorithm to play game. The flowing image shows N64 emulator graphics configuration settings.

Also the N64 emulator is positioned to Top-Left corned of screen, so it is easier to capture the images.

## Data collection for training data set

During image captures game is played as you would play normally. Also no special agent, not platform is required.

In .NET and C# it is implemented image capture from the specific position of screen, as well as it is recorded which keys are pressed during game play. In order to record keys press, the code found here is modified and used.

The flowing image shows the position of N64 emulator with playing Mario Kart game (1), the windows which is capture and transform the image (2), and the application which collect images, and key press action and generated training data set into file(3).

The data is generated on the following way:

• each image is captured, resized to 100×74 pixels and gray scaled prior to be transformed and persisted to data set training file.
• before image is persisted the hotkey of action key press is recorded and connected to image.

So the training data is persisted into CNTK format which consist of:

1. |label – which represent 5 component hot vector, indicate: Forward, Break, Forward-Left, Forward-Right and None (1 0 0 0 0)
2. |features consist of 100×74 numbers which represent pixels of the images.

The following data sample shows how training data set are persisted in the txt file:

```|label 1 0 0 0 0 |features 202 202 202 202 202 202 204 189 234 209 199...
|label 0 1 0 0 0 |features 201 201 201 201 201 201 201 201 203 18...
|label 0 0 1 0 0 |features 199 199 199 199 199 199 199 199 199 19...
|label 0 0 0 1 1 |features 199 199 199 199 199 199 199 199 199 19...
```

Since my training data is more than 300 000 MB of size, I provided just few BM sized file, but you can generate file as big as you wish with just playing the game, and running the flowing code from Program.cs file:

await GenerateData.Start();

## Training Model to play the game

Once we generate the data, we can move to the next step: training RCNN model to play the game. For training model the CNTK is used. Also since we play a game and previous sequence will determined the next sequence in the game, LSTM RNN is used. More information about CNTK and LSTM can be found in previous posts. In my case I have collected nearly 15000 images during several round of playing the same level and route. Also for more accurate model much more images should be collected, nearly 100 000. The model is trained in one hour, with 500000 iterations. The source code about whole project can be found on GitHub page. (http://github.com/bhrnjica/LSTMBotGame )

By running the following code, the training process is started with provided training data:

CNTKDeepNN.Train(DeviceDescriptor.GPUDevice(0));

## Playing the game with CNTK model

Once we trained the model, we move to the next step: playing a game. The emulator should be positioned on the same position and with the same size in order to play the game.ONce the model is trained and created in th training folder, the playing game can be achive by running:

var dev = DeviceDescriptor.CPUDevice;
MarioKartPlay.PlayGame(dev);

How it looks like on my case, you can see on this youtube video:

# Deploy CNTK model to Excel using C#

In the last blog post, we saw how to save model state (checkpoint) in order to load it and train again. Also we have seen how to save model for the evaluation and testing. In fact we have seen how to prepare the model to be production ready.

Once we finish with the modelling process, we enter in to production phase, to install the model on place where we can use it for solving real world problems. This blog post is going to describe the process how deployed CNTK model can be exported to Excel like as AddIn and be used like ordinary Excel formula.

## Preparing and deploying CNTK model

From the previous posts we saw how to train and save the model. This will be our starting point for this post.

Assume we trained and saved the model for evaluation from the previous blog post with file name as “IrisModel.model”. The model calculates Iris flower based on 4 input parameters, as we saw earlier.

1. The first step is to create  .NET Class Library and install the following Nugget packages
1. CNTK CPU Only ver. 2.3
3. Include saved IrisModel.model file in the project as Content and should be copied in Debug folder of the application.

As we can see, for this export we need Excel Dna Addin, fantastic library for making anything as Excel Addin. It can be install as Nuget package, and more information can be found at http://excel-dna.net/.

The following picture shows above 3 actions.

1. Change the Class.cs name into IrisModel.cs, and implement two methods:
1. public static string IrisEval(object arg) and
2. private static float EvaluateModel(float[] vector).

The first method is direct Excel function which will be called in the excel, and the second method is similar from the previous blog post for model evaluation. The following code snippet shows the implementation for the methods:

```[ExcelFunction(Description = "IrisEval - Prediction for the Iris flower based on 4 input values.")]
public static string IrisEval(object arg)
{
try
{
//First convert object in to array
object[,] obj = (object[,])arg;

//create list to convert values
List<float> calculatedOutput = new List<float>();
//
foreach (var s in obj)
{
var ss = float.Parse(s.ToString(), CultureInfo.InvariantCulture);
}
if (calculatedOutput.Count != 4)
throw new Exception("Incorrect number of input variables. It must be 4!");
return EvaluateModel(calculatedOutput.ToArray());
}
catch (Exception ex)
{
return ex.Message;
}

}
private static string EvaluateModel(float[] vector)
{

//extract features and label from the model
Variable feature = ffnn_model.Arguments[0];
Variable label = ffnn_model.Output;

Value xValues = Value.CreateBatch<float>(new int[] { feature.Shape[0] }, vector, DeviceDescriptor.CPUDevice);
//Value yValues = - we don't need it, because we are going to calculate it

//map the variables and values
var inputDataMap = new Dictionary<Variable, Value>();
var outputDataMap = new Dictionary<Variable, Value>();

//evaluate the model
ffnn_model.Evaluate(inputDataMap, outputDataMap, DeviceDescriptor.CPUDevice);
//extract the result  as one hot vector
var outputData = outputDataMap[label].GetDenseData<float>(label);
var actualLabels = outputData.Select(l => l.IndexOf(l.Max())).ToList();
var flower = actualLabels.FirstOrDefault();
var strFlower = flower == 0 ? "setosa" : flower == 1 ? "versicolor" : "virginica";
return strFlower;
}<span style="display: inline-block; width: 0px; overflow: hidden; line-height: 0;" data-mce-type="bookmark" class="mce_SELRES_start"></span>
```

That is all we need for model evaluation in Excel.

Notice that the project must be build with the x64 architecture, and also installed Excel must be in x64 version.  This demo will not work in Excel 32 bits.

Rebuild the project and open Excel file with Iris Data set. You can use the file included in the demo project,  specially prepared for this blog post.

• Got to Excel – > Options -> Addins,

• Start typing the Excel  formula :
• IrisEval(A1:D1) and press Enter. And the magic happen.

Complete source code for this blog post can be found here.

# Uvod u R jezik i mašinsko učenje- youtube lekcije

Do sada urađeno 7 kratkih lekcija u kojima sam pokusao objasniti osnove programskog jezika R i kako koristiti funkcije za statističku i analitičku obradu podataka. Kratkih uradaka biće još, ovo je samo početak. U narednom tekstu možete vidjeti sve do sada objavljene lekcije.