How to visualize CNTK network in C#


When building deep learning models, it is often required to check the model for consistency and proper parameters definition. In ANNdotNET, ml network models are designed using Visual Network Designer (VND), so it is easy to see the network configuration. Beside VND, in ANNdotNET there are several visualization features on different level: network preparation, model training phase, post training evaluation, performance analysis, and export results. In this blog post we will learn how to use those features when working with deep learning models

Visualization during network preparation and model training

When preparing network and training parameters, we need information about data sets, input format and output type. This information is relevant for selecting what type of network model to configure, what types of layers we will use, and what learner to select. For example the flowing image shows  network configuration containing of 2 embedding layers, 3 dense layers and 2 dropout layers. This network configuration is used to train CNTK model for mushroom data set. As can be seen network layers are arranged as listbox items, and the user has possibility to see, on the highest level, how neural networks looks like, which layers are included in the network, and how many dimensions each layer is defined. This is very helpful, since it provides the way of building network very quickly and accurately, and it requires much less times in comparisons to use traditional way of coding the network in python, or other programming language.

Image 1: ANNdotNET Network Settings 

ANNdotNET Network Settings page provides pretty much information about the network, input and output layers, what data set are defined, as well as whole network configuration arranged in layers. Beside network related information, the Network Settings tab page also provides the learning parameters for the network training. More about Visual Network Designer the ready can find on one of the previous blog post.

Since ANNdotNET implements MLEngine which is based on CNTK, so all CNTK related visualization features could be used. The CNTK  library provides rich set of visualizations. For example you can use Tensorboard in CNTK  for visualization not just computational graph, but also training history, model evaluation etc. Beside Tensorboard, CNTK provides logger module which uses Graphviz tool for visualizing network graph. The bad news of this is that all above features cannot be run on C#, since those implementation are available only in python.

This is one of the main reason why ANNdotNET provides rich set of visualizations for .NET platform. This includes: training history, model evaluation for training and validation data set, as well as model performance analysis. The following image show some of the visualization features: the training history (loss and evaluation) of minibatches during training of mushroom model:

Moreover, the following image shows evaluation of training and validation set for each iteration during training:

Those graphs are generated during training phase, so the user can see what is happening with the model.  This is of tremendous help, when deciding when to stop the training process, or are training parameters produce good model at all, or this can be helpful in case when can stop and change parameters values. In case we need to stop the training process immediately, ANNdotNET provides Stop command which stops training process at any time.

Model performance visualization

Once the model is trained, ANNdotNET provides performance analysis tool for all three types of ML problems: regression, binary and multi class classification.

Since the mushrooms project is binary ML problem the following image shows the performance of the trained model:

Using Graphviz to visualize CNTK network graph in C#

We have seen that ANNdotNET provides all types of visualizations CNTK models, and those features are provided by mouse click through the GUI interfaces. One more feature are coming to ANNdotNET v1.1 which uses Grpahviz to visualize CNTK network graph. The feature is implemented based on original CNTK python implementation with some modification and style.

In order to use Graphviz to visualize network computation graph the following requirements must be met:

  • Install Graphviz on you machine.
  • Register Graphviz path as system variable. (See image below)

Now that you have install Graphviz tool, you can generate nice image of your network model directly in ANNdotNET just by click on Graph button above the Visual Network Designer (see image 1).

Here is some of nice graphs which can be generate from ANNdotNET preclaculated models.

Graphviz generated graph of mushrooms model implemented in ANNdotNET

In case you like this nice visualization features go to http://github.com/bhrnjica/anndotnet, download the latest version from release section or just download the source code and try it with Visual Studio, but don’t forget to give a star.

Star ANNdotNET project if you found it useful.

In the next blog post I will show you how visualization of CNTK computational graph is implemented, so you will be able to use it in your custom solutions.

Announcing GPdotNET v5 and related Book


https://www.igi-global.com/Images/Covers/9781522560050.png

After one year of writing and coding, finally I  can announce my two big achievements which are related to each other:

1. The fifth version of my open source project GPdotNET – genetic programming tool, and

2. The book:Optimized Genetic Programming Applications: Emerging Research and Opportunities, published by IGI-GLobal.

Along the book, I was developing GPdotNET application which is explained in Chapter 5. Actually the Chapter 5 described in depth all aspects of the application, with real world examples.

As can be seen GPdotNET v5 is completely rewritten application, with new logo and GUI. As Introduction of the application I have prepared several videos on youtube with quick explanation how to use some of the main modules in GPdotNET.

Using ANNdotNET – GUI tool to create CNTK based model for Iris data set


In this tutorial we are going to create and train Iris model using ANNdotNET.  ANNdotNET is windows application for creating and training CNTK based models without leaving GUI.

All procedures from downloading the data set, to exporting model, can be achieved in 6 steps.

1. Step: Download the data set file from https://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data.

2. Step: Open ANNdotNET application. Press New command, select Project 1 tree item and rename the project  into Iris Data Set.

2. Step: Select Data Command from Model Preparation ribbon group, Click File button from Import experimenal data dialog and select the recently downloaded file. Check Comma check box and press Import Data button.

3. Steps: Double click on Scaling for each column, and select MinMax normalization option from the popup ComboBox list. Double click on Type for the output column, and select Category, and 1:N for encoding. More information how to prepare data for ML you can find at https://bhrnjica.net/2018/03/01/data-preparation-tool-for-machine-learning/

4. Steps: Once the data is prepared Click Create Model Command and Model Settings panel is shown. Setup parameters as shown on the image below and click Run command.

5. Steps: Once the model is trained you can evaluate model by selecting Evaluate Command. Depending on the model type (regression, Binary or Multi class classification) The appropriate Evaluation dialog appears. Since this is multi class classification model, the Confusion matrix is shows, with micro and macron performance parameters.

6. Steps: For further analysis you can export model to Excel, or into ONNX. Also you can save the project which can later be opened and retrained again.

Note: Currently ANNdotNET is in alpha version, and more feature will come in near future.

ANNdotNET – the first GUI based CNTK tool


anndotnet-logo
ANNdotNET is windows desktop application written in C# for creating and training ANN models. The application relies on Microsoft Cognitive Toolkit, CNTK, and it is supposed to be GUI tool for CNTK library with extensions in data preprocessing, model evaluation and exporting capabilities. It is hosted at GitHub and can be clone from http://github.com/bhrnjica/anndotnet

Currently, ANNdotNET supports the folowing type of ANN:

  • Simple Feed Forward NN
  • Deep Feed Forward NN
  • Recurrent NN with LSTM

The process of creating, training, evaluating and exporting models is provided from the GUI Application and does not require knowledge for supported programming languages. The ANNdotNET is ideal for engineers which are not familiar with programming languages.

Software Requirements

ANNdotNET is x64 Windows desktop application which is running on .NET Framework 4.7.1. In order to run the application, the following requirements must be met:

– Windows 7, 8 or 10 with x64 architecture
– NET Framework 4.7.1
– CPU/GPU support.

Note: The application automatically detect GPU capability on your machine and use it in training and evaluation, otherwise it will use CPU.

How to run application

In order to run the application there are two possibilities:

Clone the GitHub repository of the application and open it in Visual Studio 2017.

  1. Change build architecture into x64, build and run the application.
  2. Download released version unzip and run ANNdotNET.exe.

The following three short videos quickly show how to create, train and evaluate regression, binary and multi class classification models.

  • Training regression model. Data set is Concrete Slump Test is downloaded from the UCI ML Repository and loaded into ANNdotNET without any modification, since the data preparation module can prepare it.

2. Training and evaluation binary classifier model. Data represent Titanic data set downloaded from the public repository.

3. Training and evaluation multi class classification models. Data represents Iris data set downloaded from the same page as above.