Descriptive statistics and data normalization with CNTK and C#


As you probably know CNTK is Microsoft Cognitive Toolkit for deep learning. It is open source library which is used by various Microsoft products. Also the CNTK is powerful library for developing custom ML solutions from various fields with different platforms and languages. What is also so powerful in the CNTK is the way of the implementation. In fact the library is implemented as series of computation graphs, which  is fully elaborated into the sequence of steps performed in a deep neural network training.

Each CNTK compute graph is created with set of nodes where each node represents numerical (mathematical) operation. The edges between nodes in the graph represent data flow between operations. Such a representation allows CNTK to schedule computation on the underlying hardware GPU or CPU. The CNTK can dynamically analyze the graphs in order to to optimize both latency and efficient use of resources. The most powerful part of this is the fact thet the CNTK can calculate derivation of any constructed set of operations, which can be used for efficient learning  process of the network parameters. The flowing image shows the core architecture of the CNTK.

On the other hand, any operation can be executed on CPU or GPU with minimal code changes. In fact we can implement method which can automatically takes GPU computation if available. The CNTK is the first .NET library which provide .NET developers to develop GPU aware .NET applications.

What this exactly mean is that with this powerful library you can develop complex math computation directly to GPU in .NET using C#, which currently is not possible when using standard .NET library.

For this blog post I will show how to calculate some of basic statistics operations on data set.

Say we have data set with 4 columns (features) and 20 rows (samples). The C# implementation of this 2D array is show on the following code snippet:

static float[][] mData = new float[][] {
new float[] { 5.1f, 3.5f, 1.4f, 0.2f},
new float[] { 4.9f, 3.0f, 1.4f, 0.2f},
new float[] { 4.7f, 3.2f, 1.3f, 0.2f},
new float[] { 4.6f, 3.1f, 1.5f, 0.2f},
new float[] { 6.9f, 3.1f, 4.9f, 1.5f},
new float[] { 5.5f, 2.3f, 4.0f, 1.3f},
new float[] { 6.5f, 2.8f, 4.6f, 1.5f},
new float[] { 5.0f, 3.4f, 1.5f, 0.2f},
new float[] { 4.4f, 2.9f, 1.4f, 0.2f},
new float[] { 4.9f, 3.1f, 1.5f, 0.1f},
new float[] { 5.4f, 3.7f, 1.5f, 0.2f},
new float[] { 4.8f, 3.4f, 1.6f, 0.2f},
new float[] { 4.8f, 3.0f, 1.4f, 0.1f},
new float[] { 4.3f, 3.0f, 1.1f, 0.1f},
new float[] { 6.5f, 3.0f, 5.8f, 2.2f},
new float[] { 7.6f, 3.0f, 6.6f, 2.1f},
new float[] { 4.9f, 2.5f, 4.5f, 1.7f},
new float[] { 7.3f, 2.9f, 6.3f, 1.8f},
new float[] { 5.7f, 3.8f, 1.7f, 0.3f},
new float[] { 5.1f, 3.8f, 1.5f, 0.3f},};

If you want to play with CNTK and math calculation you need some knowledge from Calculus, as well as vectors, matrix and tensors. Also in CNTK any operation is performed as matrix operation, which may simplify the calculation process for you. In standard way, you have to deal with multidimensional arrays during calculations. As my knowledge currently there is no .NET library which can perform math operation on GPU, which constrains the .NET platform for implementation of high performance applications.

If we want to compute average value, and standard deviation for each column, we can do that with CNTK very easy way. Once we compute those values we can used them for normalizing the data set by computing standard score (Gauss Standardization).

The Gauss standardization is calculated by the flowing term:

nValue= \frac{X-\nu}{\sigma},
where X- is column values, \nu – column mean, and \sigma– standard deviation of the column.

For this example we are going to perform three statistic operations,and the CNTK automatically provides us with ability to compute those values on GPU. This is very important in case you have data set with millions of rows, and computation can be performed in few milliseconds.

Any computation process in CNTK can be achieved in several steps:

1. Read data from external source or in-memory data,
2. Define Value and Variable objects.
3. Define Function for the calculation
4. Perform Evaluation of the function by passing the Variable and Value objects
5. Retrieve the result of the calculation and show the result.

All above steps are implemented in the following implementation:

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using CNTK;
namespace DataNormalizationWithCNTK
{
    class Program
    {
       static float[][] mData = new float[][] {
        new float[] { 5.1f, 3.5f, 1.4f, 0.2f},
        new float[] { 4.9f, 3.0f, 1.4f, 0.2f},
        new float[] { 4.7f, 3.2f, 1.3f, 0.2f},
        new float[] { 4.6f, 3.1f, 1.5f, 0.2f},
        new float[] { 6.9f, 3.1f, 4.9f, 1.5f},
        new float[] { 5.5f, 2.3f, 4.0f, 1.3f},
        new float[] { 6.5f, 2.8f, 4.6f, 1.5f},
        new float[] { 5.0f, 3.4f, 1.5f, 0.2f},
        new float[] { 4.4f, 2.9f, 1.4f, 0.2f},
        new float[] { 4.9f, 3.1f, 1.5f, 0.1f},
        new float[] { 5.4f, 3.7f, 1.5f, 0.2f},
        new float[] { 4.8f, 3.4f, 1.6f, 0.2f},
        new float[] { 4.8f, 3.0f, 1.4f, 0.1f},
        new float[] { 4.3f, 3.0f, 1.1f, 0.1f},
        new float[] { 6.5f, 3.0f, 5.8f, 2.2f},
        new float[] { 7.6f, 3.0f, 6.6f, 2.1f},
        new float[] { 4.9f, 2.5f, 4.5f, 1.7f},
        new float[] { 7.3f, 2.9f, 6.3f, 1.8f},
        new float[] { 5.7f, 3.8f, 1.7f, 0.3f},
        new float[] { 5.1f, 3.8f, 1.5f, 0.3f},};
        static void Main(string[] args)
        {
            //define device where the calculation will executes
            var device = DeviceDescriptor.UseDefaultDevice();

            //print data to console
            Console.WriteLine($"X1,\tX2,\tX3,\tX4");
            Console.WriteLine($"-----,\t-----,\t-----,\t-----");
            foreach (var row in mData)
            {
                Console.WriteLine($"{row[0]},\t{row[1]},\t{row[2]},\t{row[3]}");
            }
            Console.WriteLine($"-----,\t-----,\t-----,\t-----");


            //convert data into enumerable list
            var data = mData.ToEnumerable<IEnumerable<float>>();

            
            //assign the values 
            var vData = Value.CreateBatchOfSequences<float>(new int[] {4},data, device);
            //create variable to describe the data
            var features = Variable.InputVariable(vData.Shape, DataType.Float);

            //define mean function for the variable
            var mean =  CNTKLib.ReduceMean(features, new Axis(2));//Axis(2)- means calculate mean along the third axes which represent 4 features
            
            //map variables and data
            var inputDataMap = new Dictionary<Variable, Value>() { { features, vData } };
            var meanDataMap = new Dictionary<Variable, Value>() { { mean, null } };

            //mean calculation
            mean.Evaluate(inputDataMap,meanDataMap,device);
            //get result
            var meanValues = meanDataMap[mean].GetDenseData<float>(mean);

            Console.WriteLine($"");
            Console.WriteLine($"Average values for each features x1={meanValues[0][0]},x2={meanValues[0][1]},x3={meanValues[0][2]},x4={meanValues[0][3]}");

            //Calculation of standard deviation
            var std = calculateStd(features);
            var stdDataMap = new Dictionary<Variable, Value>() { { std, null } };
            //mean calculation
            std.Evaluate(inputDataMap, stdDataMap, device);
            //get result
            var stdValues = stdDataMap[std].GetDenseData<float>(std);
            
            Console.WriteLine($"");
            Console.WriteLine($"STD of features x1={stdValues[0][0]},x2={stdValues[0][1]},x3={stdValues[0][2]},x4={stdValues[0][3]}");

            //Once we have mean and std we can calculate Standardized values for the data
            var gaussNormalization = CNTKLib.ElementDivide(CNTKLib.Minus(features, mean), std);
            var gaussDataMap = new Dictionary<Variable, Value>() { { gaussNormalization, null } };
            //mean calculation
            gaussNormalization.Evaluate(inputDataMap, gaussDataMap, device);

            //get result
            var normValues = gaussDataMap[gaussNormalization].GetDenseData<float>(gaussNormalization);
            //print data to console
            Console.WriteLine($"-------------------------------------------");
            Console.WriteLine($"Normalized values for the above data set");
            Console.WriteLine($"");
            Console.WriteLine($"X1,\tX2,\tX3,\tX4");
            Console.WriteLine($"-----,\t-----,\t-----,\t-----");
            var row2 = normValues[0];
            for (int j = 0; j < 80; j += 4)
            {
                Console.WriteLine($"{row2[j]},\t{row2[j + 1]},\t{row2[j + 2]},\t{row2[j + 3]}");
            }
            Console.WriteLine($"-----,\t-----,\t-----,\t-----");
        }

        private static Function calculateStd(Variable features)
        {
            var mean = CNTKLib.ReduceMean(features,new Axis(2));
            var remainder = CNTKLib.Minus(features, mean);
            var squared = CNTKLib.Square(remainder);
            //the last dimension indicate the number of samples
            var n = new Constant(new NDShape(0), DataType.Float, features.Shape.Dimensions.Last()-1);
            var elm = CNTKLib.ElementDivide(squared, n);
            var sum = CNTKLib.ReduceSum(elm, new Axis(2));
            var stdVal = CNTKLib.Sqrt(sum);
            return stdVal;
        }
    }

    public static class ArrayExtensions
    {
        public static IEnumerable<T> ToEnumerable<T>(this Array target)
        {
            foreach (var item in target)
                yield return (T)item;
        }
    }
}

The output for the source code above should look like:

Advertisements

Using CNTK with Visual Studio 2017 and Python


In the next few steps will show how to install CNTK and python environment in Visual Studio 2017.

  1. First download the latest CNTK version from the official GitHub page, or just click on the following link: https://github.com/Microsoft/CNTK/releases

The release page will show the latest bits. Click on the CPU only package, accept the license and download the zip file.

  1. Once that you have zip file on your PC, create the folder C:/local on disk and unzip the package in to it.
  2. The next step performs the installation of the library as well as installation of the Python related distribution anaconda 4.1.1.
  3. Open C:\local\cntk\Scripts\install\windows path and run install.bat file. You will need administrative rights in order to successfully install all required components.
  4. The following image shows the installation process:

  1. As can be seen first you have to run batch file (step 2), then press 1 and ENTER in order to continue with the installation process and press ‘y‘, to perform downloading required components.
  2. The installation process takes several minutes to complete. The first component to be installed is Anaconda 4.1.1 which is needed in order to setup  CNTK.

  1. Once the anaconda is installed, the process of CNTK installation starts and passes very quickly since we already download all CNTK bits.

  1. Now that we have CNTK installed, the last installation step is installation of the Visual Studio Tool for Python.
  2. Run the Visual Studio 2017 Installer and after the installed is show, just select the python components similar picture shows below:

  1. Once the installation is completed run Visual Studio 2017.
  2. From the Visual Studio 2017 Tool menu select Python and then select Python Environment:

  1. From the Python Environment window select Anaconda 4.1.1 and update symbols DB, by pressing the button pointed on the image below:

  1. Once we have environment updated, Press “Make this the default environment for the new projects” option in order to apply the environment for the future Python CNTK based projects.
  2. Also the path for Python and Python scripts should be registered in Global Environment OS.

  1. Once the previous steps are performed successfully, we can start writing CNTK aware python code in Visual Studio 2017.
  2. OPen VS 2017 and Anaconda 4.1.1 environment and type.

   import cntk

print(“CNTK verion:”, cntk__version__)

  1. Similar output should be appear
  2. print(“CNTK version:”, cntk.__version__)

 

Introduction to Microsoft Machine Learning package in Microsoft R Server 9.0


MicrosoftML package

Microsoft has released Microsoft R Server 9.0 (MRS9.0) with very interesting package called MicrosoftML. “Micrsooft ML” stands for Microsoft Machine Learning R package which you can use on R Server. R Server is commercial version of popular R Client distribution, which solves mayor problems when working with R. R Server contains set of cutting-edge technology to work with big data, as well as set of enhanced packages for parallelization and distributing computing.
MRS 9.0 is coming with “MicrosoftML” package which contains set of several Machine Learning algorithms developed in various Microsoft products in the last 10 years. You can combine the algorithms delivered in this package with pre-existing parallel external memory algorithms such as the RevoScaleR package as well as open source innovations such as CRAN R packages to deliver the best predictive analytic.
MicrosoftML package includes the following algorithms:

  • Fast linear learner, with support for L1 and L2 regularization,
  • Fast boosted decision tree,
  • Fast random forest,
  • Logistic regression, with support for L1 and L2 regularization,
  • GPU-accelerated Deep Neural Networks (DNNs) with convolutions,
  • Binary classification using a One-Class Support Vector Machine.

How to start with MicrosoftML package

In order to fully use the power of MicrosoftML, and RevoScaleR you need to download MRS 9.0 from the MSDN or Visual Studio Dev Essentials subscription. Once the zip file is downloaded, unzip it, and run setup file.

The following required components  were missing when my installation is started.  Seems the MRS contains the latest .NET Core components, which is pretty cool:

 

blog_mrs_9_11

After the prerequested components installed, the MRS installation process can start.

blog_mrs_9_01

By clicking the Next button the Installation process starts:

blog_mrs_9_02

Select the path where you want to install MRS, and press the Next button:

blog_mrs_9_03

If everything went ok, the installation process is finished after less than minute, and the final dialog window appears:

blog_mrs_9_04

By clicking the Finish button MRS is installed on you PC.

Run MRS 9.0 by using R Tool fo Visual Studio, RTVS

Now it is time to run some R code. YOu have two posibilities to run R code. The first option is that you use the R Studio proffesion tool for running R code. It is free and open source which you can download from rstudio.com. If you are MS Developer you usualy write the code in the Visual Studio. So you can download RTVS from this link and run R code from Visual Studio.

Now that you have right tool to run R code, we can start with setting the MRS environment.

First thing you should do is to point RTVS to use MRS 9.0 instead of curently using some other distribution. So open the Visual Studio, select R Tools->Edit Options

blog_mrs_9_08

The Option dialog appears. Set the R Engine to point installation folder of the MRS. Since my installation location was on Program Files folder, the picture below show my installation path.

blog_mrs_9_07

After you set the right installation folder , restart the Visual Studio:

When the Visual Studio is running, open R Open R Interactive window. You should have similar text if you set up MRS path correctly:

blog_mrs_9_13

Select New Project from the File->New menu option.

blog_mrs_9_10

Name it FirstRServerDemo and click Ok. Now you are ready to write first MRS R code:

blog_mrs_9_14

In the next post we will continue exploration the MicrosoftML library package and new set of Machine Learning algorithms added in this latest version.

Details of my session at ATD12


atd12_sl02.png

Today, I gave session at Advanced Technology Day conference in Zagreb. It was very excited to see full room of people at the presentation, mostly developers from .NET world interesting in R and Data Science. This is good sign that the Data Science and the R are becoming more and more popular at daily basis. Most popularity for the R will bring R Tool for Visual Studio, which means the R language became member of the family of the Visual Studio.

atd12_sl01.jpg

For those who were asking about my slides and demo sample here is the information:

  1. Presentation slides can be downloaded here,
  2. Source code of the demo is hosted at git hub at: http://github.com/bhrnjica/R-Workshop
  3. For more information about R , you can see my YouTube channel about R and Machine Learning at this link.

See you next time!

Visual Studio vNext – The New Installer


Download Visual Studio 15 Preview 3

The new version of Visual Studio will come with dramatically new installer, which will allow that you install only stuff you need, without gigabytes of unnecessary never used components. Current version of Visual Studio which is Visual Studio 2015 Update 3 is coming with nearly 8GB installation file. This is to much for the installer, you need special condition when you want to download the installation file. I am doing it by night, when I am sleeping. In some condition the installation process takes an hour to install everything you have specified.

In the next version the installation process will be changed and if you want to see and feel how the future visual studio installer  will look like you can download the preview of the Visual Studio vnext code name  “Visual Studio 15” at this link.

If you try to install Visual Studio 15 preview 3, it will take less than 5 minutes, with very simple installer. In the next five pictures whole installation process is completed.

After you download the installer, run it and the following pictures will appear:

  1. First picture is asking to confirm the installation process:

vs15_sl01

2. The next picture shows the progress of loading installer

vs15_sl02

3. The next picture is the main picture which you can select what to install. The whole Visual Studio installer is devided in to the development groups:

  1. Core Stuff of the Visual Studio- this component is required for all developer group
  2. There are for now 4 installer groups: .NET, C++, Python, Game dev.
  3. The more will come later.
    vs15_sl03

4. After you select right developer group/groups installation process starts by pressing Install button.

vs15_sl04

5. After the installation process is completed, the following picture appear, which you only need to close by pressing the Close button at the right top edge of the window.

vs15_sl05

As we can see the next version of the Visual Studio will dramatically changed the installation process, offering new simple and effective installer.

Objektno Orjentisano Programiranje- prvi dio


Objektno orijentisano programiranje (OOP) predstavlja način programiranja pri čemu se definišu objekti koji poprimaju osobine objekata iz realnog svijeta. Nastao na ovom konceptu, OOP s jedne strane proširuje funkcionalnosti osnovnih i izvedenih tipova na način da implementaciju enkapsulira (ugrađuje) u sami tip, dok s druge strane uvodi nove osobine tipova poput nasljeđivanja i polimorfizma koji daju nove funkcionalnosti i način implementacije. Sve tri paradigme OOP (enkapsulacija, nasljeđivanje i polimorfizam) čine objektno orjentisano programiranje efikasnije, lakše za kolaboraciju i održavanje te u većem dijelu rješava probleme proceduralnog programiranja.

Klase

Tipovi podataka u čistom OO jeziku poput C# označavaju se klasama, koje se mogu podijeliti na osnovne i izvedene tipove. Programski jezik poput C++, osnovne tipove ne tretira kao klase, dok klasama označava samo izvedene odnosno apstraktne tipove. Osnovne tipove poput integer, double, float, bool, char u čisto OOP jeziku zovemo klasama jer u sebi enkapsuliraju metode koje nam omogućuju rad nad samim tipovima, na isti način kako to radimo i sa izvedenim tipovima. Izvedeni tipovi su svi tipovi koji su definisani od strane programera.
U čistom objektno orijentisanom programskom jeziku ne postoji mogućnost izvršavanja koda koji nije dio neke klase, odnosno koji nije definisan unutar određene metode kao članice klase. S druge strane ovo implicira da se u objektno orijentisanom programiranju može izvršiti samo onaj kod koji je definisan unutar određene klase, odnosno instanciranjem objekta i poziva neke metode članice. Ovom činjenicom se shvata naziv objektno orijentisano programiranje, kao programirnje objekata.
Naravno, programski jezici nužno ne poštuju ovu strogu činjenicu, a pored toga imaju mogućnost izvršavanja koda izvan objekata klase. Kako je ranije naglađeno programski jezik koji ima ovakvu dualnu osobinu jeste C++. C++ programski jezik je objektno orijentisani jezik koji podržava sve osobine OO koncepta, dok istovremeno predstavlja klasični proceduralno orjentisani jezik. Zbog te činjenice danas C++ jezik predstavlja najrasprostranjeniji i najviše korišten programski jezik. Pogodan je kako za programiranje sistemskih komponenti operativnih sistema i drivera za uređaje, tako i za projekte zasnovane na poslovnoj logici. Deklaracija klase u C++ sa pripadajućim članovima možemo predstaviti u obliku prijera tipa Osoba.

class Osoba
{
    //konstruktori/destruktor klase
public:
    Osoba();
    ~Osoba();  

//clanovi klase /atributi
private:
	string  imePrezime;
	time_t  datumRodjenja;
	string  mjestoRodjenja;  

	//javne metode clanica
public:
	void Init (string ime, string datum);  

	//javne virtualne metode clanice
public:
	virtual string Status();
};

Prethodni listing predstavlja deklaraciju klase Osoba. Implementacija klase u smislu implementacije konstruktora, destruktora i metoda Init i Status nalazi se izvan tijela klase, a često u posebnoj cpp datoteci. Primjer implementacije klase Osoba vidimo na sljedećem listingu.

//defoltni konstruktor
Osoba::Osoba()
{
    imePrezime = "n/a";
}  

//destruktor klase
Osoba::~Osoba()
{
}  

//virtuelna metoda koja vraca status osobe
string Osoba::Status()
{
	return "Osoba";
}  

//postavljanje imena i datuma rodjenja osobe
void Osoba::Init(string ime, string datum)
{
	imePrezime = ime;  

	//konverzija string u datum
	struct tm tm = {0};
	strptime(datum.c_str(), "%d/%m/%Y", &tm);
	datumRodjenja = mktime(&tm);
}

Prethodni listing čini implementacija kontsruktora koji se poziva prilikom formiranja objekta Osoba, dok se implementacija dstruktora nalazi odmah iza. Metoda Init postavlja ime i datum rođenja za osobu, dok metoda Status vraća string sa nazivom vrste osobe.
C# i Java predstavljaju najpopularnije OO programske jezike pored C++, u kojima sve počinje sa objektima, i nije dozvoljeno definisanje metode izvan klasa. Specijalna metoda koja označava ulaznu ili početnu tačku zove se main metoda. Ovu metodu je dovoljno definisati u bilo kojoj klasi, a kompajler će je prepoznati kao početnu metodu i označiti je kao ulaznu tačku programa. Primjer listing 1 i 2 u C# izgledaju na slijedeći način:

public class Osoba
{
	//konstruktori/destruktor klase
	public Osoba()
	{
		imePrezime = "n/a";
}  

~Osoba()
	{  

	}  

	//clanovi klase /atributi
	private string      imePrezime;
	private DateTime    datumRodjenja;
	private string      mjestoRodjenja;  

	//javne metode
	public void Init(string ime, string datum)
	{
		imePrezime = ime;
		DateTime dat;
		if(DateTime.TryParse(datum, out dat))
			datumRodjenja= dat;
	}  

	//javne virtualne metode clanice
	public virtual string Status()
	{
		return "Osoba";
	}
};

Iz prethodnih listinga možemo vidjeti određenu razliku u deklarisanju i implementaciji klase i njenih članova. U C++ vidimo da je deklaracija i implementacija metoda članica klase razdvojena. Za razliku od C++ u C# deklaracija i implementacija klase u C# se ne razdvajaju, tako da se svaka metoda članica deklariše i implementira na jednom mjestu.

Using external config files in .NET applications


The config file is place where common variables, database connection strings, web page settings and other common stuff are placed. The config file is also dynamic, so you can change the value of the variable in the config file  without compiling and deploying the .NET app. In multi tenancy environment config file can be complicate for deployment, because  for each tenant different value must be set for most of the defined variables. In such a situation you have to be careful to set right value for the right tenant.

One way of handling this is to hold separate config file for each tenant. But the problem can be variables which are the same for all tenants, and also the case where some variables can be omitted for certain tenant.

One of the solution for this can be defining external config files for only connection strings or appSettings variables, or any other custom config section. In this blog post, it will be presenting how to define connection strings as well as appSettings section in separate config file.

Lets say you have appSettings and connectionStrings config sections, similar like code below:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
    <startup> 
        <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5" />
    </startup>

	<connectionStrings>
		<add name="SQLConnectionString01" connectionString="Data Source=sourcename01;Initial Catalog=cat01;Persist Security Info=True;Integrated Security=true;"/>
		<add name="SQLConnectionString02" connectionString="Data Source=sourcename02;Initial Catalog=cat02;Persist Security Info=True;Integrated Security=true;"/>
	</connectionStrings>

	<appSettings>
		<clear />
		<!-- Here are list of appsettings -->
		<add key="Var1" value="Var1 value from config01" />
		<add key="Var2" value="Varn value from config01"/>
		<add key="Var3" value="Var3 value from main config file"/>
	</appSettings>

</configuration>

There are three appSetting keys Var1 , Var2 and Var3  and two connectionstrings in the app.config.

The config file above can be split in such a way that variables Var1 and Var2 be defined in separated file, but the Var3 can be remain in the main cofing file. Separate config file may be unique for each tenant.

Now the main config file looks like the following:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
    <startup> 
        <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5" />
    </startup>

	<connectionStrings configSource="config\connString01.config"/>

	<appSettings file="config\config01.config">
		
		<add key="Var3" value="Var3 value from main config file"/>
	</appSettings>

</configuration>

In the Visual Studio Solution there is config folder in which we created two config files for appSettings section and two config files for Connectionstrings section, in case we have two separate environments for deployments.

exconfigfile01
The flowing code snippet shows the appSettings section implemented in the external file:

<appSettings file="appSettings.config">

	<!-- Here are list of appsettings -->
	<add key="Var1" value="Var1 value from config02" />
	<!-- ... -->
	<add key="Varn" value="Varn value from config02"/>
</appSettings>

The external config file for connection strings looks similar like the flowing:

exconfigfile02

The simple console application shows how to use this config variables in the code:

static void Main(string[] args)
{
    var var1Value= ConfigurationManager.AppSettings["Var1"];
    var var2Value = ConfigurationManager.AppSettings["Var2"];
    var var3Value = ConfigurationManager.AppSettings["Var3"];
    var conn1 = ConfigurationManager.ConnectionStrings["SQLConnectionString01"];
    var conn2 = ConfigurationManager.ConnectionStrings["SQLConnectionString02"];

    Console.WriteLine("Values from config01.config and connString01.config files");

    Console.WriteLine("Var1={0}",var1Value);
    Console.WriteLine("Var2={0}", var2Value);
    Console.WriteLine("Var3={0}", var3Value);
    Console.WriteLine("ConnStr01={0}", conn1);
    Console.WriteLine("ConnStr01={0}", conn2);

    Console.Read();
}

The complete source code can be downloaded from this link.