CNTK Evaluate Hidden Layers

This page describes how to expose a trained model's hidden layer’s values.

Overview

A CNTK model is built on interconnected layers. Some of these layers can be evaluated using the EvalDLL because they are tagged as being 'output' layers. In order to expose other layers through the EvalDLL, these layers must be tagged as output layers by adding them to the OutputNodes property.

For example, the 01_OneHidden.cntk configuration file refers to the 01_OneHidden.ndl file for the network definition. In this network description file, we have two layers defined:

h1 = DNNSigmoidLayer(featDim, hiddenDim, featScaled, 1)  
ol = DNNLayer(hiddenDim, labelDim, h1, 1)  

But only one layer is marked as an output:

OutputNodes = (ol)

Thus, the EvalDLL can only return values pertaining to the ol layer during evaluation.

In order to be able to evaluate the h1 hidden layer, we need to expose it first as an output node. There are 3 possible ways of doing this:

1. Re-train the model with the OutputNodes property listing the desired layers.
2. Modifying an already trained model using MEL and re-define the output nodes to create a copy of the trained model exposing the layers.
3. Modifying an already trained model while loading it for evaluation using the EvalDLL/EvalWrapper modules.

1. Re-training model

As option number one, we could just add the h1 layer as an output to the 01_OneHidden.ndl file, and that layer would be available for reading during evaluation:

OutputNodes=(h1:ol)

However this implies the model would need to be (re)trained with this configuration.

2. Modifying an already trained model

If the model has already been trained, and re-training is not an option, there is an alternate way to modify the trained model so that other layers can be exposed as output layers.

This alternate method to re-training, will modify an already trained model using a MEL (Model Edit Language) script and mark the desired layer(s) as output(s). The steps to enable a layer as an output layer on a trained model are the following:

• Add a new configuration file similar to the original file invoking an edit command involving a MEL script.
• Add the MEL script file for re-defining the OutputNodes property to include the desired layer(s).
• Run CNTK with the new configuration and MEL files against the trained model, producing a new trained model with the desired output layers.

These steps DO NOT RE-TRAIN the model, they simply expose the desired layer(s) as output layers.

The following section shows how to modify 01_OneHidden model located in the <cntk>\Examples\Images\MNIST folder as an example.

First, we need a new configuration file for 01_OneHidden based on the original one but adding a new edit command for modifying the trained network. Assume the new configuration file is named 01_OneHidden_h1.cntk, and located in the Config folder. This file contains the following definition:

# Parameters can be overwritten on the command line
# for example: cntk configFile=myConfigFile RootDir=../.. 
# For running from Visual Studio add
# currentDirectory=$(SolutionDir)/<path to corresponding data folder> 
RootDir = ".."

ConfigDir = "$RootDir$/Config"
DataDir = "$RootDir$/Data"
OutputDir = "$RootDir$/Output"
ModelDir = "$OutputDir$/Models"

deviceId = -1 #"auto"

command = edit

precision = "float"
modelPath = "$ModelDir$/01_OneHidden"
ndlMacros = "$ConfigDir$/Macros.ndl"

# comment the following line to write logs to the console 
#stderr = "$OutputDir$/01_OneHidden_out"
traceLevel=1
numMBsToShowResult=500

#networkDescription = "$ConfigDir$/01_OneHidden.ndl"
edit=[    
    action=edit
    CurModel = "$ModelDir$/01_OneHidden"
    NewModel = "$ModelDir$/01_OneHiddenMod"
    editPath = "$ConfigDir$/editOutput.mel"
]

Notice the edit action which will take the originally trained model (01_OneHidden) and modify it into a new model (01_OneHiddenMod) using the editOutput.mel script.

Second, we add a MEL script file to mark the network’s h1 layer as output (assume this file name 01_OneHidden.mel):

m1 = LoadModel($CurModel$, format=cntk)
SetDefaultModel(m1)

m1 = [
    featDim = 784
    labelDim = 10
    hiddenDim = 200

    features = Input(featDim)
    featScale = Const(0.00390625)
    featScaled = Scale(featScale, features)
    labels = Input(labelDim)

    # DNNSigmoidLayer and DNNLayer are defined in Macros.ndl
    h1 = DNNSigmoidLayer(featDim, hiddenDim, featScaled, 1)
    ol = DNNLayer(hiddenDim, labelDim, h1, 1)

    ce = CrossEntropyWithSoftmax(labels, ol)
    err = ErrorPrediction(labels, ol)

    # Special Nodes
    FeatureNodes = (features)
    LabelNodes = (labels)
    CriterionNodes = (ce)
    EvalNodes = (err)
    OutputNodes = (h1:ol)
]

SaveModel(m1, $NewModel$, format=cntk)

This file contains the network definition found in 01_OneHidden.cntk, as well as a modified OutputNodes property:

OutputNodes = (h1:ol)

In this case we prepended the h1 layer to the list of output layers. We use the colon character (‘:’) to delimit layers.

Finally, with these two files in the Config directory, we can now run CNTK to edit the network (we run from the Data directory):

cntk configFile=”../Config/01_OneConfig_h1.cntk”

After running (successfully) we have the 01_OneHiddenMod model in the Output/Models directory. Running the C# Example (CSEvalClient), its output shows in the EvaluateModelMultipleLayers method the output of both the h1 and ol layers.

3. Modifying an already trained model while loading it for evaluation using the EvalDLL/EvalWrapper modules.

If the model has already been trained, and it will be evaluated using the EvalDLL/EvalWrapper modules, then it is just a matter of adding the outputNodeNames property with a colon separated list of nodes to the network definition:

outputNodeNames="h1.z:ol.z"

When creating the network, the Eval engine will recognize the outputNodeNames property and replace the model's output nodes with the list of nodes specified in the outputNodeNames property.

Looking at the code inside the CPPEvalClient example project, shows the (uncommented) line specifying the outputNodeNames property:

networkConfiguration += "outputNodeNames=\"h1.z:ol.z\"\n";
networkConfiguration += "modelPath=\"" + modelFilePath + "\"";
model->CreateNetwork(networkConfiguration);

Running the program shows the corresponding output for the h1.z layer.