image_layer_rendering.md

Image Layer Rendering

The rendering of image layers is fully customizable by specifying GLSL fragment shader code for computing an RGBA output value for each pixel of the viewport based on the the single- or multi-channel values associated with the corresponding voxel.

The fragment shader code can be entered interactively from the dropdown menu for an image layer, or programmatically by specifying a 'shader' property of the JSON specification of the image layer.

Shader language

The shader code must conform to the OpenGL ES Shading Language (GLSL) version 3.0, specified at https://www.khronos.org/registry/OpenGL/specs/es/3.0/GLSL_ES_Specification_3.00.pdf.

You may find the WebGL reference card helpful: https://www.khronos.org/files/webgl20-reference-guide.pdf.

UI Controls

Rendering may depend on values specified by custom UI controls, specified by special #uicontrol directives supported by Neuroglancer as an extension to GLSL.

For example:

#uicontrol int channel slider(min=0, max=4)
#uicontrol vec3 color color(default="red")
#uicontrol float brightness slider(min=-1, max=1)
#uicontrol float contrast slider(min=-3, max=3, step=0.01)
void main() {
  emitRGB(color *
          (toNormalized(getDataValue(channel)) + brightness) *
          exp(contrast));
}

The directive syntax is:

#uicontrol <type> <name> <control>
#uicontrol <type> <name> <control>(<parameter>=<value>, ...)

which has the effect of defining a variable <name> of GLSL type <type> whose value is set by a UI control of type <control>. The valid parameters and <type> values depend on the <control> type. If no parameters are specified, the parentheses may be omitted.

slider controls

The slider control type specifies a slider control over an integer or float range. Directive syntax:

#uicontrol <type> <name> slider(min=<min>, max=<max>, default=<default>, step=<step>)

The <type> must be float, int, or uint. The min and max parameters are required. The step parameter is optional; if not specified, defaults to 1 for integer ranges and (<max> - <min>) / 100 for float ranges. The default parameter indicates the initial value and is optional; if not specified, defaults to <min> for integer ranges and to (<min> + <max>)/2 for float ranges.

color controls

The color control type specifies a color picker. Directive syntax:

#uicontrol vec3 <name> color(default="<color>")

The <type> must be vec3, which is set to the RGB [0, 1] representation of the color. The default parameter indicates the initial value as a CSS color string (must be quoted), and defaults to "white" if not specified.

API

Retrieving voxel channel value

The raw value for a given channel is obtained by calling the getDataValue function:

uint8_t getDataValue(int channelIndex = 0);
uint16_t getDataValue(int channelIndex = 0);
uint32_t getDataValue(int channelIndex = 0);
uint64_t getDataValue(int channelIndex = 0);
float getDataValue(int channelIndex = 0);

If no channelIndex is specified, the value for the first channel is returned. (The default value of 0 is shown in the above declarations for exposition only. As GLSL does not support default values for function parameters, the default value is actually implemented as a separate function overload.) The return type depends on the data type of the volume. Note that only float is a builtin GLSL type. The remaining types are defined as simple structs in order to avoid ambiguity regarding the nature of the value:

struct uint8_t {
  float value;
};
struct uint16_t {
  vec2 value;
};
struct uint32_t {
  vec4 value;
};
struct uint64_t {
  vec4 low, high;
};

For all of these struct types, the contained float values each specify a single byte as a normalized value in [0, 1]. To obtain the raw byte value, you must multiply by 255.

To obtain the raw value as a float, call the toRaw function:

float toRaw(float x) { return x; }
float toRaw(uint8_t x) { return x.value * 255.0; }
float toRaw(uint16_t x) { return x.value.x * 255.0 + x.value.y * 65280.0; }

To obtain a normalized value that maps the full range of integer types to [0,1], call the toNormalized function:

float toNormalized(float x) { return x; }
float toNormalized(uint8_t x) { return x.value; }
float toNormalized(uint16_t x) { return toRaw(x) / 65535.0; }

Emitting pixel values

To emit a normalized grayscale value in the range [0,1], call:

void emitGrayscale(float x);

To emit an RGB color value (each component in the range [0,1]), call:

void emitRGB(vec3 rgb);

To emit an RGBA color value (each component in the range [0,1]), call:

void emitRGBA(vec4 rgba);

Note that the specified alpha value is multiplied by the opacity value for the layer.

To emit a transparent pixel, call:

void emitTransparent();

Color maps

You can map values in the range [0,1] to an RGB color using one of the color maps defined in colormaps.glsl.

Avoiding artifacts due to lossy compression

If a discontinuous color mapping is applied to a volume that is stored or retrieved using lossy compression (e.g. JPEG), compression artifacts may be visible. Lossy compression can be disabled for individual data sources as follows:

Data source	Behavior
boss	JPEG compression is used by default for image volumes. For 16 bit images, append a ?window=INT,INT to request scaled images in 8 bit space.
brainmaps	JPEG compression is used by default for single-channel uint8 volumes. To override this, append a ?encoding=raw query string parameter to the data source URL.

Examples

The default shader, that displays the first channel as a grayscale intensity:

void main () {
  emitGrayscale(toNormalized(getDataValue()));
}

Outputting a 3-channel volume as RGB:

void main () {
  emitRGB(vec3(toNormalized(getDataValue(0)),
               toNormalized(getDataValue(1)),
               toNormalized(getDataValue(2))));
}

Outputting a single-channel volume as a solid red mask with varying alpha (e.g. to overlay a probability map over raw image data):

void main () {
  emitRGBA(vec4(1, 0, 0, toNormalized(getDataValue())));
}

Outputting a single-channel volume using the Jet colormap:

void main () {
  emitRGB(colormapJet(toNormalized(getDataValue())));
}

Thresholding a single-channel volume (see note above about avoiding artifacts due to lossy compression):

void main () {
  emitGrayscale(step(0.5, toNormalized(getDataValue())));
}

Mapping particular values to specific colors (see note above about avoiding artifacts due to lossy compression):

void main() {
  float value = toRaw(getDataValue(0));
  vec3 color = vec3(0, 0, 0);
  if (value == 2.0) color = vec3(1, 0, 0);
  if (value == 3.0) color = vec3(0, 1, 0);
  if (value == 4.0) color = vec3(0, 0, 1);
  emitRGB(color);
}