Streaming Analytics Framework (SAF)

SAF is a platform for real-time video ingestion and analytics, primarily of live camera streams, with a focus on running state-of-the-art machine learning/computer vision algorithms. The goals of SAF are twofold:

• Deploy real-time, edge-to-cloud ML/CV applications across geo-distributed cameras.
• Provide an expressive pipeline-based programming model that makes it easy to define complex applications that operate on many camera streams.

An example end-to-end analytics pipeline:

• Ingest a camera feed over RTSP.
• Decode the stream using a hardware H.264 decoder.
• Transform, scale, and normalize the frames into BGR images that can be fed into a deep neural network.
• Run an image classification DNN (e.g. GoogleNet, MobileNet).
• Store the classification results and the original stream on disk.

SAF currently supports these DNN frameworks:

• TensorFlow
• Intel Caffe
• OpenVINO
• BVLC Caffe
• OpenCL Caffe

SAF is a collaboration between Carnegie Mellon University and Intel Corporation.

Contact

Please file a GitHub Issue if you experience difficulties compiling or running SAF, have a feature request, or would like additional information. We appreciate feedback and are constantly looking for more real-world use cases for SAF.

Quick start

See these detailed instructions for more information about compiling SAF and its dependencies.

1. Compile SAF

git clone https://github.com/viscloud/saf
cd saf
mkdir build
cd build
cmake ..
make

2. Example: View a camera feed

The apps directory contains several example SAF applications. The simple app displays frames from a camera on the screen. A few example invocations are included below. See the config/cameras.toml file for more details on specifying cameras.

make simple
./apps/simple --display --camera GST_TEST
./apps/simple --display --camera webcam-linux
./apps/simple --display --camera webcam-mac
./apps/simple --display --camera file

Note 1: The --display option requires X11, but the simple app works without it and will print some basic frame statistics to the terminal.

Note 2: By default, the file camera reads from the file video.mp4 in the current directory (see config/cameras.toml).

3. Example: Classify frames using Caffe

The classifier app runs an image classification DNN on every frame and overlays the results in the viewer.

3.1. Install Caffe

On Ubuntu, an unoptimized version of Caffe is available from apt. It requires a BLAS library, such as OpenBLAS.

sudo apt install -y libcaffe-cpu-dev libopenblas-dev

Note: If you are compiling Caffe from source, then see this page for more details.

3.2. Enable Caffe support in SAF

We need to recompile SAF with Caffe support.

cmake -DUSE_CAFFE=yes ..
make

3.3. Configure the models

Next, we need to configure our Caffe models by editing the default model configuration file.

cd <path to SAF>/config
cp models.toml.example models.toml

The default models.toml file has configurations for using Caffe to perform image classification using GoogleNet and MobileNet. Note that some of the parameters (input_width, input_height, and default_output_layer) are model specific.

The models.toml file contains the URLs for the various model files. There are three files for each of the two models, but one of the files is shared--thus a total of five files for both GoogleNet and MobileNet. Download the model files into a models directory:

mkdir <path to SAF>/models
cd <path to SAF>/models
# Download the model files as described in "models.toml"
wget ...

3.4. Run the classification demo

cd <path to SAF>/build
make classifier
./apps/classifier --config-dir ../config --display --camera webcam-linux --model googlenet
./apps/classifier --config-dir ../config --display --camera webcam-linux --model mobilenet

The classifier app also works without the --display option.