NNEF: Neural Network Exchange Format

The goal of NNEF is to enable data scientists and engineers to easily transfer trained networks from their chosen training framework into a wide variety of inference engines. A stable, flexible and extensible standard that equipment manufacturers can rely on is critical for the widespread deployment of neural networks onto edge devices, and so NNEF encapsulates a complete description of the structure, operations and parameters of a trained neural network, independent of the training tools used to produce it and the inference engine used to execute it.

NNEF - Solving Neural Net Fragmentation

---

Convolutional Neural Networks (CNN) are computationally expensive, and so many companies are actively developing mobile and embedded processor architectures to accelerate neural net-based inferencing at high speed and low power. As a result of such rapid progress, the market for embedded neural net processing is in danger of fragmenting, creating barriers for developers seeking to configure and accelerate inferencing engines across multiple platforms.

Today, most neural net toolkits and inference engines use proprietary formats to describe the trained network parameters, making it necessary to construct many proprietary importers and exporters to enable a trained network to be executed across multiple inference engines.

NNEF has been designed to be reliably exported and imported across tools and engines such as Torch, Caffe, TensorFlow, Theano, Chainer, Caffe2, PyTorch, and MXNet. The NNEF 1.0 Specification covers a wide range of use-cases and network types with a rich set of operations and a scalable design that borrows syntactical elements from existing languages with formal elements to aid in correctness. NNEF includes the definition of custom compound operations that offers opportunities for sophisticated network optimizations. Future work will build on this architecture in a predictable way so that NNEF tracks the rapidly moving field of machine learning while providing a stable platform for deployment.

NNEF File Overview

---

The NNEF format splits the Network Structure and Network Data into separate files. This provides easy independent access to network structure or individual parameter data. A set of NNEF files can use a container such as tar or zip with optional compression and encryption.

Network Structure File

The NNEF network structure file is a distilled, platform independent network description that is human readable and contains syntactical elements from Python. It supports standardised operations, using rigorously defined semantics, including linear, convolution, pooling, normalization, activation, unary/binary, and supports fully connected, convolutional and recurrent architectures. The format supports two levels of expressiveness. Flat - for basic transfer of computation graphs with standardized operations that are simple to parse and translate to vendor specific formats. Compositional - defines custom compound operations and high-level graph descriptions that are more complex to parse but offer more optimization hints.

Network Data Files

NNEF can associate multiple data files with one network structure file e.g. the same data in multiple formats. The data file supports

• Binary format containing parameter tensors
• Floats and quantized (integer) data
• Flexible bit widths and quantization algorithms
• Quantization algorithms expressed as extensible compound operations
• Quantization info provided as hints for execution

NNEF Tools Ecosystem

---

A number of NNEF tools are available to generate and consume NNEF files, such as a parser (C++ and Python) that can be included in consumer applications, plus converters for deep learning frameworks.

• NNEF Parser & Validator - contains C++ and Python source code for a sample NNEF graph parser
• Conversion Tools - to convert pre-trained models in TensorFlow/caffe/caffe2/ONNX to NNEF format
• Model Zoo - contains a variety of NNEF models converted from various sources
• OpenVX Integration - ingestion and execution of OpenVX using an extension

Essential Resources & Tools

---

Thanks to the support of the Khronos membership and our developer community, there is a wide range of developer information and educational resources to help get you up and running with your NNEF development.

• Registry - specification and documentation on GitHub
• NNEF Tools - a range of tools including parsers and converters
• NNEF Launch Press Release
• OpenVX Extension - adds support for OpenVX Vision Processing
• Feedback on GitHub