ONE Release 1.30.0

Release Note 1.30.0

ONE Compiler

• Support more optimization option(s): --dynamic_batch_to_single_batch, --fuse_rmsnorm,
  --fuse_rope, --fuse_rsqrt.
• Introduce one-import-pytorch for direct PyTorch import that converts PyTorch modules
  straight to Circle.
• Support MX (microscaling) data type.
• Introduce circle-mlir that converts new ops (e.g., ConvTranspose2D, DepthwiseConv2D,
  ArgMax, Reduce*, Resize, Pooling, GELU, Sigmoid, Softmax, Slice/StridedSlice, Select,
  Gather, BatchMatMul) and validates shape inference.
• ONNX is converted from new circle-mlir/onnx2circle tool (onnx-tf tool is deprecated)
• luci-interpreter handles scalar indices in Gather and additional element types across
  several kernels.
• Ubuntu22.04 with python3.10 is official supported platform, python3.10 is necessary
  for Ubuntu20.04 and Ubuntu24.04 with python3.12 is experimentally supported.
• Python packages are upgraded to TensorFlow 2.19.0, ONNX 1.18.0, ONNXRuntime 1.21.1,
  Torch 2.7.0.

ONE Runtime

General Updates

• Ubuntu version support changes:
  • Add 24.04 LTS support
  • Deprecate 20.04 LTS support

Runtime API Updates

• NNAPI is not official API anymore. It is still supported but used for test purpose only.
• Model loading from model file without NN package is supported by C/Python APIs. Supporting
  model types are TFLite, Circle, and TVN.
• Python API supports training APIs.
• Python API supports dynamic shapes.
• Supporting custom BroadcastTo, AddV2, BatchMatMul are deprecated. These custom operations
  loading was introduced to support custom operations by compiler but now it is replaced with
  general operations.

Runtime Core Updates

• Layout conversion is supported for input and output only. It does not support for
  intermediate tensors.
• HDF5 dependency on minmax dumper is removed. Now it uses runtime's own tensor dumping
  functions.
• Block quantization type (GGML_Q4_0, GGML_Q8_0) is supported for quantized models.
• RMSNorm, RoPE and GELU operations are supported.
• Einsum and MatrixBandPart operations are deprecated. They were introduced to support
  custom operation by compiler but now they are replaced with general operations.

Backend Kernel Updates

• CPU backend supports INT64 BinaryArithmetic operations.
• CPU backend supports FullyConnected block quantized weight operations.
• CPU backend supports Gather block quantized constant input operations.
• CPU backend supports RMSNorm, RoPE, and GELU operations.
• ARM Compute Library version is updated to 24.07.

On-device Training Updates

• Memory usage is enhanced during training by usedef analysis and tensor planning.
• Checkpoint is introduced to export and load tensor and optimizer data, making it
  easier to save and restore training states.

On-device Compilation Updates

• Auto-compilation is supported when compilation threshold is met.

ONERT-MICRO 2.0.0

Release Notes for onert-micro 2.0.0

Operations updated

• New ops supported : Cast, Ceil, Elu, Fill
• New CMSIS NN accelerated ops : SVDF, Relu, Relu6

New features for on-device training

• New Trainable Operation : GRU, StridedSlice
  • limitation : You can train GRU's weights. Since input gradient is not supported now, GRU layer should be the last layer for training.
• New Loss Function : Sparse Categorical Cross Entropy (experimental)

ONE Release 1.29.0

Release Note 1.29.0

ONE Compiler

• Support more optimization option(s): --transform_sqrt_div_to_rsqrt_mul, --fold_mul,
  --fuse_add_to_fullyconnected_bias, --fuse_mul_to_fullyconnected_weights,
  --fuse_mul_with_fullyconnected.
• Add more optimization: CanonicalizePass.
• tflite2circle supports more data types: FLOAT64, UINT64, UINT32, UINT16.
• luci-interpreter supports more data types on some ops.
• Support multiple option names in command schema.

ONE Release 1.28.0

Release Note 1.28.0

ONE Runtime

Python API

• Support experimental python API
  • Refer howto document for more details

On-device Training

• Support on-device training with circle model
  • Training parameter can be passed to onert via onert`s experimental API or loading new model format including training information: circle_plus
  • Trained model can be exported to circle model via experimental API nnfw_train_export_circle
  • Supporting Transfer learning from a pre-trained circle model
• Introduce circle_plus_gen tool
  • Generates a circle_plus model file with a given training hyperparameters
  • Shows a _circle_plus model details

Runtime configuration API

• onert supports runtime configuration API for prepare and execution phase via experimental APIs
  • nnfw_set_prepare_config sets configuration for prepare phase, and nnfw_reset_prepare_config resets it to default value
  • nnfw_set_execution_config sets configuration for execution phase, and nnfw_reset_execution_config resets it to default value
  • Supporting prepare phase configuration: prepare execution time profile
  • Supporting execution phase configuration: dump minmax data, dump execution trace, dump execution time
• Introduce new API to set onert workspace directory: nnfw_set_workspace
  • onert workspace directory is used to store intermediate files during prepare and execution phase

Minmax Recorder

• Now onert's minmax recorder dumps raw file format instead of HDF5 format
• onert dumps minmax data into workspace directory

On-device Compilation

• onert supports full quantization of uint8/int16 type weight and activation.
  • To quantize activation, onert requires minmax data of activation.
• onert supports on-device code generation for special backend requiring special binary format such as DSP, NPU.
  • Introduce new experimental API for code generation: nnfw_codegen

Type-aware model I/O usage

• If loaded model is quantized model, onert allows float type I/O buffer
  • onert converts float type input buffer to quantized type internally
  • onert fills float type output buffers by converting quantized type output data to float type internally
• On multimodel package, onert allows edges between quantized model and float type model

ONE Release 1.27.2

Release Note 1.27.2

ONE Compiler

• Support target option with command args in one-profile, one-codegen.
• Update man pages of one-cmds tools.

ONE Release 1.27.1

Release Note 1.27.1

ONE Compiler

• Command schema supports multiple names.
• Fix invalid warning on boolean type option in onecc.

ONERT-MICRO 2.0.0-pre

Release Notes for onert-micro 2.0.0-pre

Overall Structure Refactored

• c++ api has been changed : onert-micro c++ api
• 60 ops supported : Abs, Add, AddN, AveragePool2D, ArgMax, ArgMin, Concatenation, BatchToSpaceD, Cos, Div, DepthwiseCov2D, Dequatize, FullyCoected, Cov2D, Logistic, Log, Gather, GatherD, Exp, Greater, GreaterEqual, ExpadDims, Equal, Floor, FloorDiv, FloorMod, Pad, Reshape, ReLU, ReLU6, Roud, Less, L2ormalize, L2Pool2D, LessEqual, LeakyReLU, LogSoftmax, Mul, Maximum, MaxPool2D, Miimum, otEqual, Si, SquaredDifferece, Slice, Sub, Split, SpaceToBatchD, StridedSlice, Square, Sqrt, SpaceToDepth, Tah, Traspose, TrasposeCov, Softmax, While, Rsqrt, Upack

onert-micro supports on-device training feature

• Trainable Operations : 5 operations ( Conv2D, FullyConnected, MaxPool2D, Reshape, Softmax )
• Loss : MSE, Categorical Cross Entropy
• Optimizer : ADAM, SGD
• C api for training feature : onert-micro c api header
• limitation : For now, you can train topologically linear model

ONE Release 1.27.0

Release Note 1.27.0

ONE Compiler

• Support more Op(s): CircleGRU, CircleRelu0To1
• Support more optimization option(s): resolve_former_customop, --forward_transpose_op,
  fold_shape, remove_gather_guard, fuse_add_with_conv, fold_squeeze, fuse_rsqrt
• Support INT4, UINT4 data types
• Support 4bit quantization of ONNX fake quantize model
• Introduce global configuration target feature
• Introduce command schema feature
• Use C++17

ONE Release 1.26.0

Release Note 1.26.0

ONE Compiler

• Support more Op(s): HardSwish, CumSum, BroadcastTo
• Support more optimization option(s): decompose_softmax, decompose_hardswish, fuse_slice_with_tconv,
  fuse_mul_with_conv, remove_unnecessary_add, fuse_horizontal_fc_layers, common_subexpression_elimination,
  remove_unnecessary_transpose
• one-quantize supports more option
  • Requantization option to convert TF2-quantized int8 model to uint8 model (--requantize)
  • A new option to automatically find mixed-precision configuration (--ampq)
  • A new option to save calibrated min/max values (--save_min_max)
  • Add new parameters for moving average calibration (--moving_avg_batch, --moving_avg_const)
• Introduce q-implant that writes quantization parameters and weights into the circle model
• Introduce minmax-embedder that embeds min/max values into the circle model

ONE Release 1.24.1

Release Note 1.24.1

ONE Compiler

• Updates error message of rawdata2hdf5 test