Release v2.12.0

Updated Docker Image

• Docker image is available on Dockerhub

Update Neo4J Backend

• Tunable connection pooling size

Updated S3 Support

• More efficient S3 connection management
• S3 support for descriptors

Upgraded Protobuf

• Upgraded protobuf to version 5.29.5 to address vulnerabilities and customer requests
• VDMS Python package updated with upgraded protobuf version

Experimental: Fast Post-Filtering Library for Vector Search

• Initial implementation of fast-filtering
• API will be added in future releases

Bug Fixes

• Resolved bugs regarding transaction failures

Improved Public Facing Documentation

• Documentation available: https://intellabs.github.io/vdms/

Full Changelog: v2.11.0...v2.12.0

Release v2.11.0

What's New

Updated Docker Image

• Docker image is available on Dockerhub

Improvements to Descriptor/Descriptor Set

• Integration of Descriptor support in Neo4J backend
• Expose runtime parameters of all VCL indices

New Experimental Feature

• Orchestration in VDMS with Kubernetes for easier setup of user-defined function environments both locally and on remote servers

Bug Fixes

• Fix provided setup script
• Refactor test suite

Updated Dependencies

• Upgrade Faiss version to v1.9.0
• Add pre-commit for formatting

Updated documentation in Wiki

• Config File
• Docker
• Install
• Kubernetes Orchestration
• Neo4J based Scale‐Out VDMS
• NeoAddDescriptor
• NeoAddDescriptorSet
• NeoFindDescriptor
• NeoFindDescriptorSet
• Remote Operations
• User-Defined Operations
• VDMS Test Suite

Full Changelog: v2.10.0...v2.11.0

Release 2.10.0

What's New

Updated Docker Image

• Docker image is available on Dockerhub

Improvements to Descriptor/Descriptor Set

• Cache DescriptorSet path to bypass database query after initial setup
• Additional capability for batch inserts of descriptors
• Add Faiss Hierarchical Navigable Small World (HNSW) Engine for indexing and searching descriptors
• Multiple optimizations within the FindDescriptor path, improving performance by up to 200x depending on the specifics on the query

Bug Fixes

• Connections for Neo4J are now being returned and re-used after transaction commit
• Removed unique constraint for link based descriptor queries

Updated Dependencies

Updated documentation in Wiki

• AddDescriptor
• AddDescriptorSet
• External integrations
• FindDescriptor

Release 2.9.0

What's New

Updated Docker Image

• Docker image is available on Dockerhub

New mutual TLS support for C++ Client

New Timing Capability (Experimental)

• The new timing capability provides ability to output a variety of timing information for use in tuning and optimization.
• Additional details are available: Timing Capability

Dynamic Metadata

• VDMS can now dynamically add metadata from images/videos by leveraging the UDF capability.
• Additional details are available: Dynamic Metadata

Improved Neo4j Handler Error Handling and Reporting

Bug Fixes

Updated Dependencies

Updated documentation in Wiki

• Dynamic Metadata
• Installation
• Remote Operations
• Timing Capability
• User Defined Operations

Release 2.8.0

What's New

Updated Docker Image

• Docker image is available on Dockerhub
• Users have the ability to override the default parameters in config-vdms.json. See Environment Variable in Docker Container for more information

New Neo4J + S3 Query Handler (Experimental)

• The new query handler takes advantage of our S3 connector and adds Neo4J for metadata store instead of default PMGD.
• Additional details are available: Neo4J Query Handler, NeoAdd, NeoFind

New mutual TLS support

• mTLS support was added to the server and Python client while retaining support for running in a TLS only (transport security only, no client authentication) or an insecure mode
• Additional details are available: VDMS Mutual TLS Support

Add Entity via File Path

• VDMS can now point to an image/video file accessible by the filesystem instead of storing the blob in the data store. The file could be on the local filesystem or a remote filesystem mounted on the server running VDMS.
• Additional details are available: Updated AddImage Wiki, Updated AddVideo Wiki

Bug Fixes

Updated documentation in Wiki

• AddImage
• AddVideo
• ConfigFile
• Developer Guide
• Installation
• Neo4J Query Handler
• NeoAdd
• NeoFind
• VDMS Mutual TLS Support

Release 2.7.0

What's New

Updated Docker Image

• Docker image is available on Dockerhub

New FindDescriptorSet Command (Experimental)

• The new command provides capability to obtain information regarding a descriptor set and to explicitly perform a store index operation.
• Additional details are available: FindDescriptorSet Command

Configurable S3 Connection

• New parameters accepted in VDMS configuration file for S3 storage address, proxy details, and AWS logging control.
• Additional details are available: ConfigFile

Bug Fixes

Updated documentation in Wiki

• Developer Guide

Release 2.6.0

What's New

Updated Docker Image

• Docker image is available on Dockerhub

New Query Handler Selector (Experimental)

• Enhancement to provide developers the ability to either modify existing query handlers (e.g. changing data and operations flow) or develop entirely new handlers from scratch (e.g. enabling a new type of metadata backend).
• Additional details are available; Query Handler Development

Redesign of Video Class (Experimental)

• Experimental enhancement to video management design to work with videos as a single entity and ensures the Out-of-Memory exceptions are caught/addressed

Upgrade to Protobuf v24.2

• Installation instructions updated in Install.md and Wiki

Improved Error Handling

• Improve how errors/exceptions are handled and provide better information to the client

Bug Fixes

Updated documentation in Wiki

• Descriptor Memory Usage
• Remote Operations
• Remote Storage in VDMS
• User Defined Operations

Release 2.5.0

What's New

• New Debian-based Dockerfile

  • Docker image is available on Dockerhub

• New User-defined Operations (UDF) and Remote Operations

  • Users can apply user-defined operations to images and execute image operations on a remote server using Flask APIs
  • Additional details are available; User Defined Operations and Remote Operations

• AWS S3 Support

  • Users can write data to AWS S3 instead of using local storage by specifying storage_type in configuration file
  • Additional details are available: ConfigFile Wiki

• Upgrade to TileDB 2.14

• Updated documentation in Wiki

Release 2.4.0

What's New

• Support for Storing and Accessing Other Data Formats

  • Users can use the new AddBlob, FindBlob, and UpdateBlob commands to add and query other data formats.
  • Additional details are available: AddBlob, FindBlob, and UpdateBlob.

• CSV Parser for C++ Client

  • Users can ingest data from a CSV file instead of writing JSON queries. This feature supports the Insert Commands.
  • Additional details are available: CSV Plugin for C++ Client

• Upgraded dependencies

• Updated documentation in Wiki

Release V2.3.0

What's New

• Filters to Identify Near-Neighbor Groups (FLINNG):

  • Integrated a new indexing and similarity search library (FLINNG) that addresses the computational challenges of high-dimensional similarity search by presenting an index with fast construction time, low memory requirement, and zero query-time distance computations
  • Instructions are provided (https://github.com/IntelLabs/vdms/wiki/FLINNG-Library-and-Performance-Details)

• Periodic backup (Auto-replication):

  • Users can automatically backup db folder of VDMS periodically and restore the database as needed
  • Instructions are provided (https://github.com/IntelLabs/vdms/wiki/VDMS-Backup-and-Restoring-Options)

• Kafka Connectors:

  • We are moving towards distributed VDMS and have provided Kafka connectors, configurations, instructions, and examples to use VDMS in a distributed manner leveraging Kafka
  • Instructions are provided (https://github.com/IntelLabs/vdms/wiki/Run-Kafka-VDMS-Example)

• Video Improvement

  • Addressed memory leak in video command

• Independent C++ Client:

  • Ability to build client independently without building VDMS server