The dataset is extracted from the Hand Gesture Dataset provided by the University of Padova, the dataset is cited below;

@article{Hand Gesture Datasets, title={kinect_leap_dataset}, author={Multimedia Technology and Telecommunications Laboratory, University of Padova}, journal={ICIP 2014 paper}, year={2014} } Dataset link: https://lttm.dei.unipd.it/downloads/gesture/kinect_leap/data/kinect_leap_dataset.zip

The dataset is extracted from the ToF depth data (480X640 frames) stored as bin files. The preprocessing includes extracting the initial frames from the bin files. Applying a pixel value threshold to reduce the background noise. Cropping around the region of interest (hand) in order to get the hand gesture with minimal amount of noise. There are 10 gestures in the dataset, from 'G1' to 'G10'.

The ROI frames are to be standardized and resampled in matlab to meet the output size of the ToF sensor (VL53L8CX) proposed by ST Microelectronics, before setting up a datastore and feeding the data to the neural network.

The proposed model is a 2D convolutional neural network with residual connections (to avoid the vanishing gradiant problem).

The model shall be fine tuned and after considerable accuracy, prunned and quantized.

The STM32 Cloud shall be used for benchmarking and inference time measured. This operation shall be performed for all versions of the model (untrained, trained, fine tuned, prunned and prunned_quantized versions) Several quantization adjustments shall be made in order to reduce computational cost and facilitate deployability on board.