Discussion on Gesture Control with OpenVINO

[vijaykr338]

Dear Ethan Yang and Zhuo Wu,

I'm Vijay Kumar, a second-year student from India, and I am highly interested in working on and contributing to the Gesture Control with OpenVINO project.

Here is what I have learned so far –

I have studied the MediaPipe Gesture Recognizer framework, which uses a hand landmark detection system capable of recognizing up to seven classes.
As for the visual processing pipeline, I have experience with OpenCV for image handling and preprocessing.
I have also researched OpenVINO's intermediate representation format, which uses a two-file structure to optimize execution on Intel chips.

I have a few queries –

The MediaPipe framework is capable of recognizing up to seven classes, but can we expand it to enable seamless hands-free computer navigation?

We can replicate mouse trackpad gestures such as:

1. tracking the cursor via the index finger
2. scrolling with two fingers
3. A tap gesture can act as a left click, while a pinch motion can act as a right click

as for text input, we can implement swipe-gesture-based typing using the on-screen keyboard, similar to Android keyboards.

I would love to discuss these ideas further and understand how I can contribute to the project.

Looking forward to your insights.

Best regards,
Vijay Kumar

[adrianboguszewski]

@openvino-dev-samples @zhuo-yoyowz