Gesture recognition #20

zycz · 2025-12-19T14:01:37Z

applications: gesture_recognition: Add gesture_recognition application

Add gesture_recognition application as it is in:
https://github.com/Neuton-tinyML/neuton-nordic-thingy53-ble-remotecontrol
on commit: b2063f2ae42fa8aead6b3c7df477b3cda7caac29

Application was devloped by Raman Rusak.

Add gesture_recognition application as it is in: https://github.com/Neuton-tinyML/neuton-nordic-thingy53-ble-remotecontrol on commit: b2063f2ae42fa8aead6b3c7df477b3cda7caac29 Application was devloped by Raman Rusak. JIRA: NCSDK-36990 Signed-off-by: Jan Zyczkowski <[email protected]>

Fix application for NCS 3.2.0. Remove dupplicated USB stack. Leave only USB_DEVICE_STACK_NEXT which is enabled for thingy53 by deafult. JIRA: NCSDK-36990 Signed-off-by: Jan Zyczkowski [email protected]

JIRA: NCSDK-36990 Signed-off-by: Jan Zyczkowski [email protected]

Use EdgeAI API and library from repository instead of using a copy from application location. Remove copy of API and library from application location. JIRA: NCSDK-36990 Signed-off-by: Jan Zyczkowski [email protected]

zycz · 2025-12-19T15:31:30Z

I still have several issues which needs to be discussed:

What licence should gesture recognition application have?
Should this repo and gesture recognition application meet NCS coding style standards?
When adding gesture recognition application should all commit history be kept or is it ok to push sample as a single commit?
Should all DKs use physical sensors even if there is no existing shield which can support them? In that case it would require us and future customers to use sensors connected to DKs by cables. Plans for supported DKs: nRF54L15, nRF54H20, nRF54LM20, Thingy53

rrusak · 2025-12-23T13:13:03Z

applications/gesture_recognition/.clang-format

@@ -0,0 +1,24 @@
+Standard: Cpp11


please use .clang-format from root of this repository
https://github.com/nrfconnect/sdk-edge-ai/blob/main/.clang-format

rrusak · 2025-12-23T13:16:55Z

applications/gesture_recognition/README.md

+
+![nordic-thingy-kit-img](resources/nordic_thingy.jpg)
+
+## Setup Software Environment <div id='setup-sw-env'/>


I think that since this application are now part of the add-on, the sections Setup Software Environment and Setup Firmware Project should be reworked to match the add-on documentation.

rrusak · 2025-12-23T13:31:15Z

applications/gesture_recognition/prj.conf

+CONFIG_BT_SMP=y
+CONFIG_BT_L2CAP_TX_BUF_COUNT=5
+CONFIG_BT_PERIPHERAL=y
+CONFIG_BT_DEVICE_NAME="Neuton NRF RemoteControl"


change CONFIG_BT_DEVICE_NAME something like "nRF Edge AI Remote Control", or please consult with @wbober for proper name

rrusak · 2025-12-23T13:38:24Z

applications/gesture_recognition/sample.yaml

@@ -0,0 +1,16 @@
+sample:
+  description: Hello World sample, the simplest Zephyr


Should match the provided application description :) I believe now it is template for hello world

rrusak · 2025-12-23T13:40:16Z

applications/gesture_recognition/src/app_version.h

@@ -0,0 +1,39 @@
+/* 2023-06-09T11:22:25Z */
+
+/* ----------------------------------------------------------------------


please change this copyright according add-on license and file tag format

rrusak · 2025-12-23T14:01:51Z

I still have several issues which needs to be discussed:

What licence should gesture recognition application have?

Should this repo and gesture recognition application meet NCS coding style standards?

When adding gesture recognition application should all commit history be kept or is it ok to push sample as a single commit?

Should all DKs use physical sensors even if there is no existing shield which can support them? In that case it would require us and future customers to use sensors connected to DKs by cables. Plans for supported DKs: nRF54L15, nRF54H20, nRF54LM20, Thingy53

My point of view:

The license should be the same as samples
Yes, I think we should follow the same style
I don't think it's worth saving the entire history of commits from the public repository, since some of the commits were before the Neuton acquisition and rebranding and there is history that shouldn't be exposed
I want to clarify that the model was trained on this board with this sensor, it depends on the physical orientation of the sensor in space, and using this model to other boards will require adapting the sensor axes and sensor settings for the model to work.
It is also possible that due to the different form factors of the boards, the gestures a person makes may differ in their trajectory and pattern, and the model recognition may be worse.

zycz · 2025-12-23T15:17:08Z

I still have several issues which needs to be discussed:

What licence should gesture recognition application have?

Should this repo and gesture recognition application meet NCS coding style standards?

When adding gesture recognition application should all commit history be kept or is it ok to push sample as a single commit?

Should all DKs use physical sensors even if there is no existing shield which can support them? In that case it would require us and future customers to use sensors connected to DKs by cables. Plans for supported DKs: nRF54L15, nRF54H20, nRF54LM20, Thingy53

My point of view:

The license should be the same as samples

Yes, I think we should follow the same style

I don't think it's worth saving the entire history of commits from the public repository, since some of the commits were before the Neuton acquisition and rebranding and there is history that shouldn't be exposed

I want to clarify that the model was trained on this board with this sensor, it depends on the physical orientation of the sensor in space, and using this model to other boards will require adapting the sensor axes and sensor settings for the model to work.
It is also possible that due to the different form factors of the boards, the gestures a person makes may differ in their trajectory and pattern, and the model recognition may be worse.

Thanks for your input.

You are right about issue that different hardware might not work correctly with model trained on data acquired from thingy53. Does it mean that we should provide for all other DKs simulated data? As an alternative I see would be having a different model for each hardware shape (I guess there is no much difference between DKs)

I'am trying to make app working on nrf54h20 with shield. I guess that when I will mange to do that we will see how big the issue is and we might then decide how to proceed.

Szynkaa · 2026-01-02T12:03:29Z

applications/gesture_recognition/src/ble/hid/ble_hid.c

+#include <zephyr/bluetooth/uuid.h>
+#include <zephyr/settings/settings.h>
+
+//////////////////////////////////////////////////////////////////////////////


This style of sectioning is not used in nrf

Szynkaa · 2026-01-02T12:04:28Z

applications/gesture_recognition/src/ble/hid/ble_hid.c

+
+static void input_ccc_changed(const struct bt_gatt_attr* attr, uint16_t value)
+{
+    printk("Input CCCD %s\n", value == BT_GATT_CCC_NOTIFY ? "enabled" : "disabled");


Use LOG subsys

Szynkaa · 2026-01-05T07:43:07Z

applications/gesture_recognition/src/ble/hid/ble_hid.c

+#define KEY_MEDIA_PREV_TRACK ( 1 << 4 )
+#define KEY_MEDIA_NEXT_TRACK ( 1 << 5 )
+
+#if CONFIG_SAMPLE_BT_USE_AUTHENTICATION


This CONFIG is not present in this sample

Szynkaa · 2026-01-05T07:59:00Z

applications/gesture_recognition/src/ble/hid/ble_hid.c

+#define KEY_ARROW_LEFT (0x50)
+#define KEY_ARROW_RIGHT (0x4F)
+#define KEY_F5 (0x3E)
+#define KEY_ESP (0x29)


Is this typo? Should it be ESC?

Szynkaa · 2026-01-05T08:28:15Z

applications/gesture_recognition/src/bsp/button/bsp_button.c

Do we need BSP? We target only Nordic chips

Szynkaa · 2026-01-05T08:47:35Z

applications/gesture_recognition/src/inference_postprocessing.c

+        [CLASS_LABEL_ROTATION_LEFT]  = "ROTATION LEFT"
+    };
+
+    static const uint8_t LABELS_CNT = sizeof(LABEL_VS_NAME) / sizeof(LABEL_VS_NAME[0]);


ARRAY_SIZE, same below

Szynkaa · 2026-01-05T08:51:52Z

applications/gesture_recognition/src/inference_postprocessing.c

+
+//////////////////////////////////////////////////////////////////////////////
+
+static const char* get_name_by_target_(uint8_t predicted_target)


I think this functions can be placed where they are declared, no need to split declaration from definition.

Szynkaa · 2026-01-05T08:53:57Z

applications/gesture_recognition/CMakeLists.txt

@@ -0,0 +1,22 @@
+#
+# Copyright (c) 2024 Nordic Semiconductor


correct year in every file

Szynkaa · 2026-01-05T08:54:22Z

applications/gesture_recognition/CMakeLists.txt

+cmake_minimum_required(VERSION 3.20.0)
+
+find_package(Zephyr REQUIRED HINTS $ENV{ZEPHYR_BASE})
+project(nrf_edgeai_thingy53)


gesture_recognition

Szynkaa · 2026-01-05T08:56:45Z

applications/gesture_recognition/thingy53_nrf5340_cpuapp.overlay

@@ -0,0 +1,17 @@
+// To get started, press Ctrl+Space to bring up the completion menu and view the available nodes.


this comment are not needed, file should be in boards dir

zycz added 4 commits December 19, 2025 14:48

applications: gesture_recognition: Remove dupplicated USB stack

e27fc51

Fix application for NCS 3.2.0. Remove dupplicated USB stack. Leave only USB_DEVICE_STACK_NEXT which is enabled for thingy53 by deafult. JIRA: NCSDK-36990 Signed-off-by: Jan Zyczkowski [email protected]

applications: gesture_recognition: Fix minor issues

3b80a37

JIRA: NCSDK-36990 Signed-off-by: Jan Zyczkowski [email protected]

applications: gesture_recognition: Use main EdgeAi API

b7eff79

Use EdgeAI API and library from repository instead of using a copy from application location. Remove copy of API and library from application location. JIRA: NCSDK-36990 Signed-off-by: Jan Zyczkowski [email protected]

rrusak reviewed Dec 23, 2025

View reviewed changes

Szynkaa reviewed Jan 5, 2026

View reviewed changes


		![nordic-thingy-kit-img](resources/nordic_thingy.jpg)

		## Setup Software Environment <div id='setup-sw-env'/>

		@@ -0,0 +1,16 @@
		sample:
		description: Hello World sample, the simplest Zephyr

		@@ -0,0 +1,39 @@
		/* 2023-06-09T11:22:25Z */

		/* ----------------------------------------------------------------------


		//////////////////////////////////////////////////////////////////////////////

		static const char* get_name_by_target_(uint8_t predicted_target)

		@@ -0,0 +1,17 @@
		// To get started, press Ctrl+Space to bring up the completion menu and view the available nodes.

Gesture recognition #20

Are you sure you want to change the base?

Gesture recognition #20

Uh oh!

Conversation

zycz commented Dec 19, 2025

Uh oh!

zycz commented Dec 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rrusak commented Dec 23, 2025

Uh oh!

zycz commented Dec 23, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

zycz commented Dec 19, 2025 •

edited

Loading