Python applications for RealSense #13647

UD31415 · 2025-01-01T10:51:27Z

The idea is to develop a high-level applications for the real sense camera.
Examples:
https://www.stereolabs.com/en-il/solutions/autonomous-robotics
https://github.com/stereolabs/zed-sdk
https://www.orbbec.com/
It is a draft with some preliminary code to test the submission process.

sysrsbuild · 2025-01-01T10:51:30Z

Can one of the admins verify this patch?

remibettan · 2025-05-04T07:14:20Z

applications/shape_detector/src/plane_detector.py

+Using depth image to compute depth planes locally.
+
+
+Usage:


please remove these lines, until the imports

remibettan · 2025-05-04T07:14:36Z

applications/shape_detector/src/plane_detector.py

+import cv2 as cv
+import unittest
+from scipy.spatial.transform import Rotation as Rot
+#from scipy import interpolate 


please remove

remibettan · 2025-05-04T07:15:28Z

applications/shape_detector/src/plane_detector.py

+ # importing common Use modules 
+import sys 
+sys.path.append(r'..\Utils\src')
+from opencv_realsense_camera import RealSense


what is opencv_realsense_camera - is it a known package, or something you built in your env?

remibettan · 2025-05-04T07:17:22Z

applications/shape_detector/src/plane_detector.py

+class PlaneDetector:
+    def __init__(self):
+
+        self.frame_size = (640,480)


must be sure that this resolution is available for the device - is there some check or some exception raised in case it is not available?

remibettan · 2025-05-04T07:17:39Z

applications/shape_detector/src/plane_detector.py

+        self.cam_distort= np.array([0,0,0,0,0],dtype = np.float32)
+
+        self.img3d      = None # contains x,y and depth plains
+        self.imgXYZ     = None  # comntains X,Y,Z information after depth image to XYZ transform


typo - contains

remibettan · 2025-05-04T07:18:13Z

applications/shape_detector/src/plane_detector.py

+        img_size = img_gray.size
+
+        # Set the percentage of pixels that should contain noise
+        #noise_percentage = 0.1  # Setting to 10%


should it be uncommented?

remibettan · 2025-05-04T07:19:05Z

applications/shape_detector/src/plane_detector.py

+        img_noised = img_gray.copy()
+
+        # Create a noise list with random placements of min and max values of the image pixels.
+        #noise = np.random.choice([img_gray.min(), img_gray.max()], noise_size)


please check if the right line is uncommented and eventually add a comment to explain the -10, 10

remibettan · 2025-05-04T07:20:04Z

applications/shape_detector/src/plane_detector.py

+            self.img = cv.resize(self.img , dsize = self.frame_size) 
+
+        elif img_type == 13:
+            self.img = cv.imread(r"C:\Data\Depth\Plane\image_ddd_000.png", cv.IMREAD_GRAYSCALE)


all the local paths must be replaced with remote ones

UD31415 · 2025-05-04T07:20:26Z

I built a wrapper (attached) for RS Library that looks like OpenCV. It also has extended functionality - output types, parameter configuration and etc... I find it useful

________________________________ From: remibettan Sent: Sunday, May 4, 2025 10:15 AM To: IntelRealSense/librealsense Cc: Dubin, Uri; Author Subject: Re: [IntelRealSense/librealsense] Python applications for RealSense (PR #13647) @remibettan commented on this pull request.

________________________________ In applications/shape_detector/src/plane_detector.py<#13647 (comment)>:

+Install :

+ + + +''' + +import numpy as np +import cv2 as cv +import unittest +from scipy.spatial.transform import Rotation as Rot +#from scipy import interpolate + + # importing common Use modules +import sys +sys.path.append(r'..\Utils\src') +from opencv_realsense_camera import RealSense what is opencv_realsense_camera - is it a known package, or something you built in your env? — Reply to this email directly, view it on GitHub<#13647 (review)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BJ6FGL3T3V4OWUDZJCJXXNL24W5CPAVCNFSM6AAAAABUOQPJLOVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDQMJTGM4DANRQGI>. You are receiving this because you authored the thread. --------------------------------------------------------------------- Intel Israel (74) Limited This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.

remibettan · 2025-05-04T07:24:44Z

applications/shape_detector/src/plane_detector.py

+            self.img = cv.imread(r"C:\Data\Depth\Plane\image_scl_000.png", cv.IMREAD_GRAYSCALE)  
+            self.img = cv.resize(self.img , dsize = self.frame_size)                                     
+
+        #self.img        = np.uint8(self.img) 


can be removed?

remibettan · 2025-05-04T07:25:36Z

applications/shape_detector/src/plane_detector.py

+        "load the test case"
+        roi = [0,0,self.frame_size[0],self.frame_size[1]]
+        if test_type == 1:
+            roi = [310,230,330,250] # xlu, ylu, xrb, yrb


can a comment be added to give a hint to the reader about the meaning of various rois

remibettan · 2025-05-04T07:28:11Z

applications/shape_detector/src/plane_detector.py

+        y3d[ii]   = v[ii]*z3d[ii] #/fy
+        z3d[ii]   = z3d[ii]
+
+        #self.img3d = np.stack((u/fx,v/fy,z3d), axis = 2)


can be removed?

remibettan · 2025-05-04T07:29:16Z

applications/shape_detector/src/plane_detector.py

+        y3d         = self.img3d[:,:,1].copy()  # v/f
+
+        # filter bad z values
+        #ii          = np.logical_and(z3d > 1e-6 , np.isfinite(z3d))


can be removed?
Please explain below value 15

remibettan · 2025-05-04T07:31:50Z

applications/shape_detector/src/plane_detector.py

+if __name__ == '__main__':
+    #print(__doc__)
+
+    # import sys


if these comments can be used for developing, please add some comment

remibettan · 2025-05-04T07:32:13Z

applications/shape_detector/src/plane_detector.py

+
+    #unittest.main()
+    suite = unittest.TestSuite()
+    #suite.addTest(TestPlaneDetector("test_Convert"))


please explain below commented lines or remove

remibettan · 2025-05-04T07:32:57Z

applications/shape_detector/src/plane_detector_open3d.py

+    pip install opencv-contrib-python
+    pip install scipy
+    pip install matplotlib
+    pip install pyrealsense2-2.55.10.6089-cp310-cp310-win_amd64.whl


please replace with pyrealsense2, without build number

remibettan · 2025-05-04T07:34:33Z

applications/shape_detector/readme.md

+
+    <your path>\Envs\planes3d\Scripts\activate.bat
+
+4. Installing realsense driver. For example, download pyrealsense2-2.55.10.089-cp310-cp310-win_amd64.whl:


please replace by pyrealsense2, without build number

remibettan

Few comments.
Please make sure that this app can be used from any pc.
Please also move the code to root/wrappers/python/examples, or explain the aim of the applications folder you want to add.

testing the contribution process

c2eeb94

Nir-Az requested a review from remibettan April 30, 2025 10:47

remibettan reviewed May 4, 2025

View reviewed changes

UD31415 added 9 commits May 7, 2025 11:41

applications added

dc2dfa0

Create README.md

4bbb4fb

update readme

3ccc0ef

adding application info

772c30e

remove non relevant links

1d2dbd6

Update README.md

f5d504a

working on readme and adding opencv wrapper

724fab2

Update README.md

af83675

Update README.md

17107c9

UD31415 added 30 commits May 11, 2025 12:01

adding depth image for planes test

1ee5527

improving and cleaning. ransac is not working good

613f3d5

plane detectors are updated

06a01e7

Update README.md

66cd1e0

fixing links

c47cf05

Update README.md

c17df7a

removing applications

c07b8d2

adding edges and corners

33a9295

removing applications

1292917

Update README.md

1e06c6d

Update README.md

e66fcfb

Update README.md

ce749e7

Update README.md

e43c363

Update README.md

ba12fa2

Update test_plane_detector.py

d382efa

fixing "your path"

92d6ef8

pose added

422909a

robot calibration data

2035e91

Update README.md

6997701

Update README.md

b497f8e

Update README.md

7ff9475

Update README.md

4cb1144

Update README.md

a563304

Update README.md

1d05ef1

Update README.md

cbdc383

Update README.md

5b29515

Update README.md

fae12f3

Update README.md

029eb6f

plane detector is running but it is not perfect

9ce2ae1

plane detector starting to work

fbc690b


		<your path>\Envs\planes3d\Scripts\activate.bat

		4. Installing realsense driver. For example, download pyrealsense2-2.55.10.089-cp310-cp310-win_amd64.whl:

Python applications for RealSense #13647

Are you sure you want to change the base?

Python applications for RealSense #13647

Uh oh!

Conversation

UD31415 commented Jan 1, 2025

Uh oh!

sysrsbuild commented Jan 1, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

UD31415 commented May 4, 2025 via email

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

remibettan left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

remibettan left a comment •

edited

Loading