Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WebARKit development #21

Open
3 of 5 tasks
kalwalt opened this issue Oct 25, 2023 · 2 comments
Open
3 of 5 tasks

WebARKit development #21

kalwalt opened this issue Oct 25, 2023 · 2 comments
Assignees
Labels
C/C++ code concerning the C/C++ code design and improvements enhancement New feature or request

Comments

@kalwalt
Copy link
Member

kalwalt commented Oct 25, 2023

I will make the point of the WebARKit library. Actually it is in the dev branch of this repository, before merging in master and to be officially released it is required to add some important features.
One of the most important feature to add is a better pose matrix solution. We receive the pose matrix from the knn matcher but it needs to be evaluated with a PnP solver. This require a camera matrix for the PnP pose computation, that for now our solution is lacking, and after that use a Kalman filter (or other filter) to stabilize the 6DOF tracking result.
Adding to this: in order to utilize the pose matrix we need a set of functions for GL graphics, to convert the pose matrix as Right hand GL matrix and also a function to compute the Camera Projection matrix.
Summing up, we need to develop:

  • camera matrix: implemented a new class WebARKitCamera Feature camera class #22
  • PnP solver for the pose matrix.
  • arGLposeMatrix and cameraProjectionMatrix
  • improve calcOpticalFlowPyrLK approach -> done in PR Improving c++ code #27
  • Kalman filter or OneEuro filter to stabilize the tracking.
@kalwalt kalwalt added enhancement New feature or request C/C++ code concerning the C/C++ code design and improvements labels Oct 25, 2023
@kalwalt
Copy link
Member Author

kalwalt commented Oct 30, 2023

A new WebARKitCamera class was created in PR #22, before procede to the second task, i considered that it is required to create a class that handle the "pattern", the image that is tracked. This class should contain the Point2d and Point3d data and of course keypoints and descriptors as well. this was implemented in OCVT artoolkitx but i saw also in another code projects.

@kalwalt
Copy link
Member Author

kalwalt commented Apr 20, 2024

I merged the PnP related code with #23 but it think there is something wrong with buildOpticalFlowPyramid and calcOpticalFlowPyrLK. Probably the steps are not in the right sequence and this may produce weird results, in computing homography and pose matrix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C/C++ code concerning the C/C++ code design and improvements enhancement New feature or request
Projects
Status: In progress
Development

No branches or pull requests

1 participant