Pointers feature setup #4

kalwalt · 2020-11-29T17:13:39Z

See #3 I'm testing an alternative approach to loading the tracker. I think that probably i will continue on this path.

…ing function, but nothing happens

src/WebARKitController.js

kalwalt · 2020-12-02T10:30:45Z

with this code i get this error:

WebARKit 0.0.0
WebARKit.js:14 [WebARKitController] WebARKit initialized
WebARKit.js:10 This is a test from WebARKitJS.cpp!
WebARKit.js:14 Width of image is: 1637
WebARKit.js:14 Height of image is: 2048
WebARKit.js:10 Allocated image2DSize: 13410304

WebARKit.js:10 Allocated image2DFrame, pointer is: 70174720

WebARKit.js:14 [WebARKitController] Got ID from setup 0
WebARKit.js:14 70174720
WebARKit.js:14 Hey, i am here!
WebARKit.js:14 1637
WebARKit.js:14 Uint8Array(13410305) [37, 29, 27, 255, 37, 29, 27, 255, 36, 28, 26, 255, 36, 28, 26, 255, 36, 28, 26, 255, 37, 29, 27, 255, 38, 30, 28, 255, 39, 31, 29, 255, 41, 33, 31, 255, 40, 32, 30, 255, 38, 30, 28, 255, 36, 28, 26, 255, 35, 27, 25, 255, 35, 27, 25, 255, 36, 28, 26, 255, 37, 29, 27, 255, 38, 34, 31, 255, 36, 32, 29, 255, 33, 29, 26, 255, 32, 28, 25, 255, 33, 29, 26, 255, 35, 31, 28, 255, 36, 32, 29, 255, 37, 33, 30, 255, 32, 28, 25, 255, …]
WebARKit.js:10 Start to initialize tracker...
WebARKit.js:10 Start!
WebARKit.js:10 2000
WebARKit.js:10 #1 0x4ed67 in wasm-function[-1]+0x4ed67 (data:application/octet-stream;base64,AGFzbQEAAAABhAZdYAF/AX9gAn9/AGABfwBgAn…f39/fwF/YAABf2AKf39/f39/f39/fwBgCH9/f39/f39/AX9gCX9/f39/f39/fwBgB3+0x4ed67)
WebARKit.js:10
WebARKit.js:10 Address 0x02a239a4 is located in stack of thread T0 at offset 4 in frame
WebARKit.js:10 #0 0x81 in wasm-function[-1]+0x81 (data:application/octet-stream;base64,AGFzbQEAAAABhAZdYAF/AX9gAn9/AGABfwBgAn…n9/f39/fwF/YAABf2AKf39/f39/f39/fwBgCH9/f39/f39/AX9gCX9/f39/f39/fwBgB3+0x81)
WebARKit.js:10
WebARKit.js:10 This frame has 3 object(s):
WebARKit.js:10 [16, 20) 'id.addr'
WebARKit.js:10 [32, 36) 'ref.tmp'
WebARKit.js:10 [48, 52) 'ref.tmp2'
WebARKit.js:10 HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork
WebARKit.js:10 (longjmp and C++ exceptions are supported)
WebARKit.js:10 SUMMARY: AddressSanitizer: stack-buffer-underflow (data:application/octet-stream;base64,AGFzbQEAAAABhAZdYAF/AX9gAn9/AGABfwBgAn…f39/fwF/YAABf2AKf39/f39/f39/fwBgCH9/f39/f39/AX9gCX9/f39/f39/fwBgB3+0x37af0) in wasm-function[-1]+0x37af0
WebARKit.js:10 Shadow bytes around the buggy address:
WebARKit.js:10 0x005446e0: 00 00 00 00 f1 f1 00 f3 f3 f3 00 00 00 00 00 00
WebARKit.js:10 0x005446f0: f1 f1 00 f2 f2 f2 f8 f2 f2 f2 f8 f8 f8 f8 f8 f8
WebARKit.js:10 0x00544700: f8 f2 f2 f2 f2 f2 f8 f8 f2 f2 f8 f8 f2 f2 f8 f2
WebARKit.js:10 0x00544710: f2 f2 f8 f2 f2 f2 f8 f2 f2 f2 f8 f2 f2 f2 f8 f2
WebARKit.js:10 0x00544720: f2 f2 f8 f2 f2 f2 f8 f2 f2 f2 f8 f3 f3 f3 00 00
WebARKit.js:10 =>0x00544730: 00 00 00 00[f1]f1 04 f2 f8 f2 f8 f3 00 00 00 00
WebARKit.js:10 0x00544740: f1 f1 04 f3 00 00 00 00 00 00 00 00 00 00 00 00
WebARKit.js:10 0x00544750: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
WebARKit.js:10 0x00544760: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
WebARKit.js:10 0x00544770: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
WebARKit.js:10 0x00544780: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
WebARKit.js:10 Shadow byte legend (one shadow byte represents 8 application bytes):
WebARKit.js:10 Addressable: 00
WebARKit.js:10 Partially addressable: 01 02 03 04 05 06 07
WebARKit.js:10 Heap left redzone: fa
WebARKit.js:10 Freed heap region: fd
WebARKit.js:10 Stack left redzone: f1
WebARKit.js:10 Stack mid redzone: f2
WebARKit.js:10 Stack right redzone: f3
WebARKit.js:10 Stack after return: f5
WebARKit.js:10 Stack use after scope: f8
WebARKit.js:10 Global redzone: f9
WebARKit.js:10 Global init order: f6
WebARKit.js:10 Poisoned by user: f7
WebARKit.js:10 Container overflow: fc
WebARKit.js:10 Array cookie: ac
WebARKit.js:10 Intra object redzone: bb
WebARKit.js:10 ASan internal: fe
WebARKit.js:10 Left alloca redzone: ca
WebARKit.js:10 Right alloca redzone: cb
WebARKit.js:10 Shadow gap: cc
WebARKit.js:10 ==42==ABORTING
example.html:1 Uncaught (in promise)
oB {name: "ExitStatus", message: "Program terminated with exit(1)", status: 1}

because i initialize the new intance of WebARKitOrbTracker in the wrong way:

WebARKitOrbTracker *tracker;
//.... other code
tracker->initialize(....)

should be instead:

WebARKitOrbTracker tracker;
//.... other code
tracker.initialize(....)

the error really point to this, spent hours to understand the output. 😄

kalwalt · 2020-12-02T10:40:06Z

Now with commit 3c99a38 that error is fixed, look at the log the program go over but stop in another location:

WebARKit  0.0.0
WebARKit.js:14 [WebARKitController] WebARKit initialized
WebARKit.js:10 This is a test from WebARKitJS.cpp!
WebARKit.js:14 Width of image is:  1637
WebARKit.js:14 Height of image is:  2048
WebARKit.js:10 Allocated image2DSize: 13410304
WebARKit.js:10 Allocated image2DFrame, pointer is: 70174720
WebARKit.js:14 [WebARKitController] Got ID from setup 0
WebARKit.js:14 70174720
WebARKit.js:14 Hey, i am here!
WebARKit.js:14 1637
WebARKit.js:14 
Uint8Array(13410305) [37, 29, 27, 255, 37, 29, 27, 255, 36, 28, 26, 255, 36, 28, 26, 255, 36, 28, 26, 255, 37, 29, 27, 255, 38, 30, 28, 255, 39, 31, 29, 255, 41, 33, 31, 255, 40, 32, 30, 255, 38, 30, 28, 255, 36, 28, 26, 255, 35, 27, 25, 255, 35, 27, 25, 255, 36, 28, 26, 255, 37, 29, 27, 255, 38, 34, 31, 255, 36, 32, 29, 255, 33, 29, 26, 255, 32, 28, 25, 255, 33, 29, 26, 255, 35, 31, 28, 255, 36, 32, 29, 255, 37, 33, 30, 255, 32, 28, 25, 255, …]
WebARKit.js:10 Start to initialize tracker...
WebARKit.js:10 Start!
WebARKit.js:10 2000
WebARKit.js:10 Orb created!
WebARKit.js:10 Gray Image!
WebARKit.js:10 =================================================================
WebARKit.js:10 ==42==ERROR: AddressSanitizer: null-pointer-dereference on address 0x00000000 at pc 0x00022137 bp 0x02a23710 sp 0x02a2371c
WebARKit.js:10 WRITE of size 4 at 0x00000000 thread T0
WebARKit.js:10     #0 0x22137 in wasm-function[-1]+0x22137 (data:application/octet-stream;base64,AGFzbQEAAAABhAZdYAF/AX9gAn9/AGABfwBgAn…f39/fwF/YAABf2AKf39/f39/f39/fwBgCH9/f39/f39/AX9gCX9/f39/f39/fwBgB3+0x22137)
WebARKit.js:10     #1 0x376ed in wasm-function[-1]+0x376ed (data:application/octet-stream;base64,AGFzbQEAAAABhAZdYAF/AX9gAn9/AGABfwBgAn…f39/fwF/YAABf2AKf39/f39/f39/fwBgCH9/f39/f39/AX9gCX9/f39/f39/fwBgB3+0x376ed)
WebARKit.js:10 
WebARKit.js:10 Address 0x00000000 is located in the shadow gap area.
WebARKit.js:10 SUMMARY: AddressSanitizer: null-pointer-dereference (data:application/octet-stream;base64,AGFzbQEAAAABhAZdYAF/AX9gAn9/AGABfwBgAn…f39/fwF/YAABf2AKf39/f39/f39/fwBgCH9/f39/f39/AX9gCX9/f39/f39/fwBgB3+0x22136) in wasm-function[-1]+0x22136
WebARKit.js:10 ==42==ABORTING
example.html:1 Uncaught (in promise) 
oB {name: "ExitStatus", message: "Program terminated with exit(1)", status: 1}

kalwalt · 2020-12-02T12:32:14Z

The last commit c7fe31b fixed the error.

// inside WebARKitOrbTracker.cpp
- WebARKitOrbTracker::WebARKitOrbTracker()	
+ WebARKitOrbTracker::WebARKitOrbTracker():corners(4)

I will implement now the other methods in WebARKitJS.cpp. 😄

kalwalt · 2020-12-03T15:36:51Z

I think i need separate the video setup and the image setup, doing the init in the same place is not recommended. So the setup https://github.com/kalwalt/webarkit-testing/blob/689585d601d363663de8dd9f727b49db61709d03/emscripten/WebARKitJS.cpp#L32 will be for video pointer initialization and passing data from javascript to WASM, and i will create a setupImage for the image to be tracked.

…Setup

kalwalt · 2020-12-03T17:06:07Z

Now i get this error:


WebARKit.js:14 1637
WebARKit.js:10 Start to initialize tracker...
WebARKit.js:10 Start!
WebARKit.js:10 2000
WebARKit.js:10 Orb created!
WebARKit.js:10 =================================================================
WebARKit.js:10 ==42==ERROR: AddressSanitizer: null-pointer-dereference on address 0x00000018 at pc 0x00083a34 bp 0x1067c890 sp 0x1067c89c
WebARKit.js:10 READ of size 1 at 0x00000018 thread T0
WebARKit.js:10     #0 0x83a34 in wasm-function[-1]+0x83a34 (data:application/octet-stream;base64,AGFzbQEAAAAB6AiAAWABfwF/YAJ/fwBgAX8AYA…AGAEf39/fwF/YAZ/f39/f38Bf2AAAX9gBH9/f3wAYAp/f39/f39/f39/AGAIf39/f3+0x83a34)
WebARKit.js:10 
WebARKit.js:10 Address 0x00000018 is located in the shadow gap area.
WebARKit.js:10 SUMMARY: AddressSanitizer: null-pointer-dereference (data:application/octet-stream;base64,AGFzbQEAAAAB6AiAAWABfwF/YAJ/fwBgAX8AYA…AGAEf39/fwF/YAZ/f39/f38Bf2AAAX9gBH9/f3wAYAp/f39/f39/f39/AGAIf39/f3+0x83a33) in wasm-function[-1]+0x83a33
WebARKit.js:10 ==42==ABORTING
WebARKit.js:10 [object Object]
WebARKit.js:10 Uncaught (in promise) RuntimeError: abort([object Object]). Build with -s ASSERTIONS=1 for more info.
    at ZA (http://127.0.0.1:5500/dist/WebARKit.js:10:310496)
    at SI (http://127.0.0.1:5500/dist/WebARKit.js:10:2907752)
    at <anonymous>:wasm-function[1026]:0x54dff
    at <anonymous>:wasm-function[107]:0x6121
    at <anonymous>:wasm-function[274]:0x10462
    at <anonymous>:wasm-function[153]:0x900b
    at <anonymous>:wasm-function[79]:0x4689
    at <anonymous>:wasm-function[1501]:0x83a34
    at <anonymous>:wasm-function[1574]:0x8f335
    at <anonymous>:wasm-function[3598]:0x14535e

Similar errors related to the 'corners', but i have not touched that code...

kalwalt · 2020-12-03T17:14:34Z

This happens because i'm passing to the _copyImageToHeap function a video HTML element that is object and not an Array of data... 😭 😄

kalwalt · 2020-12-03T17:59:36Z

Maybe the problem came because i mixing node-canvas package with standard canvas syntax., need to fix this.

- fixed odd error with pointer....

kalwalt · 2022-01-25T18:05:59Z

The stack-buffer-underflow error and other similar errors was caused because a wrong initialization of the WebARKitTracker. I will add some comment in the code/commits. See #4 (comment)
Spent a lot of time for this trivial error...

emscripten/WebARKitJS.cpp

kalwalt · 2022-01-26T15:22:09Z

with these flags: https://github.com/kalwalt/webarkit-testing/blob/f2ebc7ce28bde0c6eb18c4f3e81d123fb38070b5/tools/makem.js#L177
i get a stack underflow error:

webarkit_ES6_wasm.js:8   Global init order:       f6
webarkit_ES6_wasm.js:8   Poisoned by user:        f7
webarkit_ES6_wasm.js:8   Container overflow:      fc
webarkit_ES6_wasm.js:8   Array cookie:            ac
webarkit_ES6_wasm.js:8   Intra object redzone:    bb
webarkit_ES6_wasm.js:8   ASan internal:           fe
webarkit_ES6_wasm.js:8   Left alloca redzone:     ca
webarkit_ES6_wasm.js:8   Right alloca redzone:    cb
webarkit_ES6_wasm.js:8   Shadow gap:              cc
webarkit_ES6_wasm.js:8 ==42==ABORTING
webarkit_ES6_wasm.js:8 program exited (with status: 1), but EXIT_RUNTIME is not set, so halting execution but not exiting the runtime or preventing further async execution (build with EXIT_RUNTIME=1, if you want a true shutdown)
webarkit_ES6_wasm.js:8 [object Object]
webarkit_ES6_wasm.js:8 Uncaught (in promise) RuntimeError: abort([object Object]) at Error
    at jsStackTrace (webarkit_ES6_wasm.js:8:17546922)
    at stackTrace (webarkit_ES6_wasm.js:8:17547213)
    at abort (webarkit_ES6_wasm.js:8:59393)
    at ___sys_exit_group (webarkit_ES6_wasm.js:8:17609521)
    at __sanitizer::internal__exit(int) (031f8f8a:0x215b07)
    at __sanitizer::Die() (031f8f8a:0x20e243)
    at __asan::ScopedInErrorReport::~ScopedInErrorReport() (031f8f8a:0x208670)
    at __asan::ReportGenericError(unsigned long, unsigned long, unsigned long, unsigned long, bool, unsigned long, unsigned int, bool) (031f8f8a:0x20969c)
    at __asan_memcpy (031f8f8a:0x2045ff)
    at cv::Mat::copyTo(cv::_OutputArray const&) const (031f8f8a:0x776ff)
    at abort (webarkit_ES6_wasm.js:8:59424)
    at ___sys_exit_group (webarkit_ES6_wasm.js:8:17609521)
    at __sanitizer::internal__exit(int) (031f8f8a:0x215b07)
    at __sanitizer::Die() (031f8f8a:0x20e243)
    at __asan::ScopedInErrorReport::~ScopedInErrorReport() (031f8f8a:0x208670)
    at __asan::ReportGenericError(unsigned long, unsigned long, unsigned long, unsigned long, bool, unsigned long, unsigned int, bool) (031f8f8a:0x20969c)
    at __asan_memcpy (031f8f8a:0x2045ff)
    at cv::Mat::copyTo(cv::_OutputArray const&) const (031f8f8a:0x776ff)
    at cv::Mat::clone() const (031f8f8a:0x1301d)
    at WebARKitOrbTracker::resetTracking(unsigned char*, unsigned long, unsigned long) (031f8f8a:0x1179d)

the problem is passing data from the fameCurr to framePrev see: https://github.com/kalwalt/WebARKitLib/blob/ac267d5c57fa9a825fb57de51769f24763b74771/lib/SRC/WebARKitTrackers/WebARKitOpticalTracking/WebARKitOrbTracker.cpp#L88

if i comment out that line i did not receive this error.
And if i remove the Asan flag FLAGS += ' -gsource-map -fsanitize=address ' i did not get the undeflow error! any error at all. Probably i receive that because the videoframe data is not updating. I have to add a requestAnimationFrame inside https://github.com/kalwalt/webarkit-testing/blob/f2ebc7ce28bde0c6eb18c4f3e81d123fb38070b5/src/WebARKitController.js#L139-L151

- using emsdk 3.1.9

kalwalt · 2022-05-30T21:46:33Z

Not sure if this is correct but i will continue on this. I think #6 is not the best approach. Better create a class and intantiate it. As a reminder i'm using emsdk 3.1.9 and opencv-em 0.0.4, seee also comments in commits.
I think i will merge this soon in master and after improves will made in other PRs.

kalwalt · 2022-05-31T15:16:09Z

Finally now it track! But I needed to lower the number of frame Points detected to start the tracking process. There are a slight difference (about this aspect) in comparison to webarkit-OCVT. I think i can merge soon this PR. Other improves in other PRs.
One note as reminder: when i get out from view with the pinball image i get an exception error (or another type...): this need to be fixed as well.

kalwalt added 6 commits November 29, 2020 12:52

exclude opencv-em

80f2df3

setup function for testing

53fc7fc

installing canvas

6d4ca34

we can read data from pinball image

bd18e4f

loadTracker instead of initTracking

c13c230

data should be pass through the webARKitController into the initTrack…

c38ca92

…ing function, but nothing happens

kalwalt added the enhancement New feature or request label Nov 29, 2020

kalwalt self-assigned this Nov 29, 2020

kalwalt added 2 commits November 29, 2020 18:24

allow_raw_pointers() is not needed

6bc171a

this make a memory access out of bounds

efe055d

kalwalt commented Nov 29, 2020

View reviewed changes

src/WebARKitController.js Outdated Show resolved Hide resolved

kalwalt commented Nov 29, 2020

View reviewed changes

src/WebARKitController.js Outdated Show resolved Hide resolved

kalwalt and others added 4 commits November 30, 2020 11:17

log message when initialize the tracker

70a1600

more convenient and correct way to pass ImageData to Heap

ce8bc34

adding some log for debiugging

0b3fd8f

-fsanitize=address flags to better debug

85b6a43

fixi for wrong WebARKitOrbTracker initialization

3c99a38

kalwalt added 2 commits December 2, 2020 11:50

more debuggings printing

f50f3e1

fixing corners initialization

c7fe31b

kalwalt added 2 commits December 2, 2020 13:56

fix for no_libar option

52010f5

new track function

689585d

kalwalt added 2 commits December 3, 2020 17:54

adding video not working perfectly, and separation of setup and image…

4a91d66

…Setup

built libs and initTracking with errors

b8b3814

no errors while tracking now!

3ce5c10

- fixed odd error with pointer....

kalwalt commented Jan 25, 2022

View reviewed changes

emscripten/WebARKitJS.cpp Outdated Show resolved Hide resolved

kalwalt added 3 commits January 25, 2022 19:23

resetTracking and track after the Promise

14cf899

removing imageSetup

d479c65

adding more logs

f2ebc7c

kalwalt and others added 6 commits January 26, 2022 16:57

added requestAnimationFrame and loadedvideo event

ff85aef

better getImageData

6b18867

upgrading packages

699e7a8

change EXTRA_EXPORTED_RUNTIME_METHODS to EXPORTED_RUNTIME_METHODS

3d0b26e

adding more functions

0e457cd

testing getVideo

62213d9

kalwalt mentioned this pull request Feb 21, 2022

Testing new code #6

Closed

kalwalt and others added 5 commits April 5, 2022 01:19

using shared_ptr

ced21c8

removing old DEFINEs

72e236d

- using emsdk 3.1.9

up pkgs and clone frame only when Homog. is valid

9a810a2

updateT for tracking

c3f81e0

removing loading div when loaded all needed stuff

35a3de6

kalwalt added 2 commits May 31, 2022 16:47

new processFrame function

c450146

now it track!

41948ce

kalwalt added 2 commits May 31, 2022 17:22

basic infos and deleting test example

e21ede2

back to feature-webarkitrackers-pointers branch

4a96ade

kalwalt merged commit 253cd31 into main May 31, 2022

kalwalt mentioned this pull request Mar 28, 2023

Feature webarkitrackers kalwalt/WebARKitLib#4

Closed

4 tasks

kalwalt deleted the testing-pointers-setup branch April 6, 2023 21:54

kalwalt added javascript Emscripten / C / C++ all about Emscripten labels Apr 12, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pointers feature setup #4

Pointers feature setup #4

kalwalt commented Nov 29, 2020

kalwalt commented Dec 2, 2020 •

edited

Loading

kalwalt commented Dec 2, 2020

kalwalt commented Dec 2, 2020

kalwalt commented Dec 3, 2020

kalwalt commented Dec 3, 2020 •

edited

Loading

kalwalt commented Dec 3, 2020

kalwalt commented Dec 3, 2020 •

edited

Loading

kalwalt commented Jan 25, 2022 •

edited

Loading

kalwalt commented Jan 26, 2022

kalwalt commented May 30, 2022

kalwalt commented May 31, 2022 •

edited

Loading

Pointers feature setup #4

Pointers feature setup #4

Conversation

kalwalt commented Nov 29, 2020

kalwalt commented Dec 2, 2020 • edited Loading

kalwalt commented Dec 2, 2020

kalwalt commented Dec 2, 2020

kalwalt commented Dec 3, 2020

kalwalt commented Dec 3, 2020 • edited Loading

kalwalt commented Dec 3, 2020

kalwalt commented Dec 3, 2020 • edited Loading

kalwalt commented Jan 25, 2022 • edited Loading

kalwalt commented Jan 26, 2022

kalwalt commented May 30, 2022

kalwalt commented May 31, 2022 • edited Loading

kalwalt commented Dec 2, 2020 •

edited

Loading

kalwalt commented Dec 3, 2020 •

edited

Loading

kalwalt commented Dec 3, 2020 •

edited

Loading

kalwalt commented Jan 25, 2022 •

edited

Loading

kalwalt commented May 31, 2022 •

edited

Loading