Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Describe this pull request. Link to relevant GitHub issues, if any.
Joint PR with
ada_feeding
#151Although #106 improved video streaming by replacing the dependency on
web_video_server
with direct subscription to compressed images, the latency of this approach depends on the device. On a desktop computer, the video stream has 1-3 sec latency, but on an iPhone 11, it has increasing latency that gets into the tens of seconds.Looking into it, the most likely reason for this is that
roslibjs
uses EventEmitter2'semit
event, which is synchronous. Thus, even if the subscriber on the ROS side has no queue, a queue builds up on the web app side.The best solution to this, as documented here, is to implement WebRTC peer-to-peer streaming between the robot and web app. That is documented in issue #112 .
However, given the learning curve and time involved in implementing WebRTC, this PR and its affiliated PR
ada_feeding
#151 introduces a temporary fix to the problem, in the following ways:ada_feeding_perception/config/republisher.yaml
and inVideoFeed.jsx
)VideoFeed.jsx
)Note that these parameters can be tuned to find a subjectively optimal setting.
Explain how this pull request was tested, including but not limited to the below checkmarks.
DetectingFace
page, move your head around in front of the camera, verify that the latency is not more than 1-2 secs.Before creating a pull request
npm run format
python3 -m black .
in the top-level of this repositoryBefore merging a pull request
Squash and Merge
)