Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data_processing Switch CAN message timestamps from the msg packed time to using in-message timestamp by unwrapping the timestamp #213

Open
SuperZooper3 opened this issue Mar 3, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@SuperZooper3
Copy link
Contributor

For data processing / exporting, the timestamps used for CAN messages are based off the timestamps in the msg packed messages, which come from the timestamps of each computer on the omnibus, which are not synced. This causes issues where the same CAN message can be reported by two different systems at the same real time, but differing timestamps, so they would seem to be two different updates. As a current hack, only one "stream of time" is taken as truth, using msgpackFilterUnpacker, so it either ignores messages that come from "the past" (any messages where the time is lower than the currently seen max time, so like an earlier clock), or instead only takes these "old" messages, to ignore the "ahead of time" computer. These sometimes yield different results, with the behind mode giving more data points if it's available, but super super super hacky and bad.

To remedy this, timestamps should be decoded from the contents of the CAN/Parsley message. The only issue is that this is a small value (16 or 24 bits) so overflows a lot, and we would need to undo it.

For more info, see: https://waterloorocketry.slack.com/archives/C07MX0QDS/p1706479899045329 In here, there is also an algorithm proposed to unwrap the timestamps, but more work needs to be put into planning how this process should be implemented into the data exporting pipeline (maybe enable inserting messages at arbitrary timestamps? change the way msgpacked messages are ingested?)

@SuperZooper3 SuperZooper3 added the enhancement New feature or request label Mar 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant