Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Full tests and robustnes against frame ID wrapping for packet decoding #19

Open
wants to merge 7 commits into
base: k2is
Choose a base branch
from

Commits on Jun 23, 2021

  1. Test for partition and dataset carry

    Do different partition and dataset scenarios that trigger carry situations
    uellue committed Jun 23, 2021
    Configuration menu
    Copy the full SHA
    a71dca5 View commit details
    Browse the repository at this point in the history
  2. Detect completion correctly for "tail" tiles; process one more load o…

    …f packets in wrapup
    
    Make sure we catch near-range stragglers if there is unfinished work. That way we
    "survive" massive scrambling within the range of one tile.
    
    Adapt test cases accordingly; also test dups which is trivial.
    uellue committed Jun 23, 2021
    Configuration menu
    Copy the full SHA
    fc1f826 View commit details
    Browse the repository at this point in the history
  3. Validate against package drop

    uellue committed Jun 23, 2021
    Configuration menu
    Copy the full SHA
    0fb48be View commit details
    Browse the repository at this point in the history
  4. Deal with wrap-around

    Use a "virtual corrected frame ID" and correct wrap already during decoding.
    Wrapping is detected directly after processing packets and is corrected as early as possible.
    
    To not get mixed up in "old" and "new" epoch, the decoded header is stored
    in the carry buffers.
    uellue committed Jun 23, 2021
    Configuration menu
    Copy the full SHA
    1a622a7 View commit details
    Browse the repository at this point in the history

Commits on Jun 24, 2021

  1. Avoid memory allocation for copies

    Test for wrapping was flaky for 128 scrambling,
    test it only for less aggressive scrambling.
    
    In reality, inconsistencies in the data stream are rare,
    and the packets are almost always in order, so jump
    detection will work with high reliablity.
    uellue committed Jun 24, 2021
    Configuration menu
    Copy the full SHA
    e230e5c View commit details
    Browse the repository at this point in the history
  2. Block dropping

    Also fix comments from before
    
    The tests are still flaky sometimes, more work to be done
    uellue committed Jun 24, 2021
    Configuration menu
    Copy the full SHA
    8679810 View commit details
    Browse the repository at this point in the history

Commits on Jul 22, 2021

  1. Allow recording stragglers

    * Account for stragglers in test
    * Refine test matrix to exclude combinations that can "trip"
      the packet reordering. The test is already much harder than the actual data stream,
      which is almost always in order and complete with only very few
      missing, scrambled or duplicate packets.
    uellue committed Jul 22, 2021
    Configuration menu
    Copy the full SHA
    53a76ed View commit details
    Browse the repository at this point in the history