fix(process_video): deadlock on callback exception + improve shutdown robustness

[SkalskiP]

Description

The updated process_video function implements a three-stage threaded pipeline consisting of a reader, a main-thread processor, and a writer, while using bounded queues to strictly control memory consumption during video processing.

When the user-supplied callback raises an exception, the function logs the error along with the affected frame index and, by default, re-raises it after performing a clean shutdown of all components.

With the optional skip_on_error=True parameter, problematic frames are silently discarded instead of interrupting the entire job, which is especially useful for noisy or partially corrupted videos.

The reader thread runs as non-daemon so that cv2.VideoCapture.release() is reliably called even during early termination, preventing file handle or resource leaks.

Shutdown is managed through a combination of a stop_event, non-blocking queue operations with small timeouts, and a light sleep-based backoff that avoids both deadlocks and excessive CPU usage.

The writer thread is kept as a daemon because disk writes can usually be safely interrupted, while both threads receive sensible join timeouts with logging warnings if they fail to complete in time.

Type of Change

🐛 Bug fix (non-breaking change which fixes an issue)

Motivation and Context

This change fixes a critical bug in the process_video function where an unhandled exception thrown inside the user-provided callback would cause the entire processing pipeline to deadlock indefinitely.

The main thread would exit its processing loop but then block forever on reader_worker.join() while the reader thread (a daemon) was stuck trying to put() into a full frame_read_queue.

Testing

• I have tested this code locally
• I have added unit tests that prove my fix is effective or that my feature works
• All new and existing tests pass

[codecov]

Codecov Report

❌ Patch coverage is 1.61290% with 61 lines in your changes missing coverage. Please review.
✅ Project coverage is 51%. Comparing base (b85b41d) to head (7c8ce48).
⚠️ Report is 117 commits behind head on develop.

Additional details and impacted files

@@           Coverage Diff           @@
##           develop   #2102   +/-   ##
=======================================
- Coverage       52%     51%   -0%     
=======================================
  Files           61      61           
  Lines         7076    7117   +41     
=======================================
  Hits          3657    3657           
- Misses        3419    3460   +41     

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.