Skip to content

Conversation

@pinball83
Copy link

@pinball83 pinball83 commented Oct 29, 2025

🛠️ PR Summary

Made with ❤️ by Ultralytics Actions

WARNING ⚠️ this PR is very large, summary may not cover all changes.

🌟 Summary

Multi-model refactor enabling concurrent model loading/inference across Android and iOS, with unified overlays, improved streaming payloads, and example app updates to use multiple YOLO models. 🎛️📱

📊 Key Changes

  • Android: Added multi-model APIs (setModels) in YOLOPlatformView/YOLOView with parallel inference, shared executor, model generation guards, and unified overlay rendering for DETECT/SEGMENT/POSE/OBB.
  • Android: Enhanced streaming payloads with per-detection modelName, viewBoundingBox, combined results aggregation, throttling/frequency controls, and resilience for event channel reconnection.
  • Android: ObjectDetector small refactors (formatting, logs, method signature readability) and safer label loading logs.
  • iOS: SwiftYOLOPlatformView supports models args and adds setModels (initial support + first-model fallback), retains thresholds/UI controls syncing, and streaming setup.
  • iOS: VideoCapture now more robust in selecting camera devices across hardware types; safer setup and cleanup.
  • Example app: Controller/UI refactor to support multiple active models via YOLOModelSpec, dynamic toggling, and updated YOLOView invocation; simplified overlays to use controller-provided models.
  • iOS project: Updated Podfile.lock checksums and Xcode project Development Team ID.

🎯 Purpose & Impact

  • Enables concurrent tasks (e.g., segment + pose) in one camera pipeline, improving flexibility and demo quality.
  • Delivers richer, more UI-ready streaming data (normalized, pixel, and view-mapped boxes plus model attribution).
  • Improves stability and performance with parallel inference, output throttling, and reconnection retries.
  • Simplifies app integration via a clearer model spec (YOLOModelSpec) and unified overlay rendering for multi-model results.
  • Broadens device compatibility and reliability on iOS with improved camera device selection.

@github-actions
Copy link

github-actions bot commented Oct 29, 2025

All Contributors have signed the CLA. ✅
Posted by the CLA Assistant Lite bot.

@pinball83 pinball83 changed the title Feature/refactor loading models (feat) Parallel loading and execution models Oct 29, 2025
@UltralyticsAssistant UltralyticsAssistant added detect Object Detection issues, PR's enhancement New feature or request pose Pose/keypoint estimation models labels Oct 29, 2025
@UltralyticsAssistant
Copy link
Member

👋 Hello @pinball83, thank you for submitting a ultralytics/yolo-flutter-app 🚀 PR! To ensure a seamless integration of your work, please review the following checklist:

  • Define a Purpose: Clearly explain the purpose of your fix or feature in your PR description, and link to any relevant issues. Ensure your commit messages are clear, concise, and adhere to the project's conventions.
  • Synchronize with Source: Confirm your PR is synchronized with the ultralytics/yolo-flutter-app main branch. If it's behind, update it by clicking the 'Update branch' button or by running git pull and git merge main locally.
  • Ensure CI Checks Pass: Verify all Ultralytics Continuous Integration (CI) checks are passing. If any checks fail, please address the issues.
  • Update Documentation: Update the relevant documentation for any new or modified features.
  • Add Tests: If applicable, include or update tests to cover your changes, and confirm that all tests are passing.
  • Sign the CLA: Please ensure you have signed our Contributor License Agreement if this is your first Ultralytics PR by writing "I have read the CLA Document and I sign the CLA" in a new message.
  • Minimize Changes: Limit your changes to the minimum necessary for your bug fix or feature addition. "It is not daily increase but daily decrease, hack away the unessential. The closer to the source, the less wastage there is." — Bruce Lee

For more guidance, please refer to our Contributing Guide. This is an automated message and an engineer will be with you shortly. Thanks for contributing to Ultralytics! ✨🚀

Copy link
Member

@UltralyticsAssistant UltralyticsAssistant left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 PR Review

Made with ❤️ by Ultralytics Actions

The Android platform view no longer loads a model when only the legacy modelPath/task parameters are passed, which will break existing apps; we should restore the single-model fallback in the initializer.

💬 Posted 1 inline comment

⚠️ Large PR: Review focused on critical issues. Some details may not be covered.

📋 Skipped 1 file (lock files, minified, images, etc.)

initialized = true
}
} else {
Log.w(TAG, "No models provided in creationParams; skipping model load")

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ HIGH: creationParams used to rely on modelPath/task for single-model loading. With this branch we now skip setModel entirely whenever models isn’t supplied, so every existing caller that still sends the legacy params will end up with a camera preview and no inference. We need to keep the old fallback (e.g. call setModel(...) here) so the API remains backwards compatible.

@pinball83
Copy link
Author

I have read the CLA Document and I sign the CLA

@pinball83 pinball83 changed the title (feat) Parallel loading and execution models (feat) Parallel loading and execution models WIP Oct 29, 2025
@codecov
Copy link

codecov bot commented Oct 29, 2025

Codecov Report

❌ Patch coverage is 45.08197% with 67 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
lib/yolo_view.dart 1.78% 55 Missing ⚠️
lib/widgets/yolo_controller.dart 0.00% 5 Missing ⚠️
lib/models/yolo_model_spec.dart 91.48% 4 Missing ⚠️
lib/models/yolo_result.dart 72.72% 3 Missing ⚠️

📢 Thoughts on this report? Let us know!

@pinball83 pinball83 marked this pull request as ready for review October 29, 2025 23:52
@pinball83 pinball83 marked this pull request as draft October 30, 2025 00:03
@asabri97
Copy link
Collaborator

@pinball83 is this PR ready for review?

@pinball83
Copy link
Author

@pinball83 is this PR ready for review?

not yet actually, I need to test ios part application so I've been wait chance to get iphone, so now I have one after couple day I finalise PR

@pderrenger
Copy link
Member

Thanks for the update and for signing the CLA—please mark this PR as Draft while you finish iOS testing; when ready, add a brief device/OS and FPS/latency matrix to the PR description, consider splitting platform changes from example‑app updates to keep review focused, then tag me to begin review; see the Development Workflow for expectations. See the Development Workflow for expectations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

detect Object Detection issues, PR's enhancement New feature or request pose Pose/keypoint estimation models

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants