[camera] add video stabilization

[ruicraveiro]

Adds support for video stabilization to camera_platform_interface, camera_avfoundation, camera_android_camerax and camera packages.

The video stabilization modes are defined in the new VideoStabilizationMode enum defined in camera_platform_interface:

/// The possible video stabilization modes that can be capturing video.
enum VideoStabilizationMode {
  /// Video stabilization is disabled.
  off,

  /// Basic video stabilization is enabled.
  /// Maps to CONTROL_VIDEO_STABILIZATION_MODE_ON on Android
  /// and throws CameraException on iOS.
  on,

  /// Standard video stabilization is enabled.
  /// Maps to CONTROL_VIDEO_STABILIZATION_MODE_PREVIEW_STABILIZATION on Android
  /// (camera_android_camerax) and to AVCaptureVideoStabilizationModeStandard
  /// on iOS.
  standard,

  /// Cinematic video stabilization is enabled.
  /// Maps to CONTROL_VIDEO_STABILIZATION_MODE_PREVIEW_STABILIZATION on Android
  /// (camera_android_camerax) and to AVCaptureVideoStabilizationModeCinematic
  /// on iOS.
  cinematic,

  /// Extended cinematic video stabilization is enabled.
  /// Maps to AVCaptureVideoStabilizationModeCinematicExtended on iOS and
  /// throws CameraException on Android.
  cinematicExtended,
}

There is some subjectivity on the way with which I mapped the modes to both platforms, and here's a document that compares the several modes: https://docs.google.com/spreadsheets/d/1TLOLZHR5AcyPlr-y75aN-DbR0ssZLJjpV_OAJkRC1FI/edit?usp=sharing, which you can comment on.

List which issues are fixed by this PR. You must list at least one issue.
Partially implements flutter/flutter#89525

Pre-launch Checklist

• I read the Contributor Guide and followed the process outlined there for submitting PRs.
• I read the Tree Hygiene page, which explains my responsibilities.
• I read and followed the relevant style guides and ran the auto-formatter. (Unlike the flutter/flutter repo, the flutter/packages repo does use dart format.)
• I signed the CLA.
• The title of the PR starts with the name of the package surrounded by square brackets, e.g. [shared_preferences]
• I linked to at least one issue that this PR fixes in the description above.
• I updated pubspec.yaml with an appropriate new version according to the pub versioning philosophy, or this PR is exempt from version changes.
• I updated CHANGELOG.md to add a description of the change, following repository CHANGELOG style, or this PR is exempt from CHANGELOG changes.
• I updated/added relevant documentation (doc comments with ///).
• I added new tests to check the change I am making, or this PR is test-exempt.
• All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

[google-cla]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

[hellohuanlin]

only reviewed iOS part

[ruicraveiro]

Hi @hellohuanlin,

I have improved camera_avfoundation's implementation based on your comments. Instead of adding a new commit, I forced pushed a squash commit of those changes and another commit on top of it that adds the dependency_overrides to the pubspecs.

Assuming there will be more improvments I need to make, is it OK I keep adding commits until everything is fixed, and only then do a final squash merge, excluding the dependency_overrides commit, and force that squash commit?

Thanks!

[romainfd]

Hi, thank you and congratulations @ruicraveiro (and @stuartmorgan-g) for the amazing work on this! I am very interested in this feature, is there any way for me to help you progress on it?

[bparrishMines]

@ruicraveiro It looks like the camera_android_camerax ProxyApi update broke your changes to that plugin. If you are still working on this, let me know if you need help updating that portion or I can also quickly update the API wrapper for you.

[harryterkelsen]

Please make the change Stuart requested of not throwing UnimplementedError in the platform interface and having each platform gracefully indicate whether or not they support video stabilization.

[gyenesvi]

Hi @ruicraveiro, is this still ongoing work or is it stalled? Any ideas how much work is required / when this could become available in Flutter? Would be nice to understand if this feature is worth waiting for, as video recording in its current form is not very useful for my purposes. Thanks!

[stuartmorgan-g]

Thank you for your contribution. Since there are outstanding comments but the PR hasn’t been updated in several months, I’m going to close it so that our PR queue reflects active PRs. Please don't hesitate to submit a new PR if you have the time to address the review comments. Thanks!

[stuartmorgan-g]

@romainfd, @gyenesvi Anyone is welcome to make their own forks of PRs that are no longer being worked on, and open new PRs to complete the work.

[romainfd]

Great, thank you for clarifying things @stuartmorgan-g! I just contacted @gyenesvi to make sure we don't both work on it at the same time but, unless he wants to do it before, I should be able to give it a try late August or in September.

[ruicraveiro]

@romainfd No, I didn't mean to work on this; I was just curious about the status/progress of this feature, whether there's anything blocking it or there's a chance that it's going to happen anytime soon. My understanding is that this feature is available on the native platforms in some form, so it's mostly a matter of enabling this setting from the flutter side. From the comments it seemed that the bulk of it had already been implemented, and the past year was spent with discussing naming and default value issues. Is that right? Or are there more significant bits missing?

This feature seems like a pretty basic/necessary addition to the package, and flutter developers have been complaining about its non-existence for several years, as this puts flutter based camera apps to a big disadvantage compared to native apps. In my case, the video I can record with my flutter app using this package looks like videos from a decade ago compared to videos my iphone (12) is actually capable of recording, which kind of invalidates the whole point of my app. So I'd like to know if this whole path is viable for my app.

Do you guys know why @ruicraveiro stopped working on this / responding to requests after the initial commits?

Hi, I didn't exactly quit the PR, but I haven't found the time to try yet again to get it approved. However, since @stuartmorgan-g closed the PR, that determines that I am done with it. I would be grateful, though, if someone has the patience and energy to see this through.

[stuartmorgan-g]

However, since @stuartmorgan-g closed the PR, that determines that I am done with it.

I closed is as part of our standard PR triage process; if we didn't close PRs that were inactive for months we would have have hundreds of abandoned PRs to go through on a regular basis, which isn't a good use of time, and would make it much harder to make sure active PRs are not falling through the cracks.

If you want to pick this back up again later and would rather not open a new PR referencing this one, please feel free to leave a comment here and I'd be happy to re-open it.

[gyenesvi]

Hi, I didn't exactly quit the PR, but I haven't found the time to try yet again to get it approved. However, since @stuartmorgan-g closed the PR, that determines that I am done with it. I would be grateful, though, if someone has the patience and energy to see this through.

Thanks for the heads up anyways, I was kind of afraid that this would be the case, though it seemed to have been close to completion. Let's hope @romainfd has the knowledge and time to pick this up and finish it.

[fluttergithubbot]

An existing Git SHA, d71672f0227206bafadcbfdbbdf4a11b7ce3c36f, was detected, and no actions were taken.

To re-trigger presubmits after closing or re-opeing a PR, or pushing a HEAD commit (i.e. with --force) that already was pushed before, push a blank commit (git commit --allow-empty -m "Trigger Build") or rebase to continue.

[mdebbar]

No web changes, so the web part looks good to me.

[ruicraveiro]

Hi, I just pushed the re-implementation of this feature based on the latest commit in main. It keeps all the logic that we've discussed earlier.

You can find the old implementation here: https://github.com/ruicraveiro/packages/tree/vs_squashed_old

[stuartmorgan-g]

Why do we need to support string->enum mapping?

[ruicraveiro]

Must be leftover. I just removed it and also removed the unit tests related to this method. Indeed it was no longer being used anywhere.

[stuartmorgan-g]

"This feature is only available if [getSupportedVideoStabilizationModes] returns at least one value other than [VideoStabilizationMode.off]."

[ruicraveiro]

Fixed.

[stuartmorgan-g]

Why are these comments wrapped to such a short line length, instead of the 80 characters used by the code?

[ruicraveiro]

Fixed.

[stuartmorgan-g]

This paragraph should be removed.

[ruicraveiro]

Done.

[stuartmorgan-g]

If a new orthogonal mode were added to the platform interface in the future, which wouldn't be a breaking change for the platform interface package, this code would suddenly become wrong for anyone getting the latest version of all packages (up until the app-facing package were updated).

I think the way to implement this would be a public method in the platform interface package that has a switch mapping each mode to its next best fallback mode, and then, starting from the requested mode, call that helper function until it returns something that's in supportedModes.

That way, setting the fallback logic for a new mode happens in the same package as adding the new mode, so it all works at runtime regardless of version combinations.

[ruicraveiro]

Thanks for the suggestion. I just pushed a new commit with this and I actually like it better.

[stuartmorgan-g]

isEmpty will never be true.

[ruicraveiro]

Yeah, that's a leftover of the previous logic.

I added the guarantee that supported modes is never empty at the camera package level, not at the platform level. In other words, CameraController.getSupportedVideoStabilizationModes will never return an empty list, but CameraPlatform.instance.getSupportedVideoStabilizationModes() may and that is the method that is being called by _getVideoStabilizationModeToSet to get the actual modes available from the device . This is actually needed by CameraController.setVideoStabilizationMode() so that it knows whether it's supposed to fallback to a no-op in case the device doesn't support video stabilization at all (not even off) or if the fallback is to turn off.

This is the bit of code inside CameraController.getSupportedVideoStabilizationModes that guarantees that off is always included:

      final Set<VideoStabilizationMode> modes = <VideoStabilizationMode>{
        VideoStabilizationMode.off,
        ...await CameraPlatform.instance.getSupportedVideoStabilizationModes(
          _cameraId,
        ),
      };

[ruicraveiro]

Anyway, I ended changing this code as well while changing _getVideoStabilizationModeToSet() to use the new getFallbackVideoStabilizationMode() added to CameraPlatform. The only bit of code that I am not 100% sure of, but I think is in accordance of what we agreed is the following:

    // If it can't fallback and the specific
    // requested mode isn't available, then...
    if (!allowFallback && !supportedModes.contains(requestedMode)) {
      // if the request is off, it is a no-op
      if (requestedMode == VideoStabilizationMode.off) {
        return null;
      }
      // otherwise, it throws.
      throw ArgumentError('Unavailable video stabilization mode.', 'mode');
    }

So, basically here I am handling the case where the request is for off. In this case, even though if allowFallback is false, we're handling off as a special situation in which it is still fallback-ish in the sense that if the device doesn't even support that, it is to be handled as a no-op. This is in the spirit that setVideoStabilizationMode will always accept a mode returned by getSupportedVideoStabilizationModes() without throwing, even when allowFallback=false, and that getSupportedVideoStabilizationModes() will always return off, even when the device doesn't support any mode at all, which means that getSupportedVideoStabilizationModes() must handle off as a valid request even with allowFallback=false and the way it handles off when it is not supported is by doing nothing.