Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[avfoundation] Add camera specifics to PlatformCameraDescription #8323

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

tjamet
Copy link

@tjamet tjamet commented Dec 17, 2024

The purpose of this PR is to open a conversation how to better support the various cameras offered by modern platforms.

In particular, starting with the iOS ecosystem offering a wide range of cameras, including cameras with adaptive field of views like AVCaptureDeviceTypeBuiltInDualCamera or AVCaptureDeviceTypeBuiltInTripleCamera.

This PR envisions:

  1. Provide as amny infromation as possible to the PlatformCameraDescription objects

Those objects would exclusively act as adaptation layers to provide close to full-feature within dart.

  1. Enrich camera_platform_interface with new high level types

The goal would be to support virtual cameras composed of multiple physical ones, simulating a zoom factor.

To support this, the CameraDescription class would accept an optional, nullable, fieldOfViews list accepting an enum similar to:

/// The field of view of a single physical camera
enum CameraLensFieldOfView {
  /// An utra large field of view, typically around 100 degrees or more.
  /// This is the equivalent of a fisheye lens (~10-20mm focal length).
  ultraWide,

  /// A wide field of view, typically around 60 degrees.
  /// This is the equivalent of a ~24-35mm focal length.
  wide,

  /// A standard field of view, typically around 40 degrees.
  /// This is the default field of view for most cameras.
  /// This is the equivalent of a ~50mm focal length.
  standard,

  /// A telephoto field of view, typically around 20 degrees.
  /// This is the equivalent of a ~85-135mm focal length.
  telephoto,

  /// A super telephoto field of view, typically around 10 degrees.
  /// This is the equivalent of a ~200mm focal length or more.
  superTelephoto,
}

class CameraDescription {
  /// Creates a new camera description with the given properties.
  const CameraDescription({
    required this.name,
    required this.lensDirection,
    required this.sensorOrientation,
    this.fieldOfViews,
  });
  final List<CameraLensFieldOfView>? fieldOfViews;
  1. Once fieldOfViews is available in the CameraDescription class, cameraDescriptionFromPlatform from avfoundation, should populate the releant field of views, based on the apple documentation:
AVCaptureDeviceTypeBuiltInWideAngleCamera
A built-in wide-angle camera device type.

AVCaptureDeviceTypeBuiltInUltraWideCamera
A built-in camera device type with a shorter focal length than a wide-angle camera.

AVCaptureDeviceTypeBuiltInTelephotoCamera
A built-in camera device type with a longer focal length than a wide-angle camera.

AVCaptureDeviceTypeBuiltInDualCamera
A built-in camera device type that consists of a wide-angle and telephoto camera.

AVCaptureDeviceTypeBuiltInDualWideCamera
A built-in camera device type that consists of two cameras of fixed focal length, one ultrawide angle and one wide angle.

AVCaptureDeviceTypeBuiltInTripleCamera
A built-in camera device type that consists of three cameras of fixed focal length, one ultrawide angle, one wide angle, and one telephoto.

Similarly, all other supported platform should provide the camera field of view, if known

Replace this paragraph with a description of what this PR is changing or adding, and why. Consider including before/after screenshots.

List which issues are fixed by this PR. You must list at least one issue.

Pre-launch Checklist

If you need help, consider asking for advice on the #hackers-new channel on Discord.

The purpose of this PR is to open a conversation how to better support the various cameras offered by
modern platforms.

In particular, starting with the iOS ecosystem offering a wide range of cameras, including cameras
with adaptive field of views like `AVCaptureDeviceTypeBuiltInDualCamera` or `AVCaptureDeviceTypeBuiltInTripleCamera`.

This PR envisions:

1. Provide as amny infromation as possible to the PlatformCameraDescription objects

Those objects would exclusively act as adaptation layers to provide close to full-feature within dart.

2. Enrich `camera_platform_interface` with new high level types

The goal would be to support virtual cameras composed of multiple physical ones, simulating a zoom factor.

To support this, the `CameraDescription` class would accept an optional, nullable, fieldOfViews list accepting an enum similar to:

```dart
/// The field of view of a single physical camera
enum CameraLensFieldOfView {
  /// An utra large field of view, typically around 100 degrees or more.
  /// This is the equivalent of a fisheye lens (~10-20mm focal length).
  ultraWide,

  /// A wide field of view, typically around 60 degrees.
  /// This is the equivalent of a ~24-35mm focal length.
  wide,

  /// A standard field of view, typically around 40 degrees.
  /// This is the default field of view for most cameras.
  /// This is the equivalent of a ~50mm focal length.
  standard,

  /// A telephoto field of view, typically around 20 degrees.
  /// This is the equivalent of a ~85-135mm focal length.
  telephoto,

  /// A super telephoto field of view, typically around 10 degrees.
  /// This is the equivalent of a ~200mm focal length or more.
  superTelephoto,
}

class CameraDescription {
  /// Creates a new camera description with the given properties.
  const CameraDescription({
    required this.name,
    required this.lensDirection,
    required this.sensorOrientation,
    this.fieldOfViews,
  });
  final List<CameraLensFieldOfView>? fieldOfViews;
```

3. Once fieldOfViews is available in the CameraDescription class, `cameraDescriptionFromPlatform` from avfoundation,
should populate the releant field of views, based on the apple documentation:

```
AVCaptureDeviceTypeBuiltInWideAngleCamera
A built-in wide-angle camera device type.

AVCaptureDeviceTypeBuiltInUltraWideCamera
A built-in camera device type with a shorter focal length than a wide-angle camera.

AVCaptureDeviceTypeBuiltInTelephotoCamera
A built-in camera device type with a longer focal length than a wide-angle camera.

AVCaptureDeviceTypeBuiltInDualCamera
A built-in camera device type that consists of a wide-angle and telephoto camera.

AVCaptureDeviceTypeBuiltInDualWideCamera
A built-in camera device type that consists of two cameras of fixed focal length, one ultrawide angle and one wide angle.

AVCaptureDeviceTypeBuiltInTripleCamera
A built-in camera device type that consists of three cameras of fixed focal length, one ultrawide angle, one wide angle, and one telephoto.
```

Similarly, all other supported platform should provide the camera field of view, if known

relates to: flutter/flutter#134151
@tjamet tjamet requested a review from hellohuanlin as a code owner December 17, 2024 21:29
@flutter-dashboard
Copy link

It looks like this pull request may not have tests. Please make sure to add tests before merging. If you need an exemption, contact "@test-exemption-reviewer" in the #hackers channel in Discord (don't just cc them here, they won't see it!).

If you are not sure if you need tests, consider this rule of thumb: the purpose of a test is to make sure someone doesn't accidentally revert the fix. Ask yourself, is there anything in your PR that you feel it is important we not accidentally revert back to how it was before your fix?

Reviewers: Read the Tree Hygiene page and make sure this patch meets those guidelines before LGTMing. The test exemption team is a small volunteer group, so all reviewers should feel empowered to ask for tests, without delegating that responsibility entirely to the test exemption group.

@tjamet
Copy link
Author

tjamet commented Dec 17, 2024

Hi!

This PR is the first draft of the idea to enrich camera support.

I think it's going to need discussion, refinement, and fixes.
I am trying to figure out how to handle unit and integration testing for bindings.

Also, I wanted to discuss the idea, and this PR would be a good starting point.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant