Adapter Developer Guide

Note: This page is meant for developers who are interested in creating/maintaining new/current adapters. If you're only interested in using them, you probably want to visit the driver usage page instead.

Overview

The design for adapter/driver is heavily based on the Unix design principle, where everything is a file. Therefore, you can always think of any adapter as a file, you open the adapter, read from it, and then close it:

How to implement a new Adapter?

1. Implement Adapter interface

Adapters must implement the Adapter interface. This interface is being used to generalize different kinds of adapters. We use this interface so that we can store it in the driver manager.

type Adapter interface {
	Open() error
	Close() error
	Properties() []prop.Media
}

Open

Opens the underlying hardware through an OS interface, might also get permission when it's needed (permission to read from the hardware). At this point, the adapter user can start getting its Properties and get data from it through, VideoRecord or AudioRecord (which are described below).

Close

Cleans up, for example, free all used resources and stop the underlying hardware from transmitting data.

Properties

Returns a list of prop.Media that the adapter supports. These values will later be chosen by the adapter user for VideoRecord or AudioRecord.

2. Implement Specialized interface

Each adapter must implement either

package driver

type VideoRecorder interface {
	VideoRecord(p prop.Media) (r video.Reader, err error)
}

or

package driver

type AudioRecorder interface {
	AudioRecord(p prop.Media) (r audio.Reader, err error)
}

These methods do the work of getting the actual data from the hardware.

The methods accept a prop.Media defining what the user would like from the hardware. And is chosen from a list of prop.Media given to the user when they call Properties().

Which interacts with the hardware to get video/audio as raw bytes, decodes the raw bytes into frames, and returns video.Reader or audio.Reader. A call to Read() on the reader provides the decoded data.

Video Reader Interface

package video

type Reader interface {
	Read() (img image.Image, err error)
}

TODO: More description

Audio Reader Interface

package audio

type Reader interface {
	Read(samples [][2]float32) (n int, err error)
}

How to format samples?

Stereo Channel

• The L and the R are samples, simply floating-point numbers in the range of [0,1], this number represents the amplitude at a given time.

• In stereo, a data point is a 'LR pair' which represents what input we are giving the left ear and independent input we are giving to the right ear at this point in time.

• We consider a 'LR pair' as a data point and a frame as a list of 'LR pairs.

• Bringing these concepts together, the Read method returns a 2D array with the first array representing the pairs. And, the second dimension represents the LR values.

Mono Channel

• This is the same concept as Stereo Channel, except for we consider ourselves giving the listener the same sound to both ears, so instead of 'LR pairs' we have 'LL pairs' / 'RR pairs'.

• Basically it returns two copies of the same sample.

3. Register yourself

You must register yourself to allow your adapter to be discoverable.

To do so you must provide:

Device Instance

You should define a type for your Adapter to hold the necessary information. Then create an instance of your type.

Adapter Label

A label for your adapter (for metadata). A string.

Device Type

Your device type is exactly one of these:

• Camera
• Screen
  • for screen-sharing
• Microphone

Pass Information to Manager

driver.GetManager().Register(<DEVICE_INSTANCE>, driver.Info{
	Label: <ADAPTER_LABEL>,
	DeviceType: driver.<DEVICE_TYPE>,
})

What is a driver manager?

The driver manager manages the Adapters in various ways.

//TODO: add details

Including:

• The state of each adapter
  • ex. Adapter a is open, Adapter b is running
• Metadata about each adapter
  • ...
  • Unique id's
• A list of adapters
  • providing discoverability to adapters