@AutoValue
abstract class Frame


Represents a frame that is about to be rendered.

Use this class to draw overlay on camera output. It contains a Canvas for the drawing. It also provides metadata for positioning the overlay correctly, including sensor-to-buffer transform, size, crop rect, rotation, mirroring, and timestamp.

Summary

Public constructors

Public functions

abstract Rect

Returns the crop rect.

Canvas

Get the canvas for drawing the overlay.

abstract @IntRange(from = 0, to = 359) Int

Returns the rotation degrees of the frame.

abstract Matrix

Returns the sensor to image buffer transform matrix.

abstract Size

Returns the resolution of the frame.

abstract Long

Returns the timestamp of the frame in nanoseconds.

abstract Boolean

Returns whether the buffer will be mirrored.

Public constructors

Frame

Added in 1.4.0
Frame()

Public functions

getCropRect

Added in 1.4.0
abstract fun getCropRect(): Rect

Returns the crop rect.

The value represents how CameraX intends to crop the input frame. The crop rect specifies the region of valid pixels in the frame, using coordinates from (0, 0) to the (width, height) of getSize. Only the overlay drawn within the bound of the crop rect will be visible to the end users.

The crop rect is applied before the rotating and mirroring. The order of the operations is as follows: 1) cropping, 2) rotating and 3) mirroring.

See also
getCropRect

getOverlayCanvas

Added in 1.4.0
fun getOverlayCanvas(): Canvas

Get the canvas for drawing the overlay.

Call this method to get the Canvas for drawing an overlay on top of the frame. The Canvas is backed by a SurfaceTexture with a size equal to getSize. To draw object in camera sensor coordinates, apply getSensorToBufferTransform via setMatrix before drawing.

Using this method introduces wait times to synchronize frame updates. The caller should only invoke this method when it needs to draw overlay. For example, when an object is detected in the frame.

getRotationDegrees

Added in 1.4.0
abstract fun getRotationDegrees(): @IntRange(from = 0, to = 359) Int

Returns the rotation degrees of the frame.

This is a clockwise rotation in degrees that needs to be applied to the frame. The rotation will be determined by camera sensor orientation and UseCase configuration such as setTargetRotation. The app must draw the overlay according to the rotation degrees to ensure it is displayed correctly to the end users. For example, to overlay a text, make sure the text's orientation is aligned with the rotation degrees.

The rotation is applied after the cropping but before the mirroring. The order of the operations is as follows: 1) cropping, 2) rotating and 3) mirroring.

getSensorToBufferTransform

Added in 1.4.0
abstract fun getSensorToBufferTransform(): Matrix

Returns the sensor to image buffer transform matrix.

The value is a mapping from sensor coordinates to buffer coordinates, which is, from the rect of the camera sensor to the rect defined by (0, 0, #getSize() #getWidth(), #getSize()#getHeight()).

The value can be set on the Canvas using setMatrix API. This transforms the Canvas to the sensor coordinate system.

getSize

Added in 1.4.0
abstract fun getSize(): Size

Returns the resolution of the frame.

This is the size of the input SurfaceTexture created by the effect.

See also
getResolution

getTimestampNanos

Added in 1.4.0
abstract fun getTimestampNanos(): Long

Returns the timestamp of the frame in nanoseconds.

This value will match the frames from other streams. For example, for a ImageAnalysis output that is originated from the same frame, this value will match the value of getTimestamp.

isMirroring

Added in 1.4.0
abstract fun isMirroring(): Boolean

Returns whether the buffer will be mirrored.

This flag indicates whether the buffer will be mirrored across the vertical axis by the pipeline. For example, for front camera preview, the buffer is usually mirrored before displayed to end users.

The mirroring is applied after the cropping and the rotating. The order of the operations is as follows: 1) cropping, 2) rotating and 3) mirroring.

See also
isMirroring