@UnstableApi
public interface VideoSink

A sink that consumes decoded video frames and images from video and image renderers.

Multiple renderers can feed the same sink, but not in parallel.

Summary

Nested types
@Documented
@Retention(value = RetentionPolicy.SOURCE)
@Target(value = TYPE_USE)
@IntDef(value = )
public annotation VideoSink.InputType

Specifies how the input frames are made available to the video sink.
public interface VideoSink.Listener

Listener for VideoSink events.

Handler for a video frame.
public final class VideoSink.VideoSinkException extends Exception

Thrown by VideoSink implementations.

Constants
default static final int

Input frames come from a Bitmap.
default static final int

Input frames come from a surface.

Public methods
abstract void

Clears the set output surface info.
abstract void

Enables this video sink to render the start of the stream to its output surface even if the renderer is not started yet.
abstract void
flush(boolean resetPosition)

Flushes the video sink.
abstract Surface

Returns the input Surface where the video sink consumes input frames from.
abstract boolean
handleInputBitmap(
    Bitmap inputBitmap,
    TimestampIterator timestampIterator
)

Handles an input Bitmap.
abstract boolean
handleInputFrame(
    long framePresentationTimeUs,
    boolean isLastFrame,
    long positionUs,
    long elapsedRealtimeUs,
    VideoSink.VideoFrameHandler videoFrameHandler
)

Handles a video input frame.
abstract void
initialize(Format sourceFormat)

Initializes the video sink.
abstract boolean

Returns whether all the data has been rendered to the output surface.
abstract boolean

Returns whether the video sink is initialized.
abstract boolean
isReady(boolean rendererOtherwiseReady)

Returns whether the video sink is able to immediately render media to its output surface from the current position.
abstract void
join(boolean renderNextFrameImmediately)

Joins the video sink to a new stream.
abstract void

Informs the video sink that a new input stream will be queued.
abstract void

Called when the Renderer currently feeding this sink is disabled.
abstract void
onRendererEnabled(boolean mayRenderStartOfStream)

Called when the Renderer currently feeding this sink is enabled.
abstract void

Called when the Renderer currently feeding this sink is started.
abstract void

Called when the Renderer currently feeding this sink is stopped.
abstract void

Releases the sink.
abstract void
render(long positionUs, long elapsedRealtimeUs)

Incrementally renders processed video frames to the output surface.
abstract void
setChangeFrameRateStrategy(
    @C.VideoChangeFrameRateStrategy int changeFrameRateStrategy
)

Changes the C.VideoChangeFrameRateStrategy used when calling setFrameRate.
abstract void
setListener(VideoSink.Listener listener, Executor executor)

Sets a Listener on this sink.
abstract void
setOutputSurfaceInfo(Surface outputSurface, Size outputResolution)

Sets the output surface info.
abstract void

Sets video effects to apply after the next stream change.
abstract void
setPlaybackSpeed(@FloatRange(from = 0, fromInclusive = false) float speed)

Sets the playback speed.
abstract void
setStreamTimestampInfo(
    long streamStartPositionUs,
    long streamOffsetUs,
    long bufferTimestampAdjustmentUs,
    long lastResetPositionUs
)

Sets information about the timestamps of the current input stream.
abstract void
setVideoEffects(List<Effect> videoEffects)

Sets video effects to apply immediately.
abstract void
setVideoFrameMetadataListener(
    VideoFrameMetadataListener videoFrameMetadataListener
)

Sets the VideoFrameMetadataListener.

Constants

INPUT_TYPE_BITMAP

default static final int INPUT_TYPE_BITMAP = 2

Input frames come from a Bitmap.

INPUT_TYPE_SURFACE

default static final int INPUT_TYPE_SURFACE = 1

Input frames come from a surface.

Public methods

clearOutputSurfaceInfo

abstract void clearOutputSurfaceInfo()

Clears the set output surface info.

enableMayRenderStartOfStream

abstract void enableMayRenderStartOfStream()

Enables this video sink to render the start of the stream to its output surface even if the renderer is not started yet.

This is used to update the value of mayRenderStartOfStream passed to onRendererEnabled.

flush

abstract void flush(boolean resetPosition)

Flushes the video sink.

After calling this method, any frames stored inside the video sink are discarded.

Parameters
boolean resetPosition

Whether to reset the current position.

getInputSurface

abstract Surface getInputSurface()

Returns the input Surface where the video sink consumes input frames from.

Must be called after the sink is initialized.

handleInputBitmap

abstract boolean handleInputBitmap(
    Bitmap inputBitmap,
    TimestampIterator timestampIterator
)

Handles an input Bitmap.

Must be called after the corresponding stream is signalled.

Parameters
Bitmap inputBitmap

The Bitmap to queue to the video sink.
TimestampIterator timestampIterator

The times within the current stream that the bitmap should be shown at. The timestamps should be monotonically increasing.
Returns
boolean

Whether the bitmap was queued successfully. If false, the caller can try again later.

handleInputFrame

abstract boolean handleInputFrame(
    long framePresentationTimeUs,
    boolean isLastFrame,
    long positionUs,
    long elapsedRealtimeUs,
    VideoSink.VideoFrameHandler videoFrameHandler
)

Handles a video input frame.

Must be called after the corresponding stream is signalled.

Parameters
long framePresentationTimeUs

The frame's presentation time, in microseconds.
boolean isLastFrame

Whether this is the last frame of the video stream.
long positionUs

The current playback position, in microseconds.
long elapsedRealtimeUs

elapsedRealtime in microseconds, taken approximately at the time the playback position was positionUs.
VideoSink.VideoFrameHandler videoFrameHandler

The VideoFrameHandler used to handle the input frame.
Returns
boolean

Whether the frame was handled successfully. If false, the caller can try again later.
Throws
androidx.media3.exoplayer.video.VideoSink.VideoSinkException

initialize

abstract void initialize(Format sourceFormat)

Initializes the video sink.

Parameters
Format sourceFormat

The format of the first input video or image.
Throws
androidx.media3.exoplayer.video.VideoSink.VideoSinkException

If initializing the sink failed.

isEnded

abstract boolean isEnded()

Returns whether all the data has been rendered to the output surface.

isInitialized

abstract boolean isInitialized()

Returns whether the video sink is initialized.

isReady

abstract boolean isReady(boolean rendererOtherwiseReady)

Returns whether the video sink is able to immediately render media to its output surface from the current position.

The renderer should be ready if and only if the video sink is ready.

Parameters
boolean rendererOtherwiseReady

Whether the renderer is ready except for the video sink.

join

abstract void join(boolean renderNextFrameImmediately)

Joins the video sink to a new stream.

The sink will mask isReady as true for a short time to avoid interrupting an ongoing playback, even if the first frame hasn't yet been rendered to the output surface.

Parameters
boolean renderNextFrameImmediately

Whether the next frame should be rendered as soon as possible or only at its preferred scheduled release time.

onInputStreamChanged

abstract void onInputStreamChanged(@VideoSink.InputType int inputType, Format format)

Informs the video sink that a new input stream will be queued.

Must be called after the sink is initialized.

Parameters
@VideoSink.InputType int inputType

The InputType of the stream.
Format format

The Format of the stream.

onRendererDisabled

abstract void onRendererDisabled()

Called when the Renderer currently feeding this sink is disabled.

onRendererEnabled

abstract void onRendererEnabled(boolean mayRenderStartOfStream)

Called when the Renderer currently feeding this sink is enabled.

onRendererStarted

abstract void onRendererStarted()

Called when the Renderer currently feeding this sink is started.

onRendererStopped

abstract void onRendererStopped()

Called when the Renderer currently feeding this sink is stopped.

release

abstract void release()

Releases the sink.

render

abstract void render(long positionUs, long elapsedRealtimeUs)

Incrementally renders processed video frames to the output surface.

Parameters
long positionUs

The current playback position, in microseconds.
long elapsedRealtimeUs

elapsedRealtime in microseconds, taken approximately at the time the playback position was positionUs.
Throws
androidx.media3.exoplayer.video.VideoSink.VideoSinkException

If an error occurs during rendering.

setChangeFrameRateStrategy

abstract void setChangeFrameRateStrategy(
    @C.VideoChangeFrameRateStrategy int changeFrameRateStrategy
)

Changes the C.VideoChangeFrameRateStrategy used when calling setFrameRate.

The default value is VIDEO_CHANGE_FRAME_RATE_STRATEGY_ONLY_IF_SEAMLESS.

setListener

abstract void setListener(VideoSink.Listener listener, Executor executor)

Sets a Listener on this sink. Callbacks are triggered on the supplied Executor.

Parameters
VideoSink.Listener listener

The Listener.
Executor executor

The Executor to dispatch the callbacks.

setOutputSurfaceInfo

abstract void setOutputSurfaceInfo(Surface outputSurface, Size outputResolution)

Sets the output surface info.

setPendingVideoEffects

abstract void setPendingVideoEffects(List<Effect> videoEffects)

Sets video effects to apply after the next stream change.

setPlaybackSpeed

abstract void setPlaybackSpeed(@FloatRange(from = 0, fromInclusive = false) float speed)

Sets the playback speed.

setStreamTimestampInfo

abstract void setStreamTimestampInfo(
    long streamStartPositionUs,
    long streamOffsetUs,
    long bufferTimestampAdjustmentUs,
    long lastResetPositionUs
)

Sets information about the timestamps of the current input stream.

Parameters
long streamStartPositionUs

The start position of the buffer presentation timestamps of the current stream, in microseconds.
long streamOffsetUs

The offset that is added to the buffer presentation timestamps by the player, in microseconds.
long bufferTimestampAdjustmentUs

The timestamp adjustment to add to the buffer presentation timestamps to convert them to frame presentation timestamps, in microseconds.
long lastResetPositionUs

The renderer last reset position, in microseconds.

setVideoEffects

abstract void setVideoEffects(List<Effect> videoEffects)

Sets video effects to apply immediately.

setVideoFrameMetadataListener

abstract void setVideoFrameMetadataListener(
    VideoFrameMetadataListener videoFrameMetadataListener
)

Sets the VideoFrameMetadataListener.