UiAutomation
class UiAutomation
kotlin.Any | |
↳ | android.app.UiAutomation |
Class for interacting with the device's UI by simulation user actions and introspection of the screen content. It relies on the platform accessibility APIs to introspect the screen and to perform some actions on the remote view tree. It also allows injecting of arbitrary raw input events simulating user interaction with keyboards and touch devices. One can think of a UiAutomation as a special type of android.accessibilityservice.AccessibilityService
which does not provide hooks for the service life cycle and exposes other APIs that are useful for UI test automation.
The APIs exposed by this class are low-level to maximize flexibility when developing UI test automation tools and libraries. Generally, a UiAutomation client should be using a higher-level library or implement high-level functions. For example, performing a tap on the screen requires construction and injecting of a touch down and up events which have to be delivered to the system by a call to injectInputEvent(android.view.InputEvent,boolean)
.
The APIs exposed by this class operate across applications enabling a client to write tests that cover use cases spanning over multiple applications. For example, going to the settings application to change a setting and then interacting with another application whose behavior depends on that setting.
Summary
Nested classes | |
---|---|
abstract |
Listener for filtering accessibility events. |
abstract |
Listener for observing the |
Constants | |
---|---|
static Int |
UiAutomation suppresses accessibility services by default. |
static Int |
UiAutomation uses the accessibility subsystem by default. |
static Int |
Rotation constant: Freeze rotation to 0 degrees (natural orientation) |
static Int |
Rotation constant: Freeze rotation to 180 degrees . |
static Int |
Rotation constant: Freeze rotation to 270 degrees . |
static Int |
Rotation constant: Freeze rotation to 90 degrees . |
static Int |
Rotation constant: Freeze rotation to its current state. |
static Int |
Rotation constant: Unfreeze rotation (rotating the device changes its rotation state). |
Public methods | |
---|---|
Unit |
Adopt the permission identity of the shell UID for all permissions. |
Unit |
adoptShellPermissionIdentity(vararg permissions: String!) Adopt the permission identity of the shell UID only for the provided permissions. |
Boolean |
Clears the accessibility cache. |
Unit |
Clears the window animation rendering statistics. |
Boolean |
clearWindowContentFrameStats(windowId: Int) Clears the frame statistics for the content of a given window. |
Unit |
Drop the shell permission identity adopted by a previous call to |
AccessibilityEvent! |
executeAndWaitForEvent(command: Runnable!, filter: UiAutomation.AccessibilityEventFilter!, timeoutMillis: Long) Executes a command and waits for a specific accessibility event up to a given wait timeout. |
ParcelFileDescriptor! |
executeShellCommand(command: String!) Executes a shell command. |
Array<ParcelFileDescriptor!> |
executeShellCommandRw(command: String) Executes a shell command. |
Array<ParcelFileDescriptor!> |
executeShellCommandRwe(command: String) Executes a shell command. |
AccessibilityNodeInfo! |
Find the view that has the specified focus type. |
AccessibilityNodeInfo! |
Gets the root |
AccessibilityServiceInfo! |
Gets the an |
WindowAnimationFrameStats! |
Gets the window animation frame statistics. |
WindowContentFrameStats! |
getWindowContentFrameStats(windowId: Int) Gets the frame statistics for a given window. |
MutableList<AccessibilityWindowInfo!>! |
Gets the windows on the screen associated with the |
SparseArray<MutableList<AccessibilityWindowInfo!>!> |
Gets the windows on the screen of all displays. |
Unit |
grantRuntimePermission(packageName: String!, permission: String!) Grants a runtime permission to a package. |
Unit |
grantRuntimePermissionAsUser(packageName: String!, permission: String!, userHandle: UserHandle!) Grants a runtime permission to a package for a user. |
Boolean |
injectInputEvent(event: InputEvent!, sync: Boolean) A method for injecting an arbitrary input event. |
Boolean |
performGlobalAction(action: Int) Performs a global action. |
Unit |
revokeRuntimePermission(packageName: String!, permission: String!) Revokes a runtime permission from a package. |
Unit |
revokeRuntimePermissionAsUser(packageName: String!, permission: String!, userHandle: UserHandle!) Revokes a runtime permission from a package. |
Unit |
setAnimationScale(scale: Float) Sets the system settings values that control the scaling factor for animations. |
Unit |
Sets a callback for observing the stream of |
Boolean |
setRotation(rotation: Int) Sets the device rotation. |
Unit |
setRunAsMonkey(enable: Boolean) Sets whether this UiAutomation to run in a "monkey" mode. |
Unit |
Sets the |
Bitmap! |
Takes a screenshot. |
Bitmap? |
takeScreenshot(window: Window) Used to capture a screenshot of a Window. |
String |
toString() |
Unit |
waitForIdle(idleTimeoutMillis: Long, globalTimeoutMillis: Long) Waits for the accessibility event stream to become idle, which is not to have received an accessibility event within |
Constants
FLAG_DONT_SUPPRESS_ACCESSIBILITY_SERVICES
static val FLAG_DONT_SUPPRESS_ACCESSIBILITY_SERVICES: Int
UiAutomation suppresses accessibility services by default. This flag specifies that existing accessibility services should continue to run, and that new ones may start. This flag is set when obtaining the UiAutomation from Instrumentation#getUiAutomation(int)
.
Value: 1
FLAG_DONT_USE_ACCESSIBILITY
static val FLAG_DONT_USE_ACCESSIBILITY: Int
UiAutomation uses the accessibility subsystem by default. This flag provides an option to eliminate the overhead of engaging the accessibility subsystem for tests that do not need to interact with the user interface. Setting this flag disables methods that rely on accessibility. This flag is set when obtaining the UiAutomation from Instrumentation#getUiAutomation(int)
.
Value: 2
ROTATION_FREEZE_0
static val ROTATION_FREEZE_0: Int
Rotation constant: Freeze rotation to 0 degrees (natural orientation)
Value: 0
ROTATION_FREEZE_180
static val ROTATION_FREEZE_180: Int
Rotation constant: Freeze rotation to 180 degrees .
Value: 2
ROTATION_FREEZE_270
static val ROTATION_FREEZE_270: Int
Rotation constant: Freeze rotation to 270 degrees .
Value: 3
ROTATION_FREEZE_90
static val ROTATION_FREEZE_90: Int
Rotation constant: Freeze rotation to 90 degrees .
Value: 1
ROTATION_FREEZE_CURRENT
static val ROTATION_FREEZE_CURRENT: Int
Rotation constant: Freeze rotation to its current state.
Value: -1
ROTATION_UNFREEZE
static val ROTATION_UNFREEZE: Int
Rotation constant: Unfreeze rotation (rotating the device changes its rotation state).
Value: -2
Public methods
adoptShellPermissionIdentity
fun adoptShellPermissionIdentity(): Unit
Adopt the permission identity of the shell UID for all permissions. This allows you to call APIs protected permissions which normal apps cannot hold but are granted to the shell UID. If you already adopted all shell permissions by calling this method or adoptShellPermissionIdentity(java.lang.String...)
a subsequent call will replace any previous adoption. Note that your permission state becomes that of the shell UID and it is not a combination of your and the shell UID permissions.
Note: Calling this method adopts all shell permissions and overrides any subset of adopted permissions via adoptShellPermissionIdentity(java.lang.String...)
.
adoptShellPermissionIdentity
fun adoptShellPermissionIdentity(vararg permissions: String!): Unit
Adopt the permission identity of the shell UID only for the provided permissions. This allows you to call APIs protected permissions which normal apps cannot hold but are granted to the shell UID. If you already adopted shell permissions by calling this method, or adoptShellPermissionIdentity()
a subsequent call will replace any previous adoption.
Note: This method behave differently from adoptShellPermissionIdentity()
. Only the listed permissions will use the shell identity and other permissions will still check against the original UID
Parameters | |
---|---|
permissions |
String!: The permissions to adopt or null to adopt all. |
clearCache
fun clearCache(): Boolean
Clears the accessibility cache.
Return | |
---|---|
Boolean |
true if the cache was cleared |
clearWindowAnimationFrameStats
funclearWindowAnimationFrameStats(): Unit
Deprecated: animation-frames are no-longer used. Use Shared FrameTimeline jank metrics instead.
Clears the window animation rendering statistics. These statistics contain information about the most recently rendered window animation frames, i.e. for window transition animations.
clearWindowContentFrameStats
fun clearWindowContentFrameStats(windowId: Int): Boolean
Clears the frame statistics for the content of a given window. These statistics contain information about the most recently rendered content frames.
Parameters | |
---|---|
windowId |
Int: The window id. |
Return | |
---|---|
Boolean |
Whether the window is present and its frame statistics were cleared. |
Exceptions | |
---|---|
java.lang.IllegalStateException |
If the connection to the accessibility subsystem is not established. |
dropShellPermissionIdentity
fun dropShellPermissionIdentity(): Unit
Drop the shell permission identity adopted by a previous call to adoptShellPermissionIdentity()
. If you did not adopt the shell permission identity this method would be a no-op.
See Also
executeAndWaitForEvent
fun executeAndWaitForEvent(
command: Runnable!,
filter: UiAutomation.AccessibilityEventFilter!,
timeoutMillis: Long
): AccessibilityEvent!
Executes a command and waits for a specific accessibility event up to a given wait timeout. To detect a sequence of events one can implement a filter that keeps track of seen events of the expected sequence and returns true after the last event of that sequence is received.
Note: It is caller's responsibility to recycle the returned event.
Parameters | |
---|---|
command |
Runnable!: The command to execute. |
filter |
UiAutomation.AccessibilityEventFilter!: Filter that recognizes the expected event. |
timeoutMillis |
Long: The wait timeout in milliseconds. |
Exceptions | |
---|---|
java.util.concurrent.TimeoutException |
If the expected event is not received within the timeout. |
java.lang.IllegalStateException |
If the connection to the accessibility subsystem is not established. |
executeShellCommand
fun executeShellCommand(command: String!): ParcelFileDescriptor!
Executes a shell command. This method returns a file descriptor that points to the standard output stream. The command execution is similar to running "adb shell " from a host connected to the device.
Note: It is your responsibility to close the returned file descriptor once you are done reading.
Parameters | |
---|---|
command |
String!: The command to execute. |
Return | |
---|---|
ParcelFileDescriptor! |
A file descriptor to the standard output stream. |
See Also
executeShellCommandRw
fun executeShellCommandRw(command: String): Array<ParcelFileDescriptor!>
Executes a shell command. This method returns two file descriptors, one that points to the standard output stream (element at index 0), and one that points to the standard input stream (element at index 1). The command execution is similar to running "adb shell " from a host connected to the device.
Note: It is your responsibility to close the returned file descriptors once you are done reading/writing.
Parameters | |
---|---|
command |
String: The command to execute. This value cannot be null . |
Return | |
---|---|
Array<ParcelFileDescriptor!> |
File descriptors (out, in) to the standard output/input streams. This value cannot be null . |
executeShellCommandRwe
fun executeShellCommandRwe(command: String): Array<ParcelFileDescriptor!>
Executes a shell command. This method returns three file descriptors, one that points to the standard output stream (element at index 0), one that points to the standard input stream (element at index 1), and one points to standard error stream (element at index 2). The command execution is similar to running "adb shell " from a host connected to the device.
Note: It is your responsibility to close the returned file descriptors once you are done reading/writing.
Parameters | |
---|---|
command |
String: The command to execute. This value cannot be null . |
Return | |
---|---|
Array<ParcelFileDescriptor!> |
File descriptors (out, in, err) to the standard output/input/error streams. This value cannot be null . |
findFocus
fun findFocus(focus: Int): AccessibilityNodeInfo!
Find the view that has the specified focus type. The search is performed across all windows.
Note: In order to access the windows you have to opt-in to retrieve the interactive windows by setting the AccessibilityServiceInfo#FLAG_RETRIEVE_INTERACTIVE_WINDOWS
flag. Otherwise, the search will be performed only in the active window.
Parameters | |
---|---|
focus |
Int: The focus to find. One of AccessibilityNodeInfo#FOCUS_INPUT or AccessibilityNodeInfo#FOCUS_ACCESSIBILITY . |
Return | |
---|---|
AccessibilityNodeInfo! |
The node info of the focused view or null. |
Exceptions | |
---|---|
java.lang.IllegalStateException |
If the connection to the accessibility subsystem is not established. |
getRootInActiveWindow
fun getRootInActiveWindow(): AccessibilityNodeInfo!
Gets the root AccessibilityNodeInfo
in the active window.
Return | |
---|---|
AccessibilityNodeInfo! |
The root info. |
Exceptions | |
---|---|
java.lang.IllegalStateException |
If the connection to the accessibility subsystem is not established. |
getServiceInfo
fun getServiceInfo(): AccessibilityServiceInfo!
Gets the an AccessibilityServiceInfo
describing this UiAutomation. This method is useful if one wants to change some of the dynamically configurable properties at runtime.
Return | |
---|---|
AccessibilityServiceInfo! |
The accessibility service info. |
Exceptions | |
---|---|
java.lang.IllegalStateException |
If the connection to the accessibility subsystem is not established. |
getWindowAnimationFrameStats
fungetWindowAnimationFrameStats(): WindowAnimationFrameStats!
Deprecated: animation-frames are no-longer used.
Gets the window animation frame statistics. These statistics contain information about the most recently rendered window animation frames, i.e. for window transition animations.
A typical usage requires clearing the window animation frame statistics via clearWindowAnimationFrameStats()
followed by an interaction that causes a window transition which uses a window animation and finally getting the window animation frame statistics by calling this method.
// Start with a clean slate. uiAutimation.clearWindowAnimationFrameStats(); // Do stuff to trigger a window transition. // Get the frame statistics. WindowAnimationFrameStats stats = uiAutomation.getWindowAnimationFrameStats();
Return | |
---|---|
WindowAnimationFrameStats! |
The window animation frame statistics. |
getWindowContentFrameStats
fun getWindowContentFrameStats(windowId: Int): WindowContentFrameStats!
Gets the frame statistics for a given window. These statistics contain information about the most recently rendered content frames.
A typical usage requires clearing the window frame statistics via clearWindowContentFrameStats(int)
followed by an interaction with the UI and finally getting the window frame statistics via calling this method.
// Assume we have at least one window. final int windowId = getWindows().get(0).getId(); // Start with a clean slate. uiAutimation.clearWindowContentFrameStats(windowId); // Do stuff with the UI. // Get the frame statistics. WindowContentFrameStats stats = uiAutomation.getWindowContentFrameStats(windowId);
Parameters | |
---|---|
windowId |
Int: The window id. |
Return | |
---|---|
WindowContentFrameStats! |
The window frame statistics, or null if the window is not present. |
Exceptions | |
---|---|
java.lang.IllegalStateException |
If the connection to the accessibility subsystem is not established. |
getWindows
fun getWindows(): MutableList<AccessibilityWindowInfo!>!
Gets the windows on the screen associated with the UiAutomation
context (usually the default display).
getWindowsOnAllDisplays
fun getWindowsOnAllDisplays(): SparseArray<MutableList<AccessibilityWindowInfo!>!>
Gets the windows on the screen of all displays. This method returns only the windows that a sighted user can interact with, as opposed to all windows. For example, if there is a modal dialog shown and the user cannot touch anything behind it, then only the modal window will be reported (assuming it is the top one). For convenience the returned windows are ordered in a descending layer order, which is the windows that are higher in the Z-order are reported first.
Note: In order to access the windows you have to opt-in to retrieve the interactive windows by setting the AccessibilityServiceInfo#FLAG_RETRIEVE_INTERACTIVE_WINDOWS
flag.
Return | |
---|---|
SparseArray<MutableList<AccessibilityWindowInfo!>!> |
The windows of all displays if there are windows and the service is can retrieve them, otherwise an empty list. The key of SparseArray is display ID. This value cannot be null . |
Exceptions | |
---|---|
java.lang.IllegalStateException |
If the connection to the accessibility subsystem is not established. |
grantRuntimePermission
fun grantRuntimePermission(
packageName: String!,
permission: String!
): Unit
Grants a runtime permission to a package.
Parameters | |
---|---|
packageName |
String!: The package to which to grant. |
permission |
String!: The permission to grant. |
Exceptions | |
---|---|
java.lang.SecurityException |
if unable to grant the permission. |
grantRuntimePermissionAsUser
fun grantRuntimePermissionAsUser(
packageName: String!,
permission: String!,
userHandle: UserHandle!
): Unit
Grants a runtime permission to a package for a user.
Parameters | |
---|---|
packageName |
String!: The package to which to grant. |
permission |
String!: The permission to grant. |
Exceptions | |
---|---|
java.lang.SecurityException |
if unable to grant the permission. |
injectInputEvent
fun injectInputEvent(
event: InputEvent!,
sync: Boolean
): Boolean
A method for injecting an arbitrary input event. This method waits for all window container animations and surface operations to complete.
Note: It is caller's responsibility to recycle the event.
Parameters | |
---|---|
event |
InputEvent!: The event to inject. |
sync |
Boolean: Whether to inject the event synchronously. |
Return | |
---|---|
Boolean |
Whether event injection succeeded. |
performGlobalAction
fun performGlobalAction(action: Int): Boolean
Performs a global action. Such an action can be performed at any moment regardless of the current application or user location in that application. For example going back, going home, opening recents, etc.
Parameters | |
---|---|
action |
Int: The action to perform. |
Return | |
---|---|
Boolean |
Whether the action was successfully performed. |
Exceptions | |
---|---|
java.lang.IllegalStateException |
If the connection to the accessibility subsystem is not established. |
revokeRuntimePermission
fun revokeRuntimePermission(
packageName: String!,
permission: String!
): Unit
Revokes a runtime permission from a package.
Parameters | |
---|---|
packageName |
String!: The package to which to grant. |
permission |
String!: The permission to grant. |
Exceptions | |
---|---|
java.lang.SecurityException |
if unable to revoke the permission. |
revokeRuntimePermissionAsUser
fun revokeRuntimePermissionAsUser(
packageName: String!,
permission: String!,
userHandle: UserHandle!
): Unit
Revokes a runtime permission from a package.
Parameters | |
---|---|
packageName |
String!: The package to which to grant. |
permission |
String!: The permission to grant. |
Exceptions | |
---|---|
java.lang.SecurityException |
if unable to revoke the permission. |
setAnimationScale
fun setAnimationScale(scale: Float): Unit
Sets the system settings values that control the scaling factor for animations. The scale controls the animation playback speed for animations that respect these settings. Animations that do not respect the settings values will not be affected by this function. A lower scale value results in a faster speed. A value of 0
disables animations entirely. When animations are disabled services receive window change events more quickly which can reduce the potential by confusion by reducing the time during which windows are in transition.
Parameters | |
---|---|
scale |
Float: The scaling factor for all animations. |
See Also
android.view.accessibility.AccessibilityEvent#TYPE_WINDOWS_CHANGED
android.view.accessibility.AccessibilityEvent#TYPE_WINDOW_STATE_CHANGED
android.provider.Settings.Global#WINDOW_ANIMATION_SCALE
android.provider.Settings.Global#TRANSITION_ANIMATION_SCALE
android.provider.Settings.Global#ANIMATOR_DURATION_SCALE
setOnAccessibilityEventListener
fun setOnAccessibilityEventListener(listener: UiAutomation.OnAccessibilityEventListener!): Unit
Sets a callback for observing the stream of AccessibilityEvent
s. The callbacks are delivered on the main application thread.
Parameters | |
---|---|
listener |
UiAutomation.OnAccessibilityEventListener!: The callback. |
Exceptions | |
---|---|
java.lang.IllegalStateException |
If the connection to the accessibility subsystem is not established. |
setRotation
fun setRotation(rotation: Int): Boolean
Sets the device rotation. A client can freeze the rotation in desired state or freeze the rotation to its current state or unfreeze the rotation (rotating the device changes its rotation state).
Parameters | |
---|---|
rotation |
Int: The desired rotation. |
Return | |
---|---|
Boolean |
Whether the rotation was set successfully. |
setRunAsMonkey
fun setRunAsMonkey(enable: Boolean): Unit
Sets whether this UiAutomation to run in a "monkey" mode. Applications can query whether they are executed in a "monkey" mode, i.e. run by a test framework, and avoid doing potentially undesirable actions such as calling 911 or posting on public forums etc.
Parameters | |
---|---|
enable |
Boolean: whether to run in a "monkey" mode or not. Default is not. |
setServiceInfo
fun setServiceInfo(info: AccessibilityServiceInfo!): Unit
Sets the AccessibilityServiceInfo
that describes how this UiAutomation will be handled by the platform accessibility layer.
Parameters | |
---|---|
info |
AccessibilityServiceInfo!: The info. |
Exceptions | |
---|---|
java.lang.IllegalStateException |
If the connection to the accessibility subsystem is not established. |
takeScreenshot
fun takeScreenshot(): Bitmap!
Takes a screenshot.
Return | |
---|---|
Bitmap! |
The screenshot bitmap on success, null otherwise. |
takeScreenshot
fun takeScreenshot(window: Window): Bitmap?
Used to capture a screenshot of a Window. This can return null in the following cases: 1. Window content hasn't been layed out. 2. Window doesn't have a valid SurfaceControl 3. An error occurred in SurfaceFlinger when trying to take the screenshot.
Parameters | |
---|---|
window |
Window: Window to take a screenshot of This value cannot be null . |
Return | |
---|---|
Bitmap? |
The screenshot bitmap on success, null otherwise. |
toString
fun toString(): String
Return | |
---|---|
String |
a string representation of the object. |
waitForIdle
fun waitForIdle(
idleTimeoutMillis: Long,
globalTimeoutMillis: Long
): Unit
Waits for the accessibility event stream to become idle, which is not to have received an accessibility event within idleTimeoutMillis
. The total time spent to wait for an idle accessibility event stream is bounded by the globalTimeoutMillis
.
Parameters | |
---|---|
idleTimeoutMillis |
Long: The timeout in milliseconds between two events to consider the device idle. |
globalTimeoutMillis |
Long: The maximal global timeout in milliseconds in which to wait for an idle state. |
Exceptions | |
---|---|
java.util.concurrent.TimeoutException |
If no idle state was detected within globalTimeoutMillis. |
java.lang.IllegalStateException |
If the connection to the accessibility subsystem is not established. |