Android Automation Script

This guide will explain how to write an Android automation script to perform automated testing.

Default user interface automation in Lab uses a heuristic-based approach to detect common UI elements, e.g., login, password, registration forms. It may be necessary to write a custom UI automation script to achieve better coverage of your Android application.

Occasionally the target Android application has a complex or unique workflow that requires more sophisticated automation control. Default UI automation in Lab uses a heuristic-based approach to detect common UI elements, e.g., login, password, registration forms. It may be necessary to write a custom UI automation script to achieve better coverage of your application. To achieve this, we support two types of scripts, Javascript and DSL.

Javascript

JS automation scripts allow for advanced functionality when iterating through an application’s UI, adding in the ability to create functions as needed based on the various scenarios the app may experience while running through an automated security test.

Here is a boiler-plate JS script to begin with, for example Twitter for Android:

onPermissionRequest(function (request) {
  if( request.indexOf("to access this device's location?") !== -1 ) {
    print( "lets accept this one" );
    return true;
  }
  print('unhandled request for permission: ' + request);
  return false;
});

// returns an object
function findByResourceId( id ) {
  return find({ element: { 'resource-id': id } });
}

// returns an array of objects
function waitForByResourceId( id ) {
  return waitFor({ element: { 'resource-id': id } });
}

function DoIt()
{
  try {
    // click the "Log In" button
    waitForByResourceId('com.twitter.android:id/log_in')[0].click();

    // enter our username/password ( sometimes username is pre-populated with an email, clear it first )
    waitForByResourceId('com.twitter.android:id/login_identifier')[0].setText( "ENTER_USERNAME", {clearExistingText: true} );
    findByResourceId('com.twitter.android:id/login_password').setText( "ENTER_PASSWORD", {clearExistingText: false} );
    
    // click the "Log In" button
    findByResourceId('com.twitter.android:id/login_login').click();

  } catch (e) {
    print(e);
  }
}

DoIt();

Other Useful Functions

waitFor: used for pausing the script for a certain amount of time, waiting for a specific element to load. The current max limit is 60 seconds (60000ms). If you experience an app that takes longer than that to load a specific element, please reach out to NowSecure support.

waitFor( { 
	element: { 
		'resource-id': 'com.twitter.android:id/log_in'
	},
	'maxWaitInMs': 10000, //number of milliseconds to wait for the element (limit 60000)
	'intervalInMs': 1000
});

sleep: delays the script a certain amount of time (in milliseconds). This can be useful if the element you need to interact with is already on the screen but you need to wait a certain amount of time before interacting with it. ( Example: a username/password is entered, and the app takes a second or two to validate the content before enabling the “Log In” button ).

sleep(1000); // sleep for 1 second 

log: adds an entry to the analysis log which can be viewed under the “Messages” section in the Lab Auto UI. This can be helpful for troubleshooting app navigation, or confirmation that specific actions have occurred.

log("adds this text to Messages" );  

// OR

log( {msg:"also adds this text to Messages"} );

screenshot: take a screenshot at a given point in the script. Screenshots are set up to occur at every action, however, there could be cases where you want to specify a custom screenshot. Please note: for custom names on screenshots, only A-Z characters are accepted, so don’t use spaces, numbers, or special characters, otherwise an error will be thrown.

screenshot(); // takes a screenshot of the android UI and displays it in the assessment artifacts  

// OR

screenshot("loadingBar"); // specifies to take a screenshot and name it "loadingBar"

Discovering UI Elements

Android ADB has a built in tool for dumping an XML view of the UI hierarchy. We can use this when determining how we should call a specific element. With an android device connected, run:

adb shell uiautomator dump
adb shell cat /sdcard/window_dump.xml

This will dump the current view hierarchy that appears on the device, meaning you will need to open the specific application and load the required screen before performing the uiautomator dump command. Once performed, you can search through the output to find the needed element.

Using the Twitter sign-in page as our example, we can isolate the username and password fields, viewing all property tags associated with those elements:

<node index="1" text="" resource-id="com.twitter.android:id/login_form" class="android.widget.LinearLayout" package="com.twitter.android" content-desc="" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" scrollable="false" long-clickable="false" password="false" selected="false" bounds="[0,348][1080,987]">
	<node NAF="true" index="0" text="" resource-id="com.twitter.android:id/login_identifier" class="android.widget.EditText" package="com.twitter.android" content-desc="" checkable="false" checked="false" clickable="true" enabled="true" focusable="true" focused="true" scrollable="false" long-clickable="true" password="false" selected="false" bounds="[36,384][1044,609]" />
	<node NAF="true" index="1" text="" resource-id="com.twitter.android:id/login_password" class="android.widget.EditText" package="com.twitter.android" content-desc="" checkable="false" checked="false" clickable="true" enabled="true" focusable="true" focused="false" scrollable="false" long-clickable="true" password="true" selected="false" bounds="[36,609][1044,834]" />

Here, we will notice a few properties that can be extremely helpful when referencing different properties with a JS automation script. Property values can vary greatly depending on how they app was developed.

resource-id, text, and content-desc are typically the go-to properties you want to reference. In some rare cases, some or all of these will be blank, so you will need to leverage other properties when referencing from a script.

Using the boilerplate Twitter script, we can see that we can reference the resource-id to access those fields:

waitForByResourceId('com.twitter.android:id/login_identifier')[0].setText( "ENTER_USERNAME" );
findByResourceId('com.twitter.android:id/login_password').setText( "ENTER_PASSWORD", {clearExistingText: false} );

For properties other than resource-id, we can look for those using a similar function:

waitFor( { 
	element: { 
		'text': 'Some Text',
		'content-desc': 'Some Description',
		'class': 'android.widget.EditText',
		'index': '0'
	},
	'maxWaitInMs': 10000,
	'intervalInMs': 1000
});

DSL

DSL aims to take a more simplistic user approach to scripting UI interaction, however, this greatly depends on how the UI is laid out, in terms of determining which element to interact with. Web apps that have been converted to native mobile apps using major frameworks, like Cordova, tend to have UI elements with missing text or content-desc fields preventing our interpreter from finding the correct elements, despite what you might see within the app UI while running on a device.

Here’s a boiler-plate DSL script to begin with, for example Twitter for Android:

{
  "automation_steps": [
          {
            "action": "Sleep",
            "timeInMsToSleep": 10000
          },
          {
            "action": "Click",
            "clickItemMatching": "Log In"
          },
          {
            "action": "Sleep",
            "timeInMsToSleep": 10000
          },
          {
            "action": "EnterText",
            "matchStrings":       [ "username" ],
            "textToEnter"         : "ENTER_YOUR_EMAIL_OR_USERNAME_HERE",
            "fieldType"           : "username"
          },
          {
            "action": "EnterText",
            "matchStrings":       [ "password" ],
            "textToEnter"         : "ENTER_YOUR_PASSWORD_HERE",
            "fieldType"           : "password"
          },
          {
            "action": "Sleep",
            "timeInMsToSleep": 10000
          },
          {
            "action": "Click",
            "clickItemMatching": "Log In"
          }
  ]
}
  1. Delay 10 seconds to allow the app to finish loading and for the login screen to appear

  2. We wrap the rest of our script to ensure the login UI elements are on the screen. If we don’t, UI

  3. automation will error because the elements are not present.

  4. The automator will click the “Log In” button.

  5. Have the automator type the USERNAME OR EMAIL and the PASSWORD into the appropriate fields.

  6. The automator will finally click the “Log In” button and, hopefully, login to the account associated with that Android application.