Test Automation Approach for Mobile Games

Challenges in Testing Mobile Games

• Games developed with OpenGL or ActiveX bypassing the OS level services so mobile elements aren’t identifiable
• Elements in application cannot be identified by any locator stratergies like xpath, id, classname others
•  Inspector tools fails to identify on gaming application
• Different screen resolution for Android, hence vast range of device to test on
• Game UI is very fast as compared to mobile apps, making synchronization between the scripts and actual game is smart job.
• Image comparison library like Sikuli only works for desktop application i.e. not for mobile devices
• Automation on emulators not recommended as mostly bugs for game application is dependent on real devices
• Performance across all hardware & software combinations

 Automation Approach for Mobile Games

Mobile framework integration with below API/Tool will achieve game automation:

Sikuli Library :

http://www.sikulix.com/

Automates anything you see on the screen of your desktop computer running Windows, Mac or some Linux/Unix. It uses image recognition powered by OpenCV to identify and control GUI components. This is handy in cases when there is no easy access to a GUI’s internals or the source code of the application or web page you want to act on.

Robort framework :

https://docs.oracle.com/javase/7/docs/api/java/awt/Robot.html

This class is used to generate native system input events for the purposes of test automation, self-running demos, and other applications where control of the mouse and keyboard is needed. The primary purpose of Robot is to facilitate automated testing of Java platform implementations.

Vysor:

http://www.vysor.io/

Vysor puts a fully controllable window of your Android on your desktop. An easy install Chrome app, compatible with all operating systems.

Example Scenarios and Code

//Start the appium driver and launch the app. Driver work in game automation is only to launch the app

[sourcecode language=”javascript”]
AppiumDriver driver;

DesiredCapabilities capabilities = new DesiredCapabilities();

capabilities.setCapability("appPackage", "PACKAGE_NAME");

capabilities.setCapability("appActivity", "ACTIVITY_NAME");

capabilities.setCapability("deviceName", "DEVICE_NAME");

capabilities.setCapability("platformVersion", "DEVICE_VERSION");

capabilities.setCapability("platformName", "PLATFORM_NAME");
[/sourcecode]

//make sure you set large value for COMMAND TIMEOUT because no command will be given by appium driver once app is launched

[sourcecode language=”javascript”]
capabilities.setCapability ("newCommandTimeout", 360000);

driver = new AndroidDriver<MobileElement>(new URL("http://127.0.0.1:4725/wd/hub"), capabilities);
[/sourcecode]

//Function will take screenshot of desktop using sikuli library and return buffered image

[sourcecode language=”javascript”]
Screen screen = new Screen();

public BufferedImage takeScreenshot() {

File scrFile = new File(screen.saveScreenCapture(System.getProperty("user.dir") + "/src/main/resources/", "FILE_NAME.png"));

BufferedImage bufferedImage = null;

try {

bufferedImage = ImageIO.read(scrFile);

} catch (IOException e) {

e.printStackTrace();

}

return bufferedImage;

}
[/sourcecode]

//Sikuli library will compare both the screen and return the centre of co-ordinates of match on desktop

[sourcecode language=”javascript”]
Point2D coords = getCoords(takeScreenshot(), targetImgPath);

System.out.println("———-"+ coords.getX());

System.out.println("———-"+ coords.getY());
[/sourcecode]

//Use Robot class to click on match found on destop screen i.e. will be inside mobile cast image

Robot class can be used to perform various action on mobile cast on Desktop which will be replicated in mobile by Vysor like moveMouse, mouseWheel, keyPress, drag&Drop etc

[sourcecode language=”javascript”]
if ((coords.getX() >= 0) && (coords.getY() >= 0)) {

Robot bot = new Robot();

bot.mouseMove((int) coords.getX(),(int) coords.getY());

bot.mousePress(InputEvent.BUTTON1_MASK);

bot.mouseRelease(InputEvent.BUTTON1_MASK);

100);

}
[/sourcecode]

Limitations

OpenGL or ActiveX bypassing the OS level services.

OCR is possible but not able to confirm same as no much info needed;