Unified Testing Criteria

(1)

1

This is the Unified Testing Criteria (UTC) against which Android applications can be tested.

It follows the same format as the UTC for Java ME applications, and has been compiled by members of the Unified Testing Initiative (UTI).

The UTC will be updated on an ongoing basis as a result of input from the community and changing platform requirements. UTI welcomes input via the UTI blog at www.unifiedtestinginitiative.org/blog

Please note that whereas applications that pass the Java ME UTC can then gain the Java Verified Signature through UTI’s Java Verified programme, there is no known testing and validation programme for Android apps. However, if your Android apps pass the relevant tests outlined within this UTC, you can be confident that they will have reached a high quality standard.

The tests should be performed on a device to which a factory reset has been applied prior to the installation of the application to be tested. This will ensure that there is a known base with only pre-installed applications and any errors will be attributable to the application under test.

It is not within the scope of these criteria to be able to test the performance of the application on devices with multiple applications installed.

Unified Testing Criteria

for Android applications version 1.0: March 2011

(2)

2 Test definitions

1 Install and Launch ... 4

1.1 OTA install ... 4

1.2 Long Launch Time ... 5

2 Memory Use ... 6

2.1 Memory during run ... 6

3 Connectivity ... 7

3.1 Invalid web access setup ... 7

3.2 Send/receive data ... 8

3.3 Network delays and loss of connection ... 9

3.4 Network connectivity – Airplane mode ... 10

4 Event Handling ... 11

4.1 Messaging auto start ... 11

4.2 Message queuing ... 12

4.3 Timed event expiry ... 13

4.4 Timed event expiry during suspend ... 14

4.5 Timed event expiry during application exit ... 15

5 Messaging & calls ... 16

5.1 Send ... 16

5.2 Receive ... 17

5.3 Incoming call ... 18

6 External Influence ... 19

6.1 Memory card insertion ... 19

6.2 Memory card insertion and removal ... 20

6.3 Memory card screen behaviour... 21

7 User Interface ... 22

7.1 Readability ... 22

7.2 Read time ... 23

7.3 Screen repainting ... 24

7.4 Consistency ... 25

7.5 Key layout ease of use ... 26

7.6 Application speed ... 27

7.7 Error messages ... 28

7.8 Function progress ... 29

7.9 Actions while rendering ... 30

7.10 Multiple display format handling... 31

7.11 Differing screen sizes ... 32

7.12 Multiple format input handling ... 33

7.13 Accelerometer/motion sensor responses ... 34

7.14 Spelling errors ... 35

7.15 Technical text errors ... 36

8 Language... 37

8.1 Correct operation ... 37

8.2 Manual selection ... 38

8.3 Supported formats ... 39

8.4 International characters ... 40

9 Performance ... 41

9.1 Suspend/resume from main menu ... 41

9.2 Suspend while executing... 42

9.3 Resume ... 43

9.4 Influence on terminal system features ... 44

9.5 Resource sharing - database ... 46

10 Media ... 47

10.1 Application mute option ... 47

(3)

3

10.2 Settings statuses understandable ... 48

10.3 Settings do not impair application ... 49

10.4 Settings combinations ... 50

10.5 Saving settings... 51

10.6 Specific functions ... 52

11 Menu ... 53

11.1 Help and about ... 53

11.2 Valid actions ... 55

12 Functionality ... 56

12.1 Functionality sanity check... 56

12.2 Application hidden features ... 57

13 Keys ... 58

13.1 Scrolling in menus ... 58

13.2 Selection key ... 59

13.3 Text field scrolling ... 60

13.4 Pause ... 61

13.5 Simultaneous key presses ... 62

13.6 Multi key presses ... 63

14 Device Specific Tests ... 64

14.1 Device close ... 64

14.2 Device open ... 65

15 Stability ... 66

15.1 Application stability ... 66

15.2 Application behaviour after forced close ... 67

16 Data Handling ... 68

16.1 Save game state ... 68

16.2 Data deletion ... 69

16.3 Modify record ... 70

17 Security ... 71

17.1 Encryption ... 71

17.2 Passwords ... 72

(4)

4

1 Install and Launch 1.1 OTA install Test ID

1.1 Test Title

Lifecycle – OTA install Test Description

The Application must install via OTA.

Required for:

All applications.

Testing Note

1. If errors occur at installation time, corresponding messages must be reported by the tester in the test report.

2. If the device does not display the icon, then the user must be able to start the Application using other means.

3. For carriers that will only accept the installation of Applications from Android Market, this test cannot be performed until the Application is in Android Market.

Testing Steps

1. Open the browser Application of the device;

2. Type the URL of the Application file, or navigate to it graphically;

3. Connect to the typed URL / application icon;

4. Accept the installation of the Application.

RESULT:

1. The Application installs to the device.

2. The icon for the Application can be found from the device.

Result of Test

PASS FAIL

(5)

5

1.2 Long Launch Time

Test ID

Lifecycle – Long launch time Test Description

Ensure that the Application notifies the user about a long launch time.

Required for:

Testing Note Testing Steps

1. Launch the Application.

2. Observe launch time.

RESULT:

If the Application takes longer than 5 seconds to launch, a progress bar or a message must be displayed to tell the user what is happening.

Result of Test

PASS FAIL

(6)

6 2 Memory Use

2.1 Memory during run Test ID

File System – Memory during run Test Description

Ensure that the Application correctly handles out of memory exceptions during Application execution.

Required for:

Application which writes to file system.

Not required for:

Application which does not write to file system.

1. Operate the Application in such a way so as to force the Application to write files into the file system.

2. Exit the Application. Fill the file system to its capacity or near it.

3. Operate the Application - try to explore screens and functions, which access the file system.

RESULT:

1. The Application should handle any out of memory exceptions correctly.

2. Ensure that there is a warning to the user advising about lack of memory when file is trying to be stored.

Result of Test

PASS FAIL EXCEPTION(S)

Application does not write to file system.

(7)

7 3 Connectivity

3.1 Invalid web access setup Test ID

HTTP – Invalid Network Connection Test Description

Ensure that the Application can handle the network connection being invalid / unusable.

Required for:

Application using HTTP network connection.

Not required for:

Application not using HTTP network connection.

1. Create and use an invalid APN entry - Open Settings

- Navigate to Wireless Controls – Mobile Networks – Access Point Names.

- Create a new APN with invalid details.

- Save this and select it with the radio button on the Access Point Names screen.

- Exit the Settings menu 2. Launch the Application

3. Initiate an HTTP network connection from the Application.

RESULT:

The Application should handle this situation and all exceptions correctly.

(8)

8

3.2 Send/receive data Test ID

HTTP – Send/receive data Test Description

Ensure that the Application can connect via a valid Web Access session setup and send/receive data via an HTTP network session.

Required for:

Application using HTTP network connection.

Not required for:

Application not using HTTP network connection.

1. Create a valid Web Access session setup.

3. Initiate an HTTP network connection from the Application.

4. Conduct some action which ensures a data transfer action via the Network Connection.

RESULT:

1. The Application data is properly sent/received over the network (check it for each Application screen or feature that uses data services).

Result of Test

Application does not use HTTP network connection.

(9)

9

3.3 Network delays and loss of connection Test ID

Network connectivity - Network delays and the loss of connection

Test Description

When the Application uses network capabilities, it must be able to handle network delays and any loss of connection.

Required for:

Application which uses Network Connection.

Not required for:

Application which does not use Network Connection.

2. Start the network access from the Application.

3. Put the phone in a place where there connection will be lost.

4. Observe the result.

RESULT:

The Application will work until time out and then give an error message to the user indicating there was an error with the connection.

Result of Test

The Application does not use Network Connection.

(10)

10

3.4 Network connectivity – Airplane mode Test ID

Network connectivity - Airplane mode Test Description

When the Application uses network capabilities, it must be able to handle the device being in Airplane mode

Required for:

Application which uses Network Connection.

Not required for:

Application which does not use Network Connection.

1. Set the device to Airplane mode 2. Start the Application.

3. Observe the result.

RESULT:

The Application will give a meaningful error message to indicate that the device is in Airplane mode and the application cannot run successfully.

Result of Test

The Application does not use Network Connection.

(11)

11

4 Event Handling

4.1 Messaging auto start Test ID

Messaging – Auto start and process Test Description

Ensure that the Application starts correctly on receipt of Application specific SMS

Required for:

Application which is started by Application-specific SMS.

Not required for:

Application which is not started by Application-specific SMS.

1. Ensure that the Application is not running.

2. Send an SMS message which meets the Application specification to the test handset on the correct port number.

RESULT:

1. The Application should launch correctly.

2. The Application should process the incoming message correctly.

Result of Test

The Application does not use Application-specific SMS to start.

(12)

12

4.2 Message queuing Test ID

Messaging – Message queuing Test Description

Ensure that the Application Queues Application-specific SMS messages for processing.

Required for:

Application which uses Application-specific SMS messages.

Not required for:

Application which does not use Application-specific SMS messages.

Repeat test Messaging – Auto start and process several times in quick succession.

RESULT:

The Application should queue the messages and then correctly process the queued messages.

Result of Test

The Application does not use Application-specific SMS messages.

(13)

13

4.3 Timed event expiry Test ID

Timed Event – Expiry during Application Test Descriptionrun

Ensure that the Application behaves correctly on expiry of a timed event while the Application is running.

Required for:

Application which uses timed events.

Not required for:

Application which does not use timed events.

1.Set a timed event in the Application for a specific “future” time 2. Keep the Application in an active state.

3. Allow the “future” time to pass.

RESULT:

Ensure that Application reacts correctly once the designated time has expired.

Result of Test

The Application does not use timed events.

(14)

14

4.4 Timed event expiry during suspend Test ID

Timed Event – Expiry during Application suspend

Test Description

Ensure that the Application resumes correctly from a suspended state on expiry of a timed event.

Required for:

Not required for:

1. Set a timed event in the Application for a specific “future” time 2. Suspend the Application

3. Allow the “future” time to pass.

RESULT:

Ensure that the application resumes correctly once the designated time has expired, and then ensure that the Application behaves correctly after being resumed.

Result of Test

(15)

15

4.5 Timed event expiry during application exit Test ID

Timed Event – Expiry during Application Test Descriptionexit

Ensure that the Application starts correctly from an exited state on expiry of a timed event.

Required for:

Not required for:

1. Set a timed event in the Application for a specific time 2. Exit the Application

RESULT:

1. Application starts, or user is presented with a start option once the designated time has expired.

2. Application behaves correctly when started.

Result of Test

(16)

16

5 Messaging & calls

5.1 Send Test ID

Message – Send Test Description

Ensure that the Application can send messages successfully.

Required for:

Application which sends SMS or MMS messages as part of its functions.

Not required for:

Application which does not send SMS or MMS messages as part of its functions.

1. Launch Application.

2. Send a message from the Application to another handset – if both SMS and MMS are supported, test both formats.

RESULT:

1. Notification of new message is given where enabled on the receiving handset.

2. Message is in the correct format, and for MMS contains the correct payload.

Result of Test

Application does not send messages as part of its functions.

(17)

17 5.2 Receive Test ID

Message – Receive Test Description

Ensure that the Application can receive messages successfully.

Required for:

Application which receives SMS or MMS messages as part of its functions.

Not required for:

Application which does not receive SMS or MMS messages as part of its functions.

1. Launch the Application (with sound on).

2. Compose message at another phone and send it to the test handset – if the application supports both SMS and MMS, test both formats.

RESULT:

1. Notification of new message is given where enabled on the receiving handset.

2. Message is in the correct format, and for MMS contains the correct payload.

Result of Test

Application does not receive messages as part of its functions.

(18)

18

5.3 Incoming call Test ID

Telephone call – incoming while application in use

Test Description

If the user accepts an incoming phone call while the Application is running, it should be possible to resume from the same point in the Application at the end of the call, or a logical re-starting point.

Required for:

1. While Application is running, make an incoming call to the test handset.

2. Accept the incoming call.

3. End the incoming call.

4. Return to the Application.

RESULT:

1. The incoming call dialog is shown.

2. After the call is taken and ended, the Application should resume to either the point of interruption, or a point which neither inconveniences the user nor causes data loss.

Result of Test

PASS FAIL

(19)

19

6 External Influence

6.1 Memory card insertion Test ID

Memory Card - Insertion Test Description

Ensure that the Application works correctly following a memory card insertion action when the Application is suspended and resumed.

Required for:

All applications, on device which supports removable memory cards.

Not Required for:

Device which does not support removable memory cards.

2. Suspend Application

3. Insert the memory card into the phone, and mount the card.

4. Fill the card to its capacity 5. Unmount the memory card.

6. Resume and operate the Application RESULT:

The Application continues to operate as designed based on the Application specification and is not affected by the memory card insertion or

mounting/unmounting.

Result of Test

Device does not support removable memory cards.

(20)

20

6.2 Memory card insertion and removal Test ID

Memory Card – Insertion and removal Test Description

Ensure that the Application works correctly during a memory card insertion and removal.

Required for:

All applications, on device which supports removable memory cards.

Not Required for:

Device which does not support removable memory cards.

2. Insert and remove the memory card.

3. Verify that Application works correctly.

RESULT:

1. The Application should work correctly following memory card insertion.

2. The Application should work correctly with memory card removed.

Result of Test

(21)

21

6.3 Memory card screen behaviour Test ID

Memory Card – screen behaviour Test Description

Ensure that the Application with memory card functional screens works correctly with memory card inserted and removed.

Required for:

Application which uses memory card.

Not Required for:

1. Device which does not support removable memory cards;

2. Application which does not use memory card, regardless of device support.

2. Navigate to screen where Application works with memory card.

3. Insert the memory card.

5. Remove the memory card.

RESULT:

1. The Application should work correctly following memory card insertion.

2. The Application should work correctly following memory card removal.

Result of Test

Application does not use memory card.

(22)

22 7 User Interface

7.1 Readability Test ID

7.1 Test Title Readability Test Description

Ensure that the application content is readable.

Required for:

Applications on all devices with user display.

Not Required for:

Devices without user display.

Testing Note

An exception to the requirement for naked-eye legibility may be made where the applications allows a high level graphical view of an item (eg map or web page) and the device zoon facility is used to make areas legible, providing that usability is not impaired by any limiting of the visible areas when zoomed sufficiently to be legible.

Testing Steps

All screen content must be clear (e.g. screen not crowded with content) and readable to the naked eye regardless of information displayed, or choice of font, colour scheme etc.

RESULT:

The application content should be readable.

Result of Test

Device does not have user display

(23)

23 7.2 Read time Test ID

7.2 Test Title UI – Read time Test Description

Comfortable time for content reading.

Required for:

Use the application, moving between screens.

RESULT:

Each screen must be visible for the time necessary to comfortably read all its information.

Result of Test

PASS FAIL

(24)

24

7.3 Screen repainting Test ID

UI – Screen repainting Test Description

Correct screen repainting.

Required for:

RESULT:

1. The Application screens must be correctly repainted, including cases when edit boxes and dialog boxes are dismissed.

2. There must be no blinking of moving objects and background. If the Application objects overlap they must still render correctly.

Result of Test

PASS FAIL

(25)

25 7.4 Consistency Test ID

UI - Consistency Test Description

UI consistency.

Required for:

RESULT:

The Application UI should be consistent and understandable throughout, e.g.

common series of actions, action sequences, terms, layouts, soft button definitions and sounds that are clear and understandable

Result of Test

PASS FAIL

(26)

26

7.5 Key layout ease of use Test ID

UI – Key layout ease of use Test Description

Key layout ease of use.

Required for:

All Apps.

Testing Note

1. Key layout ease of use should only be tested to the extent that it can be influenced by the application. Any limitations of the device that cannot be overcome by application design should be disregarded.

2. Where the device offers multiple input methods (e.g. hardware keypad / touch screen keypad), all the input methods available during normal use of the application should be tested.

Testing Steps

RESULT:

1. The buttons should be easy to use.

2. Button usage should be suitable for both a left-handed and right-handed person, within the physical constraints of the device design.

Result of Test

PASS FAIL

(27)

27

7.6 Application speed Test ID

UI - Application speed Test Description

The Application works in the device it was targeted for, and it is usable on the device: the speed of the Application is acceptable to the purpose of the Application and must not alter the user experience by being uncontrollable.

Required for:

Testing Note

The developer / publisher is expected to test the entire Application: for example, play through the entire game on the target handset. The tester will only conduct a representative sample test of the Application in different areas if possible, for a 15 minutes period only.

Testing Steps

1. Use the Application.

2. Observe how fast the Application is to use, and if it is too slow or too fast in its operation for good usability.

3. If the Application behavior is incontrollable due to its speed, please report such findings.

RESULT:

1. The Application is usable on the device.

2. The speed of the Application is good enough for the Application usage (i.e.

the Application frame rate or response to user input must remain adequate, and must not compromise the Application usage, or prevent the user from progressing normally).

Result of Test

PASS FAIL

(28)

28

7.7 Error messages Test ID

UI – Error messages Test Description

Error messages.

Required for:

RESULT:

1. Any error messages in the Application must be clearly understandable.

2. Error messages must clearly explain to a user the nature of the problem, and indicate what action needs to be taken (where appropriate).

Result of Test

PASS FAIL

(29)

29

7.8 Function progress Test ID

UI – Function progress Test Description

Visual indication of the function execution progress.

Required for:

RESULT:

1. Any function selected in the Application should start within 5 seconds.

2. There must be some visual indication that the function is being performed.

3. The visual indication can be anything that the user would understand as a response, e.g.

- prompting for user input;

- displaying splash screens or progress bars;

- displaying text such as “Please wait...”, etc.

Result of Test

PASS FAIL

(30)

30

7.9 Actions while rendering Test ID

UI – Actions while rendering Test Description

Application must not perform inappropriate actions while thinking or rendering

Required for:

Make user input while the Application or handset is busy processing or rendering.

RESULT:

There must be no inappropriate reaction by the Application.

Result of Test

PASS FAIL

(31)

31

7.10 Multiple display format handling Test ID

UI – Multiple Display Format Handling Test Description

Where the device and Application can display in multiple formats (e.g.

portrait / landscape, internal / external display), the elements of the application should be correctly formatted in all display environments.

Required for:

Applications that support multiple display formats, on device with multiple display formats support.

Not required for:

1. Device which does not have multiple display formats;

2. Applications that do not support multiple display formats, regardless of device support.

Testing Note

For this test, a failure would be a gross error that makes the application difficult to use, or is seriously misleading in some way. Minor errors that do not impede functionality should be passed, but the details added to the Result of Test information as a text note.

Testing Steps

Operate the Application and make use of all available display formats in multiple functions.

RESULT:

The Application should display correctly without obvious errors in all formats.

Result of Test

Device does not support multiple display formats.

Application does not support multiple display formats by design.

(32)

32

7.11 Differing screen sizes Test ID

UI – Differing screen sizes Test Description

Where the application is designed to work on multiple devices it must be able to display correctly on differing screen sizes

Required for:

Applications that support multiple devices Not required for:

Applications that target specific devices Testing Note

For this test, a failure would be the inability to display correctly on devices with different screen sizes.

Testing Steps

Operate the Application on two devices with differing screen sizes.

RESULT:

The Application should display correctly without obvious errors.

Result of Test

Application does not support multiple devices.

(33)

33

7.12 Multiple format input handling Test ID

UI – Multiple Format Input Handling Test Description

Where the device and application can accept input in multiple formats (e.g. external touchscreen / external keypad / internal touchscreen / internal keypad / QWERTY layout / 12-key layout and others), the application must work correctly with all supported input methods.

Required for:

Applications that support multiple input formats, on device with multiple input format support.

Not required for:

1. Device which does not have multiple input formats;

2. Applications that do not support multiple input formats, regardless of device support.

Testing Note

For this test, a failure would be a gross error that makes the application difficult to use, or is seriously misleading in some way. Minor errors that do not impede functionality should be passed, but the details added to the Result of Test information as a text note.

Testing Steps

Operate the Application and make use of all input methods in all functions.

RESULT:

The Application should accept input correctly in all supported formats.

Result of Test

Device does not support multiple input formats.

Application does not support multiple input formats by design.

(34)

34

7.13 Accelerometer/motion sensor responses Test ID

UI – Accelerometer / Motion Sensor Responses

Test Description

The response of the application to movement or change of alignment of the device should not impair use of the application, nor be likely to confuse the user.

Required for:

All Applications, except where both device and Application lack accelerometer / motion sensor support.

Not required for:

Application where both it and the device lack accelerometer / motion sensor support.

Testing Note

1. Testing should be performed even where either the device or the

Application (but not both) lack accelerometer support, in order to determine any unexpected reaction to the presence or absence of motion sensor responses.

2. Minor hesitations or inaccuracies are permissible. To fail, the problems must be serious enough to make it difficult to use the application.

3. The device could have adjustable orientation (accelerometer rotation) found in Settings\Sound & Display\Display Settings. Try checking the box labelled “Orientation” to switch orientation automatically when rotating phone.

Testing Steps

Operate the Application and make use of functions while changing the position, angle and alignment of the device and subjecting it to slow, rapid, and random movements.

RESULT:

The response of the application to movement or change of alignment of the device should not impair use of the application, nor be likely to confuse the user. Application should change between portrait and landscape modes without confusing errors being displayed to user.

Result of Test

Device not equipped with accelerometer / motion sensor.

Application does not make use of accelerometer / motion sensor.

(35)

35

7.14 Spelling errors Test ID

UI - Spelling errors Test Description

The Application must be free of spelling errors.

Required for:

Testing Note

1. A spelling error is defined as a strict mis-spelling of a word (no grammar or punctuation rules will be applied). Missing diacritics and accents (e.g.

acutes, cedillas, umlauts etc) will not be reported as spelling errors.

2. The tester will perform the test as specified below, but the developer must ensure that this requirement is fulfilled throughout the Application.

3. In all cases, spelling shall be acceptable if it conforms to the norm for a selected language or location.

4. For generic English, US spelling is to be regarded as the norm, but British spelling will be acceptable so long as the chosen spelling is used

throughout.

Testing Steps

1. Launch Application in target language.

2. Check text appearing in:

a) Splash/Title/Logo/Loading Screen;

b) Main Menu and all its subsidiary menus;

c) Help/Instructions Screen(s);

d) About screen;

e) Application Pause Menu and all its subsidiary menus (if present).

RESULT:

No spelling errors must be present in the defined areas.

Result of Test

PASS FAIL

(36)

36

7.15 Technical text errors Test ID

UI - Technical text errors Test Description

The text in the Application must be clear and readable.

The Application must be free of technical text display issues such as:

Text cut off / Text overlapping.

Required for:

Testing Note

1. The tester will perform the test as specified below, but the developer must ensure that this requirement is fulfilled throughout the Application.

2. All text in each target language is displayed without corruption, distortion or other display problems. Examples of failures may include:

a) Menu item text labels incorrectly aligned with cursor;

b) Button text label over-running the button area or truncated such that its meaning is not clear;

c) Text over-running or being truncated in other bounded text display areas (e.g. speech bubbles, user interface elements etc);

d) Text not wrapping at the edge of the screen resulting in words being cut off;

e) Multiple pieces of text overlapping each other, or text overlapping user interface elements (but see note 3 below);

f) Text being cut horizontally.

3. Text overlapping user interface elements may be allowable where a) The developer has stated that this is by design, and b) There is no impairment of the user experience.

Testing Steps

1. Launch Application in target language.

2. Check text appearing in:

a) Splash/Title/Logo/Loading Screen;

b) Main Menu and all its subsidiary menus;

c) Help/Instructions Screen(s) d) About screen;

e) Application Pause Menu and all its subsidiary menus (if present).

RESULT:

All text located in the specified areas is shown without technical display issues that hinder legibility.

Result of Test

PASS FAIL

(37)

37 8 Language

8.1 Correct operation Test ID

Language – Correct operation Test Description

Ensure that the Application works correctly with all appropriate languages.

Required for:

Testing Note

1. Translation table with all text used in UI should be provided to Test team before testing starts.

Testing Steps

1. If handset supports more than one language, set handset to a language not already tested in certification testing.

2. Launch the Application and perform brief testing with aim to go through all possible screens, menu, messages.

3. If Application Specification specifies that Application detects selected handset language, ensure Application displays appropriate for each supported language.

4. Exit the Application

5. Repeat steps 1 to 4 for each supported language.

RESULT:

1. All text content is rendered in the correct/expected language.

2. Ensure Application detects correct language and renders content as appropriate (if applicable).

Result of Test

PASS FAIL

(38)

38

8.2 Manual selection Test ID

Language – Manual selection Test Description

Ensure that the Application allows selection of languages.

Required for:

Application which allows selection of languages within the Application.

Not Required for:

Application that does not permit selection of languages within the Application.

1. Set Application to each language using language selection facility of the Application.

2. Perform brief testing.

RESULT:

1. User is able to select all desired languages.

2. All text content is rendered in the correct/expected language.

Result of Test

Application does not permit selection of languages.

(39)

39

8.3 Supported formats Test ID

Language – Supported formats Test Description

Ensure that the Application supports all date/time/numeric/currency features for supported languages

Required for:

Verify that date, time, time zone, week start, numeric separators and

currency, are formatted appropriately for the implemented language’s target country and supported throughout the Application.

RESULT:

All text content relating to date/time/numeric/currency fields are rendered in the correct/expected language format.

Result of Test

PASS FAIL

(40)

40

8.4 International characters Test ID

Language – International characters Test Description

Ensure that the Application accepts and displays all appropriate international characters correctly.

Required for:

Verify that all data entry fields accept and properly display all International characters.

RESULT:

All international text characters are rendered in the correct/expected language.

Result of Test

PASS FAIL

(41)

41 9 Performance

9.1 Suspend/resume from main menu Test ID

Lifecycle – Suspend / resume from main Test Descriptionmenu

Ensure that the Application suspends when at the Application main menu.

Required for:

2. Go to the main menu of the Application.

3. Suspend the Application 4. Resume the Application RESULT:

Application should suspend and resume correctly, and resume at a point that does not impair the user experience.

Result of Test

PASS FAIL

(42)

42

9.2 Suspend while executing Test ID

Lifecycle – Suspend while executing Test Description

Check for Suspend in the middle of Application execution.

Required for:

2. During Application execution, suspend the Application 3. Resume the Application

RESULT:

Application should suspend and resume correctly, and resume at a point that does not impair the user experience.

Result of Test

PASS FAIL

(43)

43 9.3 Resume Test ID

Lifecycle - Resume Test Description

Ensure that the Application resumes correctly.

Required for:

Testing Note

The objective of this test is to confirm the application’s stability when suspended and resumed multiple times from different locations in one test cycle.

Testing Steps

1. Perform Lifecycle – Suspend / resume from main menu 2. Resume the Application

4. Perform Lifecycle – Suspend while executing 5. Repeat step 2.

RESULT:

The Application resumes to the point where it was suspended, or to a point that does not impair the user experience.

Result of Test

PASS FAIL

(44)

44

9.4 Influence on terminal system features Test ID

Lifecycle - Influence on Terminal System Features

Test Description

Application must correctly handle situations where following user input, or some external event (e.g. a phone call), it is switched to the

background by the terminal. Upon returning to foreground the Application must resume its execution correctly. While in the

background the Application must not emit any audio (unless it is part of its specific purpose to do so) and all handset functions should remain intact.

While being in the background, the Application must either not affect the use of the system features or other Applications or, if the

Application does so, such behaviour must be described in the help file.

Not Required for:

Application which is not written to run as a Service.

Required for:

Application which is written to run as a Service.

Testing Note

1. When performing the test below, the Application either needs to be switched to background or foreground. The actual method used depends on the functionality of the target terminal

2. If features are found to be disabled or not able to be used and are not listed in the help file the Application must fail this test.

3. The developer should be asked whether the Application is written to run as a Service, and the answer should determine whether this test is applicable.

Testing Steps

2. Familiarize yourself with the help file.

3. Switch Application to background while the Application is running and in each of the following locations within the Application:

- During initial loading of the Application - Main Menu

- In the process of normal Application usage

- In the process of loading data from the network (where applicable) - In pause state (where applicable).

4. Try using system features and Applications of the terminal (Phone Application, Calendar, Clock, Contacts, Browser, etc). In particular try the following:

- make a voice call

- make a video call (if supported by terminal) - send an SMS message

- send an MMS message

- open a WAP and WEB page (if supported by terminal)

- start a streaming session using a WEB browser or media player.

5. Verify that terminal’s system features and Applications can still be used normally, and where this is not the case, the Application’s help file describes the situation adequately to the user. Verify also that the Application does not emit any audio (unless this is its specific purpose by design).

6. Switch the Application back to the foreground.

(45)

45 Test ID

Lifecycle - Influence on Terminal System Features

7. Verify that the Application operates normally by using it for a time period of 5 minutes.

RESULT:

1. Terminal’s system features and Applications can be used normally

2. In case the Application execution causes some changes to normal use of system features and Applications, this is adequately explained in the help file of the Application.

3. After the Application is brought back to foreground, it continues to operate normally.

Result of Test

Application is not written to run as a Service.

(46)

46

9.5 Resource sharing - database Test ID

Lifecycle - Resource Sharing – Database Test Description

Check that database resources are properly shared between Application and a competing Application.

Required for:

Applications which make use of Contacts database.

Not required for:

Applications which do not make use of Contacts database.

Testing Note

Application under test should not be the device native Contacts application.

Testing Steps

2. Suspend Application

3. Launch the device Contacts application 4. Add a new entry into contacts.

5. Remove an existing entry from contacts.

6. Resume the Application under test 7. Check Application state.

RESULT:

1. Application should continue from the previous state prior to being suspended.

2. Application should see the new entry and the deleted entry.

Result of Test

PASS FAIL

Application does not use Contacts database.