Ubuntu on ARM Test Plan
Overview
Purpose
The purpose of this document is to define a test plan for Ubuntu Desktop on ARM processors. This test plan is based on Blueprint specifications, Ubuntu test procedures, and wiki pages. The objective is to identify target areas for testing, and guide the process of test design and creation. The test plan, at this time, is not to be used as a criteria for limiting or blocking release.
The ARM port is based on the traditional i386/x86_64 Ubuntu Desktop, but adds some components unique to ARM for hardware enablement, toolchain, and a kernel specific to the system on chip (SoC). Requirement changes will be added as needed. Features and functionality will be available as the schedule dictates in the project.
Scope
This test plan covers the ARM port of Ubuntu Desktop for the Oneiric Ocelot (11.10) release.
Environment
The systems used for this test will include all ARM based SoCs supported for the 11.10 release.
Test Approach and Methodology
Tests will be designed to exercise the system as a whole, as well as focusing on individual components and applications. Manual, tests will be used, but should be automated where possible or scripted otherwise.
Installation testing
Installation testing will focus on testing the install procedures for the target devices. Installation testing should target the options available during installation, as well as smoke testing that the installation completed successfully and boots after install. Validation of the state of the system post-installation will be covered by the rest of the test procedures.
Basic Feature testing
Basic feature testing covers the areas that users are likely to see on a daily basis. Defects in basic functionality will be noticed immediately by the user. Basic feature testing is broken down into core applications shipped by default.
System Configuration and Utilities
Testing of system configuration and utilities covers non-application specific areas of the distribution. Target areas include:
- Software Installation
- Power management
- Network manager
- User theme customization
- Volume controls
- System Information
- Date and Time settings
- Update Manager
Device Testing
The following areas will be tested implicitly, or directly, by running test procedures for the devices where they are available.
- WLAN support
- SDIO support
- Bluetooth support
- USB support
- 2d/3d video support
- Audio support
- Keyboard
- Mouse/Touchpad
- Hard disks
- Physical ports
Test Scenarios
Most of the tests mentioned above are already covered in the checkbox suite. A few things are not currently able to be covered there, and will be described here instead.
Installation
- Install to SD card
- Install to USB drive
- Install to SATA drive
Bootloader
- Update the kernel package, check that the initrd and bootloader are updated and that the new kernel boots correctly. Check lsmod before and after update/reboot to see that the same list of modules exist.
- For systems where the bootloader can be updated:
- check the time, it should be reasonable and should not have been reset to the beginning of epoch after a reboot without network attached
- check the mac address for all network devices before and after reboot to ensure they are the same
Entry and Exit Criteria
Entry criteria
- The kernel for the SoC to be tested must build and boot on the target platform
- Install image should boot and installer should run
Exit criteria
- All tests in plan have been attempted, or have a documented reason for deferment
- All defects found during testing have been reported
- All critical bugs have been addressed (investigated, triaged, etc)
