Before you Begin
You should ensure that you have a development enviroment setup on launchpad and you can check out and check in code using bzr. If you need help at any point, please ask on freenode @ #ubuntu-testing.
Getting the tests for checkbox
All of the manual application tests for checkbox are in a branch in Launchpad. Get it with:
bzr branch lp:~nskaggs/checkbox/checkbox-app-testing-qt
Inside the branch you will see a jobs directory. To create a job, you need to create a new file in this directory in the proper format.
How Checkbox Tests Work
Have a look at these two wiki pages, they will give you an overview of checkbox and how the test work:
https://wiki.ubuntu.com/Testing/Automation/Checkbox
https://wiki.ubuntu.com/Testing/Automation/Checkbox/Walkthrough
Note, as of checkbox-qt the above pages are in need of an update to show the new test format; see the example tests in the jobs folder inside the branch itself.
Writing a Checkbox Test
It's important you follow the format shown in the checkbox walk-through page above. In addition, there is are examples in the jobs folder for you to copy. You can duplicate and change one of the tests to help you get started. Let's walk-through doing a simple test.
- Copy a test file *.txt.in file in the jobs subfolder.
- Edit the steps, the requires, the test name, the depends and the description.
- Add an entry for your new file in local.txt.in. You can follow the format as show in the file, updating to reference your filename and description.
- Run the checkbox application and verify tests show up in the menu and work properly
Making sure your tests work
First you need to change the bin file to set the variables to point to your local directory. The /bin/checkbox-app-testing file needs modified as follows:
Change
export CHECKBOX_SHARE=${CHECKBOX_SHARE:-/usr/share/checkbox-app-testing}to
export CHECKBOX_SHARE=${CHECKBOX_SHARE:-.}Change
export CHECKBOX_APP_TESTING_SHARE=${CHECKBOX_APP_TESTING_SHARE:-/usr/share/checkbox-app-testing}to
export CHECKBOX_APP_TESTING_SHARE=${CHECKBOX_APP_TESTING_SHARE:-.}Now, to 'test' your new test files, cd to the root directory of the branch and execute the following:
./bin/checkbox-app-testing
This executes checkbox and you should see your new tests appear in the menu.
Contributing Your Tests
Before you contribute your branch, please ensure you have:
- updated the jobs/local.txt.in file -- this serves up the menu of tests inside checkbox
- created a new *.txt.in file for your test
- ran and execute your tests successfully using ./bin/checkbox-app-testing
To contribute your accomplishment, cd to the root of the branch and,
Add your new files:
bzr add .
Commit your changes:
bzr commit -m "commit message"
Finally push your changes to Launchpad:
bzr push lp:~your-lp-username/checkbox-app-testing-qt/your-branch-name
Now go to the branch in Launchpad and propose it for merging into `lp:~nskaggs/checkbox/checkbox-app-testing-qt. You can do so from this page: https://code.launchpad.net/~nskaggs/checkbox/checkbox-app-testing-qt/+register-merge
