Quality Assurance

Working with the Xubuntu quality assurance team has three main aspects, testing, triaging and filing bugs. Like any piece of software, Xubuntu needs people to track bugs, test at static points and continually test the development version.

Please note that many of the tasks mentioned below require you to have registered a Launchpad account, it will be assumed that you have, or intend to get, one.


The Xubuntu testing infrastructure is made up of three key parts:

  • Xubuntu Testers team on Launchpad, which you should join once you’ve started helping with testing
  • The xubuntu-devel mailing list, where further information on new packages, known bugs and much more can be gathered and where discussion about critical bugs take place
  • Testing trackers (for Images and packages), with information on tests to run along with instructions how to run them

Using the trackers

While the QA trackers have all the necessary information on how to do testing, for a quick overview, here’s what you need to do after adding yourself to the Xubuntu Testers page and team:

  • File bugs when you notice them while testing
  • Report the tests to the tracker along with the list of bugs you have found


  • To start helping with our testing, download an image; download links can be found in the QA tracker, along with zsync commands. Zsync will check that the checksum matches. If you, however, are downloading we suggest that you check the image integrity against the MD5 checksum found in the tracker
  • If you are intending to test for some while, rather than using links on the tracker to grab images, there are constant links for both of the current dailies at Current build

Remember that testing doesn’t help the developers unless the tests are reported!

Testing and Bug reporting

Exploratory testing

In addition to testing the images and software with predefined testing actions (static testing), community members can take part in exploratory testing as well.

  • In essence, exploratory testing means running the development release and doing your daily tasks with the system and finally, filing bugs when you find them. This allows Xubuntu to get a much larger spread of testing than is possible with predefined tests.
  • A useful way to deal with this is dual (or multiple) boots. Where you can ensure you are able to access at all times a working version. In this way you can do as much work in the development release as is practical – all the while watching for bugs and regressions in the development release.
  • Issues which the development team are keen to address include, in addition to normally reported issues, usability bugs, missing icons, inconsistent functionality.

Development PPAs

We have 3 PPAs which we use regularly to test packages. These are:

      • Shimmer Themes for daily builds of the Shimmer Project’s projects
      • Xubuntu Staging for packages and package versions that are being prepared for inclusion in Xubuntu
      • Xubuntu daily builds for daily Git and Bzr builds for packages related to Xubuntu and/or Xfce4

Installing these three PPAs means that you will be using and testing packages that developers are currently working with, this means that regressions found by you will not be present once they are released.

At times it is useful to remove a PPA. Using ppa-purge causes apt to disable a PPA source list and then change affected packages back to the default versions.

  • Run sudo apt-get install ppa-purge
  • Run sudo ppa-purge ppa:ppaname

Reporting bugs with PPAs can be problematic, please see further information on reporting bugs with PPAs enabled below.

Static Testing

Testing the development versions of Xubuntu and packages in it helps keep up the every day quality. If you have spare hardware resources or are able to run a virtualized testing environment, you can help.

Image Testing

      • Check the devel mailing list close to the start of the cycle, where the decision as to which milestones we will participate in will be discussed.
      • While milestone testing is in progress, please watch for rebuilds. These will take place either when we have reason to rebuild, eg following a bug fix landing, or more likely when the Ubuntu Release Team have cause to do so.

Milestone Testing

      • Stand alone image testing for milestones is useful in the 2 days before the release. To see how these milestones are scheduled, refer to the Ubuntu release schedule.
      • Live Session: Boot with the image and ensure that basic application testing (open, close, saving etc.) passes.
      • Installation tests required pass. These test only the installation.

Daily Testing

      • The importance of daily image testing lies in the main in knowing that boot or installation regressions haven’t appeared. Where it appears they have, if you are able to boot with a different flavours image, this can help prove a global or local to Xubuntu issue. If unsure contact the Xubuntu team, preferrably on IRC.
      • Live Session: Boot with the image and ensure that basic application testing (open, close, saving etc.) passes.
      • Installation tests required pass. These test only the installation.

Package Testing

For releases where we are making use of the Package Tracker, calls for package testing will be made to the xubuntu-devel mailing list as required. This could be a call from QA or Developers.

      • A schedule of planned tests is mailed to the Xubuntu Devel mailing list close to the cycle start
      • The schedule is shown on the QA Trello page
      • Reminders of upcoming package test requirements are mailed to the Xubuntu devel mailing list as required
      • Specific developer testing requirement are mailed as they are available, often these packages will be those found on one of our PPAs

Filing bugs

In addition to the links mentioned below, the Ubuntu wiki has some good pages on bugs, including Helping with bugs, Standard responses, How to debug bugs and Common tasks.

After you’ve filed a bug report, be ready to cooperate with developers and triagers to give them more information if needed to help them debug and fix the issue.

Please note that feature request and development ideas do not count as bugs, and thus shouldn’t be filed as bugs.

Bug tagging

      • Bug tagging. Please ensure that while you are reporting bugs during a development cycle, that you add specific tags to any automatically generated ones. This allows us to search Launchpad for bugs affecting us during a cycle.
      • For general bugs during a cycle please use xubuntu-exp cyclename.
      • For bugs found with versions supplied by PPA please use xubuntu-exp cyclename ppa.

For example a bug found during the wily cycle in a PPA would have xubuntu-exp wily ppa as it’s tag.

Apport configuration file

At some point during the cycle, Canonical enables crash reporting with apport. It can be useful to pre-empt this and manually edit the file concerned.

      • Run pkexec mousepad /etc/apport/crashdb.conf
      • Locate the line which includes ‘problem_types’: [‘Bug’, ‘Package’]
      • Add a # to the beginning of the line #’problem_types’: [‘Bug’, ‘Package’],
      • Save the file

Normal bug reporting

Anybody using Xubuntu can file (or in other words, report) bugs, even with no experience at all. Filing bugs is important because in some cases, bugs can go unnoticed unless an end user files them.

Before submitting a bug, you should look at the existing bug reports and release notes to verify the bug hasn’t been reported already. If the bug has not already been reported, you should file a new bug report. It is sensible to read through the bug reporting guidelines before filing your first bug.

In the most situations, it is easiest to file a bug by opening a new Terminal and type ubuntu-bug package-name, where package-name is the package you want to file bug against. If you don’t know which package you should file the bug against, refer to the instructions on finding the right package. When filing the bug, it is better to have too much than too little information.

Reporting bugs with PPAs

When you encounter a bug with a package from a PPA, you’ll need to file the bug report manually. Once you’ve got the offending package name, go to http://bugs.launchpad.net/ubuntu/+source/PACKAGENAME/+filebug.

Crash logs related to the package can be found in /var/crash/. This directory requires superuser (sudo) permissions to view and extract the contents. These logs can be particularly valuable to include in your bug report, but be sure to review them as they may contain sensitive or personally identifiable material.

Be sure to follow the above general guidelines, and also add the ppa tag so it is clear to developers that this is an unsupported package.

Triaging bugs

Triaging bugs means to get bug reports in a state where they are useful for developers by making sure bug reports have useful titles, descriptions, appropriate logs and more. To get started, read the page on How to triage bugs in the Ubuntu wiki.

After you have made yourself familiar with the aspects of triaging bugs, you can start triaging Xubuntu bugs. Please note that the bug status and importance can be only be changed by members of the Ubuntu Bug Control team – in the beginning, you will need to report triaged bugs for the team in #ubuntu-bugs on Freenode. After you have demonstrated your ability to triage bugs, you will gain more responsibilities by the Bug Control team.

As you gain experience on triaging Xubuntu bugs, you may want to take a look at new bugs that mention Xubuntu. Triaging new bugs is recommended for those who are more familiar with both triaging and Xubuntu generally, since not all of the new bugs mentioning Xubuntu are actually Xubuntu bugs.

Forwarding bugs upstream

When bugs are forwarded upstream, all users of the relevant software will benefit from the debugging, triaging and patching work carried out by bug triagers and developers.

Once it has been determined that a bug is not caused by a change in Xubuntu, bugs can be forwarded to their respective upstream projects to be reviewed. For example, all appropriate Xfce bugs should be filed in the Xfce bug tracker. In addition, to track the status of the bug in Ubuntu, the bug in Launchpad should be linked with the upstream bug.

The Ubuntu wiki has an extensive page on forwarding bugs upstream along with instructions on how to report bugs in the upstream bug trackers. Detailed information on the way to add an upstream bug to Launchpad can be found on the Bugs/Watches page on the Ubuntu wiki.