The Ubuntu Continuous Integration engine runs all the tests against various Ubuntu images for various releases.

This project provides a central place for utilities common to many use cases so we don't have to re-implement them for multiple projects.

It builds on techniques used for many years in lp:bzr, lp:testtools, lp:subunit, lp:selenium-simple-test and of course python unittest itself.

Test Loaders
============

The provided test loader standardize on test IDs and address current and known issues with unittest discover.

It can be used to load arbitrary tests from an arbitrary tree, relying on simple hooks defined in python packages.

Examples include lp:selenium-simple-test scripts, shell scripts as well as python tests organized in unusual ways. It also allows mixing different kinds of tests in the same tree.

The emphasis is on allowing the user to fully customize test loading.

Test selection
==============

While a test suite is most commonly run in full, there are times when only a subset of a suite needs to run:
 * a developer may want to focus on a very reduced set when fixing or developing a piece of code,
 * a CI engine may want to run only hardware specific tests or any specific configuration (network, file systems, OS).

Using regular expressions to select or exclude tests has been addressing all these needs in a surprisingly easy way.

Test runners
============

See test results ;)

Test runners generally mix test loading, test selection and test reults production.

uci-tests aim is not to provide a one-size fits all test runner but instead allow people to use their preferred one or define their own by offering customizable test loaders, test filters and test results.

''uci-run-tests'' is a simple test runner that demonstrate how to use the various pieces. It may or may not address your needs ;)

Test Results
============

The test results provided here address the Ubuntu Continuous Integration engine needs.

This is mainly a simple text result displaying the test IDs and their timings and is good enough for interactive use or to provide feedback during remote execution.

subunit is used when collecting failure related data is needed, this project focus on *testing* the integration with subunit.

Test fixtures
=============

A common pattern in TestCase is to add fixtures in the base class so they are accessible to all tests. This pollutes the base classes with methods or attributes not used by most of the tests.

Fixtures are better used via composition. But requiring an additional '.fixtures.feature_x.attribute_y' make them harder to use.

We use a middle ground here by just joining a simpler (and a bit surprising at first) syntax and using python's ability to handle arbitrary attributes.

Apart from that modest invasise use of test objects namespace, the only constraint is that the test object supports the 'addCleanup' and the most usual assertX methods. Since 'addCleanup' is provided by unittest.TestCase itself, this is a light constraint, most python test frameworks inheriting from it.

The benefit is that these fixtures can be used on any TestCase class without requiring inheritance which can then be used for other needs without interferences.