Testrepository

Download project files

0.0.19 release from the trunk series released 2014-08-23

Release information

Release notes:

CHANGES
-------

* Passing --subunit to all testr commands will now consistently output subunit
v2. Previously it would output v1 for stored streams and v2 for live
streams. (Robert Collins)

* ``run`` was outputting bad MIME types - test/plain, not text/plain.
(Robert Collins)

* Test filtering was failing under python3 and would only apply the
filters to the first test listed by discover. (Clark Boylan, #1317607)

* Tests that are enumerated but not executed will no longer reset the test
  timing data. Enumeration was incorrectly recording a 0 timestamp for
  enumerated tests. This leads to poor scheduling after an interrupted test
  run. (Robert Collins, #1322763)

* Version 0.0.18 of subunit is now a hard dependency - the v2 protocol solves
key issues in concurrency and st...

File	Description	Downloads
testrepository-0.0.19.tar.gz (md5, sig)	Python source release	369 last downloaded 63 weeks ago
Total downloads:		369

0.0.9 release from the trunk series released 2012-12-18

Release information

Release notes:

0.0.9
+++++

IMPROVEMENTS
------------

* On OSX the ``anydbm`` module by default returns an implementation that
doesn't support update(). Workaround that by falling back to a loop.
(Robert Collins, #1091500)

* ``testr --analyze-improvements`` now honours test regex filters and only
analyzes matching tests. (Robert Collins)

File	Description	Downloads
testrepository-0.0.9.tar.gz (md5, sig)	Source tarball.	3,200 last downloaded 75 weeks ago
Total downloads:		3,200

0.0.8 release from the trunk series released 2012-12-07

Release information

Release notes:

Major new release, with better help, subunit support for more commands, and helpful helpers for 'testr run' - you can select tests to run from testr itself, run tests in a loop until failure with --until-failure and peform bisection to detect test interactions using --analyze-isolation.

Changelog:

############################
testrepository release notes
############################

NEXT (In development)
+++++++++++++++++++++

0.0.8
+++++

CHANGES
-------

* As a side effect of fixing bug #597060 additional arguments passed to testr
run or testr list are only passed to the underlying test runner if they are
preceeded by '--'. (Robert Collins, #597060)

* ``testr run --failing`` will no longer run any tests at all if there are
no failing tests. (Robert Collins, #904400)

IMPROVEMENTS
------------

* ``AbstractArgument`` now handles the case where additional arguments are
   present that the argument type cannot parse, but enough have been parsed for
   it to be valid. This allows optional arguments to be in the middle of a
   grammar. (Robert Collins)

* ``cli.UI`` now passed '--' down to the argument layer for handling rather
than implicitly stripping it. (Robert Collins)

* ``DoubledashArgument`` added to allow fine grained control over the impact of
-- in command lines. (Robert Collins)

* New argument type ``ExistingPathArgument`` for use when commands want to take
the name of a file. (Robert Collins)

* ``testr`` will now show the version. (Robert Collins)

* ``testr last`` when just one test run has been run works again.
(Robert Collins)

* ``testr help command`` now shows the docstring for commands (Robert Collins)

* ``testr --help command`` or ``testr command --help`` now shows the options
for the command. (Robert Collins)

* ``testr run --analyze-isolation`` will search the current failing tests for
spurious failures caused by interactions with other tests.
(Robert Collins, #684069)

* ``testr run --until-failure`` will repeat a test run until interrupted by
ctrl-C or until a failure occurs. (Robert Collins, #680995)

* ``Repository.get_test_run`` now raises KeyError if asked for a missing or
nonexistant test run. (Robert Collins)

* Sphinx has been added to tie the documentation toghether (And it is available
on testrepository.readthedocs.org). (Robert Collins)

* ``StringArgument`` now rejects '--' - it should be handled by the use of a
  ``DoubledashArgument`` where one is expected. This is a bit awkward and does
  not permit passing '--' down to a child process, so further work may be
  needed - file a bug if this affects you. (Robert Collins)

* ``test failing --subunit`` now exits 0 unless there was a problem generating
the stream. This is consistent with the general processing model of subunit
generators. (Robert Collins)

* ``testr last`` now supports ``--subunit`` and when passed will output
  the stored subunit stream. Note that the exit code is always 0 when this
  is done (unless an exception occurs reading the stream) - subunit consumers
  should parse the subunit to determine success/failure. (Robert Collins)

* ``testr load`` now supports passing filenames to subunit streams to load.
(Robert Collins, #620386)

* ``testr run`` will now fail a test run if the test process exits non-zero.
  As a side effect of this change, if the test program closes its stdout but
  does not exit, ``testr run`` will hang (waiting for the test program to
  exit). (Robert Collins)

* ``testr run --load-list FILENAME`` will limit the tests run to the test ids
  supplied in the list file FILENAME. This is useful for manually specifying
  the tests to run, or running testr subordinate to testr (e.g. on remote
  machines). (Robert Collins, partial fix for #597060)

* ``testr run foo`` now applies foo as a regex filter against the tests
  found by doing a listing of the test runners tests. Likewise
  ``testr list-tests foo`` will apply foo as a filter against the found tests.
  This makes it easy to limit the tests that will be requested for running by
  the backend test process - simply pass one or more regex filters into testr
  run. (Robert Collins, #597060)

* Test tags are now shown in failures. Of particular interest for folk debgging
cross-test interactions will be the worker-N tags which indicate which
backend test process executed a given test. (Robert Collins)

File	Description	Downloads
testrepository-0.0.8.tar.gz (md5, sig)	Source tarball.	381 last downloaded 75 weeks ago
Total downloads:		381

0.0.6 release from the trunk series released 2012-05-08

Release information

Release notes:

Testrepository 0.0.6 adds a new command 'slowest' which reports the slowest tests seen. --parallel has had a lot of improvements (many done via upstream libraries subunit and testtools) to make diagnosing test issues in the workers better and more reliable. As usual, see the NEWS or manual for details. The run and load commands now report how long the test run took, and the change in duration and in number of tests.

Changelog:

0.0.6
+++++

CHANGES
-------

* Now relies on subunit 0.0.8 or better and testtools 0.9.15 or better.

IMPROVEMENTS
------------

* Much better handling of unicode input from subunit streams. Specifically,
we won't crash when we can't figure something out. (Francesco Banconi,
Martin Packman, #955006)

* Parallel tests now record their worker thread number as tags in tests.
This makes identifying test ordering problems much easier.
(Benji York, #974622)

* Python2.7 changed the interface for DBM, this has been worked around.
(Robert Collins, #775214, #961103)

* Subunit 0.0.7 Changes its TestResultFilter implementation, requiring the
subclass in testrepository.filter to be come more robust.
(Robert Collins)

* A horrible thinko in the testrepository test suite came to light and has been
fixed. How the tests ever ran is a mystery. (Robert Collins, #881497)

* ''failing'', ''run'' and ''load'' now both take a ''--subunit'' option, which
displays output in raw subunit format. If ''--full-results'' is passed too,
then all subunit information is displayed. (Brad Crittenden, #949950)

* Setting ''filter_tags'' in ''.testr.conf'' will cause tests tagged with those
  tags to be hidden unless the fail/error. This requires Subunit 0.0.8. If
  an older version of subunit is configured, testr will return an error.
  (Robert Collins, #914166)

* ``testr`` will drop into PDB from its command line UI if the environment
variable TESTR_PDB is set. (Robert Collins)

* Test partitioning now handles a corner case where multiple tests have a
  reported duration of 0. Previously they could all accumulate into one
  partition, now they split across partitions (the length of a partition is
  used as a tie breaker if two partitions have the same duration).
  (Robert Collins, #914359)

* The test 'test_outputs_results_to_stdout' was sensitive to changes in
testtools and has been made more generic. (Robert Collins)

File	Description	Downloads
testrepository-0.0.6.tar.gz (md5, sig)	Source tarball.	933 last downloaded 75 weeks ago
Total downloads:		933

0.0.5 release from the trunk series released 2011-02-26

Release information

Release notes:

This long overdue release:
- depends on testtools 0.9.8
- adds parallel testing support using instrumented partitioning of the tests
- adds support for listing tests
- improves variable expansion
- has better win32 support

For details see the changelog.

Changelog:

0.0.5
+++++

CHANGES
-------

* The testrepository test suite depends on testtools 0.9.8. (Robert Collins)

* If interrupted while updating the ``failing`` list, temp files are now
cleaned up - previously a carefully timed interrupt would leave the
temporary failing file in place. (Robert Collins, #531665)

* Local implementation of MatchesException has been removed in favour of the
testtools implementation. All ``self.assertRaises`` have been migrated to
this new testing interface.

* ``setup.py`` will read the version number from PKG-INFO when it is running
without a bzr tree : this makes it easier to snapshot without doing a
release. (Jonathan Lange)

* Testrepository should be more compatible with win32 environments.
(Martin [gz])

* ``testr init-repo`` now has a ``--force-init`` option which when provided
will cause a repository to be created just-in-time. (Jonathan Lange)

* ``testr load`` and ``testr run`` now have a flag ``--partial``. When set
  this will cause existing failures to be preserved. When not set, doing a
  load will reset existing failures. The ``testr run`` flag ``--failing``
  implicitly sets ``--partial`` (so that an interrupted incremental test run
  does not incorrectly discard a failure record). The ``--partial`` flag exists
  so that deleted or renamed tests do not persist forever in the database.
  (Robert Collins)

* ``testr load`` now loads all input streams in parallel. This has no impact
on the CLI as yet, but permits API users to load from parallel processes.
(Robert Collins)

* ``testr list-tests`` is a new command that will list the tests for a project
when ``.testr.conf`` has been configured with a ``test_list_option``.
(Robert Collins)

* ``test run --parallel`` will parallelise by running each test command once
  per CPU in the machine (detection for this only implemented on Linux so far).
  An internally parallelising command will not benefit from this, but for
  many projects it will be a win either from simplicity or because getting
  their test runner to parallise is nontrivial. The observed duration of tests
  is used to inform the partitioning algorithm - so each test runner should
  complete at approximately the same time, minimising total runtime.
  (Robert Collins)

* ``testr run`` no longer attempts to expand unknown variables. This permits
  the use of environmen variables to control the test run. For instance,
  ${PYTHON:-python} in the test_command setting will run the command with
  $PYTHON or python if $PYTHON is not set. (Robert Collins, #595295)

* ``testr run`` now resets the SIGPIPE handler to default - which is what
most Unix processes expect. (Robert Collins)

* ``testr run`` now uses a unique file name rather than hard coding
failing.list - while not as clear, this permits concurrent testr invocations,
or parallel testing from within testr, to execute safely. (Robert Collins)

* ``testr run`` uses an in-process load rather than reinvoking testr. This
should be faster on Windows and avoids the issue with running the wrong
testr when PYTHONPATH but not PATH is set. (Robert Collins, #613129)

* ``testr run`` will now pass -d to the ``testr load`` invocation, so that
running ``testr run -d /some/path`` will work correctly.
(Robert Collins, #529698)

* ``testr run`` will now pass ``-q`` down to ``testr load``.
(Robert Collins, #529701)

* The ``testrepository.repository.Repository`` interface now tracks test times
for use in estimating test run duration and parallel test partitioning.
(Robert Collins)

* There are the beginnings of a samba buildfarm backend for testrepository,
though it is not hooked into the UI yet, so is only useful for API users.
(Jelmer Vernooij)

* Updates to next-stream are done via a temporary file to reduce the chance
of an empty next-stream being written to disk. (Robert Collins, #531664)

* Variable expansion no longer does python \ escape expansion.
(Robert Collins, #694800)

* When next-stream is damaged testr will report that it is corrupt rather than
reporting an invalid literal. (Robert Collins, #531663)

File	Description	Downloads
testrepository-0.0.5.tar.gz (md5, sig)	Source tarball.	625 last downloaded 75 weeks ago
Total downloads:		625

0.0.4 release from the trunk series released 2010-09-11

Release information

Release notes:

0.0.4
+++++

IMPROVEMENTS
------------

* ``failing`` now supports ``--list`` to list the failing tests.
(Jonathan Lange)

* Repository not found errors are now clearer. (Jonathan Lange, #530010)

* The summary of a test run is now formatted as foo=NN rather than foo: NN,
which some folk find easier to read.

* The file implementation of Repository.open now performs ~ expansion.
(Jonathan Lange, #529665)

Changelog:

0.0.4
+++++

IMPROVEMENTS
------------

* ``failing`` now supports ``--list`` to list the failing tests.
(Jonathan Lange)

* Repository not found errors are now clearer. (Jonathan Lange, #530010)

* The summary of a test run is now formatted as foo=NN rather than foo: NN,
which some folk find easier to read.

* The file implementation of Repository.open now performs ~ expansion.
(Jonathan Lange, #529665)

File	Description	Downloads
testrepository-0.0.4.tar.gz (md5, sig)	Source tarball.	589 last downloaded 75 weeks ago
Total downloads:		589

0.0.3 release from the trunk series released 2010-02-28

Release information

Release notes:

* ``testr failing`` now tracks all failures rather than just the most recent ones.

* ``testr run`` can run a test suite and automatically gather the results.

* ``test last`` no longer errors on new repositories.

Changelog:

0.0.3
-----

IMPROVEMENTS
~~~~~~~~~~~~

* ``failing`` now correctly calls ``repository.get_failing`` and will this
track all seen failures rather than just the latest observed failures.

* New argument type ``StringArgument`` for use when a supplied argument is just
a string, rather than a typed argument.

* New subcommand 'failing' added.

* New subcommand ``run`` added which reads a .testr.conf file to figure out how
  to run tests with subunit output. It then runs them and pipes into testr
  load. This allows simpler integration and permits a programming interface so
  that tools like Tribunal/Eclipe etc can refresh tests in a testrepository.
  ``run`` also passes arguments and options down to the child process. ``run``
  can also supply test ids on the command, for test runners that want that.

* The command 'last' will no longer error on a new repository.

File	Description	Downloads
dist-testrepository-0.0.3.tar.gz (md5, sig)	release tarball	34 last downloaded 75 weeks ago
Total downloads:		34

0.0.2 release from the trunk series released 2010-01-10

Release information

Release notes:

Bugfix for 0.0.1 - include all the commands.

File	Description	Downloads
testrepository-0.0.2.tar.gz (md5, sig)	Source tarball.	201 last downloaded 75 weeks ago
Total downloads:		201

0.0.1 release from the trunk series released 2010-01-10

Release information

Release notes:

First sketch

File	Description	Downloads
testrepository-0.0.1.tar.gz (md5, sig)	Source tarball.	27 last downloaded 82 weeks ago
Total downloads:		27