News and announcements

0.8.0 Released

Written for git-ubuntu by Bryce Harrington on 2019-11-26

It's been a long while since git-ubuntu's last release[1], 0.7.4. The
team has been doing git snapshot releases in the interim, but we figure
it'd be good to start formal releases once again.

The git-ubuntu snap package has been updated to 0.8.0 for the 'edge' and
'beta' channels. Assuming all goes well, we'll update the 'stable'
channel to 0.8.0 within a few weeks.

Installing the Beta Release
---------------------------

  $ snap install --beta --classic git-ubuntu

Snap is still the recommended way to install git-ubuntu, however note
that we're adjusting our channel usage:

  - Use the 'stable' channel for production usage.

  - The 'beta' channel is for testers, and will be what we run on the
    backend server.

  - The 'edge' channel will track current development, and is not
    recommended for most users.

If you have been running git-ubuntu from edge, please take a moment to
re-install to beta, and please track that going forward:

  $ snap refresh --beta --classic git-ubuntu

Changes from 0.7.4 to 0.8.0
---------------------------

The lion's share of effort since 0.7.4 has gone towards bug fixing and
general stabilization. Documentation and tests received a fair share of
attention, as did the snap and setup.py packaging. The number of these
changes is too much to mention with any brevity, but are summarized in
shortlog below.

One noteworthy bug fix involved a case where a package has no quilt
patches, LP: #1801168. git-ubuntu build would still attempt to copy the
non-existant .pc directory and fail. We had several dupes (and patch
offerings -- thank you!) of this issue.

The importer now uses a sqlite3 database to store persistent information
such as the pending package import status. This is designed to serve as
a cache rather than a definitive data source; the database can be
dropped at will without impacting overall behavior.

A new --only-request-new-imports-once option is added for the backend
source package importer. This makes the importer exit immediately after
entering new imports to the database.

--deconstruct has been changed to --split, to prevent confusion that led
people to assume --deconstruct meant the opposite of "reconstruct".
"Split" offers a more sensible metaphor.

Launchpad object fetches are cached using Python's cachetools module, as
a performance improvement that reduces the excessive number of API calls
to the Launchpad service.

Finally, the backend service is now managed using a systemd watchdog
daemon. Prior to this the service would need to be manually restarted
whenever it hung or crashed, such as due to Launchpad service outages or
network instabilities.

1: https://lists.ubuntu.com/archives/ubuntu-server/2018-March/007669.html

Commits included in this Release
--------------------------------

Andreas Hasenack (10):
      Add package cabextract to whitelist
      Add more pmdk related packages to the whitelist
      Add some kernels to the blacklist
      Import zeromq3
      Update the Debian keyring package
      blacklist linux-oem (~150Mb large source package)
      Add debian-multimedia to whitelist
      Add wireguard to the whitelist
      Add containerd and runc to the whitelist
      blacklist linux-hwe (155MiB)

Bryce Harrington (25):
      .gitignore: Ignore eggs & build/
      Fix check warnings and document
      Use AUTHORS instead of authors field in setup.py
      README: Update instructions regarding preference for snap installation
      doc: Copyedit grammar
      man: Fix typo 'paramaters'
      man: Fix grammar (a/an)
      man: Fix reference to algorithm
      man: Fix typo 'resultig'
      man: Fix description for git-ubuntu import-ppa
      man: Improve git-ubuntu-lint short desc
      man: Highlight the 'Ubuntu delta' terminology
      man: Fix indentation for onto
      man: Revise the git-ubuntu man page description
      source_information: Add tests for derive_codename_from_series()
      source_information: Fix ubuntu series lookup
      source_information: Document derive_codename_from_series()
      Implement a systemd watchdog daemon to run import-source-packages.py
      snap: Fix build of systemd python package
      Improve testsuite to verify dependencies and provide coverage of python scripts.
      .gitignore: Ignore files generated during snap packaging
      Cleanup obsolete integration test code
      setup.py: Bump keyring version for upstream fix
      setup.py: Bump lazr.restfulclient version requirement
      version: bump to 0.8.0

Christian Ehrhardt (1):
      add ua-client related packges to whitelist

Claudio Matsuoka (1):
      Fix snap package build

Karl Stenerud (1):
      Author: Karl Stenerud <karl.stenerud@canonical.com> Date: Thu Mar 29 10:09:05 2019 +0100

Kyle Fazzari (2):
      Move wrappers out of snap dir
      Place wrappers back into wrappers subdirectory

Nishanth Aravamudan (69):
      import: fix case of reimport of a not-yet imported repository
      import: handle orphan tags as changelog parents
      update-repository-alias: do not quote package
      import-source-packages: add --num-days-ago option
      whitelist: fix typo in kopanocore
      whitelist: add packages manually imported
      whitelist: add two missed manual imports
      whitelist: add more manually imported packages
      whitelist: fix typo
      import-source-packages: call sendmail from ORIG_PATH
      snap: fix future compatibility issues with snapcraft 2.40
      snap: avoid duplication lazr.restfulclient contents
      snap: remove duplicate libgpg-error and libgcrypt so files
      snap: fix typo in filelist
      blacklist: add oxide-qt
      blacklist: add texlive-lang
      snap: order xdelta3 after xz-utils
      snap: also do not stage lib/*/gpg-error*
      repo_builder: rename test_placeholder -> test_replace_placeholders
      repo_builder: strengthen check in test_replace_placeholders
      repo_builder: increase code coverage to 100%
      repo_builder: rename CommitGraph to Repo
      repo_builder.Repo: handle empty repository
      repo_builder: add Branch and Tag classes
      repo_builder: specify UTC offset
      repo_builder: add Repo.copy
      dsc_to_tree_hash: add ability to get final patches-applied tree
      repo_builder.SourceTree: allow for patches to be applied
      git_repository: add docstring to pristine_tar methods
      git_repository: drop ensure_importer_branches_exist
      source_builder: add second assertion
      source_builder_test: fix docstrings
      source_builder: add file_contents attribute to SourceSpec
      test_fixtures: fix typos in comments
      test_fixtures: for simplicity chdir to the repo local directory
      source_builder: fix changelog_versions handling
      importer: drop unused variable
      importer: stub out and test tag and commit message functions
      Add SPECIFICATION.tags
      git_repository: add reimport_tag infrastructure support
      importer: add get_existing_{import,applied}_tags API
      git_repository: Add create_tag API
      repo_comparator: add basic equals() method
      importer: add create_{import,applied}_tag API
      importer: remove import_patches_unapplied_tree
      git_repository: add docstrings
      git_repository: formatting cleanups
      git_repository: do not pass spi to commit_tree_hash
      importer: do not pass spi to override_parents
      importer: do not pass spi to get_commit_msg
      importer: do not pass spi to _commit_import
      blacklist: add a few more large source packages
      importer: abstract out get_changelog_parent_commit
      importer._commit_import: use repo.create_tag
      importer._commit_import: use get_import_tag_msg()
      importer: refactor _commit_import to return the commit hash
      importer: abstract out import_{un,}applied_dsc
      importer: do not shadow imported methods
      blacklist: add libreoffice-l10n
      blacklist: add texlive-base
      blacklist: add openoffice.org
      blacklist: add texlive-extra
      blacklist: add more large source packages
      versioning: add helper method to split version string
      git_repository: add find_ubuntu_merge_base method
      find_ubuntu_merge_base: fix the code for the new imported Git layout
      snap: drop one-off oauth part
      setup.py: lazr.restfulclient bump
      snap: order gnupg2 to be before devscripts

Rafael David Tinoco (1):
      Add corosync-qdevice, pcs and sbd to whitelist

Robie Basak (66):
      Add http-parser requested by Andreas
      Enhance dpkg tarball reproduction instructions
      Update people.c.c vendored tarball URLs
      Make sure launchpadlib builds before git-ubuntu
      Bump and move vendored launchpadlib Launchpad
      Document external dependency management policy
      Whitelist brotli
      snap: fix lazr.restfulclient stage path
      Add default values to main function arguments
      Fix documentation typo
      git_repository: add close method and invoke it in a try/finally from importer
      Drop inferred log message
      Add skytools3 to whitelist
      Drop catch-all exception
      Add dpkg-parsechangelog behaviour variance test
      Test edge case changelog assertion
      Fix edge case changelog assertion check
      Merge branch 'master' of https://git.launchpad.net/~tsimonq2/usd-importer/+git/add-lubuntu
      Move changelog tests to dedicated test file
      Avoid cross-test-file dependency
      Rename test files
      Merge branch 'changelog-assertions'
      Add pmdk and ndctl to whitelist
      wip: automated upload tag pusher
      wip: fix whitespace wrap
      Merge branch 'bugfix/move_sources_out_of_snap_dir' of git+ssh://git.launchpad.net/~kyrofa/usd-importer
      tag: improve dirty tree detection
      Add rustc to whitelist
      Test that a keyring backend can be found
      Drop keyrings.alt version restriction
      Add awk integration test
      Fix call into core snap awk binary
      Add cargo to whitelist
      Merge ~smoser/usd-importer:cleanup/build-cleanups
      Revert "Merge ~smoser/usd-importer:cleanup/build-cleanups"
      Add nsd to whitelist
      Merge remote-tracking branch 'smoser/cleanup/build-cleanups-redux' into HEAD
      Use the keyring package from git-ubuntu only
      Pin keyring version to work around regression
      Use persistent state directory
      Move timestamp storage to sqlite3 database
      Store pending package import status
      Drop --dry-run function
      Add --only-request-new-imports-once option
      Mark "running" imports as errored on restart
      Test that signing keys are present by default
      Bump ubuntu-keyring keyrings used in the snap
      Restore dry_run parameter in API call
      Drop use of dry_run that is no longer defined
      Merge remote-tracking branch 'smoser/fix/1801168-build-when-no-patches'
      Add pgaudit to whitelist
      Sort ahasenack section of whitelist
      Add lmdb to whitelist
      pylint: ignore the py.path module
      Stop pinning pylint
      Friendlier self-test output
      Add bind-dyndb-ldap to whitelist
      Add php7.3 to import whitelist
      Complete Certbot-related whitelist packages
      Cache some Launchpad object fetches
      Blacklist wheezy as an active series
      Merge remote-tracking branch 'bryce/tests-fixing'
      Add debian-archive-keyring to whitelist
      Merge remote-tracking branch 'bryce/fix_derive_codename_from_series.4'
      Work around pylint/astroid bug
      Add pandas to whitelist

Scott Moser (19):
      Fix 'git ubuntu build' when user id does not match container id.
      Change internal user to 'buildd' from build-user.
      build: Be more explicit about creating user in container.
      build: build in Ubuntu devel for 'unstable', 'testing', 'devel'.
      build: Explicitly wait for the container to finish booting.
      Rework installing dependencies.
      arguments are a list, not a set.
      run_in_lxd: do not use a shell in sudo.
      Fix native build.
      build: Be more explicit about creating user in container.
      build: build in Ubuntu devel for 'unstable', 'testing', 'devel'.
      build: Explicitly wait for the container to finish booting.
      Rework installing dependencies.
      arguments are a list, not a set.
      run_in_lxd: do not use a shell in sudo.
      Fix native build.
      build: Fix build when there are no quilt patches.
      build: Use ubuntu images for build unless image provided.
      add test

Sergio Schvezov (1):
      snap: make use of the source-checksum field for sources

Simon Quigley (1):
      Add Lubuntu packages.

Past Releases
-------------

For a list of previous release announcements, please see:

    https://launchpad.net/usd-importer/+announcements