Se connecter S'inscrire

Discover new versions of your packages or dependencies

Publié le 
Discover new versions of your packages or dependencies

z3c.checkversions

As part of my work on the the Zope Toolkit 1.0 and BlueBream 1.0, I've released several versions of z3c.checkversions.

z3c.checkversions
Find newer package versions on PyPI
PyPI

This package is part of the z3c (Zope 3 Community) namespace, but it does not depend on any Zope package and has zero dependencies (except setuptools). It provides you with a single checkversions script which can be used with any project. This is particularly useful, for instance, on large Plone projects, with many packages. The more packages you use, the more bugs you find. You shouldn't loose time on bugs that are already fixed by newer versions of packages! In the Zope community, it's generally safe to upgrade to the latest minor versions of packages. If you're using package foo version 1.0.2, and discover that version 1.0.6 and 1.1.0 are available, you can upgrade immediately to 1.0.6 without risk.

As a summary, versions are commonly formed with 3 numbers : X.Y.Z:

  • a new minor Z version only provides bugfixes. No new features are allowed.
  • a new intermediate Y version should provide new (backward compatible) features
  • a new major X version is expected to cause breakages and is probably backward-incompatible.

Discovering new versions on the system Python

Install z3c.checkversions as an administrator. This is safe because there is no dependencies, and you can uninstall it easily with pip afterwards:

$ sudo pip install z3c.checkversions  # or sudo easy_install z3c.checkversions
$ checkversions
# Checking your installed distributions
pida=0.6.2
python-dateutil=1.5
genshi=0.6
buildbot=0.8.1
pyparsing=1.5.3
(...)

If you add the -v option, all parsed versions will be displayed, as well as the current installed versions:

$ checkversions -v
# Checking your installed distributions
twisted-words=10.0.0
pida=0.6.2 # was: 0.5.0
foolscap=0.5.1
distribute=0.6.14
twisted-names=10.0.0
bicyclerepair=0.9
python-dateutil=1.5 # was: 1.4.1
genshi=0.6 # was: 0.5.1
buildbot=0.8.1 # was: 0.7.12
pyparsing=1.5.3 # was: 1.5.2
(...)

If you only want the new intermediate versions, add -l 1. If you only want the new minor (bugfix) versions, add -l 2 instead. If you ever have some packages with 4 version digits, you can even use -l 3:

$ checkversions -v -l 2
# Checking your installed distributions
twisted-words=10.0.0
pida=0.5.1 # was: 0.5.0
foolscap=0.5.1
distribute=0.6.14
twisted-names=10.0.0
bicyclerepair=0.9
python-dateutil=1.5
genshi=0.6
buildbot=0.8.1
pyparsing=1.5.3
(...)

Now you can uninstall it:

$ sudo pip uninstall z3c.checkversions

Working with a virtualenv

Do the same, but install it in a virtualenv:

$ virtualenv --no-site-packages --distribute sandbox
$ sandbox/bin/pip install z3c.checkversions

Then the script is available only in the virtualenv:

$ sandbox/bin/checkversions -v
# Checking your installed distributions
pip=0.8 # was: 0.7.1
python=2.6
distribute=0.6.14 # was: 0.6.10
z3c.checkversions=0.4
wsgiref=0.1.2
(...)

Working with a buildout

If the [versions] section of your buildout is huge, finding new versions one by one can be painful, particularly if you want to find only new minor versions. In that case just install z3c.checkversions in your buildout. Don't forget to enable the [buildout] extra:

[some_part]
recipe = zc.recipe.egg
eggs = z3c.checkversions [buildout]

Then run your buildout again, and run the checkversions script by passing the buildout.cfg (or versions.cfg) file as an argument:

$ bin/checkversions -v -l 2  versions.cfg
# Checking buildout file versions.cfg
zope.app.error=3.5.2
distribute=0.6.14 # was: 0.6.10
zope.dottedname=3.4.6
Twisted=10.0.0
zope.hookable=3.4.1
z3c.viewtemplate=0.3.2
zope.app.pagetemplate=3.4.1
zope.decorator=3.4.0
jquery.javascript=1.0.0
zope.app.container=3.5.6 # was: 3.5.4
hachoir-parser=1.2.1
z3c.template=1.1.0
(...)

Blacklist and incremental options

While setting up the AFPY buildbots for the Zope Toolkit, I've been experimenting with a special buildbot that would be able to create a new Zope Toolkit version list automatically, by testing arbitrary new versions. For this purpose I've added two options to z3c.checkversions:

  • -1 or --incremental
  • -b or --blacklist

The first option allows to suggest only one new version:

$ checkversions -v -l 2 --incremental versions.cfg
# Checking buildout file versions.cfg
zope.app.error=3.5.2
distribute=0.6.14 # was: 0.6.10
zope.dottedname=3.4.6
Twisted=10.0.0
zope.hookable=3.4.1
z3c.viewtemplate=0.3.2
zope.app.pagetemplate=3.4.1
zope.decorator=3.4.0
jquery.javascript=1.0.0
zope.app.container=3.5.4
hachoir-parser=1.2.1
z3c.template=1.1.0
(...)

The second option allows to not suggest a version which is in a blacklist version file. Say we have the following blacklist.cfg file:

zope.app.container=3.5.6

Then the script will suggest all the highest new versions, except the one in the blacklist:

$ checkversions -v -l 2 -b blacklist.cfg versions.cfg
# Checking buildout file versions.cfg
zope.app.error=3.5.2
distribute=0.6.14 # was: 0.6.10
zope.dottedname=3.4.6
Twisted=10.0.0
zope.hookable=3.4.1
z3c.viewtemplate=0.3.2
zope.app.pagetemplate=3.4.1
zope.decorator=3.4.0
jquery.javascript=1.0.0
zope.app.container=3.5.5 # was: 3.5.4
hachoir-parser=1.2.1
z3c.template=1.1.0
(...)

Since zope.app.container=3.5.6 is part of the blacklist, the script suggests the version just below it: zope.app.container=3.5.5.

Automatic discovery of the Zope Toolkit version list

I'm using these two options in this special Zope Toolkit buildbot: http://buildbot.afpy.org:8018/waterfall

This buildbot automatically upgrades one package version (builder at the left), then run the tests. If tests fail, the package is added in the blacklist (builder at the right). In the next run, the buildbot will choose the version just below. If tests fail, it will downgrade again. If tests pass, it will remember the current versions and try to upgrade another package. When the buildbot finally reach a stable state, we automatically get all the latest working versions.

During the first cycles, the buildbot has given the following behaviour :

  • it started with ZTK 1.0a2
  • it upgraded (arbitrary) zope.dublincore 3.6.0 -> 3.6.3
  • tests failed
  • it added zope.dublincore 3.6.3 in the blacklist
  • it upgraded zope.dublincore 3.6.0 -> 3.6.2
  • tests failed
  • it added zope.dublincore 3.6.2 in the blacklist
  • it upgraded zope.dublincore 3.6.0 -> 3.6.1
  • tests failed
  • it added zope.dublincore 3.6.1 in the blacklist
  • so it keeps zope.dublincore 3.6.0 (until 3.6.4 is released)
  • it upgraded lxml 2.2.6 -> 2.2.7
  • tests passed
  • lxml 2.2.7 is kept for future builds
  • etc....

The cycle is not optimal, because zope.dublincore 3.6.3 maybe works well when upgraded with another package. So maybe I should try the opposite: upgrading packages bottom up, one by one, until the highest one is reached.