Advanced configuration

factor-based, multi-dimensional, platform-specific configuration

Generate and select variants

Suppose you want to test your package against python2.6, python2.7 and on the windows and linux platforms. Today you would have to write down 2*2 = 4 [testenv:*] sections and then instruct tox to run a specific list of environments on each platform.

With tox-1.X you can directlys specify combinations:

# combination syntax gives 2 * 2 = 4 testenv names
#
envlist = {py26,py27}-{win,linux}

[testenv]
deps = pytest
platform =
       win: windows
       linux: linux
basepython =
       py26: python2.6
       py27: python2.7
commands = pytest

Let’s go through this step by step:

envlist = {py26,py27}-{windows,linux}

This is bash-style syntax and will create 2*2=4 environment names like this:

py26-windows
py26-linux
py27-windows
py27-linux

Our [testenv] uses a new templating style for the platform definition:

platform=
       windows: windows
       linux: linux

These two conditional settings will lead to either windows or linux as the platform string. When the test environment is run, its platform string needs to be contained in the string returned from platform.platform(). Otherwise the environment will be skipped.

The next configuration item in the testenv section deals with the python interpreter:

basepython =
       py26: python2.6
       py27: python2.7

This defines a python executable, depending on if py26 or py27 appears in the environment name.

The last config item is simply the invocation of the test runner:

commands = pytest

Nothing special here :)

Note

tox provides good defaults for platform and basepython settings, so the above ini-file can be further reduced:

[tox]
envlist = {py26,py27}-{win,linux}

[testenv]
deps = pytest
commands = pytest

Voila, this multi-dimensional tox.ini configuration defines 2*2=4 environments.

The platform setting

A testenv can define a platform setting. If its value is not contained in the string obtained from calling sys.platform the environment will be skipped.

Expand the envlist setting

The envlist setting allows to use {} bash-style expressions. XXX explanation or pointer to bash-docs

Templating based on environment names

For a given environment name, all lines in a testenv section which start with “NAME: …” will be checked for being part in the environment name. If they are part of it, the remainder will be the new line. If they are not part of it, the whole line will be left out. Parts of an environment name are obtained by --splitting it.

Variant specification with [variant:VARNAME]

Show all sections

To help with understanding how the variants will produce section values, you can ask tox to show their expansion with a new option:

$ tox -[v]l

To show all environments (including ones not in envlist):

$ tox -[v]a

note: adding the v flag outputs the descriptions of environments if they are set via the description key.

Make sure your packages installs with easy_install

The “installer” testenv setting allows to specify the tool for installation in a given test environment:

[testenv]
installer =
    easy: easy_install
    pip: pip

If you want to have your package installed with both easy_install and pip, you can list them in your envlist likes this:

[tox]
envlist = py[26,27,32]-django[13,14]-[easy,pip]

If no installer is specified, pip will be used.

Use more bash-style syntax

tox leverages bash-style syntax if you specify mintoxversion = 1.4:

  • $VARNAME or ${…} syntax instead of the older {} substitution.
  • XXX go through config.rst and see how it would need to be changed

Example: django-rest

The original django-rest-framework tox.ini file has 159 lines and a lot of repetition, the new one would +have 20 lines and almost no repetition:

[tox]
envlist = {py25,py26,py27}-{django12,django13}{,-example}

[testenv]
deps=
    coverage==3.4
    unittest-xml-reporting==1.2
    Pyyaml==3.10
    django12: django==1.2.4
    django13: django==1.3.1
    # some more deps for running examples
    example: wsgiref==0.1.2
    example: Pygments==1.4
    example: httplib2==0.6.0
    example: Markdown==2.0.3

commands =
   !example: python setup.py test
   example: python examples/runtests.py

Note that {,-example} in the envlist denotes two values, an empty one and a example one. The empty value means that there are no specific settings and thus no need to define a variant name.

Example: django-treebeard

Another tox.ini has 233 lines and runs tests against multiple Postgres and Mysql engines. It also performs backend-specific test commands, passing different command line options to the test script. With the new tox-1.X we not only can do the same with 32 non-repetive configuration lines but we also produce 36 specific testenvs with specific dependencies and test commands:

[tox]
envlist = {py24,py25,py26,py27}-{django11,django12,django13}-{nodb,pg,mysql}, docs

[testenv:docs]
changedir = docs
deps =
    Sphinx
    Django
commands =
    make clean
    make html

[testenv]
deps=
    coverage
    pysqlite
    django11: django==1.1.4
    django12: django==1.2.7
    django13: django==1.3.1
    django14: django==1.4
    nodb: pysqlite
    pg: psycopg2
    mysql: MySQL-python

commands =
    nodb: {envpython} runtests.py {posargs}
    pg: {envpython} runtests.py {posargs} \
                    --DATABASE_ENGINE=postgresql_psycopg2 \
                    --DATABASE_USER=postgres {posargs}
    mysql: {envpython} runtests.py --DATABASE_ENGINE=mysql \
                                   --DATABASE_USER=root {posargs}