tox - testing out of the box

Welcome to the tox automation project - helping to standardize testing in Python since 2012

What is tox?

tox aims to automate and standardize testing and task automation in Python. It is part of a larger vision to be unifying frontend between CI systems and local development activity therfore easing the packaging, testing and release process.

Most typical usages are:

  • Build your package and check if it installs correctly with different Python versions and interpreters
  • Run your tests in each of the environments, configuring your test tool of choice
  • Build and deploy the documentation of the project

In a nutshell

Supported Pythons: CPython 2.6-3.6, jython, pypy

Supported operating systems: Linux, Windows, macOS, Unix

License: MIT

development: https://github.com/tox-dev

Installation

pip install tox

It is fine to install tox itself into a virtualenv environment.

Basic example

First, install tox with pip install tox. Then put basic information about your project and the test environments you want your project to run in into a tox.ini file residing right next to your setup.py file:

# content of: tox.ini , put in same dir as setup.py
[tox]
envlist = py26,py27
[testenv]
deps=pytest       # install pytest in the venvs
commands=pytest  # or 'nosetests' or ...

You can also try generating a tox.ini file automatically, by running tox-quickstart and then answering a few simple questions.

To sdist-package, install and test your project against Python2.6 and Python2.7, just type:

tox

… and watch things happening (you must have python2.6 and python2.7 installed in your environment otherwise you will see errors). When you run tox a second time you’ll note that it runs much faster because it keeps track of virtualenv details and will not recreate or re-install dependencies. You also might want to checkout the Examples to get some more ideas.

Current features

  • automation of tedious Python related test activities

  • test your Python package against many interpreter and dependency configs

    • automatic customizable (re)creation of virtualenv test environments
    • installs your setup.py based project into each virtual environment
    • test-tool agnostic: runs pytest, nose or unittests in a uniform manner
  • plugin system to modify tox execution with simple hooks.

  • uses pip and setuptools by default. Support for configuring the installer command through install_command=ARGV.

  • cross-Python compatible: CPython-2.6, 2.7, 3.2 and higher, Jython and pypy.

  • cross-platform: Windows and Unix style environments

  • integrates with continuous integration servers like Jenkins and helps you to avoid boilerplatish and platform-specific build-step hacks.

  • full interoperability with devpi: is integrated with and is used for testing in the devpi system, a versatile pypi index server and release managing tool.

  • driven by a simple ini-style config file

  • documented examples and configuration

  • concise reporting about tool invocations and configuration errors

  • professionally Paid professional support

  • supports using different / multiple PyPI index servers

CHANGELOG

Versions follow Semantic Versioning (<major>.<minor>.<patch>).

Backward incompatible (breaking) changes will only be introduced in major versions with advance notice in the Deprecations section of releases.

Changes that already arrived in master, but are not released yet can be found as fragments in the “changelog” directory.

If you want to know more details: since version 2.8 releases are organized as Github projects. To see all closed issues and merged pull requests, contained in a release, visit the projects on Github:

2.9.0rc1 (2017-09-14)

Features

  • tox --version now shows information about all registered plugins - by @obestwalter (#544)

Bugfixes

  • skip_install overrides usedevelop (usedevelop is an option to choose the installation type if the package is installed and skip_install determines if it should be installed at all) - by @ferdonline (#571)

Improved Documentation

  • add towncrier to allow adding changelog entries with the pull requests without generating merge conflicts; with this release notes are now grouped into four distinct collections: Features, Bugfixes, Improved Documentation and Deprecations and Removals. (#614)

2.8.2 (2017-10-09)

  • #466: stop env var leakage if popen failed with resultjson or redirect

2.8.1 (2017-09-04)

  • pull request 599: fix problems with implementation of #515. Substitutions from other sections were not made anymore if they were not in envlist. Thanks to Clark Boylan (@cboylan) for helping to get this fixed (pull request 597).

2.8.0 (2017-09-01)

  • #276: Remove easy_install from docs (TL;DR: use pip). Thanks Martin Andrysík (@sifuraz).
  • #301: Expand nested substitutions in tox.ini. Thanks @vlaci. Thanks to Eli Collins (@eli-collins) for creating a reproducer.
  • #315: add --help and --version to helptox-quickstart. Thanks @vlaci.
  • #326: Fix OSError ‘Not a directory’ when creating env on Jython 2.7.0. Thanks Nick Douma (@LordGaav).
  • #429: Forward MSYSTEM by default on Windows. Thanks Marius Gedminas (@mgedmin) for reporting this.
  • #449: add multi platform example to the docs. Thanks Aleks Bunin (@sashkab) and @rndr.
  • #474: Start using setuptools_scm for tag based versioning.
  • #484: Renamed py.test to pytest throughout the project. Thanks Slam (@3lnc).
  • #504: With -a: do not show additional environments header if there are none. Thanks @rndr.
  • #515: Don’t require environment variables in test environments where they are not used. Thanks André Caron (@AndreLouisCaron).
  • #517: Forward NUMBER_OF_PROCESSORS by default on Windows to fix multiprocessor.cpu_count(). Thanks André Caron (@AndreLouisCaron).
  • #518: Forward USERPROFILE by default on Windows. Thanks André Caron (@AndreLouisCaron).
  • pull request 528: Fix some of the warnings displayed by pytest 3.1.0. Thanks Bruno Oliveira (@nicoddemus).
  • pull request 547: Add regression test for #137. Thanks Martin Andrysík (@sifuraz).
  • pull request 553: Add an XFAIL test to reproduce upstream bug #203. Thanks Bartolomé Sánchez Salado (@bartsanchez).
  • pull request 556: Report more meaningful errors on why virtualenv creation failed. Thanks @vlaci. Also thanks to Igor Sadchenko (@igor-sadchenko) for pointing out a problem with that PR before it hit the masses ☺
  • pull request 575: Add announcement doc to end all announcement docs (using only CHANGELOG and Github issues since 2.5 already).
  • pull request 580: Do not ignore Sphinx warnings anymore. Thanks Bernát Gábor (@gaborbernat).
  • pull request 585: Expand documentation to explain pass through of flags from deps to pip (e.g. -rrequirements.txt, -cconstraints.txt). Thanks Alexander Loechel (@loechel).
  • pull request 588: Run pytest wit xfail_strict and adapt affected tests.

2.7.0 (2017-04-02)

  • pull request 450: Stop after the first installdeps and first testenv create hooks succeed. This changes the default behaviour of tox_testenv_create and tox_testenv_install_deps to not execute other registered hooks when the first hook returns a result that is not None. Thanks Anthony Sottile (@asottile).

  • #271 and #464: Improve environment information for users.

    New command line parameter: -a show all defined environments - not just the ones defined in (or generated from) envlist.

    New verbosity settings for -l and -a: show user defined descriptions of the environments. This also works for generated environments from factors by concatenating factor descriptions into a complete description.

    Note that for backwards compatibility with scripts using the output of -l it’s output remains unchanged.

    Thanks Bernát Gábor (@gaborbernat).

  • #464: Fix incorrect egg-info location for modified package_dir in setup.py. Thanks Selim Belhaouane (@selimb).

  • #431: Add ‘LANGUAGE’ to default passed environment variables. Thanks Paweł Adamczak (@pawelad).

  • #455: Add a Vagrantfile with a customized Arch Linux box for local testing. Thanks Oliver Bestwalter (@obestwalter).

  • #454: Revert pull request 407, empty commands is not treated as an error. Thanks Anthony Sottile (@asottile).

  • #446: (infrastructure) Travis CI tests for tox now also run on OS X now. Thanks Jason R. Coombs (@jaraco).

2.6.0 (2017-02-04)

  • add “alwayscopy” config option to instruct virtualenv to always copy files instead of symlinking. Thanks Igor Duarte Cardoso (@igordcard).
  • pass setenv variables to setup.py during a usedevelop install. Thanks Eli Collins (@eli-collins).
  • replace all references to testrun.org with readthedocs ones. Thanks Oliver Bestwalter (@obestwalter).
  • fix #323 by avoiding virtualenv14 is not used on py32 (although we don’t officially support py32). Thanks Jason R. Coombs (@jaraco).
  • add Python 3.6 to envlist and CI. Thanks Andrii Soldatenko (@andriisoldatenko).
  • fix glob resolution from TOX_TESTENV_PASSENV env variable Thanks Allan Feldman (@a-feld).

2.5.0 (2016-11-16)

  • slightly backward incompatible: fix #310: the {posargs} substitution now properly preserves the tox command line positional arguments. Positional arguments with spaces are now properly handled. NOTE: if your tox invocation previously used extra quoting for positional arguments to work around #310, you need to remove the quoting. Example: tox – “‘some string’” # has to now be written simply as tox – “some string” thanks holger krekel. You can set minversion = 2.5.0 in the [tox] section of tox.ini to make sure people using your tox.ini use the correct version.
  • fix #359: add COMSPEC to default passenv on windows. Thanks @anthrotype.
  • add support for py36 and py37 and add py36-dev and py37(nightly) to travis builds of tox. Thanks John Vandenberg.
  • fix #348: add py2 and py3 as default environments pointing to “python2” and “python3” basepython executables. Also fix #347 by updating the list of default envs in the tox basic example. Thanks Tobias McNulty.
  • make “-h” and “–help-ini” options work even if there is no tox.ini, thanks holger krekel.
  • add {:} substitution, which is replaced with os-specific path separator, thanks Lukasz Rogalski.
  • fix #305: downloadcache test env config is now ignored as pip-8 does caching by default. Thanks holger krekel.
  • output from install command in verbose (-vv) mode is now printed to console instead of being redirected to file, thanks Lukasz Rogalski
  • fix #399. Make sure {envtmpdir} is created if it doesn’t exist at the start of a testenvironment run. Thanks Manuel Jacob.
  • fix #316: Lack of commands key in ini file is now treated as an error. Reported virtualenv status is ‘nothing to do’ instead of ‘commands succeeded’, with relevant error message displayed. Thanks Lukasz Rogalski.

2.4.1 (2016-10-12)

  • fix #380: properly perform substitution again. Thanks Ian Cordasco.

2.4.0 (2016-10-12)

  • remove PYTHONPATH from environment during the install phase because a tox-run should not have hidden dependencies and the test commands will also not see a PYTHONPATH. If this causes unforeseen problems it may be reverted in a bugfix release. Thanks Jason R. Coombs.
  • fix #352: prevent a configuration where envdir==toxinidir and refine docs to warn people about changing “envdir”. Thanks Oliver Bestwalter and holger krekel.
  • fix #375, fix #330: warn against tox-setup.py integration as “setup.py test” should really just test with the current interpreter. Thanks Ronny Pfannschmidt.
  • fix #302: allow cross-testenv substitution where we substitute with {x,y} generative syntax. Thanks Andrew Pashkin.
  • fix #212: allow escaping curly brace chars “{” and “}” if you need the chars “{” and “}” to appear in your commands or other ini values. Thanks John Vandenberg.
  • addresses #66: add –workdir option to override where tox stores its “.tox” directory and all of the virtualenv environment. Thanks Danring.
  • introduce per-venv list_dependencies_command which defaults to “pip freeze” to obtain the list of installed packages. Thanks Ted Shaw, Holger Krekel.
  • close #66: add documentation to jenkins page on how to avoid “too long shebang” lines when calling pip from tox. Note that we can not use “python -m pip install X” by default because the latter adds the CWD and pip will think X is installed if it is there. “pip install X” does not do that.
  • new list_dependencies_command to influence how tox determines which dependencies are installed in a testenv.
  • (experimental) New feature: When a search for a config file fails, tox tries loading setup.cfg with a section prefix of “tox”.
  • fix #275: Introduce hooks tox_runtest_pre` and tox_runtest_post which run before and after the tests of a venv, respectively. Thanks to Matthew Schinckel and itxaka serrano.
  • fix #317: evaluate minversion before tox config is parsed completely. Thanks Sachi King for the PR.
  • added the “extras” environment option to specify the extras to use when doing the sdist or develop install. Contributed by Alex Grönholm.
  • use pytest-catchlog instead of pytest-capturelog (latter is not maintained, uses deprecated pytest API)

2.3.2 (2016-02-11)

  • fix #314: fix command invocation with .py scripts on windows.
  • fix #279: allow cross-section substitution when the value contains posargs. Thanks Sachi King for the PR.

2.3.1 (2015-12-14)

  • fix #294: re-allow cross-section substitution for setenv.

2.3.0 (2015-12-09)

  • DEPRECATE use of “indexservers” in tox.ini. It complicates the internal code and it is recommended to rather use the devpi system for managing indexes for pip.
  • fix #285: make setenv processing fully lazy to fix regressions of tox-2.2.X and so that we can now have testenv attributes like “basepython” depend on environment variables that are set in a setenv section. Thanks Nelfin for some tests and initial work on a PR.
  • allow “#” in commands. This is slightly incompatible with commands sections that used a comment after a “” line continuation. Thanks David Stanek for the PR.
  • fix #289: fix build_sphinx target, thanks Barry Warsaw.
  • fix #252: allow environment names with special characters. Thanks Julien Castets for initial PR and patience.
  • introduce experimental tox_testenv_create(venv, action) and tox_testenv_install_deps(venv, action) hooks to allow plugins to do additional work on creation or installing deps. These hooks are experimental mainly because of the involved “venv” and session objects whose current public API is not fully guranteed.
  • internal: push some optional object creation into tests because tox core doesn’t need it.

2.2.1 (2015-12-09)

  • fix bug where {envdir} substitution could not be used in setenv if that env value is then used in {basepython}. Thanks Florian Bruhin.

2.2.0 (2015-11-11)

  • fix #265 and add LD_LIBRARY_PATH to passenv on linux by default because otherwise the python interpreter might not start up in certain configurations (redhat software collections). Thanks David Riddle.
  • fix #246: fix regression in config parsing by reordering such that {envbindir} can be used again in tox.ini. Thanks Olli Walsh.
  • fix #99: the {env:…} substitution now properly uses environment settings from the setenv section. Thanks Itxaka Serrano.
  • fix #281: make –force-dep work when urls are present in dependency configs. Thanks Glyph Lefkowitz for reporting.
  • fix #174: add new ignore_outcome testenv attribute which can be set to True in which case it will produce a warning instead of an error on a failed testenv command outcome. Thanks Rebecka Gulliksson for the PR.
  • fix #280: properly skip missing interpreter if {envsitepackagesdir} is present in commands. Thanks BB:ceridwenv

2.1.1 (2015-06-23)

  • fix platform skipping for detox
  • report skipped platforms as skips in the summary

2.1.0 (2015-06-19)

  • fix #258, fix #248, fix #253: for non-test commands (installation, venv creation) we pass in the full invocation environment.
  • remove experimental –set-home option which was hardly used and hackily implemented (if people want home-directory isolation we should figure out a better way to do it, possibly through a plugin)
  • fix #259: passenv is now a line-list which allows to intersperse comments. Thanks stefano-m.
  • allow envlist to be a multi-line list, to intersperse comments and have long envlist settings split more naturally. Thanks Andre Caron.
  • introduce a TOX_TESTENV_PASSENV setting which is honored when constructing the set of environment variables for test environments. Thanks Marc Abramowitz for pushing in this direction.

2.0.2 (2015-06-03)

  • fix #247: tox now passes the LANG variable from the tox invocation environment to the test environment by default.
  • add SYSTEMDRIVE into default passenv on windows to allow pip6 to work. Thanks Michael Krause.

2.0.1 (2015-05-13)

  • fix wheel packaging to properly require argparse on py26.

2.0.0 (2015-05-12)

  • (new) introduce environment variable isolation: tox now only passes the PATH and PIP_INDEX_URL variable from the tox invocation environment to the test environment and on Windows also SYSTEMROOT, PATHEXT, TEMP and TMP whereas on unix additionally TMPDIR is passed. If you need to pass through further environment variables you can use the new passenv setting, a space-separated list of environment variable names. Each name can make use of fnmatch-style glob patterns. All environment variables which exist in the tox-invocation environment will be copied to the test environment.
  • a new --help-ini option shows all possible testenv settings and their defaults.
  • (new) introduce a way to specify on which platform a testenvironment is to execute: the new per-venv “platform” setting allows to specify a regular expression which is matched against sys.platform. If platform is set and doesn’t match the platform spec in the test environment the test environment is ignored, no setup or tests are attempted.
  • (new) add per-venv “ignore_errors” setting, which defaults to False.
    If True, a non-zero exit code from one command will be ignored and further commands will be executed (which was the default behavior in tox < 2.0). If False (the default), then a non-zero exit code from one command will abort execution of commands for that environment.
  • show and store in json the version dependency information for each venv
  • remove the long-deprecated “distribute” option as it has no effect these days.
  • fix #233: avoid hanging with tox-setuptools integration example. Thanks simonb.
  • fix #120: allow substitution for the commands section. Thanks Volodymyr Vitvitski.
  • fix #235: fix AttributeError with –installpkg. Thanks Volodymyr Vitvitski.
  • tox has now somewhat pep8 clean code, thanks to Volodymyr Vitvitski.
  • fix #240: allow to specify empty argument list without it being rewritten to “.”. Thanks Daniel Hahler.
  • introduce experimental (not much documented yet) plugin system based on pytest’s externalized “pluggy” system. See tox/hookspecs.py for the current hooks.
  • introduce parser.add_testenv_attribute() to register an ini-variable for testenv sections. Can be used from plugins through the tox_add_option hook.
  • rename internal files – tox offers no external API except for the experimental plugin hooks, use tox internals at your own risk.
  • DEPRECATE distshare in documentation

1.9.2 (2015-03-23)

  • backout ability that –force-dep substitutes name/versions in requirement files due to various issues. This fixes #228, fixes #230, fixes #231 which popped up with 1.9.1.

1.9.1 (2015-03-23)

  • use a file instead of a pipe for command output in “–result-json”. Fixes some termination issues with python2.6.
  • allow –force-dep to override dependencies in “-r” requirements files. Thanks Sontek for the PR.
  • fix #227: use “-m virtualenv” instead of “-mvirtualenv” to make it work with pyrun. Thanks Marc-Andre Lemburg.

1.9.0 (2015-02-24)

  • fix #193: Remove --pre from the default install_command; by default tox will now only install final releases from PyPI for unpinned dependencies. Use pip_pre = true in a testenv or the --pre command-line option to restore the previous behavior.
  • fix #199: fill resultlog structure ahead of virtualenv creation
  • refine determination if we run from Jenkins, thanks Borge Lanes.
  • echo output to stdout when --report-json is used
  • fix #11: add a skip_install per-testenv setting which prevents the installation of a package. Thanks Julian Krause.
  • fix #124: ignore command exit codes; when a command has a “-” prefix, tox will ignore the exit code of that command
  • fix #198: fix broken envlist settings, e.g. {py26,py27}{-lint,}
  • fix #191: lessen factor-use checks

1.8.1 (2014-10-24)

  • fix #190: allow setenv to be empty.
  • allow escaping curly braces with “”. Thanks Marc Abramowitz for the PR.
  • allow “.” names in environment names such that “py27-django1.7” is a valid environment name. Thanks Alex Gaynor and Alex Schepanovski.
  • report subprocess exit code when execution fails. Thanks Marius Gedminas.

1.8.0 (2014-09-24)

  • new multi-dimensional configuration support. Many thanks to Alexander Schepanovski for the complete PR with docs. And to Mike Bayer and others for testing and feedback.
  • fix #148: remove “__PYVENV_LAUNCHER__” from os.environ when starting subprocesses. Thanks Steven Myint.
  • fix #152: set VIRTUAL_ENV when running test commands, thanks Florian Ludwig.
  • better report if we can’t get version_info from an interpreter executable. Thanks Floris Bruynooghe.

1.7.2 (2014-07-15)

  • fix #150: parse {posargs} more like we used to do it pre 1.7.0. The 1.7.0 behaviour broke a lot of OpenStack projects. See PR85 and the issue discussions for (far) more details, hopefully resulting in a more refined behaviour in the 1.8 series. And thanks to Clark Boylan for the PR.
  • fix #59: add a config variable skip-missing-interpreters as well as command line option --skip-missing-interpreters which won’t fail the build if Python interpreters listed in tox.ini are missing. Thanks Alexandre Conrad for PR104.
  • fix #164: better traceback info in case of failing test commands. Thanks Marc Abramowitz for PR92.
  • support optional env variable substitution, thanks Morgan Fainberg for PR86.
  • limit python hashseed to 1024 on Windows to prevent possible memory errors. Thanks March Schlaich for the PR90.

1.7.1 (2014-03-28)

  • fix #162: don’t list python 2.5 as compatibiliy/supported
  • fix #158 and fix #155: windows/virtualenv properly works now: call virtualenv through “python -m virtualenv” with the same interpreter which invoked tox. Thanks Chris Withers, Ionel Maries Cristian.

1.7.0 (2014-01-29)

  • don’t lookup “pip-script” anymore but rather just “pip” on windows as this is a pip implementation detail and changed with pip-1.5. It might mean that tox-1.7 is not able to install a different pip version into a virtualenv anymore.
  • drop Python2.5 compatibility because it became too hard due to the setuptools-2.0 dropping support. tox now has no support for creating python2.5 based environments anymore and all internal special-handling has been removed.
  • merged PR81: new option –force-dep which allows to override tox.ini specified dependencies in setuptools-style. For example “–force-dep ‘django<1.6’” will make sure that any environment using “django” as a dependency will get the latest 1.5 release. Thanks Bruno Oliveria for the complete PR.
  • merged PR125: tox now sets “PYTHONHASHSEED” to a random value and offers a “–hashseed” option to repeat a test run with a specific seed. You can also use –hashsheed=noset to instruct tox to leave the value alone. Thanks Chris Jerdonek for all the work behind this.
  • fix #132: removing zip_safe setting (so it defaults to false) to allow installation of tox via easy_install/eggs. Thanks Jenisys.
  • fix #126: depend on virtualenv>=1.11.2 so that we can rely (hopefully) on a pip version which supports –pre. (tox by default uses to –pre). also merged in PR84 so that we now call “virtualenv” directly instead of looking up interpreters. Thanks Ionel Maries Cristian. This also fixes #140.
  • fix #130: you can now set install_command=easy_install {opts} {packages} and expect it to work for repeated tox runs (previously it only worked when always recreating). Thanks jenisys for precise reporting.
  • fix #129: tox now uses Popen(…, universal_newlines=True) to force creation of unicode stdout/stderr streams. fixes a problem on specific platform configs when creating virtualenvs with Python3.3. Thanks Jorgen Schäfer or investigation and solution sketch.
  • fix #128: enable full substitution in install_command, thanks for the PR to Ronald Evers
  • rework and simplify “commands” parsing and in particular posargs substitutions to avoid various win32/posix related quoting issues.
  • make sure that the –installpkg option trumps any usedevelop settings in tox.ini or
  • introduce –no-network to tox’s own test suite to skip tests requiring networks
  • introduce –sitepackages to force sitepackages=True in all environments.
  • fix #105 – don’t depend on an existing HOME directory from tox tests.

1.6.1 (2013-09-04)

  • fix #119: {envsitepackagesdir} is now correctly computed and has a better test to prevent regression.
  • fix #116: make 1.6 introduced behaviour of changing to a per-env HOME directory during install activities dependent on “–set-home” for now. Should re-establish the old behaviour when no option is given.
  • fix #118: correctly have two tests use realpath(). Thanks Barry Warsaw.
  • fix test runs on environments without a home directory (in this case we use toxinidir as the homedir)
  • fix #117: python2.5 fix: don’t use --insecure option because its very existence depends on presence of “ssl”. If you want to support python2.5/pip1.3.1 based test environments you need to install ssl and/or use PIP_INSECURE=1 through setenv. section.
  • fix #102: change to {toxinidir} when installing dependencies. this allows to use relative path like in “-rrequirements.txt”.

1.6.0 (2013-08-15)

  • fix #35: add new EXPERIMENTAL “install_command” testenv-option to configure the installation command with options for dep/pkg install. Thanks Carl Meyer for the PR and docs.
  • fix #91: python2.5 support by vendoring the virtualenv-1.9.1 script and forcing pip<1.4. Also the default [py25] environment modifies the default installer_command (new config option) to use pip without the “–pre” option which was introduced with pip-1.4 and is now required if you want to install non-stable releases. (tox defaults to install with “–pre” everywhere).
  • during installation of dependencies HOME is now set to a pseudo location ({envtmpdir}/pseudo-home). If an index url was specified a .pydistutils.cfg file will be written with an index_url setting so that packages defining setup_requires dependencies will not silently use your HOME-directory settings or https://pypi.python.org/pypi.
  • fix #1: empty setup files are properly detected, thanks Anthon van der Neuth
  • remove toxbootstrap.py for now because it is broken.
  • fix #109 and fix #111: multiple “-e” options are now combined (previously the last one would win). Thanks Anthon van der Neut.
  • add –result-json option to write out detailed per-venv information into a json report file to be used by upstream tools.
  • add new config options usedevelop and skipsdist as well as a command line option --develop to install the package-under-test in develop mode. thanks Monty Tailor for the PR.
  • always unset PYTHONDONTWRITEBYTE because newer setuptools doesn’t like it
  • if a HOMEDIR cannot be determined, use the toxinidir.
  • refactor interpreter information detection to live in new tox/interpreters.py file, tests in tests/test_interpreters.py.

1.5.0 (2013-06-22)

  • fix #104: use setuptools by default, instead of distribute, now that setuptools has distribute merged.
  • make sure test commands are searched first in the virtualenv
  • re-fix #2 - add whitelist_externals to be used in [testenv*] sections, allowing to avoid warnings for commands such as make, used from the commands value.
  • fix #97 - allow substitutions to reference from other sections (thanks Krisztian Fekete)
  • fix #92 - fix {envsitepackagesdir} to actually work again
  • show (test) command that is being executed, thanks Lukasz Balcerzak
  • re-license tox to MIT license
  • depend on virtualenv-1.9.1
  • rename README.txt to README.rst to make bitbucket happier

1.4.3 (2013-02-28)

  • use pip-script.py instead of pip.exe on win32 to avoid the lock exe file on execution issue (thanks Philip Thiem)
  • introduce -l|–listenv option to list configured environments (thanks Lukasz Balcerzak)
  • fix downloadcache determination to work according to docs: Only make pip use a download cache if PIP_DOWNLOAD_CACHE or a downloadcache=PATH testenv setting is present. (The ENV setting takes precedence)
  • fix #84 - pypy on windows creates a bin not a scripts venv directory (thanks Lukasz Balcerzak)
  • experimentally introduce –installpkg=PATH option to install a package rather than create/install an sdist package. This will still require and use tox.ini and tests from the current working dir (and not from the remote package).
  • substitute {envsitepackagesdir} with the package installation directory (closes #72) (thanks g2p)
  • issue #70 remove PYTHONDONTWRITEBYTECODE workaround now that virtualenv behaves properly (thanks g2p)
  • merged tox-quickstart command, contributed by Marc Abramowitz, which generates a default tox.ini after asking a few questions
  • fix #48 - win32 detection of pypy and other interpreters that are on PATH (thanks Gustavo Picon)
  • fix grouping of index servers, it is now done by name instead of indexserver url, allowing to use it to separate dependencies into groups even if using the same default indexserver.
  • look for “tox.ini” files in parent dirs of current dir (closes #34)
  • the “py” environment now by default uses the current interpreter (sys.executable) make tox’ own setup.py test execute tests with it (closes #46)
  • change tests to not rely on os.path.expanduser (closes #60), also make mock session return args[1:] for more precise checking (closes #61) thanks to Barry Warsaw for both.

1.4.2 (2012-07-20)

  • fix some tests which fail if /tmp is a symlink to some other place
  • “python setup.py test” now runs tox tests via tox :) also added an example on how to do it for your project.

1.4.1 (2012-07-03)

  • fix #41 better quoting on windows - you can now use “<” and “>” in deps specifications, thanks Chris Withers for reporting

1.4 (2012-06-13)

  • fix #26 - no warnings on absolute or relative specified paths for commands
  • fix #33 - commentchars are ignored in key-value settings allowing for specifying commands like: python -c “import sys ; print sys” which would formerly raise irritating errors because the “;” was considered a comment
  • tweak and improve reporting
  • refactor reporting and virtualenv manipulation to be more accessible from 3rd party tools
  • support value substitution from other sections with the {[section]key} syntax
  • fix #29 - correctly point to pytest explanation for importing modules fully qualified
  • fix #32 - use –system-site-packages and don’t pass –no-site-packages
  • add python3.3 to the default env list, so early adopters can test
  • drop python2.4 support (you can still have your tests run on
  • fix the links/checkout howtos in the docs python-2.4, just tox itself requires 2.5 or higher.

1.3 2011-12-21

  • fix: allow to specify wildcard filesystem paths when specifying dependencies such that tox searches for the highest version
  • fix issue #21: clear PIP_REQUIRES_VIRTUALENV which avoids pip installing to the wrong environment, thanks to bb’s streeter
  • make the install step honour a testenv’s setenv setting (thanks Ralf Schmitt)

1.2 2011-11-10

  • remove the virtualenv.py that was distributed with tox and depend on >=virtualenv-1.6.4 (possible now since the latter fixes a few bugs that the inlining tried to work around)
  • fix #10: work around UnicodeDecodeError when invoking pip (thanks Marc Abramowitz)
  • fix a problem with parsing {posargs} in tox commands (spotted by goodwill)
  • fix the warning check for commands to be installed in testenvironment (thanks Michael Foord for reporting)

1.1 (2011-07-08)

  • fix #5 - don’t require argparse for python versions that have it
  • fix #6 - recreate virtualenv if installing dependencies failed
  • fix #3 - fix example on frontpage
  • fix #2 - warn if a test command does not come from the test environment
  • fixed/enhanced: except for initial install always call “-U –no-deps” for installing the sdist package to ensure that a package gets upgraded even if its version number did not change. (reported on TIP mailing list and IRC)
  • inline virtualenv.py (1.6.1) script to avoid a number of issues, particularly failing to install python3 environments from a python2 virtualenv installation.
  • rework and enhance docs for display on readthedocs.org

1.0

  • move repository and toxbootstrap links to https://bitbucket.org/hpk42/tox
  • fix #7: introduce a “minversion” directive such that tox bails out if it does not have the correct version.
  • fix #24: introduce a way to set environment variables for for test commands (thanks Chris Rose)
  • fix #22: require virtualenv-1.6.1, obsoleting virtualenv5 (thanks Jannis Leidel) and making things work with pypy-1.5 and python3 more seamlessly
  • toxbootstrap.py (used by jenkins build slaves) now follows the latest release of virtualenv
  • fix #20: document format of URLs for specifying dependencies
  • fix #19: substitute Hudson for Jenkins everywhere following the renaming of the project. NOTE: if you used the special [tox:hudson] section it will now need to be named [tox:jenkins].
  • fix issue 23 / apply some ReST fixes
  • change the positional argument specifier to use {posargs:} syntax and fix issues #15 and #10 by refining the argument parsing method (Chris Rose)
  • remove use of inipkg lazy importing logic - the namespace/imports are anyway very small with tox.
  • fix a fspath related assertion to work with debian installs which uses symlinks
  • show path of the underlying virtualenv invocation and bootstrap virtualenv.py into a working subdir
  • added a CONTRIBUTORS file

0.9

  • fix pip-installation mixups by always unsetting PIP_RESPECT_VIRTUALENV (thanks Armin Ronacher)
  • #1: Add a toxbootstrap.py script for tox, thanks to Sridhar Ratnakumar
  • added support for working with different and multiple PyPI indexservers.
  • new option: -r|–recreate to force recreation of virtualenv
  • depend on py>=1.4.0 which does not contain or install the py.test anymore which is now a separate distribution “pytest”.
  • show logfile content if there is an error (makes CI output more readable)

0.8

  • work around a virtualenv limitation which crashes if PYTHONDONTWRITEBYTECODE is set.
  • run pip/easy installs from the environment log directory, avoids naming clashes between env names and dependencies (thanks ronny)
  • require a more recent version of py lib
  • refactor and refine config detection to work from a single file and to detect the case where a python installation overwrote an old one and resulted in a new executable. This invalidates the existing virtualenvironment now.
  • change all internal source to strip trailing whitespaces

0.7

  • use virtualenv5 (my own fork of virtualenv3) for now to create python3 environments, fixes a couple of issues and makes tox more likely to work with Python3 (on non-windows environments)
  • add sitepackages option for testenv sections so that environments can be created with access to globals (default is not to have access, i.e. create environments with --no-site-packages.
  • addressing #4: always prepend venv-path to PATH variable when calling subprocesses
  • fix #2: exit with proper non-zero return code if there were errors or test failures.
  • added unittest2 examples contributed by Michael Foord
  • only allow ‘True’ or ‘False’ for boolean config values (lowercase / uppercase is irrelevant)
  • recreate virtualenv on changed configurations

0.6

  • fix OSX related bugs that could cause the caller’s environment to get screwed (sorry). tox was using the same file as virtualenv for tracking the Python executable dependency and there also was confusion wrt links. this should be fixed now.
  • fix long description, thanks Michael Foord

0.5

  • initial release

CONTRIBUTING

Contributions are highly welcomed and appreciated. Every little help counts, so do not hesitate! If you like tox, also share some love on Twitter or in your blog posts.

Feature requests and feedback

We’d also like to hear about your propositions and suggestions. Feel free to submit them as issues and:

  • Explain in detail how they should work.
  • Keep the scope as narrow as possible. This will make it easier to implement.

Report bugs

Report bugs for tox in the issue tracker.

If you are reporting a bug, please include:

  • Your operating system name and version.
  • Any details about your local setup that might be helpful in troubleshooting, specifically the Python interpreter version, installed libraries, and tox version.
  • Detailed steps to reproduce the bug, or - even better, a n xfaling test reproduces the bug

If you can write a demonstration test that currently fails but should pass (xfail), that is a very useful commit to make as well, even if you cannot fix the bug itself (e.g. something like this in test_config

Fix bugs

Look through the GitHub issues for bugs. Here is a filter you can use: https://github.com/tox-dev/tox/labels/bug

Don’t forget to check the issue trackers of your favourite plugins, too!

Implement features

Look through the GitHub issues for enhancements. Here is a filter you can use: https://github.com/tox-dev/tox/labels/enhancement

Write documentation

tox could always use more documentation. What exactly is needed?

  • More complementary documentation. Have you perhaps found something unclear?
  • Docstrings. There can never be too many of them.
  • Blog posts, articles and such – they’re all very appreciated.

You can also edit documentation files directly in the GitHub web interface, without using a local copy. This can be convenient for small fixes.

Note

Build the documentation locally with the following command:

$ tox -e docs

The built documentation should be available in the doc/_build/.

Preparing Pull Requests

Short version

  1. Fork the repository

  2. Make your changes.

  3. open a pull request targeting the master branch.

  4. Follow PEP-8. There’s a tox command to help fixing it: tox -e fix-lint. You can also add a pre commit hook to your local clone to run the style checks and fixes (see hint after running tox -e fix-lint)

  5. Tests for tox are (obvioulsy) run using tox:

    tox -e fix-lint,py27,py36
    

    The test environments above are usually enough to cover most cases locally.

  6. Consider the checklist in the pull request form

Long version

What is a “pull request”? It informs the project’s core developers about the changes you want to review and merge. Pull requests are stored on GitHub servers. Once you send a pull request, we can discuss its potential modifications and even add more commits to it later on. There’s an excellent tutorial on how Pull Requests work in the GitHub Help Center.

Here is a simple overview, with tox-specific bits:

  1. Fork the tox GitHub repository. It’s fine to use tox as your fork repository name because it will live under your user.

  2. Clone your fork locally using git and create a branch:

    $ git clone git@github.com:YOUR_GITHUB_USERNAME/tox.git
    $ cd tox
    # now, to fix a bug create your own branch off "master":
    
        $ git checkout -b your-bugfix-branch-name master
    
    # or to instead add a feature create your own branch off "features":
    
        $ git checkout -b your-feature-branch-name features
    

    If you need some help with Git, follow this quick start guide: https://git.wiki.kernel.org/index.php/QuickStart

  3. Install tox in development mode

    Of course tox is used to run all the tests of itself:

    $ cd </path/to/your/tox/clone>
    $ pip install [-e] .
    
  4. Run all the tests

    You need to have Python 2.7 and 3.6 available in your system. Now running tests is as simple as issuing this command:

    $ tox -e fix-lint,py27,py36
    

    This command will run tests via the “tox” tool against Python 2.7 and 3.6 and also perform style checks with some automatic fixes.

  5. You can now edit your local working copy. Please follow PEP-8.

    You can now make the changes you want and run the tests again as necessary.

    $ tox -e py27 – –pdb

    Or to only run tests in a particular test module on Python 3.6:

    $ tox -e py36 -- testing/test_config.py
    

    You can also use the dev environment:

    $ tox -e dev

    To get information about all environements, type:

    $ tox -av

  6. Commit and push once your tests pass and you are happy with your change(s):

    $ git commit -a -m "<commit message>"
    $ git push -u
    
  7. submit a pull request through the GitHub website and and consider the checklist in the pull request form:

    head-fork: YOUR_GITHUB_USERNAME/tox
    compare: your-branch-name
    
    base-fork: tox-dev/tox
    base: master
    

Submitting plugins to tox-dev

tox development of the core, some plugins and support code happens in repositories living under the tox-dev organisation:

All tox-dev team members have write access to all contained repositories. tox core and plugins are generally developed using pull requests to respective repositories.

The objectives of the tox-dev organisation are:

  • Having a central location for popular tox plugins
  • Sharing some of the maintenance responsibility (in case a maintainer no longer wishes to maintain a plugin)

You can submit your plugin by opening an issue requesting to add you as a member of tox-dev to be able to integrate the plugin. As a member of the or you can then transfer the plugin yourself.

The plugin must have the following:

  • PyPI presence with a setup.py that contains a license, tox- prefixed name, version number, authors, short and long description.
  • a tox.ini for running tests using tox.
  • a README.txt describing how to use the plugin and on which platforms it runs.
  • a LICENSE.txt file or equivalent containing the licensing information, with matching info in setup.py.
  • an issue tracker for bug reports and enhancement requests.
  • a CHANGELOG (e.g see http://keepachangelog.com)

If no contributor strongly objects, the repository can then be transferred to the tox-dev organisation. For details see about repository transfers

Memebrs of the tox organization have write access to all projects. We recommend that each plugin has at least three people who have the right to release to PyPI.

Repository owners can rest assured that no tox-dev administrator will ever make releases of your repository or take ownership in any way, except in rare cases where someone becomes unresponsive after months of contact attempts. As stated, the objective is to share maintenance and avoid “plugin-abandon”.