mirror of
https://github.com/enpaul/tox-poetry-installer.git
synced 2025-11-04 07:46:06 +00:00
Overhaul basic usage documentation to improve clarity
Add intra document links Update to document new/updated features Add badge for downloads per month Update badge order
This commit is contained in:
330
README.md
330
README.md
@@ -6,22 +6,23 @@ dependencies to be installed using [Poetry](https://python-poetry.org/) from its
|
|||||||
⚠️ **This project is alpha software and should not be used in production environments** ⚠️
|
⚠️ **This project is alpha software and should not be used in production environments** ⚠️
|
||||||
|
|
||||||
[](https://github.com/enpaul/tox-poetry-installer/actions)
|
[](https://github.com/enpaul/tox-poetry-installer/actions)
|
||||||
[](https://opensource.org/licenses/MIT)
|
|
||||||
[](https://pypi.org/project/tox-poetry-installer/)
|
[](https://pypi.org/project/tox-poetry-installer/)
|
||||||
|
[](https://libraries.io/pypi/tox-poetry-installer)
|
||||||
|
[](https://opensource.org/licenses/MIT)
|
||||||
[](https://www.python.org)
|
[](https://www.python.org)
|
||||||
[](https://github.com/psf/black)
|
[](https://github.com/psf/black)
|
||||||
|
|
||||||
**Documentation**
|
**Documentation**
|
||||||
|
|
||||||
* [Installation](#installation)
|
* [Introduction](#introduction)
|
||||||
* [Quick Start](#quick-start)
|
* [Install](#install)
|
||||||
* [Plugin Usage](#reference-and-usage)
|
* [Quick Start](#quick-start)
|
||||||
|
* [Why would I use this?](#why-would-i-use-this) (What problems does this solve?)
|
||||||
* [Reference](#reference)
|
* [Reference](#reference)
|
||||||
* [Configuration Options](#configuration-options)
|
* [Configuration Options](#configuration-options)
|
||||||
* [Command-line Arguments](#command-line-arguments)
|
* [Command-line Arguments](#command-line-arguments)
|
||||||
* [Errors](#errors)
|
* [Errors](#errors)
|
||||||
* [Advanced Usage](#advanced-usage)
|
* [Advanced Usage](#advanced-usage)
|
||||||
* [Why would I use this?](#why-would-i-use-this) (What problems does this solve?)
|
|
||||||
* [Developing](#developing)
|
* [Developing](#developing)
|
||||||
* [Contributing](#contributing)
|
* [Contributing](#contributing)
|
||||||
* [Roadmap](#roadmap)
|
* [Roadmap](#roadmap)
|
||||||
@@ -38,44 +39,88 @@ Similar projects:
|
|||||||
* [Poetry Tox Plugin](https://github.com/tkukushkin/tox-poetry)
|
* [Poetry Tox Plugin](https://github.com/tkukushkin/tox-poetry)
|
||||||
|
|
||||||
|
|
||||||
## Installation
|
## Introduction
|
||||||
|
|
||||||
Add the plugin as a development dependency of a Poetry project:
|
This is a plugin to unify two great projects in the Python ecosystem: the
|
||||||
|
[Tox](https://tox.readthedocs.io/en/latest/) automation project and the
|
||||||
|
[Poetry](https://python-poetry.org) project/dependency manager. Specifically it allows
|
||||||
|
the repeatable dependency resolution and installation tools that Poetry uses to benefit
|
||||||
|
the isolated environments that Tox uses to run automated tests. The motivation to write
|
||||||
|
this plugin came from a need for a single source of truth for the versions of all
|
||||||
|
packages that should be installed to an environment.
|
||||||
|
|
||||||
```
|
When in use this plugin will allow a Tox environment to install its required
|
||||||
~ $: poetry add tox-poetry-installer --dev
|
dependencies using the versions specified in the Poetry lockfile. This eliminates
|
||||||
|
needing to specify package versions in multiple places as well as ensures that the Tox
|
||||||
|
environment has the exact same versions of a given package as the Poetry environment.
|
||||||
|
This reduces (or hopefully eliminates) hard to debug problems caused by subtle
|
||||||
|
differences in the dependency graph of the active development environment (the one managed
|
||||||
|
by Poetry) and the automated test environment(s) created by Tox.
|
||||||
|
|
||||||
|
To learn more about the problems this plugin aims to solve jump ahead to
|
||||||
|
[What problems does this solve?](#what-problems-does-this-solve).
|
||||||
|
Otherwise keep reading to get started.
|
||||||
|
|
||||||
|
### Install
|
||||||
|
|
||||||
|
The recommended way to install the plugin is to add it to a project's `pyproject.toml`
|
||||||
|
and lockfile using Poetry:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
poetry add tox-poetry-installer[poetry] --dev
|
||||||
```
|
```
|
||||||
|
|
||||||
Confirm that the plugin is installed, and Tox recognizes it, by checking the Tox version:
|
**WARNING:** The below installation methods are vulnerable to the
|
||||||
|
[transient dependency issues this plugin aims to avoid](#why-would-i-use-this). It is
|
||||||
|
always recommended to install dependencies using Poetry whenever possible.
|
||||||
|
|
||||||
|
The plugin can also be installed with pip directly, though it is recommended to always
|
||||||
|
install to a virtual environment and pin to a specific version:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
source my-venv/bi/activate
|
||||||
|
pip install tox-poetry-installer[poetry] == 0.6.0
|
||||||
|
```
|
||||||
|
|
||||||
|
The plugin can also be installed using the Tox
|
||||||
|
[`requires`]((https://tox.readthedocs.io/en/latest/config.html#conf-requires))
|
||||||
|
configuration option. Note however that dependencies installed via the `requires` option
|
||||||
|
are not handled by the plugin and will be installed the same way as a `pip install ...`
|
||||||
|
above. For this reason it is also recommended to always pin to a specific version when
|
||||||
|
using this installation method:
|
||||||
|
|
||||||
|
```ini
|
||||||
|
# tox.ini
|
||||||
|
[tox]
|
||||||
|
requires
|
||||||
|
tox-poetry-installer[poetry] == 0.6.0
|
||||||
|
```
|
||||||
|
|
||||||
|
Check that the plugin is registered by checking the Tox version:
|
||||||
|
|
||||||
```
|
```
|
||||||
~ $: poetry run tox --version
|
~ $: poetry run tox --version
|
||||||
3.20.0 imported from .venv/lib64/python3.8/site-packages/tox/__init__.py
|
3.20.0 imported from .venv/lib64/python3.8/site-packages/tox/__init__.py
|
||||||
registered plugins:
|
registered plugins:
|
||||||
tox-poetry-installer-0.5.0 at .venv/lib64/python3.8/site-packages/tox_poetry_installer.py
|
tox-poetry-installer-0.6.0 at .venv/lib64/python3.8/site-packages/tox_poetry_installer.py
|
||||||
```
|
```
|
||||||
|
|
||||||
If using Pip, ensure that the plugin is installed to the same environment as Tox:
|
**NOTE:** Installing the `tox-poetry-installer[poetry]` extra will add the `poetry`
|
||||||
|
package as a managed environment dependency which can cause problems when the Poetry
|
||||||
|
installation is externally managed (such as in a CI or container environment). See
|
||||||
|
[Advanced Usage](#installing-alongside-an-existing-poetry-installation) for more
|
||||||
|
information on this use case.
|
||||||
|
|
||||||
```
|
### Quick Start
|
||||||
# Calling the virtualenv's 'pip' binary directly will cause pip to install to that virtualenv
|
|
||||||
~ $: /path/to/my/automation/virtualenv/bin/pip install tox
|
|
||||||
~ $: /path/to/my/automation/virtualenv/bin/pip install tox-poetry-installer
|
|
||||||
```
|
|
||||||
|
|
||||||
**Note:** While it is possible to install this plugin using Tox's
|
Before making any changes to `tox.ini` the project is already benefiting from having
|
||||||
[`requires`](https://tox.readthedocs.io/en/latest/config.html#conf-requires)
|
the plugin installed: all dependencies of the root project package are installed using
|
||||||
configuration option, it is not recommended. Dependencies from the `requires` option are
|
the Poetry backend to all Tox environments that install the root package without any
|
||||||
installed using the default Tox installation backend which opens up the
|
configuration changes.
|
||||||
[possibility of transient dependency problems](#why-would-i-use-this) in your automation
|
|
||||||
environment.
|
|
||||||
|
|
||||||
|
To add dependencies from the lockfile to a Tox environment, add the option
|
||||||
## Quick Start
|
[`locked_deps`](#locked_deps) to the environment configuration and list names of
|
||||||
|
dependencies (with no version specifier) under it:
|
||||||
To add dependencies from the lockfile to a Tox environment, add the option `locked_deps`
|
|
||||||
to the environment configuration and list names of dependencies (with no version
|
|
||||||
specifier) under it:
|
|
||||||
|
|
||||||
```ini
|
```ini
|
||||||
[testenv]
|
[testenv]
|
||||||
@@ -87,9 +132,9 @@ locked_deps =
|
|||||||
commands = ...
|
commands = ...
|
||||||
```
|
```
|
||||||
|
|
||||||
The standard `deps` option can be used in parallel with the `locked_deps` option to
|
The standard [`deps`](https://tox.readthedocs.io/en/latest/config.html#conf-deps) option
|
||||||
install unlocked dependencies (dependencies not in the lockfile) alongside locked
|
can be used in parallel with the `locked_deps` option to install unlocked dependencies
|
||||||
dependencies:
|
(dependencies not in the lockfile) alongside locked dependencies:
|
||||||
|
|
||||||
```ini
|
```ini
|
||||||
[testenv]
|
[testenv]
|
||||||
@@ -105,10 +150,105 @@ commands = ...
|
|||||||
```
|
```
|
||||||
|
|
||||||
Alternatively, to quickly install all Poetry dev-dependencies to a Tox environment, add the
|
Alternatively, to quickly install all Poetry dev-dependencies to a Tox environment, add the
|
||||||
`install_dev_deps = true` option to the environment configuration.
|
[`install_dev_deps`](#install_dev_deps) option to the environment configuration:
|
||||||
|
|
||||||
**Note:** Regardless of the settings outlined above, all dependencies of the project package (the
|
```ini
|
||||||
one Tox is testing) will always be installed from the lockfile.
|
[testenv]
|
||||||
|
description = Some very cool tests
|
||||||
|
install_dev_deps = true
|
||||||
|
```
|
||||||
|
|
||||||
|
See the [Plugin Usage](#plugin-usage) section for more details on available
|
||||||
|
configuration options and the [Advanced Usage](#advanced-usage) section for some
|
||||||
|
unusual use cases.
|
||||||
|
|
||||||
|
### Why would I use this?
|
||||||
|
|
||||||
|
**The Problem**
|
||||||
|
|
||||||
|
By default Tox uses [Pip](https://docs.python.org/3/tutorial/venv.html) to install the
|
||||||
|
[PEP-508](https://www.python.org/dev/peps/pep-0508/) compliant dependencies to a test
|
||||||
|
environment. This plugin extends the default Tox dependency installation behavior to
|
||||||
|
support installing dependencies using a Poetry-based installation method that makes use
|
||||||
|
of the dependency metadata from Poetry's lockfile.
|
||||||
|
|
||||||
|
Environment dependencies for a Tox environment are usually specified in PEP-508 format, like
|
||||||
|
the below example:
|
||||||
|
|
||||||
|
```ini
|
||||||
|
[testenv]
|
||||||
|
description = Some very cool tests
|
||||||
|
deps =
|
||||||
|
foo == 1.2.3
|
||||||
|
bar >=1.3,<2.0
|
||||||
|
baz
|
||||||
|
```
|
||||||
|
|
||||||
|
Let's assume these dependencies are also useful during development, so they can be added to the
|
||||||
|
Poetry environment using this command:
|
||||||
|
|
||||||
|
```
|
||||||
|
poetry add --dev \
|
||||||
|
foo==1.2.3 \
|
||||||
|
bar>=1.3,<2.0 \
|
||||||
|
baz
|
||||||
|
```
|
||||||
|
|
||||||
|
However there is a potential problem that could arise from each of these environment
|
||||||
|
dependencies that would _only_ appear in the Tox environment and not in the Poetry
|
||||||
|
environment in use by a developer:
|
||||||
|
|
||||||
|
* **The `foo` dependency is pinned to a specific version:** let's imagine a security
|
||||||
|
vulnerability is discovered in `foo` and the maintainers release version `1.2.4` to fix
|
||||||
|
it. A developer can run `poetry remove foo` and then `poetry add foo^1.2` to get the new
|
||||||
|
version, but the Tox environment is left unchanged. The development environment, as defined by
|
||||||
|
the lockfile, is now patched against the vulnerability but the Tox environment is not.
|
||||||
|
|
||||||
|
* **The `bar` dependency specifies a dynamic range:** a dynamic range allows a range of
|
||||||
|
versions to be installed, but the lockfile will have an exact version specified so that
|
||||||
|
the Poetry environment is reproducible; this allows versions to be updated with
|
||||||
|
`poetry update` rather than with the `remove` and `add` commands used above. If the
|
||||||
|
maintainers of `bar` release version `1.6.0` then the Tox environment will install it
|
||||||
|
because it is valid for the specified version range. Meanwhile the Poetry environment will
|
||||||
|
continue to install the version from the lockfile until `poetry update bar` explicitly
|
||||||
|
updates it. The development environment is now has a different version of `bar` than the Tox
|
||||||
|
environment.
|
||||||
|
|
||||||
|
* **The `baz` dependency is unpinned:** unpinned dependencies are
|
||||||
|
[generally a bad idea](https://python-poetry.org/docs/faq/#why-are-unbound-version-constraints-a-bad-idea),
|
||||||
|
but here it can cause real problems. Poetry will interpret an unbound dependency using
|
||||||
|
[the carrot requirement](https://python-poetry.org/docs/dependency-specification/#caret-requirements)
|
||||||
|
but Pip (via Tox) will interpret it as a wildcard. If the latest version of `baz` is `1.0.0`
|
||||||
|
then `poetry add baz` will result in a constraint of `baz>=1.0.0,<2.0.0` while the Tox
|
||||||
|
environment will have a constraint of `baz==*`. The Tox environment can now install an
|
||||||
|
incompatible version of `baz` and any errors that causes cannot be replicated using `poetry update`.
|
||||||
|
|
||||||
|
All of these problems can apply not only to the dependencies specified for a Tox environment,
|
||||||
|
but also to the dependencies of those dependencies, those dependencies' dependencies, and so on.
|
||||||
|
|
||||||
|
**The Solution**
|
||||||
|
|
||||||
|
This plugin allows dependencies specified in Tox environment take their version directly from
|
||||||
|
the Poetry lockfile without needing an independent version to be specified in the Tox
|
||||||
|
environment configuration. The modified version of the example environment given below appears
|
||||||
|
less stable than the one presented above because it does not specify any versions for its
|
||||||
|
dependencies:
|
||||||
|
|
||||||
|
```ini
|
||||||
|
[testenv]
|
||||||
|
description = Some very cool tests
|
||||||
|
require_locked_deps = true
|
||||||
|
locked_deps =
|
||||||
|
foo
|
||||||
|
bar
|
||||||
|
baz
|
||||||
|
```
|
||||||
|
|
||||||
|
However with the `tox-poetry-installer` plugin installed Tox will install these
|
||||||
|
dependencies from the Poetry lockfile so that the version installed to the Tox
|
||||||
|
environment exactly matches the version Poetry is managing. When `poetry update` updates
|
||||||
|
the lockfile with new versions of these dependencies, Tox will automatically install
|
||||||
|
these new versions without needing any changes to the configuration.
|
||||||
|
|
||||||
|
|
||||||
## Reference
|
## Reference
|
||||||
@@ -337,126 +477,6 @@ python -c '\
|
|||||||
the [`--require-poetry`](#--require-poetry) option.
|
the [`--require-poetry`](#--require-poetry) option.
|
||||||
|
|
||||||
|
|
||||||
## Why would I use this?
|
|
||||||
|
|
||||||
**Introduction**
|
|
||||||
|
|
||||||
The lockfile is a file generated by a package manager for a project that records what
|
|
||||||
dependencies are installed, the versions of those dependencies, and any additional metadata that
|
|
||||||
the package manager needs to recreate the local project environment. This allows developers
|
|
||||||
to have confidence that a bug they are encountering that may be caused by one of their
|
|
||||||
dependencies will be reproducible on another device. In addition, installing a project
|
|
||||||
environment from a lockfile gives confidence that automated systems running tests or performing
|
|
||||||
builds are using the same environment as a developer.
|
|
||||||
|
|
||||||
[Poetry](https://python-poetry.org/) is a project dependency manager for Python projects, and
|
|
||||||
so it creates and manages a lockfile so that its users can benefit from all the features
|
|
||||||
described above. [Tox](https://tox.readthedocs.io/en/latest/#what-is-tox) is an automation tool
|
|
||||||
that allows Python developers to run tests suites, perform builds, and automate tasks within
|
|
||||||
self-contained [Python virtual environments](https://docs.python.org/3/tutorial/venv.html).
|
|
||||||
To make these environments useful Tox supports installing dependencies in each environment.
|
|
||||||
However, since these environments are created on the fly and Tox does not maintain a lockfile,
|
|
||||||
there can be subtle differences between the dependencies a developer is using and the
|
|
||||||
dependencies Tox uses.
|
|
||||||
|
|
||||||
This is where this plugin comes into play.
|
|
||||||
|
|
||||||
By default Tox uses [Pip](https://docs.python.org/3/tutorial/venv.html) to install the
|
|
||||||
PEP-508 compliant dependencies to a test environment. This plugin extends the default
|
|
||||||
Tox dependency installation behavior to support installing dependencies using a Poetry-based
|
|
||||||
installation method that makes use of the dependency metadata from Poetry's lockfile.
|
|
||||||
|
|
||||||
**The Problem**
|
|
||||||
|
|
||||||
Environment dependencies for a Tox environment are usually specified in PEP-508 format, like
|
|
||||||
the below example:
|
|
||||||
|
|
||||||
```ini
|
|
||||||
# from tox.ini
|
|
||||||
...
|
|
||||||
|
|
||||||
[testenv]
|
|
||||||
description = Some very cool tests
|
|
||||||
deps =
|
|
||||||
foo == 1.2.3
|
|
||||||
bar >=1.3,<2.0
|
|
||||||
baz
|
|
||||||
|
|
||||||
...
|
|
||||||
```
|
|
||||||
|
|
||||||
Let's assume these dependencies are also useful during development, so they can be added to the
|
|
||||||
Poetry environment using this command:
|
|
||||||
|
|
||||||
```
|
|
||||||
poetry add --dev \
|
|
||||||
foo==1.2.3 \
|
|
||||||
bar>=1.3,<2.0 \
|
|
||||||
baz
|
|
||||||
```
|
|
||||||
|
|
||||||
However there is a potential problem that could arise from each of these environment
|
|
||||||
dependencies that would _only_ appear in the Tox environment and not in the Poetry
|
|
||||||
environment in use by a developer:
|
|
||||||
|
|
||||||
* **The `foo` dependency is pinned to a specific version:** let's imagine a security
|
|
||||||
vulnerability is discovered in `foo` and the maintainers release version `1.2.4` to fix
|
|
||||||
it. A developer can run `poetry remove foo` and then `poetry add foo^1.2` to get the new
|
|
||||||
version, but the Tox environment is left unchanged. The development environment, as defined by
|
|
||||||
the lockfile, is now patched against the vulnerability but the Tox environment is not.
|
|
||||||
|
|
||||||
* **The `bar` dependency specifies a dynamic range:** a dynamic range allows a range of
|
|
||||||
versions to be installed, but the lockfile will have an exact version specified so that
|
|
||||||
the Poetry environment is reproducible; this allows versions to be updated with
|
|
||||||
`poetry update` rather than with the `remove` and `add` commands used above. If the
|
|
||||||
maintainers of `bar` release version `1.6.0` then the Tox environment will install it
|
|
||||||
because it is valid for the specified version range. Meanwhile the Poetry environment will
|
|
||||||
continue to install the version from the lockfile until `poetry update bar` explicitly
|
|
||||||
updates it. The development environment is now has a different version of `bar` than the Tox
|
|
||||||
environment.
|
|
||||||
|
|
||||||
* **The `baz` dependency is unpinned:** unpinned dependencies are
|
|
||||||
[generally a bad idea](https://python-poetry.org/docs/faq/#why-are-unbound-version-constraints-a-bad-idea),
|
|
||||||
but here it can cause real problems. Poetry will interpret an unbound dependency using
|
|
||||||
[the carrot requirement](https://python-poetry.org/docs/dependency-specification/#caret-requirements)
|
|
||||||
but Pip (via Tox) will interpret it as a wildcard. If the latest version of `baz` is `1.0.0`
|
|
||||||
then `poetry add baz` will result in a constraint of `baz>=1.0.0,<2.0.0` while the Tox
|
|
||||||
environment will have a constraint of `baz==*`. The Tox environment can now install an
|
|
||||||
incompatible version of `baz` and any errors that causes cannot be replicated using `poetry update`.
|
|
||||||
|
|
||||||
All of these problems can apply not only to the dependencies specified for a Tox environment,
|
|
||||||
but also to the dependencies of those dependencies, those dependencies' dependencies, and so on.
|
|
||||||
|
|
||||||
**The Solution**
|
|
||||||
|
|
||||||
This plugin allows dependencies specified in Tox environment take their version directly from
|
|
||||||
the Poetry lockfile without needing an independent version to be specified in the Tox
|
|
||||||
environment configuration. The modified version of the example environment given below appears
|
|
||||||
less stable than the one presented above because it does not specify any versions for its
|
|
||||||
dependencies:
|
|
||||||
|
|
||||||
```ini
|
|
||||||
# from tox.ini
|
|
||||||
...
|
|
||||||
|
|
||||||
[testenv]
|
|
||||||
description = Some very cool tests
|
|
||||||
require_locked_deps = true
|
|
||||||
locked_deps =
|
|
||||||
foo
|
|
||||||
bar
|
|
||||||
baz
|
|
||||||
|
|
||||||
...
|
|
||||||
```
|
|
||||||
|
|
||||||
However with the `tox-poetry-installer` plugin installed the `require_locked_deps = true`
|
|
||||||
setting means that Tox will install these dependencies from the Poetry lockfile so that the
|
|
||||||
version installed to the Tox environment exactly matches the version Poetry is managing. When
|
|
||||||
`poetry update` updates the lockfile with new versions of these dependencies, Tox will
|
|
||||||
automatically install these new versions without needing any changes to the configuration.
|
|
||||||
|
|
||||||
|
|
||||||
## Developing
|
## Developing
|
||||||
|
|
||||||
This project requires a developer to have Poetry version 1.0+ installed on their workstation, see
|
This project requires a developer to have Poetry version 1.0+ installed on their workstation, see
|
||||||
|
|||||||
Reference in New Issue
Block a user