QA checks and warnings

This section explains Python-related QA checks and the resulting QA warnings that can be output while running the package manager or related tooling.

Improved QA warning reporting in Portage

Normally, Portage outputs QA warnings at specific phases of the build process. They are usually interspersed with other verbose output, and they are easy to miss, especially when building multiple packages in a single batch.

To make them easier to catch, Portage’s elog system can be used to repeat all the QA warnings once emerge exits. The required “echo” module is already enabled by default, however it skips QA warnings by default. To change that, set in your make.conf:

PORTAGE_ELOG_CLASSES="log warn error qa"

For more information on using Portage’s elog system, please refer to make.conf.example included in the Portage distribution.

Stray top-level files in site-packages

distutils-r1 checks for the common mistake of installing unexpected files that are installed top-level into the site-packages directory. An example error due to that looks like the following:

* The following unexpected files/directories were found top-level
* in the site-packages directory:
*
*   /usr/lib/python3.10/site-packages/README.md
*   /usr/lib/python3.10/site-packages/LICENSE
*   /usr/lib/python3.10/site-packages/CHANGELOG
*
* This is most likely a bug in the build system.  More information
* can be found in the Python Guide:
* https://projects.gentoo.org/python/guide/qawarn.html#stray-top-level-files-in-site-packages

In general, it is desirable to prepare a fix for the build system and submit it upstream. However, it is acceptable to remove the files locally in the ebuild while waiting for a release with the fix.

The subsequent sections describe the common causes and the suggested fixes.

Example for test packages installed by setuptools

Many packages using the setuptools build system utilize the convenient find_packages() method to locate the Python sources. In some cases, this method also wrongly grabs top-level test directories or other files that were not intended to be installed.

For example, the following invocation will install everything that looks like a Python package from the source tree:

setup(
    packages=find_packages())

The correct fix for this problem is to add an exclude parameter that restricts the installed package list, for example:

setup(
    packages=find_packages(exclude=["tests", "tests.*"]))

Note that if the top-level tests package has any subpackages, both tests and tests.* need to be listed.

If setup.cfg is used instead, the excludes are specified as follows:

[options.packages.find]
exclude =
    tests
    tests.*

If pyproject.toml is used:

[tool.setuptools.packages.find]
exclude = [
    "tests",
    "tests.*",
]

For reference, see custom discovery in setuptools documentation.

Documentation files installed by Poetry

It is a relatively common problem that packages using the Poetry build system are installing documentation files (such as README) to the site-packages directory. This is because of incorrect include use in pyproject.toml. For example, consider the following configuration:

include = [
    "CHANGELOG",
    "README.md",
    "LICENSE"
]

The author meant to include these files in the source distribution packages. However, the include key applies to wheels as well, effectively including them in files installed into site-packages.

To fix that, you need to specify file formats explicitly, for every entry:

include = [
    { path = "CHANGELOG", format = "sdist" },
    { path = "README.md", format = "sdist" },
    { path = "LICENSE", format = "sdist" },
]

For reference, see include and exclude in Poetry documentation.

Deprecated PEP 517 backends

flit.buildapi

Some packages are still found using the historical flit build backend. Their pyproject.toml files contain a section similar to the following:

[build-system]
requires = ["flit"]
build-backend = "flit.buildapi"

This backend requires installing the complete flit package manager. Instead, the package should be fixed upstream to use flit_core per flit build system section documentation instead:

[build-system]
requires = ["flit_core"]
build-backend = "flit_core.buildapi"

flit_core produces identical artifacts to flit. At the same time, it reduces the build-time dependency footprint and therefore makes isolated PEP 517 builds faster.

poetry.masonry.api

A similar problem applies to the packages using poetry. The respective pyproject.toml files contain:

[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"

Instead, the lightweight poetry-core module should be used per poetry PEP-517 documentation:

[build-system]
requires = ["poetry_core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

poetry-core produces identical artifacts to poetry. It has smaller dependency footprint and makes isolated builds much faster.

setuptools.build_meta:__legacy__

Some packages using setuptools specify the following:

[build-system]
requires = ["setuptools>=40.8.0", "wheel"]
build-backend = "setuptools.build_meta:__legacy__"

This is incorrect, as the legacy backend is intended to be used only as an implicit fallback. All packages should be using the regular backend instead:

[build-system]
requires = ["setuptools>=40.8.0"]
build-backend = "setuptools.build_meta"

Please also note that the wheel package should not be listed as a dependency, as it is an implementation detail and it was always implicitly returned by the backend. Unfortunately, due to prolonged documentation error, a very large number of packages still specifies it, and other packages tend to copy that mistake.