Don't run regen-sbom in regen-all - it requires internet access

Bug report

Bug description:

make regen-all should be possible to run when building without the internet access. regen-sbom hits PyPI to determine pip's metadata and exits if not successful, so it shouldn't be a part of the regen-all target.

In previous cases like this, the problematic targets were removed (#112120)

CPython versions tested on:

3.13

Operating systems tested on:

Linux

Linked PRs

gh-114240: Skip verifying pip SBOM entry against PyPI if offline outside CI #114255

Note that make regen-configure has required internet access for quite a while; this is nothing new.

@befeleme

@befeleme Apologies for the added friction! I would like to keep regen-sbom as a part of regen-all, I can make the generate_sbom.py script more resilient in the case of no internet connection and reuse pip's SBOM entry in that case.

I've implemented my proposal in this PR: #114255

Note that make regen-configure has required internet access for quite a while; this is nothing new.

It is, but it's not a part of make regen-all target, see: https://github.com/python/cpython/blob/main/Makefile.pre.in#L1370

00418: Don't generate sbom in make regen-all

This is still not working as expected. When the build happens offline:

[cpython (main)]$ LANG=C.utf-8 make regen-sbom 
python3.13 ./Tools/build/generate_sbom.py
Packages defined in SBOM tool don't match those defined in SBOM file: {'libb2', 'pip', 'macholib', 'msgpack', 'certifi', 'resolvelib', 'tenacity', 'pyparsing', 'chardet', 'webencodings', 'packaging', 'expat', 'urllib3', 'six', 'hacl-star', 'rich', 'platformdirs', 'idna', 'typing_extensions', 'requests', 'distlib', 'tomli', 'mpdecimal', 'distro', 'truststore', 'pygments', 'colorama', 'cachecontrol', 'setuptools', 'pyproject-hooks'}, {'mpdecimal', 'libb2', 'hacl-star', 'pip', 'expat', 'macholib'}
See 'https://devguide.python.org/developer-workflow/sbom' for more information.
make: *** [Makefile:2842: regen-sbom] Error 1

This behavior was released in Python 3.12.2.

The problem is from https://github.com/python/cpython/pull/114450/files#r1482058937

@befeleme

To clarify ours (= Fedora, i.e. including @befeleme) use case:

we run the builds offline
we (usually) remove the ensurepip pip wheel and use our own wheel from /usr/share/python-wheels

That combination indeed works, as was confirmed by @befeleme in #114450 (comment)

However, we sometimes don't remove the ensurepip pip wheel. E.g. when we bootstrap new Python version or when the particular pip wheel version for a Fedora release is too old (or too new) for the given Python version. This happens for example with Python 3.12 on Fedora Linux 38 (our oldest still supported release). When we tried to update to 3.12.2, we had this combination:

we run the builds offline
we keep the ensurepip pip wheel

That is when we have experienced the error.

The same problem will likely happen when we try to update to 3.13.0a4 in Fedora Linux 38, unless fixed before the release.

@hroncok

Apologies @hroncok, this would get fixed by moving the entire pip discovery process into the python/release-tools repository, something that I want to do for other reasons too.

I've created this issue to track that work.

00418: Don't generate sbom in make regen-all

Navigation Menu

Repository navigation

Don't run regen-sbom in regen-all - it requires internet access #114240

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Don't run regen-sbom in regen-all - it requires internet access #114240

Description

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

Activity

erlend-aasland commented on Jan 18, 2024

sethmlarson commented on Jan 18, 2024

sethmlarson commented on Jan 18, 2024

befeleme commented on Jan 19, 2024

hroncok commented on Feb 7, 2024

hroncok commented on Feb 7, 2024

hroncok commented on Feb 7, 2024

sethmlarson commented on Feb 7, 2024

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions