Skip to content

Undefined references to HACL symbols when statically linking Python 3.12 #131298

Description

@luis605

Bug report

Bug description:

I am developing an application that uses Python 3.12's static library (libpython3.12.a). Due to some restrictions, I am required to use a package manager to install Python and cannot compile it from source. Additionally, I can't use dynamic libraries, so I must link python statically.

When linking libpython3.12.a to my project, I encounter undefined references to HACL (HACL*) symbols, such as:

/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/13/../../../x86_64-linux-gnu/libpython3.12.a(sha2module.o): in function `update_256':
(.text.unlikely+0x9ea): undefined reference to `python_hashlib_Hacl_Hash_SHA2_update_256'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/13/../../../x86_64-linux-gnu/libpython3.12.a(sha2module.o): in function `SHA512Type_copy':
(.text.unlikely+0xcd3): undefined reference to `python_hashlib_Hacl_Hash_SHA2_copy_512'
[etc.]

These symbols appear to be related to the HACL library, which is included in Python 3.12's development. However, I cannot locate the corresponding object files or static libraries for HACL in the Python installation provided by the package manager.

I was previously using python3.11 - which worked flawlessly -, however the dev package has been removed from apt and other similar package managers. For this specific reason, I had to upgrade.

A quick note, the command nm -gA /usr/lib/x86_64-linux-gnu/*.a 2>/dev/null | grep python_hashlib_Hacl_Hash_SHA2_update_512 shows that these symbols are shipped with the static package: /usr/lib/x86_64-linux-gnu/libpython3.12.a:sha2module.o: U python_hashlib_Hacl_Hash_SHA2_update_512.

Steps to Reproduce:

  1. Install Python 3.12 using your system's package manager (e.g., apt on Ubuntu).
  2. Attempt to statically link the Python static library libpython3.12.a into a C/C++ project.
  3. Observe linker errors related to undefined references to HACL symbols.

Expected Behavior:
The static library libpython3.12.a should include all necessary dependencies, including HACL, or provide a way to link against HACL statically.

Actual Behavior:
The linker fails with undefined references to HACL symbols, as the required HACL objects or static libraries are not provided.

Questions:

  1. Are the HACL object files or static libraries supposed to be included in the Python 3.12 package from the package manager?
  2. If not, is there a recommended way to obtain the necessary HACL static library for linking with libpython3.12.a? And if so, is it possible to do so without having to compile Python or any other large project?
  3. Could this be an issue with the packaging of Python 3.12 in the package manager, or is it a broader issue with the static linking setup?
  4. My primary goal is to link python statically. If there isn't any solution available that follows the requirements of using a package manager, are there any work arounds to bypass the linker issue?

Thank you for your assistance.

CPython versions tested on:

3.12

Operating systems tested on:

Linux

Linked PRs

Activity

  1. picnixz commented on Mar 15, 2025

    @picnixz
    Member

    I think this issue is known but I can't remember where it is. I've statically built HACL* sources for HMAC in #130157. What should we do for 3.12? cc @gpshead @msprotz

  2. msprotz commented on Mar 16, 2025

    @msprotz
    Contributor

    #116043 is the original report

    I proposed a PR at the time #119320 but I don't think it was deemed critical enough for the PR to be merged

  3. luis605 commented on Mar 16, 2025

    @luis605
    Author

    If the PR wasn't deemed necessary then it won't probably be implemented. Is there any workaround that doesn't include compiling Python from source?

  4. luis605 commented on Mar 19, 2025

    @luis605
    Author

    A possible workaround, if you don't need HACL is to create and compile the following .cpp file in your project:

    #include <cstddef>
    #include <cstdlib>
    
    extern "C" {
        void* python_hashlib_Hacl_Hash_SHA2_malloc_512() {}
        void python_hashlib_Hacl_Hash_SHA2_free_512(void* ptr) {}
        void python_hashlib_Hacl_Hash_SHA2_update_512(void*, const void*, size_t) {}
        void python_hashlib_Hacl_Hash_SHA2_digest_512(void*, void*) {}
        void python_hashlib_Hacl_Hash_SHA2_copy_512(void*, void*) {}
        void* python_hashlib_Hacl_Hash_SHA2_malloc_256() {}
        void python_hashlib_Hacl_Hash_SHA2_free_256(void* ptr) {}
        void python_hashlib_Hacl_Hash_SHA2_update_256(void*, const void*, size_t) {}
        void python_hashlib_Hacl_Hash_SHA2_digest_256(void*, void*) {}
        void python_hashlib_Hacl_Hash_SHA2_copy_256(void*, void*) {}
        void* python_hashlib_Hacl_Hash_SHA2_malloc_224() {}
        void* python_hashlib_Hacl_Hash_SHA2_malloc_384() {}
    }
  5. picnixz commented on Apr 12, 2025

    @picnixz
    Member

    Ah. I think I should have complicated the builds even more :D in my latest HMAC PR, I actually made all objects as being static because it was easier. I'll eliminate those for the upcoming 3.14 beta and make them fully dynamic (I think we can mostly do this now). My very very bad.

  6. added a commit that references this issue on Apr 20, 2025
  7. picnixz commented on Apr 26, 2025

    @picnixz
    Member

    There is #133012 that is pending but now it should work for Python 3.14 and later. I don't want to backport the fix though because 3.12 is security-only. Maybe a backport to 3.13 is possible though but the build process has been changed too much and I fear too many conflicts.

  8. added a commit that references this issue on Apr 26, 2025
  9. added a commit that references this issue on Jun 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    buildThe build process and cross-buildextension-modulesC modules in the Modules dirtype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions