Skip to content

test_ctypes failing on Linux SPARC64 #78952

New issue
New issue
Closed as not planned
Closed as not planned
test_ctypes failing on Linux SPARC64#78952
Labels
testsTests in the Lib/test dirtopic-ctypestype-bugAn unexpected behavior, bug, or error

Description

@kelledin
kelledin
mannequin
opened on Sep 22, 2018
BPO 34771
Nosy @vadmium, @Kelledin

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2018-09-22.17:56:12.924>
labels = ['ctypes', 'type-bug']
title = 'test_ctypes failing on Linux SPARC64'
updated_at = <Date 2018-09-30.18:22:03.191>
user = 'https://github.com/kelledin'

bugs.python.org fields:

activity = <Date 2018-09-30.18:22:03.191>
actor = 'kelledin-3'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['ctypes']
creation = <Date 2018-09-22.17:56:12.924>
creator = 'kelledin-3'
dependencies = []
files = []
hgrepos = []
issue_num = 34771
keywords = []
message_count = 5.0
messages = ['326102', '326504', '326636', '326656', '326740']
nosy_count = 2.0
nosy_names = ['martin.panter', 'kelledin-3']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue34771'
versions = ['Python 3.6']

Activity

  1. kelledin commented on Sep 22, 2018

    @kelledin
    kelledinmannequin
    on Sep 22, 2018
    MannequinAuthor

    Python 3.6.6 on Linux 4.16.18 SPARC64 fails test_ctypes. Specifically, it appears to be due to the _testfunc_large_struct_update_value() or _testfunc_reg_struct_update_value():

    0:00:44 load avg: 46.24 [137/407/1] test_ctypes failed -- running: test_socket (44 sec), test_subprocess (35 sec), test_venv (43 sec), test_normalization (43 sec), test_signal (43 sec), test_multiprocessing_spawn (43 sec), test_concurrent_futures (43 sec), test_email (34 sec), test_cmd_line_script (43 sec), test_tools (43 sec), test_pickletools (34 sec), test_zipfile (30 sec), test_multiprocessing_fork (33 sec), test_pyclbr (31 sec), test_math (42 sec), test_calendar (35 sec), test_datetime (33 sec), test_distutils (30 sec)
test test_ctypes failed -- Traceback (most recent call last):
  File "/usr/src/dist/new/Python-3.6.6/Lib/ctypes/test/test_structures.py", line 416, in test_pass_by_value
    self.assertEqual(s.first, 0xdeadbeef)
AssertionError: 195948557 != 3735928559

    Obviously, the "0xbadf00d" field setting is propagating back up through something that's supposed to be passed-by-value, and the test case quite rightly picks up on it. I suspect this bug exists in 2.7.15 as well (2.7 just doesn't have the testcase to catch it).

    This is built with gcc-8.2.0, glibc-2.27, kernel 4.16.18, CFLAGS="-O1 -mcpu=v9 -mtune=v9". (FYI I had to turn down optimization to resolve another test failure, hence the "-O1".)

    I'm guessing SPARC64 calling conventions are still passing certain large values by reference, and libffi isn't dealing with this? I'm still investigating. I've tried it with and without --with-system-libffi, with no difference (my system libffi is 3.2.1).

  2. kelledin
    added
    topic-ctypes
    type-bugAn unexpected behavior, bug, or error
    on Sep 22, 2018

  3. kelledin commented on Sep 26, 2018

    @kelledin
    kelledinmannequin
    on Sep 26, 2018
    MannequinAuthor

    Further details:

    I cloned libffi from a few days ago to see if I had any different behavior. So far the test fails the same way with the updated libffi.

    I'll also see about contacting libffi upstream and see what they can suggest here.

  4. vadmium commented on Sep 28, 2018

    @vadmium
    vadmium
    on Sep 28, 2018
    Member

    Seems to be a common theme on various 64-bit ABIs. There is already a fix for Python’s Windows copy of the FFI library (bpo-29565), and a “hack” for Arm and x86 Windows (again!): bpo-30353.

  5. kelledin commented on Sep 28, 2018

    @kelledin
    kelledinmannequin
    on Sep 28, 2018
    MannequinAuthor

    FYI the libffi bug report is open here:

    libffi/libffi#451

    As noted in the bug report, this issue actually doesn't appear to impact ARM64 (or ARM32 GNUEABI/GNUEABIHF).

  6. kelledin commented on Sep 30, 2018

    @kelledin
    kelledinmannequin
    on Sep 30, 2018
    MannequinAuthor

    Well, after perusing the ctypes callproc.c code, I found the hacks referenced by martin.panter and tried activating them with some SPARC64 #ifdefs:

    --- python3.6-3.6.6.orig/Modules/_ctypes/callproc.c
+++ python3.6-3.6.6/Modules/_ctypes/callproc.c
@@ -1041,6 +1041,7 @@ GetComError(HRESULT errcode, GUID *riid,
 #endif
 
 #if (defined(__x86_64__) && (defined(__MINGW64__) || defined(__CYGWIN__))) || \
+    (defined(__sparc_v9__) || (defined(__sparc__) && defined(__arch64__))) || \
     defined(__aarch64__)
 #define CTYPES_PASS_BY_REF_HACK
 #define POW2(x) (((x & ~(x - 1)) == x) ? x : 0)

    This is based on #ifdef checks in libffi, but somewhat more generalized. The good news is, this appears to resolve all test_ctypes failures. So I'm guessing this is necessary on Linux/SPARC64, though I can't tell if it's necessary for Solaris/SPARC64. I don't even know what built-in compiler defines get turned on for Solaris, though someone else might.

    It might also be advisable to backport this to Python 2.7, but obviously we should also backport the additional ctypes tests if we do that.

    My biggest concern is, do these hacks have a purely performance-centric impact, or do they potentially degrade functionality as well?

  7. ezio-melotti
    transferred this issue fromon Apr 10, 2022
  8. iritkatriel
    added
    testsTests in the Lib/test dir
    on Nov 24, 2023

  9. kulikjak commented on Dec 7, 2023

    @kulikjak
    kulikjak
    on Dec 7, 2023
    Contributor

    I just tested this with Solaris 11.4 SPARC, and interestingly, we don't see any issues with test_ctypes. I tried running it both with and without your suggested change, and all tests passed in both cases. That said, this bug is pretty old now, and it's possible that things changed since then.

    I tested this with Python 3.11 and libffi 3.4.4.

  10. serhiy-storchaka
    added this to Ctypes issueson Jan 1, 2024
  11. encukou
    added this to Unsupported platformson Sep 17, 2024

  12. encukou commented on Sep 17, 2024

    @encukou
    encukou
    on Sep 17, 2024
    Member

    SPARC is unsupported per PEP-11. We don't have the means to test this issue. I'd be happy to merge a PR to add something clearly SPARC-specific to that #if, if a SPARC mainainer told me it fixes the tests. But, let's not track the issue here.

    I'll close the bug and PR.
    Like other issues for unsupported platforms, it'll be listed on the GitHub project for anyone who wants to support the platform unofficially or bring support back.
    I encourage such people to add themselves to the Experts list so we know whom to ping.

  13. encukou
    closed this as not plannedon Sep 17, 2024
  14. github-project-automation
    moved this to Done in Unsupported platformson Sep 17, 2024
  15. github-project-automation
    moved this to Done in Ctypes issueson Sep 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    testsTests in the Lib/test dirtopic-ctypestype-bugAn unexpected behavior, bug, or error

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions