memcpy() usage without #include <string.h> in pwdmodule.c

Bug report

Bug description:

For a non-gcc/clang compiler, _Py_TYPEOF is not defined.

Lines 548 to 551 in c08a302

    
           // The macro is only defined if GCC or clang compiler is used. 
        
           #if defined(__GNUC__) || defined(__clang__) 
        
           #  define _Py_TYPEOF(expr) __typeof__(expr) 
        
           #endif

Which leads to Py_CLEAR() fallback to an implementation with memcpy() call.

cpython/Include/object.h

Lines 1016 to 1027 in c08a302

    
           #else 
        
           #define Py_CLEAR(op) \ 
        
               do { \ 
        
                   PyObject **_tmp_op_ptr = _Py_CAST(PyObject**, &(op)); \ 
        
                   PyObject *_tmp_old_op = (*_tmp_op_ptr); \ 
        
                   if (_tmp_old_op != NULL) { \ 
        
                       PyObject *_null_ptr = _Py_NULL; \ 
        
                       memcpy(_tmp_op_ptr, &_null_ptr, sizeof(PyObject*)); \ 
        
                       Py_DECREF(_tmp_old_op); \ 
        
                   } \ 
        
               } while (0) 
        
           #endif

Py_CLEAR() is used in ./Modules/pwdmodule.c

cpython/Modules/pwdmodule.c

Lines 355 to 358 in c08a302

    
           static int pwdmodule_clear(PyObject *m) { 
        
               Py_CLEAR(get_pwd_state(m)->StructPwdType); 
        
               return 0; 
        
           }

without including string.h

cpython/Modules/pwdmodule.c

Lines 12 to 16 in c08a302

    
           #include <errno.h>                // ERANGE 
        
           #include <pwd.h>                  // getpwuid() 
        
           #include <unistd.h>               // sysconf()

For a non-gcc/clang compiler, this may fail due to missing memcpy() declaration.

./Include/object.h:1017: #define Py_CLEAR(op)     do {         PyObject **_tmp_op_ptr = _Py_CAST(PyObject**, &(op));         PyObject *_tmp_old_op = (*_tmp_op_ptr);         if (_tmp_old_op != NULL) {             PyObject *_null_ptr = _Py_NULL;             memcpy(_tmp_op_ptr, &_null_ptr, sizeof(PyObject*));             Py_DECREF(_tmp_old_op);         }     } while (0)
                                                                                                                                                                                                                                                                ^ implicit declaration of a function
./Modules/pwdmodule.c:356:     Py_CLEAR(get_pwd_state(m)->StructPwdType);
                               ^ in expansion of macro

On a side note, typeof is widely implemented among alternative C compilers like TinyCC and cproc, kefir, chibicc, and is standardized in C23. It could be beneficial to enable typeof usage through configure option or probing, instead of hard-coded off on non-gcc/clang compilers.

CPython versions tested on:

3.13

Operating systems tested on:

Linux

Linked PRs

For now, GCC and Clang are the officially supported toolchains for Python. I'm not sure we need support extra compiler set.

But for now, this is expected behavior

For now, GCC and Clang are the officially supported toolchains

From what I'm seeing, there are new code written as recent as 3.13 that explicitly provide support for 1. standard C11 compiler and 2. MSVC.

cpython/Include/cpython/pyatomic.h

Lines 488 to 498 in 2bd960b

    
           #if _Py_USE_GCC_BUILTIN_ATOMICS 
        
           #  define Py_ATOMIC_GCC_H 
        
           #  include "cpython/pyatomic_gcc.h" 
        
           #  undef Py_ATOMIC_GCC_H 
        
           #elif __STDC_VERSION__ >= 201112L && !defined(__STDC_NO_ATOMICS__) 
        
           #  define Py_ATOMIC_STD_H 
        
           #  include "cpython/pyatomic_std.h" 
        
           #  undef Py_ATOMIC_STD_H 
        
           #elif defined(_MSC_VER) 
        
           #  define Py_ATOMIC_MSC_H

The define logic of _Py_TYPEOF() clearly has "implementations other than GCC and Clang" in mind,

cpython/Include/pyport.h

Lines 548 to 551 in c08a302

    
           // The macro is only defined if GCC or clang compiler is used. 
        
           #if defined(__GNUC__) || defined(__clang__) 
        
           #  define _Py_TYPEOF(expr) __typeof__(expr) 
        
           #endif

even if the guarding condition only exists for MSVC, it would simply be !defined(_MSC_VER).

If fact, were it be !defined(_MSC_VER) this issue would not exist, since the tested compiler actually support __typeof__.

pythongh-121617: Include string.h in pwdmodule.c if _Py_TYPEOF is und…

@vstinner

@vstinner, AFAIK you both rewrote Py_CLEAR and removed string.h. Are you interested in this issue?

I don't think fixing only pwdmodule.h is correct; Py_CLEAR should work for all users of Python.h.

pythongh-121617: Include <string.h> for Py_CLEAR()

pythongh-121617: Include <string.h> for Py_CLEAR() macro

I don't think fixing only pwdmodule.h is correct; Py_CLEAR should work for all users of Python.h.

I agree. I wrote #144666 to include <string.h> header if the _Py_TYPEOF macro is not defined.

On a side note, typeof is widely implemented among alternative C compilers like TinyCC and cproc, kefir, chibicc, and is standardized in C23. It could be beneficial to enable typeof usage through configure option or probing, instead of hard-coded off on non-gcc/clang compilers.

My PR also modify _Py_TYPEOF() macro to use C23 typeof() if available.

I wrote a configure check for typeof() but it didn't work as expected. It says that typeof() is available, but then building Python fails because typeof() is undefined. The problem is that configure adds -std=c11 to CFLAGS_NODIST, and typeof() is only defined in C23 or newer. I abandoned the configure change, it's too tricky for me to get it right (run the checks with the same C flags than the ones used to build Python).

On a side note, typeof is widely implemented among alternative C compilers like TinyCC and cproc, kefir, chibicc, and is standardized in C23. It could be beneficial to enable typeof usage through configure option or probing, instead of hard-coded off on non-gcc/clang compilers.

For fun, I tried building Python 3.15 (main branch) with tcc (TinyCC), but it failed to compile _Py_thread_local.

The following program fails to build with: error: _Thread_local is not implemented (using tcc -std=c11).

int main()
{
    _Thread_local int myvar;
    return 0;
}

gh-121617: Include <string.h> for Py_CLEAR() macro (#144666)

@fuhsnn

I fixed the Py_CLEAR() issue in the main branch: Python.h now always includes <string.h>.

I don't think that it's worth it to backport the change to 3.13 and 3.14 branches. I close the issue.

Thanks for the bug report @fuhsnn.

@vstinner

Thanks @vstinner!

pythongh-121617: Include <string.h> for Py_CLEAR() macro (python#144666)

gh-121617: Fix Py_CLEAR() in C++: replace NULL with _Py_NULL (#157067)

pythongh-121617: Fix Py_CLEAR() memcpy in C++: replace NULL with _Py_…

pythongh-121617: Fix Py_CLEAR() in C++: replace NULL with _Py_NULL (p…

[3.14] gh-121617: Fix Py_CLEAR() in C++: replace NULL with _Py_NULL (#…

[3.13] gh-121617: Fix Py_CLEAR() in C++: replace NULL with _Py_NULL (#…

gh-121617: Fix Py_CLEAR() memcpy in C++: replace NULL with _Py_NULL (#…

Merge branch 'main' into tachyon-native-callable

pythongh-121617: Fix Py_CLEAR() in C++: replace NULL with _Py_NULL (p…

pythongh-121617: Fix Py_CLEAR() memcpy in C++: replace NULL with _Py_…

pythongh-121617: Fix Py_CLEAR() in C++: replace NULL with _Py_NULL (p…

pythongh-121617: Fix Py_CLEAR() memcpy in C++: replace NULL with _Py_…

[3.15] gh-121617: Fix Py_CLEAR() in C++: replace NULL with _Py_NULL (G…

Navigation Menu

Repository navigation

`memcpy()` usage without `#include <string.h>` in `pwdmodule.c` #121617

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

3 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

	// The macro is only defined if GCC or clang compiler is used.
	#if defined(__GNUC__) \|\| defined(__clang__)
	# define _Py_TYPEOF(expr) __typeof__(expr)
	#endif

	#else
	#define Py_CLEAR(op) \
	do { \
	PyObject _tmp_op_ptr = _Py_CAST(PyObject, &(op)); \
	PyObject _tmp_old_op = (_tmp_op_ptr); \
	if (_tmp_old_op != NULL) { \
	PyObject *_null_ptr = _Py_NULL; \
	memcpy(_tmp_op_ptr, &_null_ptr, sizeof(PyObject*)); \
	Py_DECREF(_tmp_old_op); \
	} \
	} while (0)
	#endif

	static int pwdmodule_clear(PyObject *m) {
	Py_CLEAR(get_pwd_state(m)->StructPwdType);
	return 0;
	}


	#include <errno.h> // ERANGE
	#include <pwd.h> // getpwuid()
	#include <unistd.h> // sysconf()


	#if _Py_USE_GCC_BUILTIN_ATOMICS
	# define Py_ATOMIC_GCC_H
	# include "cpython/pyatomic_gcc.h"
	# undef Py_ATOMIC_GCC_H
	#elif __STDC_VERSION__ >= 201112L && !defined(__STDC_NO_ATOMICS__)
	# define Py_ATOMIC_STD_H
	# include "cpython/pyatomic_std.h"
	# undef Py_ATOMIC_STD_H
	#elif defined(_MSC_VER)
	# define Py_ATOMIC_MSC_H

memcpy() usage without #include <string.h> in pwdmodule.c #121617

Description

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

Activity

Zheaoli commented on Jul 11, 2024

fuhsnn commented on Jul 11, 2024

encukou commented on Feb 10, 2026

vstinner commented on Feb 10, 2026

vstinner commented on Feb 10, 2026

vstinner commented on Feb 12, 2026

fuhsnn commented on Feb 12, 2026

3 remaining items

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

`memcpy()` usage without `#include <string.h>` in `pwdmodule.c` #121617