Repository navigation
Py_Finalize() function, why it does not finalise Python?? #98524
Description
The following code shows a bug in Py_Finalize(). It does not finalize python interpreter, the second round fires an error.
Thanks in advance to all of those of you that want's a better python
Alexei
Python code (copy it and save it as a .py file)
[Python code]
import numpy as np
The variable <char *pcharPathName> change it to the proper pathname of the python code. Test it
[c]
#ifdef _DEBUG
#define ALEX_PYTHON_TWEEK _DEBUG
#undef _DEBUG
#endif
//#define PY_SSIZE_T_CLEAN
//needed to define the following directive to avoid static linking
//#define Py_LIMITED_API 0x03110000
#include <Python.h>
//needed to avoid static linking
//#pragma comment (lib,"python311.lib")
//restore _DEBUB macro after python include
#ifdef ALEX_PYTHON_TWEEK
#define _DEBUG ALEX_PYTHON_TWEEK
#endif
#include <stdlib.h>
char* charFromWStr(wchar_t* pWChar,SIZE_T uLen)
{
char* pChar;
if (uLen)
{
//Alocare string len including End of Str char
pChar = (char*)malloc(uLen + 1);
if (pChar)
{
memset(pChar, 0, uLen + 1);
if (pChar)
{
SIZE_T I;
for (I = 0; I < uLen; I++)
{
pChar[I] = (char)pWChar[I];
}
return pChar;
}
}
}
return NULL;
}
void PythonObject_DecodeString(char** ppcharValue, PyObject* pPyValue)
{
SIZE_T sLen;
wchar_t* pwcDymmy;
if (ppcharValue)
{
if (pPyValue)//
{
sLen = (PyUnicode_GetLength(pPyValue) + 1);
SIZE_T byteSize = sLen * sizeof(wchar_t);
pwcDymmy = malloc(byteSize);
SIZE_T Z = PyUnicode_AsWideChar(pPyValue, pwcDymmy, byteSize);
if (Z) *ppcharValue = charFromWStr(pwcDymmy, sLen);
free(pwcDymmy);
return;
}
else *ppcharValue = NULL;
}
}
void ReportPythonError()
{
PyObject* pType, * pValue, * pTraceBack, * PyObjAsStr;
PyErr_Fetch(&pType, &pValue, &pTraceBack);
PyObjAsStr = PyObject_Str(pValue);
if (PyObjAsStr)
{
char* pwc;
PythonObject_DecodeString(&pwc, PyObjAsStr);
if (pwc)
{
printf(pwc);
}else printf("Could not retrieve error!");
}
}
void RunCodeFromFile(PyObject* pModule,char* charPathName)
{
int f = open(charPathName, 0,"r");
if (f)
{
int l = 1024 * 1024;
char* pBuff = malloc(l);
if (pBuff)
{
memset(pBuff, 0, l);
int ll = read(f, pBuff, l);
if (ll)
{
PyErr_Clear();
PyObject *pPyCode = Py_CompileString(pBuff,"", Py_file_input);
if (pPyCode)
{
PyObject *pPyGlobals = PyModule_GetDict(pModule);//borrowwed Ref
PyObject *pPyResult = PyEval_EvalCode(pPyCode, pPyGlobals, pPyGlobals);
if (pPyResult)
{
Py_DecRef(pPyResult);
}
Py_DecRef(pPyCode);
}
else ReportPythonError();
}
free(pBuff);
}
close(f);
}
}
char* pcharPathName = "D:\\Programming\\VC\\2019\\_COMCLASSES\\Scripting\\Demo\\test.py";
int main(int argc, char* argv[])
{
Py_InitializeEx(0);
if (Py_IsInitialized())
{
PyObject *pPyMain = PyModule_New("__main__");
if (pPyMain)
{
RunCodeFromFile(pPyMain, pcharPathName);
Py_DecRef(pPyMain);
}
Py_Finalize();
}
//2nd round
Py_InitializeEx(0);
if (Py_IsInitialized())
{
PyObject* pPyMain = PyModule_New("__main__");
if (pPyMain)
{
RunCodeFromFile(pPyMain, pcharPathName);
Py_DecRef(pPyMain);
}
Py_Finalize();
}
return 0;
}
The 2nd round reports the following:
[ERROR Report]
Traceback (most recent call last):
File "", line 3, in
File "C:\Python\Python311\Lib\site-packages\numpy_init_.py", line 140, in
from . import core
File "C:\Python\Python311\Lib\site-packages\numpy\core_init_.py", line 23, in
from . import multiarray
File "C:\Python\Python311\Lib\site-packages\numpy\core\multiarray.py", line 10, in
from . import overrides
File "C:\Python\Python311\Lib\site-packages\numpy\core\overrides.py", line 6, in
from numpy.core._multiarray_umath import (
SystemError: D:_w\1\s\Objects\structseq.c:476: bad argument to internal function
Activity
kumaraditya303 commentedon Jan 14, 2023
numpydoes not supports multiple interpreter initializations, it is not a bug in CPython. You may report this to numpy tracker.AlexSoft73 commentedon Feb 3, 2023
Thanks kumaraditya303
CPython does have memory leaks, it does not need to be tested with numpy, that was just another sample of memoryt leaks.
By just executing:
you get memory leaks. But these days the CPyton "core developers"coalition is committee to close every thread that talks about CPython memory leaks taken any escuse as reason, this time numpy.
arhadthedev commentedon Feb 3, 2023
Because memory leaks in legacy code left for older plugins.
CPython now confidently transitions from global variables that actually cause memory leaks (because nobody cleaned objects that are supposed to die with the program anyway) to dynamically allocated structures destroyed with a single, simple and thus always presented function call. These structures are "heap types" and "support of multiple interpreters inside the same process".
Unfortunately for all of us such a transition is a mess that will be here for a long time until everything involved (both all CPython modules and every third party module) gets converted so nothing uses that legacy code anymore. In the worst case it can be akin to Python 2-to-3 transition lasting for a decade.
So this problem is well known (so reports repeating over and over again are closed) and is being steadely resolved, but not as fast as desired.
kumaraditya303 commentedon Feb 4, 2023
It is a known problem and the work is ongoing, if you want to help read https://peps.python.org/pep-0687/ and work on it, creating issues for the same thing over and over is not helpful.