gh-140067: Fix memory leak in sub-interpreter creation #140111

ashm-dev · 2025-10-14T14:03:19Z

This patch fixes a memory leak detected by AddressSanitizer during subinterpreter creation.
The original allocation pointer is now stored before the aligned memory block, removing
the need for the _malloced field in PyInterpreterState.

Thanks to @kumaraditya303 for highlighting the pointer overwrite issue.

Fixes #140067

Issue: Memory leak in test_sys with subinterpreters creation (AddressSanitizer detection) #140067

malloced pointer)

Misc/NEWS.d/next/Core_and_Builtins/2025-10-14-17-07-37.gh-issue-140067.ID2gOm.rst

Python/pystate.c

…e-140067.ID2gOm.rst Co-authored-by: Kumar Aditya <[email protected]>

Co-authored-by: Kumar Aditya <[email protected]>

kumaraditya303 · 2025-10-14T14:17:03Z

On main without fix:

Using random seed: 1178707616
0:00:00 load avg: 0.00 Run 1 test sequentially in a single process
0:00:00 load avg: 0.00 [1/1] test_sys
0:00:15 load avg: 0.30 [1/1] test_sys passed

== Tests result: SUCCESS ==

1 test OK.

Total duration: 15.3 sec
Total tests: run=92 skipped=7
Total test files: run=1/1
Result: SUCCESS

=================================================================
==72608==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 4290637 byte(s) in 19 object(s) allocated from:
    #0 0x55ccf021119d in calloc (/home/realkumaraditya/cpython/python+0x32f19d) (BuildId: 8c0c0e48d6c92676ff9a5253a45afb26c8d18926)
    #1 0x55ccf0549d3e in _PyMem_DebugRawAlloc /home/realkumaraditya/cpython/Objects/obmalloc.c:2884:24
    #2 0x55ccf0549d3e in _PyMem_DebugRawCalloc /home/realkumaraditya/cpython/Objects/obmalloc.c:2929:12
    #3 0x55ccf091d6d5 in alloc_interpreter /home/realkumaraditya/cpython/Python/pystate.c:462:17
    #4 0x55ccf091d6d5 in _PyInterpreterState_New /home/realkumaraditya/cpython/Python/pystate.c:660:18
    #5 0x55ccf08d5f30 in new_interpreter /home/realkumaraditya/cpython/Python/pylifecycle.c:2425:14
    #6 0x55ccf08d5cb2 in Py_NewInterpreterFromConfig /home/realkumaraditya/cpython/Python/pylifecycle.c:2521:12
    #7 0x55ccf080f6a8 in _PyXI_NewInterpreter /home/realkumaraditya/cpython/Python/crossinterp.c:3204:23
    #8 0x7f4cbc4dc62f in _interpreters_create_impl /home/realkumaraditya/cpython/./Modules/_interpretersmodule.c:880:13
    #9 0x7f4cbc4dc62f in _interpreters_create /home/realkumaraditya/cpython/./Modules/clinic/_interpretersmodule.c.h:91:20
    #10 0x55ccf03fafc2 in _PyObject_VectorcallTstate /home/realkumaraditya/cpython/./Include/internal/pycore_call.h:169:11
    #11 0x55ccf0783a7f in _PyEval_EvalFrameDefault /home/realkumaraditya/cpython/Python/generated_cases.c.h:3188:35
    #12 0x55ccf075c6e7 in _PyEval_EvalFrame /home/realkumaraditya/cpython/./Include/internal/pycore_ceval.h:121:16
    #13 0x55ccf075c6e7 in _PyEval_Vector /home/realkumaraditya/cpython/Python/ceval.c:2001:12
    #14 0x55ccf0406d12 in _PyObject_VectorcallTstate /home/realkumaraditya/cpython/./Include/internal/pycore_call.h:169:11
    #15 0x55ccf0404631 in method_vectorcall /home/realkumaraditya/cpython/Objects/classobject.c:95:18
    #16 0x55ccf077790d in _PyEval_EvalFrameDefault /home/realkumaraditya/cpython/Python/generated_cases.c.h:2616:32
    #17 0x55ccf075c6e7 in _PyEval_EvalFrame /home/realkumaraditya/cpython/./Include/internal/pycore_ceval.h:121:16
    #18 0x55ccf075c6e7 in _PyEval_Vector /home/realkumaraditya/cpython/Python/ceval.c:2001:12
    #19 0x55ccf03fb440 in _PyObject_VectorcallDictTstate /home/realkumaraditya/cpython/Objects/call.c:135:15
    #20 0x55ccf03fe1b2 in _PyObject_Call_Prepend /home/realkumaraditya/cpython/Objects/call.c:504:24
    #21 0x55ccf05ade31 in call_method /home/realkumaraditya/cpython/Objects/typeobject.c:3076:19
    #22 0x55ccf03fb9f3 in _PyObject_MakeTpCall /home/realkumaraditya/cpython/Objects/call.c:242:18
    #23 0x55ccf078353d in _PyEval_EvalFrameDefault /home/realkumaraditya/cpython/Python/generated_cases.c.h:4021:35
    #24 0x55ccf075c6e7 in _PyEval_EvalFrame /home/realkumaraditya/cpython/./Include/internal/pycore_ceval.h:121:16
    #25 0x55ccf075c6e7 in _PyEval_Vector /home/realkumaraditya/cpython/Python/ceval.c:2001:12
    #26 0x55ccf0406d12 in _PyObject_VectorcallTstate /home/realkumaraditya/cpython/./Include/internal/pycore_call.h:169:11
    #27 0x55ccf0404631 in method_vectorcall /home/realkumaraditya/cpython/Objects/classobject.c:95:18
    #28 0x55ccf077790d in _PyEval_EvalFrameDefault /home/realkumaraditya/cpython/Python/generated_cases.c.h:2616:32
    #29 0x55ccf075c6e7 in _PyEval_EvalFrame /home/realkumaraditya/cpython/./Include/internal/pycore_ceval.h:121:16
    #30 0x55ccf075c6e7 in _PyEval_Vector /home/realkumaraditya/cpython/Python/ceval.c:2001:12
    #31 0x55ccf03fb440 in _PyObject_VectorcallDictTstate /home/realkumaraditya/cpython/Objects/call.c:135:15
    #32 0x55ccf03fe1b2 in _PyObject_Call_Prepend /home/realkumaraditya/cpython/Objects/call.c:504:24
    #33 0x55ccf05ade31 in call_method /home/realkumaraditya/cpython/Objects/typeobject.c:3076:19
    #34 0x55ccf03fb9f3 in _PyObject_MakeTpCall /home/realkumaraditya/cpython/Objects/call.c:242:18
    #35 0x55ccf078353d in _PyEval_EvalFrameDefault /home/realkumaraditya/cpython/Python/generated_cases.c.h:4021:35
    #36 0x55ccf075c6e7 in _PyEval_EvalFrame /home/realkumaraditya/cpython/./Include/internal/pycore_ceval.h:121:16
    #37 0x55ccf075c6e7 in _PyEval_Vector /home/realkumaraditya/cpython/Python/ceval.c:2001:12

Direct leak of 225823 byte(s) in 1 object(s) allocated from:
    #0 0x55ccf021119d in calloc (/home/realkumaraditya/cpython/python+0x32f19d) (BuildId: 8c0c0e48d6c92676ff9a5253a45afb26c8d18926)
    #1 0x55ccf0549d3e in _PyMem_DebugRawAlloc /home/realkumaraditya/cpython/Objects/obmalloc.c:2884:24
    #2 0x55ccf0549d3e in _PyMem_DebugRawCalloc /home/realkumaraditya/cpython/Objects/obmalloc.c:2929:12
    #3 0x55ccf091d6d5 in alloc_interpreter /home/realkumaraditya/cpython/Python/pystate.c:462:17
    #4 0x55ccf091d6d5 in _PyInterpreterState_New /home/realkumaraditya/cpython/Python/pystate.c:660:18
    #5 0x55ccf08d5f30 in new_interpreter /home/realkumaraditya/cpython/Python/pylifecycle.c:2425:14
    #6 0x55ccf08d5cb2 in Py_NewInterpreterFromConfig /home/realkumaraditya/cpython/Python/pylifecycle.c:2521:12
    #7 0x55ccf080f6a8 in _PyXI_NewInterpreter /home/realkumaraditya/cpython/Python/crossinterp.c:3204:23
    #8 0x7f4cbc4dc62f in _interpreters_create_impl /home/realkumaraditya/cpython/./Modules/_interpretersmodule.c:880:13
    #9 0x7f4cbc4dc62f in _interpreters_create /home/realkumaraditya/cpython/./Modules/clinic/_interpretersmodule.c.h:91:20
    #10 0x55ccf03fafc2 in _PyObject_VectorcallTstate /home/realkumaraditya/cpython/./Include/internal/pycore_call.h:169:11
    #11 0x55ccf078af2d in _PyEval_EvalFrameDefault /home/realkumaraditya/cpython/Python/generated_cases.c.h:2920:35
    #12 0x55ccf075c6e7 in _PyEval_EvalFrame /home/realkumaraditya/cpython/./Include/internal/pycore_ceval.h:121:16
    #13 0x55ccf075c6e7 in _PyEval_Vector /home/realkumaraditya/cpython/Python/ceval.c:2001:12
    #14 0x55ccf0406d12 in _PyObject_VectorcallTstate /home/realkumaraditya/cpython/./Include/internal/pycore_call.h:169:11
    #15 0x55ccf0404631 in method_vectorcall /home/realkumaraditya/cpython/Objects/classobject.c:95:18
    #16 0x55ccf077790d in _PyEval_EvalFrameDefault /home/realkumaraditya/cpython/Python/generated_cases.c.h:2616:32
    #17 0x55ccf075c6e7 in _PyEval_EvalFrame /home/realkumaraditya/cpython/./Include/internal/pycore_ceval.h:121:16
    #18 0x55ccf075c6e7 in _PyEval_Vector /home/realkumaraditya/cpython/Python/ceval.c:2001:12
    #19 0x55ccf03fb440 in _PyObject_VectorcallDictTstate /home/realkumaraditya/cpython/Objects/call.c:135:15
    #20 0x55ccf03fe1b2 in _PyObject_Call_Prepend /home/realkumaraditya/cpython/Objects/call.c:504:24
    #21 0x55ccf05ade31 in call_method /home/realkumaraditya/cpython/Objects/typeobject.c:3076:19
    #22 0x55ccf03fb9f3 in _PyObject_MakeTpCall /home/realkumaraditya/cpython/Objects/call.c:242:18
    #23 0x55ccf078353d in _PyEval_EvalFrameDefault /home/realkumaraditya/cpython/Python/generated_cases.c.h:4021:35
    #24 0x55ccf075c6e7 in _PyEval_EvalFrame /home/realkumaraditya/cpython/./Include/internal/pycore_ceval.h:121:16
    #25 0x55ccf075c6e7 in _PyEval_Vector /home/realkumaraditya/cpython/Python/ceval.c:2001:12
    #26 0x55ccf0406d12 in _PyObject_VectorcallTstate /home/realkumaraditya/cpython/./Include/internal/pycore_call.h:169:11
    #27 0x55ccf0404631 in method_vectorcall /home/realkumaraditya/cpython/Objects/classobject.c:95:18
    #28 0x55ccf077790d in _PyEval_EvalFrameDefault /home/realkumaraditya/cpython/Python/generated_cases.c.h:2616:32
    #29 0x55ccf075c6e7 in _PyEval_EvalFrame /home/realkumaraditya/cpython/./Include/internal/pycore_ceval.h:121:16
    #30 0x55ccf075c6e7 in _PyEval_Vector /home/realkumaraditya/cpython/Python/ceval.c:2001:12
    #31 0x55ccf03fb440 in _PyObject_VectorcallDictTstate /home/realkumaraditya/cpython/Objects/call.c:135:15
    #32 0x55ccf03fe1b2 in _PyObject_Call_Prepend /home/realkumaraditya/cpython/Objects/call.c:504:24
    #33 0x55ccf05ade31 in call_method /home/realkumaraditya/cpython/Objects/typeobject.c:3076:19
    #34 0x55ccf03fb9f3 in _PyObject_MakeTpCall /home/realkumaraditya/cpython/Objects/call.c:242:18
    #35 0x55ccf078353d in _PyEval_EvalFrameDefault /home/realkumaraditya/cpython/Python/generated_cases.c.h:4021:35
    #36 0x55ccf075c6e7 in _PyEval_EvalFrame /home/realkumaraditya/cpython/./Include/internal/pycore_ceval.h:121:16
    #37 0x55ccf075c6e7 in _PyEval_Vector /home/realkumaraditya/cpython/Python/ceval.c:2001:12

SUMMARY: AddressSanitizer: 4516460 byte(s) leaked in 20 allocation(s).

With fix:

Using random seed: 1947560357
0:00:00 load avg: 1.07 Run 1 test sequentially in a single process
0:00:00 load avg: 1.07 [1/1] test_sys
0:00:16 load avg: 1.12 [1/1] test_sys passed

== Tests result: SUCCESS ==

1 test OK.

Total duration: 16.3 sec
Total tests: run=92 skipped=7
Total test files: run=1/1
Result: SUCCESS

Include/internal/pycore_interp_structs.h

bedevere-app · 2025-10-14T14:49:18Z

GH-140118 is a backport of this pull request to the 3.14 branch.

bedevere-app · 2025-10-14T14:49:34Z

GH-140118 is a backport of this pull request to the 3.14 branch.

…ythonGH-140111) Fix memory leak in sub-interpreter creation caused by overwriting of the previously used `_malloced` field. Now the pointer is stored in the first word of the memory block to avoid it being overwritten accidentally. (cherry picked from commit 59547a2) Co-authored-by: Shamil <[email protected]> Co-authored-by: Kumar Aditya <[email protected]>

…ython#140111)" This reverts commit 59547a2.

ZeroIntensity · 2025-10-15T00:02:23Z

It seems this broke the TSan CI, see #140138. I have a revert up at #140140.

…)" (#140140) This reverts commit 59547a2.

bpo-140067: Fix memory leak in subinterpreter creation (remove redundant

595adea

malloced pointer)

ashm-dev requested review from ZeroIntensity and ericsnowcurrently as code owners October 14, 2025 14:03

bedevere-app bot added the awaiting review label Oct 14, 2025

add blurb

0fd2715

StanFromIreland changed the title ~~bpo-140067: Fix memory leak in subinterpreter creation (remove redundant~~ gh-140067: Fix memory leak in subinterpreter creation (remove redundant Oct 14, 2025

bedevere-app bot mentioned this pull request Oct 14, 2025

Memory leak in test_sys with subinterpreters creation (AddressSanitizer detection) #140067

Open

Merge branch 'main' into pythongh-140067

b96102b

kumaraditya303 changed the title ~~gh-140067: Fix memory leak in subinterpreter creation (remove redundant~~ gh-140067: Fix memory leak in sub-interpreter creation Oct 14, 2025

kumaraditya303 reviewed Oct 14, 2025

View reviewed changes

Misc/NEWS.d/next/Core_and_Builtins/2025-10-14-17-07-37.gh-issue-140067.ID2gOm.rst Outdated Show resolved Hide resolved

kumaraditya303 reviewed Oct 14, 2025

View reviewed changes

Python/pystate.c Outdated Show resolved Hide resolved

kumaraditya303 reviewed Oct 14, 2025

View reviewed changes

Python/pystate.c Outdated Show resolved Hide resolved

ashm-dev and others added 3 commits October 14, 2025 17:16

Update Misc/NEWS.d/next/Core_and_Builtins/2025-10-14-17-07-37.gh-issu…

e9c6735

…e-140067.ID2gOm.rst Co-authored-by: Kumar Aditya <[email protected]>

Update Python/pystate.c

0afd99e

Co-authored-by: Kumar Aditya <[email protected]>

Update Python/pystate.c

47fef02

Co-authored-by: Kumar Aditya <[email protected]>

ZeroIntensity reviewed Oct 14, 2025

View reviewed changes

Include/internal/pycore_interp_structs.h Show resolved Hide resolved

kumaraditya303 approved these changes Oct 14, 2025

View reviewed changes

bedevere-app bot added awaiting merge and removed awaiting review labels Oct 14, 2025

kumaraditya303 requested a review from ZeroIntensity October 14, 2025 14:25

ZeroIntensity approved these changes Oct 14, 2025

View reviewed changes

kumaraditya303 self-assigned this Oct 14, 2025

kumaraditya303 added interpreter-core (Objects, Python, Grammar, and Parser dirs) topic-subinterpreters labels Oct 14, 2025

kumaraditya303 enabled auto-merge (squash) October 14, 2025 14:38

kumaraditya303 merged commit 59547a2 into python:main Oct 14, 2025
53 checks passed

bedevere-app bot removed the awaiting merge label Oct 14, 2025

ashm-dev deleted the gh-140067 branch October 14, 2025 14:43

nascheme mentioned this pull request Oct 14, 2025

TSAN failure with free-threaded build, test_daemon_threads_fatal_error #140138

Open

ZeroIntensity mentioned this pull request Oct 14, 2025

[3.14] gh-140067: Fix memory leak in sub-interpreter creation (GH-140111) #140118

Draft

ZeroIntensity added a commit to ZeroIntensity/cpython that referenced this pull request Oct 14, 2025

Revert "pythongh-140067: Fix memory leak in sub-interpreter creation (p…

eac1c71

…ython#140111)" This reverts commit 59547a2.

kumaraditya303 pushed a commit that referenced this pull request Oct 15, 2025

Revert "gh-140067: Fix memory leak in sub-interpreter creation (#140111…

0bcb1c2

…)" (#140140) This reverts commit 59547a2.

ashm-dev restored the gh-140067 branch October 15, 2025 07:13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

gh-140067: Fix memory leak in sub-interpreter creation #140111

gh-140067: Fix memory leak in sub-interpreter creation #140111

ashm-dev commented Oct 14, 2025 •

edited by bedevere-app bot

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kumaraditya303 commented Oct 14, 2025

Uh oh!

Uh oh!

Uh oh!

bedevere-app bot commented Oct 14, 2025

Uh oh!

bedevere-app bot commented Oct 14, 2025

Uh oh!

ZeroIntensity commented Oct 15, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

gh-140067: Fix memory leak in sub-interpreter creation #140111

gh-140067: Fix memory leak in sub-interpreter creation #140111

Conversation

ashm-dev commented Oct 14, 2025 • edited by bedevere-app bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kumaraditya303 commented Oct 14, 2025

Uh oh!

Uh oh!

Uh oh!

bedevere-app bot commented Oct 14, 2025

Uh oh!

bedevere-app bot commented Oct 14, 2025

Uh oh!

ZeroIntensity commented Oct 15, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ashm-dev commented Oct 14, 2025 •

edited by bedevere-app bot

Loading