🌐 AI搜索 & 代理 主页
Skip to content

Conversation

@charris
Copy link
Member

@charris charris commented Sep 30, 2025

Backport of #29811.

Otherwise, a malloc failure in init_gesv(...) is not acted upon, and the python return value is silently wrong---inv(non_zero_array) returns an array of zeros.

On main, using the script under the fold,

  • limiting the memory to 30mb emits numpy._core._exceptions._ArrayMemoryError: Unable to allocate 191. MiB for an array with shape (5000, 5000) and data type float64 from somewhere else in the gufunc machinery;
  • limiting the memory to 300mb makes res be an array of all zeros, no error.

Note that I don't know what I'm doing with NPY_ALLOW_C_API_DEF etc ---here I simply parrot the macros from elsewhere in this source file, e.g. https://github.com/numpy/numpy/blob/main/numpy/linalg/umath_linalg.cpp#L1202
The same issue seems present in other linalg functions, so if this fix looks reasonable, I'll extend to other functions.

import os
import psutil     # need to `pip install psutil`
import resource
import numpy as np


def set_limit(buffer):
     """
     Set a limit on process virtual memory space.
     """
     process = psutil.Process(os.getpid())
     memory_info = process.memory_info()
     used = memory_info.vms # current virtual memory space
     old_limit = resource.getrlimit(resource.RLIMIT_AS)
     new_limit = (used + buffer, old_limit[1]) # set new
     resource.setrlimit(resource.RLIMIT_AS, new_limit)


np.random.seed(123467)
arr = np.random.rand(25_000_000).reshape(5000, 5000)

set_limit(300 * 1024 * 1024) # ~300MB

res = np.linalg.inv(arr)
assert not (res == 0).all()

Otherwise, a malloc failure in `init_gesv(...)` etc. is not acted upon,
and the python return value is silently wrong.
@charris charris added this to the 2.3.4 release milestone Sep 30, 2025
@charris charris added 00 - Bug 08 - Backport Used to tag backport PRs labels Sep 30, 2025
@charris charris merged commit 48b536d into numpy:maintenance/2.3.x Sep 30, 2025
79 of 81 checks passed
@charris charris deleted the backport-29811 branch September 30, 2025 19:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

00 - Bug 08 - Backport Used to tag backport PRs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants