Mailman 3 - NumPy-Discussion

odd behaviour of meshgrid with 3 arrays
by Slavin, Jonathan Nov. 21, 2024

Nov. 21, 2024

Hi all, I was trying to use meshgrid with three arrays and got some odd results. Here's a simple example: xt = np.array([1,2,3,4]) yt = np.array([6,7,8]) zt = np.array([12,13]) xxx,yyy,zzz = np.meshgrid(xt,yt,zt) So I would expect that xxx[0,0,:] = array([1,2,3,4]) instead I get xxx[0,0,:] = array([1,1]) and xxx[0,:,0] = array([1,2,3,4]) also yyy[:,0,0] = array([6,7,8]), whereas I would expect yyy[0,:,0] = array([6,7,8]) So what's going on? This seems like a bug to me. Any suggestions for getting what I wanted -- i.e. xxx.shape = (2,3,4), with values as appropriate? Thanks, Jon -- Jonathan D. Slavin (he/him) Astrophysicist - High Energy Astrophysics Division Center for Astrophysics | Harvard & Smithsonian Office: (617) 496-7981 | Cell: (781) 363-0035 60 Garden Street | MS 04 | Cambridge, MA 02138

2 2

Add bitwise deposit/extract functions
by Freddie Witherden Nov. 15, 2024

Nov. 15, 2024

NumPy currently has a reasonable selection of bitwise operators. However, one missing class are bit deposit and extract functions (also known as expand and compress). The general prototype is deposit(mask, x) and extract(mask, x). The extract function goes through each entry in mask and, if it is set, extracts the corresponding bit from x and concatenates it into the return value. The deposit function does the opposite: the first bit in x goes to the location of the first non-zero in the mask, the second bit in x goes to the location of the second non-zero in the mask, and so on, with everything being or'ed together and returned. These functions have a variety of uses. A particularly nice one is if you want to Morton order a set of integers in N dimensions which reduces down to N deposit calls followed by some | and <<. More importantly, these functions are very difficult to efficiently emulate in NumPy as a large number of explicit <<, &, and | operations are required. Indeed, these functions are so important that recent CPUs have support for them in hardware (x86 has PDEP and PEXT for ~10 years, AVX-512 has similar vector versions but called compress and expand, SVE also has variants, and they'll be part of RISC-V, too). The functions are also under consideration for future revisions of the C++ standard (see https://eisenwave.github.io/cpp-proposals/bit-permutations.html#application… which also gives some other use cases). It would therefore be nice if they were supported by NumPy even if, initially, without hardware support (straight emulation will still be 10~20 times faster than what one can accomplish in NumPy simply because everything happens on a per-element basis in registers and these methods expose a fair amount of instruction level parallelism). The implementation should not be too complicated (the above link shows a reference implementation in C++). Regards, Freddie.

1 0

Feature request: Add out-of-place (i.e. pure) versions of all existing in-place operations
by Carlos Martin Nov. 14, 2024

Nov. 14, 2024

Feature request: Add out-of-place (i.e. pure) versions of all existing in-place operations, such as the following: - https://numpy.org/doc/stable/reference/generated/numpy.fill_diagonal.html - https://numpy.org/doc/stable/reference/generated/numpy.put.html - https://numpy.org/doc/stable/reference/generated/numpy.place.html - https://numpy.org/doc/stable/reference/generated/numpy.put_along_axis.html - https://numpy.org/doc/stable/reference/generated/numpy.putmask.html This can be done by adding a boolean argument to each function called "inplace", which is True by default, but when set to False, makes the function return a modified copy of the array. This is the approach taken by JAX for its counterparts of these functions: - https://jax.readthedocs.io/en/latest/_autosummary/jax.numpy.fill_diagonal.h… - https://jax.readthedocs.io/en/latest/_autosummary/jax.numpy.put.html - https://jax.readthedocs.io/en/latest/_autosummary/jax.numpy.place.html This would have the following advantages: - Making it easier to perform these operations in a functional way (potentially less error-prone). - Harmonizing the interfaces of NumPy and JAX.

1 0

How do I use the numpy on spyder?
by Joao Pereira Oct. 31, 2024

Oct. 31, 2024

I don´t know how to use the numpy on spyder. I have tried to use this line: "import numpy as np" but i dont know if it's the only necessary thing to do. If you could explain it to me I'll appreciate it.

2 1

C API could be more consistent with returning NPY_SUCCEED/NPY_FAIL
by sverre.hassing＠gmail.com Oct. 31, 2024

Oct. 31, 2024

While using the C API to work with Numpy arrays, I came across some inconsistencies regarding the return of a success state from various Numpy functions. For example, the array iterator functions return either NPY_SUCCEED (defined as 1) or NPY_FAIL (defined as 0) to indicate whether the function successfully finished. However, various of the array functions return -1 on success and 0 on failure. The result is similar enough, but I would think it more consistent to use NPY_SUCCEED/NPY_FAIL everywhere. As an example, consider the documentation pages for the Array API: https://numpy.org/doc/stable/reference/c-api/array.html and the Array iterator API: https://numpy.org/doc/stable/reference/c-api/iterator.html#array-iterator-a… I hope this is the right place to place this. It is the first time I really publicly engage with this kind of thing. This did not seem like a bug and the Github page indicated that feature requests should go to the mailing list first.

2 1

Suggestion to show the shape in repr for summarized arrays
by Marten van Kerkwijk Oct. 30, 2024

Oct. 30, 2024

Hi All, When the repr of an array is shown, currently the dtype and shape are explicitly listed if these cannot be directly inferred from the list that is shown, i.e., if the dtype is not float64 or int64, and if the size of the array is zero, but the shape not the simple (0,). For instance, ``` np.empty((10,2,0), dtype="i2") array([], shape=(10, 2, 0), dtype=int16) ``` I propose to also show the shape for the (rare) case that an array is summarized, i.e., when it has more than the default threshold of 1000 elements, and elements are replaced by `...`. The logic is that also in that case it is no longer clear what the shape actually is, which is useful information (e.g., if working in a notebook -- which is the original use case at https://github.com/numpy/numpy/issues/27461). I have a PR for that at https://github.com/numpy/numpy/pull/27482 which would lead to the following: ``` np.arange(1001) array([ 0, 1, 2, ..., 998, 999, 1000], shape=(1001,)) ``` Just to be sure: this PR causes *no* change for any arrays with sizes less than a 1000, so I do not believe this change will lead to a lot of unnecessary churn for down-stream packages. Indeed, between numpy and astropy (which has lots of doctests), the only changes to (doc)tests that were needed are the very few for arrays where the "threshold" is explicitly exceeded. One irritant is that the shape is not an argument that can be passed in to an `np.array` call. While this is just as much the case for zero-sized arrays, perhaps a better solution would be to move the shape information out of the parentheses, e.g., using ``...) # shape=(...)``. I can change the PR to do that if that's the consensus. All the best, Marten

5 5

Canonical way of serialising Generators
by Andrew Nelson Oct. 28, 2024

Oct. 28, 2024

Hi all, is there a canonical way of serialising Generators (not via pickle). Would the following be reasonable for saving and restoring state: ``` def serialize_rng(rng): klass = rng.bit_generator.state['bit_generator'] entropy = rng.bit_generator.seed_seq.entropy return klass, entropy def deserialize_rng(klass, entropy, rng=None): if rng is not None and klass == rng.bit_generator.state['bit_generator']: rng.bit_generator.seed_seq.entropy = entropy return rng BG = getattr(np.random, klass) bg = BG(np.random.SeedSequence(entropy)) return np.random.default_rng(bg) ```

2 2

Endorsing SPECs 1, 6, 7, and 8
by Sebastian Berg Oct. 26, 2024

Oct. 26, 2024

Hi all, TL;DR: NumPy should endorse some or all of the new SPECs if we like them. If you don't or do like them, please discuss, otherwise I suspect we will propose and endorsing them soon and do it if a few core maintainers agree. --- The Scientific Python project has the SPEC process to write up helpful tips/patterns [1] with the idea that projects "endorse" them to indicate that we think following the SPEC is a good idea. One example probably known best is SPEC 0, which is NEP 27 (the suggested minimum supported versions for dependencies, e.g. Python/library versions). There has been a lot of work recently to write some new SPECs, and the following ones are up for endorsement by NumPy: * 1 -- Lazy Loading of Submodules and Functions https://scientific-python.org/specs/spec-0001 * 6 -- Keys to the Castle https://scientific-python.org/specs/spec-0006/ * 7 -- Seeding Pseudo-Random Number Generation https://scientific-python.org/specs/spec-0007/ * 8 -- Securing the Release Process https://scientific-python.org/specs/spec-0008/ Endorsing them means we think they are good guidance for projects, not necessarily strictly following them [2]. Without input, I suspect we may endorse them soon (if a few maintainers give a thumbs up). But of course it would be great to discuss and maybe improve the SPECs in the progress! SPEC 7 is important, because it describes how we would like other libraries to use the new random number generation API in the future (but actually doesn't apply directly). Cheers, Sebastian [1] The SPECs are not necessarily fixed. If you think they are missing a note/solution, we can modify them. [2] With it's flat namespace, I am not sure e.g. lazy-loading is of much use to NumPy (just like we have longer compatibility than SPEC 0).

8 9

Re: np.ndenumerate doesn't obey mask?
by Stefano Miccoli Oct. 23, 2024

Oct. 23, 2024

> On 22 Oct 2024, at 18:00, numpy-discussion-request(a)python.org wrote: > > From: Neal Becker <ndbecker2(a)gmail.com> > Subject: [Numpy-discussion] np.ndenumerate doesn't obey mask? > Date: 21 October 2024 at 18:52:41 CEST > To: Discussion of Numerical Python <numpy-discussion(a)python.org> > Reply-To: Discussion of Numerical Python <numpy-discussion(a)python.org> > > > I was using ndenuerate with a masked array, and it seems that the mask is ignored. Is this true? If so, isn't that a bug? > This is expected behaviour: you should use the function from the “.ma" namespace “numpy.ma.ndenumerate" <https://numpy.org/devdocs/reference/generated/numpy.ma.ndenumerate.html#num…> for skipping masked elements. By design numpy.enumerate does not skip masked elements. Stefano

1 0

np.ndenumerate doesn't obey mask?
by Neal Becker Oct. 21, 2024

Oct. 21, 2024

I was using ndenuerate with a masked array, and it seems that the mask is ignored. Is this true? If so, isn't that a bug?

1 0