Mailman 3 March 2017 - NumPy-Discussion

mixed mode arithmetic
by Neal Becker July 11, 2023

July 11, 2023

I've been browsing the numpy source. I'm wondering about mixed-mode arithmetic on arrays. I believe the way numpy handles this is that it never does mixed arithmetic, but instead converts arrays to a common type. Arguably, that might be efficient for a mix of say, double and float. Maybe not. But for a mix of complex and a scalar type (say, CDouble * Double), it's clearly suboptimal in efficiency. So, do I understand this correctly? If so, is that something we should improve?

4 6

Invalid value encoutered : how to prevent numpy.where to do this?
by Eric Emsellem Feb. 18, 2023

Feb. 18, 2023

Dear all, I have a code using lots of "numpy.where" to make some constrained calculations as in: data = arange(10) result = np.where(data == 0, 0., 1./data) # or data1 = arange(10) data2 = arange(10)+1.0 result = np.where(data1 > data2, np.sqrt(data1-data2), np.sqrt(data2-data2)) which then produces warnings like: /usr/bin/ipython:1: RuntimeWarning: invalid value encountered in sqrt or for the first example: /usr/bin/ipython:1: RuntimeWarning: divide by zero encountered in divide How do I avoid these messages to appear? I know that I could in principle use numpy.seterr. However, I do NOT want to remove these warnings for other potential divide/multiply/sqrt etc errors. Only when I am using a "where", to in fact avoid such warnings! Note that the warnings only happen once, but since I am going to release that code, I would like to avoid the user to get such messages which are irrelevant here (because I am testing, with the where, when NOT to divide by zero or take a sqrt of a negative number). thanks! Eric

5 4

Re: [Numpy-discussion] example reading binary Fortran file
by Neil Martinsen-Burrell July 22, 2021

July 22, 2021

David Froger <david.froger.info <at> gmail.com> writes: > Hy,My question is about reading Fortran binary file (oh no this question > again...) I've posted this before, but I finally got it cleaned up for the Cookbook. For this purpose I use a subclass of file that has methods for reading unformatted Fortran data. See http://www.scipy.org/Cookbook/FortranIO/FortranFile. I'd gladly see this in numpy or scipy somewhere, but I'm not sure where it belongs. > program makeArray > implicit none > integer,parameter:: nx=10,ny=20 > real(4),dimension(nx,ny):: ux,uy,p > integer :: i,j > open(11,file='uxuyp.bin',form='unformatted') > do i = 1,nx > do j = 1,ny > ux(i,j) = real(i*j) > uy(i,j) = real(i)/real(j) > p (i,j) = real(i) + real(j) > enddo > enddo > write(11) ux,uy > write(11) p > close(11) > end program makeArray When I run the above program compiled with gfortran on my Intel Mac, I can read it back with:: >>> import numpy as np >>> from fortranfile import FortranFile >>> f=FortranFile('uxuyp.bin', endian='<') >>> uxuy = f.readReals(prec='f') # 'f' for default reals >>> len(uxuy) 400 >>> ux = np.array(uxuy[:200]).reshape((20,10)).T >>> uy = np.array(uxuy[200:]).reshape((20,10)).T >>> p = f.readReals('f').reshape((20,10)).T >>> ux array([[ 1., 2., 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20.], [ 2., 4., 6., 8., 10., 12., 14., 16., 18., 20., 22., 24., 26., 28., 30., 32., 34., 36., 38., 40.], [ 3., 6., 9., 12., 15., 18., 21., 24., 27., 30., 33., 36., 39., 42., 45., 48., 51., 54., 57., 60.], [ 4., 8., 12., 16., 20., 24., 28., 32., 36., 40., 44., 48., 52., 56., 60., 64., 68., 72., 76., 80.], [ 5., 10., 15., 20., 25., 30., 35., 40., 45., 50., 55., 60., 65., 70., 75., 80., 85., 90., 95., 100.], [ 6., 12., 18., 24., 30., 36., 42., 48., 54., 60., 66., 72., 78., 84., 90., 96., 102., 108., 114., 120.], [ 7.,Proxy-Connection: keep-alive Cache-Control: max-age=0 14., 21., 28., 35., 42., 49., 56., 63., 70., 77., 84., 91., 98., 105., 112., 119., 126., 133., 140.], [ 8., 16., 24., 32., 40., 48., 56., 64., 72., 80., 88., 96., 104., 112., 120., 128., 136., 144., 152., 160.], [ 9., 18., 27., 36., 45., 54., 63., 72., 81., 90., 99., 108., 117., 126., 135., 144., 153., 162., 171., 180.], [ 10., 20., 30., 40., 50., 60., 70., 80., 90., 100., 110., 120., 130., 140., 150., 160., 170., 180., 190., 200.]]) >>> uy array([[ 1. , 0.5 , 0.33333334, 0.25 , 0.2 , 0.16666667, 0.14285715, 0.125 , 0.11111111, 0.1 , 0.09090909, 0.08333334, 0.07692308, 0.07142857, 0.06666667, 0.0625 , 0.05882353, 0.05555556, 0.05263158, 0.05 ], [ 2. , 1. , 0.66666669, 0.5 , 0.40000001, 0.33333334, 0.2857143 , 0.25 , 0.22222222, 0.2 , 0.18181819, 0.16666667, 0.15384616, 0.14285715, 0.13333334, 0.125 , 0.11764706, 0.11111111, 0.10526316, 0.1 ], [ 3. , 1.5 , 1. , 0.75 , 0.60000002, 0.5 , 0.42857143, 0.375 , 0.33333334, 0.30000001, 0.27272728, 0.25 , 0.23076923, 0.21428572, 0.2 , 0.1875 , 0.17647059, 0.16666667, 0.15789473, 0.15000001], [ 4. , 2. , 1.33333337, 1. , 0.80000001, 0.66666669, 0.5714286 , 0.5 , 0.44444445, 0.40000001, 0.36363637, 0.33333334, 0.30769232, 0.2857143 , 0.26666668, 0.25 , 0.23529412, 0.22222222, 0.21052632, 0.2 ], [ 5. , 2.5 , 1.66666663, 1.25 , 1. , 0.83333331, 0.71428573, 0.625 , 0.55555558, 0.5 , 0.45454547, 0.41666666, 0.38461539, 0.35714287, 0.33333334, 0.3125 , 0.29411766, 0.27777779, 0.2631579 , 0.25 ], [ 6. , 3. , 2. , 1.5 , 1.20000005, 1. , 0.85714287, 0.75 , 0.66666669, 0.60000002, 0.54545456, 0.5 , 0.46153846, 0.42857143, 0.40000001, 0.375 , 0.35294119, 0.33333334, 0.31578946, 0.30000001], [ 7. , 3.5 , 2.33333325, 1.75 , 1.39999998, 1.16666663, 1. , 0.875 , 0.77777779, 0.69999999, 0.63636363, 0.58333331, 0.53846157, 0.5 , 0.46666667, 0.4375 , 0.41176471, 0.3888889 , 0.36842105, 0.34999999], [ 8. , 4. , 2.66666675, 2. , 1.60000002, 1.33333337, 1.14285719, 1. , 0.8888889 , 0.80000001, 0.72727275, 0.66666669, 0.61538464, 0.5714286 , 0.53333336, 0.5 , 0.47058824, 0.44444445, 0.42105263, 0.40000001], [ 9. , 4.5 , 3. , 2.25 , 1.79999995, 1.5 , 1.28571427, 1.125 , 1. , 0.89999998, 0.81818181, 0.75 , 0.69230771, 0.64285713, 0.60000002, 0.5625 , 0.52941179, 0.5 , 0.47368422, 0.44999999], [ 10. , 5. , 3.33333325, 2.5 , 2. , 1.66666663, 1.42857146, 1.25 , 1.11111116, 1. , 0.90909094, 0.83333331, 0.76923078, 0.71428573, 0.66666669, 0.625 , 0.58823532, 0.55555558, 0.52631581, 0.5 ]]) >>> p array([[ 2., 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21.], [ 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22.], [ 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23.], [ 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24.], [ 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25.], [ 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26.], [ 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27.], [ 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28.], [ 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28., 29.], [ 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24., 25., 26., 27., 28., 29., 30.]]) Note that you have to provide the shape information for ux and uy because fortran writes them together as a stream of 400 numbers. -Neil

10 10

Fwd: [numfocus] Grants up to $3k available to NumFOCUS projects (sponsored & affiliated)
by Ralf Gommers April 3, 2017

April 3, 2017

Hi all, For those who did not see the call on the NumFOCUS list, the below may be of interest. I also have a proposal idea: a redesign of numpy.org. Our website is very poor, both in terms of content and design. If a good designer spends 40 hours on it, that should be enough to change it into something nice (or at least not embarassing). What do you think? Other ideas? Cheers, Ralf ---------- Forwarded message ---------- From: Gina Helfrich <gina(a)numfocus.org> Date: Fri, Mar 17, 2017 at 10:51 AM Subject: Re: [numfocus] Grants up to $3k available to NumFOCUS projects (sponsored & affiliated) To: numfocus(a)googlegroups.com, devs(a)numfocus.org There is no specific template, but proposals should be kept under 2 pages. Maximum 1 submission per project. Required elements of the proposal are: - title - project description - benefit to project/community - project team - and budget Submit proposals to info(a)numfocus.org Best, Gina On Thu, Mar 16, 2017 at 1:13 PM, Bob Carpenter <carp(a)alias-i.com> wrote: > Is there a format you're expecting for proposals? > > Are applications limited to one per project? > > Hard to imagine a project won't have something to do with $3K, > so I imagine you'll get applications from all of the projects if > the proposals take less than an hour or two to put together. > > - Bob > > > On Mar 16, 2017, at 12:14 PM, Gina Helfrich <gina(a)numfocus.org> wrote: > > > > Call for Proposals - Small Development Grants > > > > NumFOCUS is asking for proposals from its sponsored and affiliated > projects for targeted small development projects with a clear benefit to > those projects. This call is motivated by the success of our 2016 > end-of-year fundraising drive; we want to direct the donated funds to our > projects in a way that has impact and visibility to donors and the wider > community. > > > > There are no restrictions on what the funding can be used for. Whether > it’s code development, documentation work, an educational, sustainability > or diversity initiative, or yet something else, we trust the projects > themselves to understand what they need and explain that need in the > proposal. > > > > Available Funding: > > • Up to $3,000 per proposal > > • Allocated funding is $9,000; depending on the number and quality > of proposals this may be adjusted up or down. > > > > Eligibility: > > • Proposals must be approved by the leadership of a NumFOCUS > sponsored or affiliated project. > > • Proposed work must have a clear outcome, achievable within 2017. > > • The call is open to applicants from any nationality and can be > performed at any university, institute or business worldwide (US export > laws permitting). > > > > Timeline: > > • Mid-March 2017: Call for Proposals released > > • 3 April 2017: deadline for proposal submissions > > • 17 April: successful proposals announced > > > > -- > > Dr. Gina Helfrich > > Communications Director, NumFOCUS > > gina(a)numfocus.org > > 512-222-5449 > > > > > > > > > > -- > > You received this message because you are subscribed to the Google > Groups "NumFOCUS" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to numfocus+unsubscribe(a)googlegroups.com. > > For more options, visit https://groups.google.com/d/optout. > > -- > You received this message because you are subscribed to the Google Groups > "NumFOCUS" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to numfocus+unsubscribe(a)googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- Dr. Gina Helfrich Communications Director, NumFOCUS gina(a)numfocus.org 512-222-5449 <(512)%20222-5449> <http://www.numfocus.org> <https://twitter.com/NumFOCUS> <https://www.facebook.com/NumFOCUS/> -- You received this message because you are subscribed to the Google Groups "NumFOCUS" group. To unsubscribe from this group and stop receiving emails from it, send an email to numfocus+unsubscribe(a)googlegroups.com. For more options, visit https://groups.google.com/d/optout.

6 10

speed of random number generator compared to Julia
by Pierre Haessig April 3, 2017

April 3, 2017

Hello, I have a simulation code which uses intensively random number generation (e.g. normal rng). I wanted to compare the performance of this simulation across some Numpy implementations and some Julia implementation. In the end, in both languages, the respective best performing implementations are dominated by the rng, but Julia wins by a factor of 4-5, because the rng is 4-5x faster. Here are some timing results: I*n IPython (Python 3.5.2 from Anaconda)* import numpy as np N = 10**6 %timeit np.random.normal(size=N) 10 loops, best of 3: 37.1 ms per loop %timeit np.random.uniform(size=N) 100 loops, best of 3: 10.2 ms per loop *In Julia (0.4.7 x86_64 from Debian testing)* N = 10^6 @time randn(N); 0.007802 seconds (6 allocations: 7.630 MB) @time rand(N); 0.002059 seconds (8 allocations: 7.630 MB) (with some variations between trials) Results are consistent in the sense that generating Gaussian numbers is 3-4 times slower than uniform in Python and in Julia. But how come Julia is 4-5x faster since Numpy uses C implementation for the entire process ? (Mersenne Twister -> uniform double -> Box-Muller transform to get a Gaussian https://github.com/numpy/numpy/blob/master/numpy/random/mtrand/randomkit.c). Also I noticed that Julia uses a different algorithm (Ziggurat Method from Marsaglia and Tsang , https://github.com/JuliaLang/julia/blob/master/base/random.jl#L700) but this doesn't explain the difference for uniform rng. best, Pierre

3 7

Fwd: SciPy2017 Sprints FinAid for sprint leaders/core devs
by Nathaniel Smith March 31, 2017

March 31, 2017

In case anyone is interested in helping run a NumPy sprint at SciPy this year: ---------- Forwarded message ---------- From: Jonathan Rocher <jonathanrocher(a)gmail.com> Date: Mon, Mar 27, 2017 at 7:06 AM Subject: SciPy2017 Sprints FinAid for sprint leaders/core devs To: Nathaniel Smith <njs(a)pobox.com>, charlesr.harris(a)gmail.com, jtaylor.debian(a)googlemail.com, matthew.brett(a)gmail.com Dear Nathaniel, Chuck, Julian and Matt, cc: SciPy2017 FinAid and Sprints co-chairs This year, SciPy2017 Sprint and Financial Aid comittees would like like to encourage more Scientific Python leaders to lead sprints at the conference and grow our foundations and the contributor base! As such, we're launching, this year, a new "Sprint Leader FinAid", which will cover two nights of lodging for sprint leaders who can offer their knowledge of a package central to our community and pedagogical skills to help make sprints more accessible and grow the contributor pool. Would you be interested in applying? If not, would you be kind enough to help us encourage qualified individuals to lead a sprint for NumPy? Please feel free to forward them our emails and this note! Cheers, Scott Collis, Eric Ma & Jonathan Rocher -- Jonathan Rocher Austin TX, USA twitter:@jonrocher, linkedin:jonathanrocher ------------------------------------------------------------- -- Nathaniel J. Smith -- https://vorpus.org

5 6

__array_ufunc__ counting down to launch, T-24 hrs.
by Charles R Harris March 31, 2017

March 31, 2017

Hi All, Just a note that the __array_ufunc__ PR is ready to merge. If you are interested, you can review here <https://github.com/numpy/numpy/pull/8247>. Chuck

5 5

Optimize evaluation of function on matrix
by Florian Lindner March 31, 2017

March 31, 2017

Hello, I have this function: def eval_BF(self, meshA, meshB): """ Evaluates single BF or list of BFs on the meshes. """ if type(self.basisfunction) is list: A = np.empty((len(meshA), len(meshB))) for i, row in enumerate(meshA): for j, col in enumerate(meshB): A[i, j] = self.basisfunction[j](row - col) else: mgrid = np.meshgrid(meshB, meshA) A = self.basisfunction( np.abs(mgrid[0] - mgrid[1]) ) return A meshA and meshB are 1-dimensional numpy arrays. self.basisfunction is e.g. def Gaussian(radius, shape): """ Gaussian Basis Function """ return np.exp( -np.power(shape*abs(radius), 2)) or a list of partial instantations of such functions (from functools.partial). How can I optimize eval_BF? Esp. in the case of basisfunction being a list. Thanks! Florian

3 5

Re: [Numpy-discussion] migration of all scipy.org mailing lists
by Pauli Virtanen March 28, 2017

March 28, 2017

Sun, 26 Mar 2017 21:05:47 +1300, Ralf Gommers kirjoitti: [clip] > The move is complete, and I have updated the mailing list addresses at: > - http://scipy.org/scipylib/mailing-lists.html (done) > - numpy codebase (PR https://github.com/numpy/numpy/pull/8840) > - scipy codebase (PR https://github.com/scipy/scipy/pull/7226) > - Nabble (opened issue > http://support.nabble.com/All-scipy-org-mailing-list-moved-to-python- org-td7597902.html), > also for ipython-dev > > If anyone knows of other places, please let me know (or fix it). Should the Read/Search links be removed from this page: https://scipy.org/scipylib/mailing-lists.html They point to Gmane, which currently appears to just serve a blank page. So maybe Gmane really is going away. The Gmane NNTP server still works, but apparently you cannot post via it because the To-address still points to @scipy.org. Not sure if there is someone who can fix this any more. -- Pauli Virtanen

2 2

Re: [Numpy-discussion] Fwd: [numfocus] Grants up to $3k available to NumFOCUS projects, (sponsored & affiliated)
by Pauli Virtanen March 27, 2017

March 27, 2017

Mon, 27 Mar 2017 08:21:37 -0700, Chris Barker kirjoitti: > On Mon, Mar 27, 2017 at 3:33 AM, Julian Taylor > <jtaylor.debian(a)googlemail.com> wrote: > >> - add ascii/latin1 dtype to support a compact python3 string array, >> deprecate 's' dtype which has different meaning in python2 and 3 This >> one is probably too big for 3k though. > > probably -- but not THAT big -- it seems pretty straightforward to me. > > The bigger challenge is deciding what to do -- the bikeshedding -- and > the backward incompatibility issues. IIRC, when this came up on the > list, there was nothing like consensus on exactly what to do and how > to do it. TBH, I don't see why 's' should be deprecated --- the operation is well-specified (byte strings + null stripping) and has the same meaning in python2 and 3. Of course, a true 1-byte unicode subset string may be more useful type for some applications, so it could indeed be added. -- Pauli Virtanen

2 1