Mailman 3 February 2012 - NumPy-Discussion

avoiding loops when downsampling arrays
by Moroney, Catherine M (388D) Feb. 7, 2012

Feb. 7, 2012

Hello, I have to write a code to downsample an array in a specific way, and I am hoping that somebody can tell me how to do this without the nested do-loops. Here is the problem statement: Segment a (MXN) array into 4x4 squares and set a flag if any of the pixels in that 4x4 square meet a certain condition. Here is the code that I want to rewrite avoiding loops: shape_out = (data_in.shape[0]/4, data_in.shape[1]/4) found = numpy.zeros(shape_out).astype(numpy.bool) for i in xrange(0, shape_out[0]): for j in xrange(0, shape_out[1]): excerpt = data_in[i*4:(i+1)*4, j*4:(j+1)*4] mask = numpy.where( (excerpt >= t1) & (excerpt <= t2), True, False) if (numpy.any(mask)): found[i,j] = True Thank you for any hints and education! Catherine

4 11

Structure of polynomial module
by Stéfan van der Walt Feb. 7, 2012

Feb. 7, 2012

Hi all, I noticed the following docstring on ``np.polynomial.polyval``: In [116]: np.polynomial.polyval? File: /home/stefan/src/numpy/numpy/lib/utils.py Definition: np.polynomial.polyval(*args, **kwds) Docstring: `polyval` is deprecated! Please import polyval from numpy.polynomial.polynomial I guess we don't expect users to do "from numpy.polynomial.polynomial import polyval, Polynomial", so what is the suggested API for getting hold of the polynomial functions? Also, why is numpy.polynomial.polynomial.polyfit different from numpy.polyfit? Regards Stéfan

2 2

fast method to to count a particular value in a large matrix
by Naresh Pai Feb. 6, 2012

Feb. 6, 2012

I am somewhat new to Python (been coding with Matlab mostly). I am trying to simplify (and expedite) a piece of code that is currently a bottleneck in a larger code. I have a large array (7000 rows x 4500 columns) titled say, abc, and I am trying to find a fast method to count the number of instances of each unique value within it. All unique values are stored in a variable, say, unique_elem. My current code is as follows: import numpy as np #allocate space for storing element count elem_count = zeros((len(unique_elem),1)) #loop through and count number of unique_elem for i in range(len(unique_elem)): elem_count[i]= np.sum(reduce(np.logical_or,(abc== x for x in [unique_elem[i]]))) This loop is bottleneck because I have about 850 unique elements and it takes about 9-10 minutes. Can you suggest a faster way to do this? Thank you, Naresh

8 8

datetime64 format parameter?
by John Salvatier Feb. 6, 2012

Feb. 6, 2012

Hello, Is there a way to specify a format for the datetime64 constructor? The constructor doesn't have a doc. I have dates in a file with the format "MM/dd/YY". datetime64 used to be able to parse these in 1.6.1 but the dev version throws an error. Cheers, John

2 2

(no subject)
by Debashish Saha Feb. 6, 2012

Feb. 6, 2012

basic difference between the commands: import numpy as np from numpy import *

2 1

R: Re: "ValueError: total size of new array must be unchanged" only on Windows
by Paolo Zaffino Feb. 6, 2012

Feb. 6, 2012

Yes, I understand this but I don't know because on Linux and Mac it works well. If the matrix size is different it should be different indipendently from os type. Am I wrong? Thanks for your support!

6 8

dtype related deprecations
by Ralf Gommers Feb. 5, 2012

Feb. 5, 2012

Hi, I'm having some trouble cleaning up tests to deal with these two deprecations: DeprecationWarning: Setting NumPy dtype names is deprecated, the dtype will become immutable in a future version DeprecationWarning: DType strings 'O4' and 'O8' are deprecated because they are platform specific. Use 'O' instead They seem fairly invasive, judged by the test noise in both numpy and scipy. Record arrays rely on setting dtype names. There are tests for picking and the buffer protocol that generate warnings for O4/O8. Can anyone comment on the necessity of these deprecations and how to deal with them? Ralf

4 5

"ValueError: total size of new array must be unchanged" only on Windows
by Paolo Feb. 5, 2012

Feb. 5, 2012

Hello, I wrote a function that works on a numpy matrix and it works fine on Mac OS and GNU/Linux (I didn't test it on python 3). Now I have a problem with numpy: the same python file doesn't work on Windows (Windows xp, python 2.7 and numpy 2.6.1). I get this error: matrix=matrix.reshape(a, b, c) ValueError: total size of new array must be unchanged Why? Do anyone have an idea about this? Thank you very much.

2 1

GSOC
by Charles R Harris Feb. 5, 2012

Feb. 5, 2012

Hi All, I thought I'd raise this topic just to get some ideas out there. At the moment I see two areas that I'd like to see addressed. 1. Documentation editor. This would involve looking at the generated documentation and it's organization/coverage as well such things as style and maybe reviewing stuff on the documentation site. This would be more technical writing than coding. 2. Test coverage. There are a lot of areas of numpy that are not well tested as well as some tests that are still doc tests and should probably be updated. This is a substantial amount of work and would require some familiarity with numpy as well as a willingness to ping developers for clarification of some topics. Thoughts? Chuck

6 12

Re: [Numpy-discussion] Trick for fast
by santhu kumar Feb. 3, 2012

Feb. 3, 2012

Hi Josef, I am unclear on what you want to say, but all I am doing in the code is getting inertia tensor for a bunch of particle masses. (http://en.wikipedia.org/wiki/Moment_of_inertia#Moment_of_inertia_tensor) So the diagonals are not actually zeros but would have z^2 + y^2 .. The reason which I said 3secs could be misunderstood .. This code is called many times over a loop and the bulk time is taken in computing this inertial tensor. After code change, the entire loop finishes off in 3 ses. Thanks for alertness, Santhosh On Fri, Feb 3, 2012 at 12:47 PM, <numpy-discussion-request(a)scipy.org> wrote: > Send NumPy-Discussion mailing list submissions to > numpy-discussion(a)scipy.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://mail.scipy.org/mailman/listinfo/numpy-discussion > or, via email, send a message with subject or body 'help' to > numpy-discussion-request(a)scipy.org > > You can reach the person managing the list at > numpy-discussion-owner(a)scipy.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of NumPy-Discussion digest..." > > > Today's Topics: > > 1. Re: Trick for fast (santhu kumar) > 2. Re: Trick for fast (josef.pktd(a)gmail.com) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 3 Feb 2012 12:29:26 -0600 > From: santhu kumar <mesanthu(a)gmail.com> > Subject: Re: [Numpy-discussion] Trick for fast > To: numpy-discussion(a)scipy.org > Message-ID: > <CA+7TRsszzxPLAOz_9pgU5e9mYcHTDQZMzRoMPzpfhrRkcStZEQ(a)mail.gmail.com > > > Content-Type: text/plain; charset="iso-8859-1" > > On Fri, Feb 3, 2012 at 1:29 PM, santhu kumar <mesanthu(a)gmail.com> wrote: > > Hello all, > > > > Thanks for lovely solutions. I have sat on it for some time and wrote it > > myself : > > > > n =x.shape[0] > > ea = np.array([1,0,0,0,1,0,0,0,1]) > > inert = ((np.tile(ea,(n,1))*((x*x).sum(axis=1)[:,np.newaxis]) - > > > np.hstack([x*x[:,0][:,np.newaxis],x*x[:,1][:,np.newaxis],x*x[:,2][:,np.newaxis]]))*mass[:,np.newaxis]).sum(axis=0) > > inert.shape = 3,3 > > > > Does the trick and reduces the time from over 45 secs to 3 secs. > > I do want to try einsum but my numpy is little old and it does not have > it. > > > > Thanks Sebastian (it was tricky to understand your code for me) and Josef > > (clean). > > Isn't the entire substraction of the first term just to set the > diagonal of the result to zero. > > It looks to me now just like the weighted dot product and setting the > diagonal to zero. That shouldn't take 3 secs unless you actual > dimensions are huge. > > Josef > > > > >

3 4