Mailman 3 November 2010 - NumPy-Discussion

Error in API docs?
by Henry Gomersall Nov. 4, 2010

Nov. 4, 2010

There is an inconsistency in the documentation for NPY_INOUT_ARRAY. cf. http://docs.scipy.org/doc/numpy/user/c-info.how-to-extend.html#NPY_INOUT_AR… http://docs.scipy.org/doc/numpy/reference/c-api.array.html#NPY_INOUT_ARRAY The first link includes the flag NPY_UPDATEIFCOPY. Checking the code seems to confirm that the correct version is that in the first link, and also that NPY_OUT_ARRAY is wrong in the API docs. I haven't checked NPY_INOUT_FARRAY or NPY_OUT_FARRAY. Cheers, Henry

2 2

large float32 array issue
by Vincent Schut Nov. 4, 2010

Nov. 4, 2010

Hi, I'm running in this strange issue when using some pretty large float32 arrays. In the following code I create a large array filled with ones, and calculate mean and sum, first with a float64 version, then with a float32 version. Note the difference between the two. NB the float64 version is obviously right :-) In [2]: areaGrid = numpy.ones((11334, 16002)) In [3]: print(areaGrid.dtype) float64 In [4]: print(areaGrid.shape, areaGrid.min(), areaGrid.max(), areaGrid.mean(), areaGrid.sum()) ((11334, 16002), 1.0, 1.0, 1.0, 181366668.0) In [5]: areaGrid = numpy.ones((11334, 16002), numpy.float32) In [6]: print(areaGrid.dtype) float32 In [7]: print(areaGrid.shape, areaGrid.min(), areaGrid.max(), areaGrid.mean(), areaGrid.sum()) ((11334, 16002), 1.0, 1.0, 0.092504406598019437, 16777216.0) Can anybody confirm this? And better: explain it? Am I running into a for me till now hidden ieee float 'feature'? Or is it a bug somewhere? Btw I'd like to use float32 arrays, as precision is not really an issue in this case, but memory usage is... This is using python 2.7, numpy from git (yesterday's checkout), on arch linux 64bit. Best, Vincent.

6 9

~2**32 byte tofile()/fromfile() limit in 64-bit Windows?
by Martin Spacek Nov. 4, 2010

Nov. 4, 2010

I just opened a new ticket (http://projects.scipy.org/numpy/ticket/1660), but I thought I'd bring it up here as well. I can't seem to get tofile() or save() to write anything much bigger than a 2**32 byte array to a file in Py 2.6.6 on 64-bit Windows. They both hang with no errors. Also, fromfile() throws an "IOError: could not seek in file" on > 2**32-1 byte files, although load() seems to work fine on any size of file. I'm a bit surprised I haven't stumbled across these problems before. I've tested as far back as 1.4.1. Is this a known issue? (the ticket has much more detail) Cheers, Martin

4 3

float16/half-precision module
by Mark Wiebe Nov. 3, 2010

Nov. 3, 2010

I wrote a float16/half-precision module here: https://github.com/m-paradox/numpy_half I've tested it with NumPy 1.3.0 and the latest trunk. With it, you can do things like this: >>> import numpy as np, half as h >>> np.array([0,0.1,1.0/3.0], dtype='float16') array([ 0. , 0.09997559, 0.33325195], dtype=float16) >>> a = h.float16(1.5) >>> print a 1.5 >>> a.dtype dtype('float16') Many functions aren't implemented, so things like this don't work: >>> np.arange(10, dtype='float16') Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: no fill-function for data-type. Also, because of bug #809, it looks like there's no way to nicely support '<f2' or '>f2'. >>> np.array([10], dtype='f2') array([ 10.], dtype=float16) >>> np.array([10], dtype='<f2') Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: data type not understood What would need to be done to build it in as a supported NumPy data type? -Mark

1 0

strange behavior of ravel() and flatnonzero() on matrix
by braingateway Nov. 3, 2010

Nov. 3, 2010

>>> aa=matrix([[-1, 2, 0],[0, 0, 3]]) >>> aa matrix([[-1, 2, 0], [ 0, 0, 3]]) >>> aa.nonzero() (matrix([[0, 0, 1]], dtype=int64), matrix([[0, 1, 2]], dtype=int64)) *********OK********* >>> npy.nonzero(aa.flat) (array([0, 1, 5], dtype=int64),) *********OK********* >>> flatnonzero(aa) matrix([[0, 0, 0]], dtype=int64) *******This is Wrong********** If I convert aa to an ndarray, it is OK then aaa=asarray(aa) >>> flatnonzero(aaa) array([0, 1, 5], dtype=int64) Then I figure it out that it might be induced by the behavior of ravel() >>> aaa.shape (2L, 3L) >>> aaa.ravel().shape (6L,) >>> aa.ravel() matrix([[-1, 2, 0, 0, 0, 3]]) >>> _.shape (1L, 6L) Why not make ravel() behaviors consistent under both ndarray and matrix contexts? Or make different flatnonzero() for the matrix context? m.ravel().nonzero()[1]# for matrix a.ravel().nonzero()[0]# for ndarray

2 2

Path to numpy installation
by Juanjo Gomez Navarro Nov. 3, 2010

Nov. 3, 2010

Hi, I have just updated my old version of numpy 1.01 to the version 1.5 *in my Mac*. The problem is that the system does not seem to recognize the new version. When I type print numpy.__path__ I get /System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python/numpy But the new version I have just installed is in /Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/numpy I don't know how to change the path of the installation, or how to say python to search numpy in the new path Any idea??? 2010/10/18 Ralf Gommers <ralf.gommers(a)googlemail.com> > On Mon, Oct 18, 2010 at 9:55 PM, Vincent Davis <vincent(a)vincentdavis.net> > wrote: > > On Sun, Oct 17, 2010 at 5:35 AM, Ralf Gommers > > <ralf.gommers(a)googlemail.com> wrote: > >> Hi, > >> > >> I am pleased to announce the availability of the first release > >> candidate of NumPy 1.5.1. This is a bug-fix release with no new > >> features compared to 1.5.0. > >> > >> Binaries, sources and release notes can be found at > >> https://sourceforge.net/projects/numpy/files/. > >> A note on the available binaries for OS X: these are known to not work > >> on Intel-based OS X 10.5. We hope to have that problem fixed within a > >> week. > >> > >> On Windows there are still two known test failures: > >> - #1610, in fromfile (under Python 2.x) > >> - #1633, a failing test for ldexp (under Python 2.5 only) > >> Please report any other issues on the Numpy-discussion mailing list. > > > > Test pass for me. > > osx 10.6 py27 > > > > OK (KNOWNFAIL=4, SKIP=2) > > <nose.result.TextTestResult run=2648 errors=0 failures=0> > > > Glad it works for you. But I just reopened the OS X gfortran issue, > http://projects.scipy.org/numpy/ticket/1399. > > RC2 in one week. Two other issues to be fixed by then: > http://projects.scipy.org/numpy/ticket/1610 (Pauli has suggested a fix > already) > http://projects.scipy.org/numpy/ticket/1633 > > Cheers, > Ralf > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion(a)scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion > -- Juan José Gómez Navarro Departamento de Física Centro de Investigacion en Óptica y Nanofísica (CIOyN) Universidad de Murcia Campus Espinardo E-30100 Murcia España Tel : +34 968 398552 Fax : +34 968 39 8568 Email: juanjo.gomeznavarro(a)gmail.com, jjgomeznavarro(a)um.es

3 5

List with numpy semantics
by Nikolaus Rath Nov. 3, 2010

Nov. 3, 2010

Hello, I have a couple of numpy arrays which belong together. Unfortunately they have different dimensions, so I can't bundle them into a higher dimensional array. My solution was to put them into a Python list instead. But unfortunately this makes it impossible to use any ufuncs. Has someone else encountered a similar problem and found a nice solution? Something like a numpy list maybe? Best, -Nikolaus -- »Time flies like an arrow, fruit flies like a Banana.« PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C

3 5

Precision difference between dot and sum
by Joon Nov. 2, 2010

Nov. 2, 2010

Hi, I just found that using dot instead of sum in numpy gives me better results in terms of precision loss. For example, I optimized a function with scipy.optimize.fmin_bfgs. For the return value for the function, I tried the following two things: sum(Xb) - sum(denominator) and dot(ones(Xb.shape), Xb) - dot(ones(denominator.shape), denominator) Both of them are supposed to yield the same thing. But the first one gave me -589112.30492110562 and the second one gave me -589112.30492110678. In addition, with the routine using sum, the optimizer gave me "Warning: Desired error not necessarily achieved due to precision loss." With the routine with dot, the optimizer gave me "Optimization terminated successfully." I checked the gradient value as well (I provided analytical gradient) and gradient was smaller in the dot case as well. (Of course, the the magnitude was e-5 to e-6, but still) I was wondering if this is well-known fact and I'm supposed to use dot instead of sum whenever possible. It would be great if someone could let me know why this happens. Thank you, Joon

8 9

Knownfails due to platform and numpy.c99
by Friedrich Romstedt Nov. 2, 2010

Nov. 2, 2010

Hello, This is a NEPP (proposal for a numpy enhancement proposal). It's quite near to Depp, which means idiot in german, but I hope that's not the case. Some tests fail currently because of platform issues. I.e., the platform clib or similar supplies some partly broken implementation which we use. The current approach is to mark those tests on those platforms as K fail. On the other hand, there are platforms which *don't* supply the functions, and there a numpy own implementation chimes in. (Afaik.) 1. We want to test the numpy own implementations, and not the OS supplied ones. This can be seen by the fact that we regard a test not as failing (or, as K fail) if it is due to OS issues. Meaning we only really want to take care about those systems where *our* implementation is used. 2. I propose to export the numpy own implementations *always* under a distinct name, e.g. numpy.c99.exp(). 3. And to test those functions on all platforms, they never have to fail. 4. And to not test the OS supplied versions, since we don't care about them failing anyway. Implementation a) always define the C level functions, but under another name not conflicting with the OS defined ones. E.g.: numpy_cexp(). b) If the OS supplies a function, okay, use it. c) If the OS doesn't supply the function, then define it as a wrapper, which just calls numpy_*(). If a Python user needs corner case safety, he would be advised to use the numpy.c99.*, e.g. numpy.c99.exp(). For C level users, it would be to use numpy_*(), e.g. numpy_cexp() instead of cexp(). Both functions would always be supplied, but do not need to be identical. Using the C99 compliant version might cause speed tradeoff. :: // Include the system header: #include <...> // Define the C99 compliant version: void numpy_cexp(float r, float i, float *or, float *oi){ ...; } // Wrap the C99 compliant version in the usual-named function if necessary ... #ifndef HAVE_CEXP void cexp(...){ wrap(); } #endif ... export numpy_cexp() as ``numpy.c99.exp`` ... ... export cexp() as ``numpy.exp`` -- The following might not be feasible if we *have to* include the system headers in any way. Ralf suggested the very good addition, to support a compilation flag -DC99COMPLIANT or similar, in that case, numpy headers would not incude the system ones (assumed they do), and would always define the functions using the numpy_*() ones. :: // Do not include the system headers if requested so: #ifndef C99COMPLIANT #include <...> #endif // Define the C99 compliant version: void numpy_cexp(float r, float i, float *or, float *oi){ ...; } // Wrap the C99 compliant version in the usual-named function if necessary ... #ifdef C99COMPLIANT void cexp(...){ wrap(); } #else #ifndef HAVE_CEXP void cexp(...){ wrap(); } #endif #endif ... export numpy_cexp() as ``numpy.c99.exp`` ... ... export cexp() as ``numpy.exp`` -- Friedrich

1 0

ANN: NumPy 1.5.1 release candidate 1
by Ralf Gommers Nov. 2, 2010

Nov. 2, 2010

Hi, I am pleased to announce the availability of the first release candidate of NumPy 1.5.1. This is a bug-fix release with no new features compared to 1.5.0. Binaries, sources and release notes can be found at https://sourceforge.net/projects/numpy/files/. A note on the available binaries for OS X: these are known to not work on Intel-based OS X 10.5. We hope to have that problem fixed within a week. On Windows there are still two known test failures: - #1610, in fromfile (under Python 2.x) - #1633, a failing test for ldexp (under Python 2.5 only) Please report any other issues on the Numpy-discussion mailing list. Enjoy, Ralf

7 14