Mailman 3 December 2009 - NumPy-Discussion

Re: [Numpy-discussion] [Nipy-devel] Impossibility to build nipy on recent numpy?
by Fernando Perez Dec. 10, 2009

Dec. 10, 2009

On Thu, Dec 10, 2009 at 3:59 AM, David Cournapeau <cournape(a)gmail.com> wrote: > You could also try playing with the order of the -Ipath to make sure > the right one us picked up before /usr/include, but I am not sure it > is even possible, as /usr/include may always be the first one gcc > looks in. Gael, orthogonal to reporting this to Ubuntu, I think you can do what David suggests by configuring your variables correctly. I have a pretty strictly manipulated set of valid installation $PREFIX location, all of which I can use for ./configure --prefix=$PREFIX or ./setup.py install --prefix=$PREFIX and that's because for each one of these, I correctly configure ALL of these: PATH: binary execution LD_LIBRARY_PATH: dynamic linker search path LIBRARY_PATH: static linking by gcc (like -L) CPATH: generic include path for gcc (like -I), used for all languages C_INCLUDE_PATH: C-specific include path, after CPATH CPLUS_INCLUDE_PATH: C++-specific include path, after CPATH PYTHONPATH: search path for python packages I have some bash code to do this automatically, I can send it your way if you want. This doesn't out of the box work for in-place installs, because it creates pythonpath with pythonX.Y/site-packages, so it would not cover your in-place headers. But you could easily use the utilities in there to configure some of the *PATH variables with the location of your numpy source tree, so the in-place installs work as you expect them. I haven't tested it, but I think it should work. Cheers, f

2 2

Histogram - removing the "new" keyword for 1.4
by David Huard Dec. 10, 2009

Dec. 10, 2009

Hi all, A long time ago, it was decided to change the default behaviour of the histogram function. The new behaviour has been the default in 1.3 and usage of the old behaviour has raised a warning. According to the timeline discussed at the time, version 1.4 would be the time to remove the old stuff entirely. I was wondering if this was satisfactory for all, or if someone still depends on "new=False" ? I don't think there is any harm in keeping it around until 1.5, we'd just have to update the docstring to reflect this. Of course, following the original plan would be better. I'm sorry to bring this so late in the release cycle. Cheers, David Huard

1 0

Assigning complex values to a real array
by Stéfan van der Walt Dec. 10, 2009

Dec. 10, 2009

Hi all, The following code succeeds, while I thought it should fail: a = np.zeros(6) # real b= np.arange(6)*(2+3j) # complex a[1] = b[1] # shouldn't this break? What is the rationale behind this behaviour? Cheers Stéfan

14 32

c++ code
by Neal Becker Dec. 10, 2009

Dec. 10, 2009

Here is my c++ code if you want to play with it. You'll need to find constrained_value for boost (or I can provide it as a patch if you request). As for semantics, different people actually want/expect different behavior. I chose the 'pythonic' approach. If you do fpa @ fpb, where fpa,b are fixed point numbers of same/different sizes and @ is a binary op, my library will _not_ promote the output. Explicit is better. If you want a(32 bit) x b(32 bit) -> c(64 bit), you promote the operands explicitly first.

1 0

Numpy not reconising PA-RISC CPU on HP-UX workstation.
by David Kirkby Dec. 10, 2009

Dec. 10, 2009

See here for a fuller errror message, http://trac.sagemath.org/sage_trac/ticket/7166 but basically I see: gcc: build/src.hp-ux-B.11.11-9000-785-2.6/numpy/core/src/_sortmodule.c In file included from numpy/core/include/numpy/npy_endian.h:22, from numpy/core/include/numpy/ndarrayobject.h:26, from numpy/core/include/numpy/noprefix.h:7, from numpy/core/src/_sortmodule.c.src:29: numpy/core/include/numpy/npy_cpu.h:49:6: error: #error Unknown CPU, please report this to numpy maintainers with information about your platform (OS, CPU and compiler) In file included from numpy/core/include/numpy/ndarrayobject.h:26, The computer is a HP C3600 workstation The CPU is a 64-bit 552 MHz PA-RISC device. The OS is HP-UX 11i also known as HP-UX 11.11. I'm not sure of the cache sizes on this, though I expect data can be found on the web. If a numpy maintainer wants access to the HP-UX machine, let me know your preferred login name by email, and I'll create you an account. Dave

3 3

Re: [Numpy-discussion] [Nipy-devel] Impossibility to build nipy on recent numpy?
by Gael Varoquaux Dec. 10, 2009

Dec. 10, 2009

On Thu, Dec 10, 2009 at 10:59:30AM +0100, Alexis Roche wrote: > Apparently you are trying to link nipy with an in-place build of > numpy, is that correct? Are you confident that the nipy setup does not > mix bits of your local numpy with another site-package install? My > question might be naive: I never build anything in-place (and never > had this bug, although it might be unrelated). It does seem like it might be a mess related to having two numpy installed: the local one, and the system one. It seems that during compilation of nipy, numpy.distutils picks up the wrong header, that is the system ones, and not those of my local instal of numpy (I tried with a 'develop' and with an 'install'). When I compile nipy, the compile options are: compile options: '-DATLAS_INFO="\"3.6.0\"" -I/usr/include -I/home/varoquau/usr/lib/python2.6/site-packages/numpy/core/include -I/home/varoquau/dev/nipy-neurospin/libcstat/fff -I/home/varoquau/dev/nipy-neurospin/libcstat/randomkit -I/home/varoquau/dev/nipy-neurospin/libcstat/wrapper -I/home/varoquau/usr/lib/python2.6/site-packages/numpy/core/include -I/usr/include -I/usr/include/python2.6 -c' In /usr/include, there is (under ubuntu Karmic) a numpy directory, with the numpy headers, corresponding to the system numpy. I don't want to remove these headers, as that would mean removing the python-numpy package on which many other packages depend. Does anybody have a suggestion on how to cleanly solve the problem? I fear I won't be the only one to have it. Gaël

1 0

Flattening an array
by Jake VanderPlas Dec. 10, 2009

Dec. 10, 2009

Hello, I have a function -- call it f() -- which takes a length-N 1D numpy array as an argument, and returns a length-N 1D array. I want to pass it the data in an N-D array, and obtain the N-D array of the result. I've thought about wrapping it as such: #python code: from my_module import f # takes a 1D array, raises an exception otherwise def f_wrap(A): A_1D = A.ravel() B = f(A_1D) return B.reshape(A.shape) #end code I expect A to be contiguous in memory, but I don't know if it will be C_CONTIGUOUS or F_CONTIGUOUS. Is there a way to implement this such that 1) the data in the arrays A and B_1D are not copied (memory issues) 2) the function f is only called once (speed issues)? The above implementation appears to copy data if A is fortran-ordered. Thanks for the help -Jake

3 2

Impossibility to build nipy on recent numpy?
by Gael Varoquaux Dec. 9, 2009

Dec. 9, 2009

In the nipy project, we have cython-generated C files checked in. With the latest numpy I get the following run-time failure: "/home/varoquau/dev/nipy-trunk/nipy/neurospin/register/iconic_matcher.py", line 6, in <module> from routines import _joint_histogram, _similarity, similarity_measures File "numpy.pxi", line 74, in nipy.neurospin.register.routines (nipy/neurospin/register/routines.c:6042) ValueError: numpy.dtype does not appear to be the correct type object Where routines.c is a cython-generated file. I suspect that this is due to the recent change in the size of the struct representing numpy arrays. With other projects, it was sufficient to recompile the project to avoid this problem. With nipy it is not. Could that be because the cython-generated C file encodes the size of the numpy array struct, and has been compiled with a different numpy? If this is the case, I think that it means that we need to all use the same version of numpy (1.3). Does anybody know if the cython folks are going to work around that anytime soon (I am not on the cython mailing list, so I am not asking there)? Nipy folks, does that mean that we should change our policy, and build the cython-generated c files at compile time, thus requiring cython as a build dependency? Any comments appreciated. Gaël

4 6

General Array -> Into Index Array + Value Array of Nonzero Elements
by Benjamin Kern Dec. 9, 2009

Dec. 9, 2009

Hello everyone, at the moment i like to create a numpy interface to a library for numerical optimization. This library uses a special format to include symmetric matrices, basically if you have A = np.array([ [1.0, 0.0, 2.0] [0.0, 3.0, 0.0] [2.0, 0.0, 5.0] ] ) you would have to create 2 arrays which specify the position as well as the the non-zero elements of the lower triangular part of the matrix. So in this example you would need the following arrays to specify the matrix completely A_pos = np.array([0, 2, 3, 5], dtype=int ) A_val = np.array([1.0, 3.0, 2.0, 5.0]) So now to my question. Is there a clever way to extract these two arrays A_pos and A_val from an arbirtrary A (where A.ndim=2) Another question would be if there is the possibility to do something similiar if you are using sparse matrices (from scipy.sparse). Best Benjamin

2 2

numpy distutils breaks scipy install on mac
by Mark Sienkiewicz Dec. 9, 2009

Dec. 9, 2009

When I compile scipy on a mac, the build fails with: ... gfortran:f77: scipy/fftpack/src/dfftpack/dcosqb.f f951: error: unrecognized command line option "-arch" f951: error: unrecognized command line option "-arch" f951: error: unrecognized command line option "-arch" f951: error: unrecognized command line option "-arch" error: Command "/sw/bin/gfortran -Wall -ffixed-form -fno-second-underscore -arch i686 -arch x86_64 -fPIC -O3 -funroll-loops -I/usr/stsci/pyssgdev/2.5.4/numpy/core/include -c -c scipy/fftpack/src/dfftpack/dcosqb.f -o build/temp.macosx-10.3-i386-2.5/scipy/fftpack/src/dfftpack/dcosqb.o" failed with exit status 1 I have % gfortran --version GNU Fortran (GCC) 4.3.0 Copyright (C) 2008 Free Software Foundation, Inc. % which gfortran /sw/bin/gfortran (This /sw/bin apparently means it was installed by "fink". My IT department did this. This is not the recommended compiler from AT&T, but it seems a likely configuration to encounter in the wild, and I didn't expect a problem. ) I traced the problem to numpy/distutils/fcompiler/gnu.py in the class Gnu94FCompiler. The function _universal_flags() tries to detect which processor types are recognized by the compiler, presumably in an attempt to make a macintosh universal binary. It adds "-arch whatever" for each architecture that it thinks it detected. Since gfortran does not recognize "-arch", the compile fails. ( Presumably, some other version of gfortan does accept -arch, or this code wouldn't be here, right? ) The function _can_target() attempts to recognize what architectures the compiler is capable of by passing in -arch parameters with various known values, but gfortran does not properly indicate a problem in a way that _can_target() can detect: % gfortran -arch i386 hello_world.f f951: error: unrecognized command line option "-arch" % gfortran -arch i386 -v Using built-in specs. Target: i686-apple-darwin9 Configured with: ../gcc-4.3.0/configure --prefix=/sw --prefix=/sw/lib/gcc4.3 --mandir=/sw/share/man --infodir=/sw/share/info --enable-languages=c,c++,fortran,objc,java --with-arch=nocona --with-tune=generic --build=i686-apple-darwin9 --with-gmp=/sw --with-libiconv-prefix=/sw --with-system-zlib --x-includes=/usr/X11R6/include --x-libraries=/usr/X11R6/lib --disable-libjava-multilib Thread model: posix gcc version 4.3.0 (GCC) % echo $status 0 % That is, when you say "-v", it gives no indication that it doesn't understand the -arch flag. I didn't ask for a universal binary and I don't need one, so I'm surprised that it is trying to make one for me. I think the correct solution is that _universal_flag() should not add -arch flags unless the user specifically requests one. Unfortunately, I can't write a patch, because I don't have the time it would take to reverse engineer distutils well enough to know how to do it. As is usual when a setup.py auto-detects the wrong compiler flags, the easiest solution is to create a shell script that looks like the compiler, but add/removes flags as necessary: % cat /eng/ssb/auto/prog/binhacks/scipy.osx/gfortran #!/bin/sh args="" for x in $* do case "$x" in -arch) shift shift ;; *) args="$args $x" ;; esac done /sw/bin/gfortran $args Mark S.

3 4