Mailman 3 July 2010 - NumPy-Discussion

Difference between shape=() and shape=(1,)
by John Reid July 17, 2010

July 17, 2010

Hi, I have some arrays of various shapes in which I need to set any NaNs to 0. I have been doing the following: a[numpy.where(numpy.isnan(a)] = 0. as you can see here: In [20]: a=numpy.ones(2) In [21]: a[1]=numpy.log(-1) In [22]: a Out[22]: array([ 1., NaN]) In [23]: a[numpy.where(numpy.isnan(a))]=0. In [24]: a Out[24]: array([ 1., 0.]) Unfortunately, I've just discovered that when a.shape == () this doesn't work at all. For example: In [41]: a=numpy.array((1.)) In [42]: a.shape Out[42]: () In [43]: a[numpy.where(numpy.isnan(a))]=0. In [44]: a Out[44]: array(0.0) but if the shape is (1,), everything is ok: In [47]: a=numpy.ones(1) In [48]: a.shape Out[48]: (1,) In [49]: a[numpy.where(numpy.isnan(a))]=0. In [50]: a Out[50]: array([ 1.]) What's the difference between the 2 arrays with different shapes? If I pass a scalar into numpy.asarray() why do I get an array of shape () back? In my case this has caused a subtle bug. Is there a better way to set NaNs in an array to 0? Thanks for any tips, John.

6 9

__array__struct__: about using PyCapsule instead of PyCObject for Python 2.7
by Lisandro Dalcin July 17, 2010

July 17, 2010

Do we really need this for NumPy 2? What about using the old PyCObject for all Py 2.x versions? If this is not done, perhaps NumPy 2 on top of Py 2.x should still accept the __array_struct__ being a PyCObject? As reference, Cython still uses PyCObject for Py<3.1 http://hg.cython.org/cython-devel/rev/8a58f1544bd8#l1.33 . -- Lisandro Dalcin --------------- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169

5 9

_wrapit bug?
by Benjamin Root July 17, 2010

July 17, 2010

Hi, I was just digging through the fromnumeric.py code when I noticed that the function _wrapit used asarray() rather than asanyarray(). I don't know if this function really gets used anymore (as it gets called in special situations), but would seem to be an issue as it would cause arrays such as masked_array to lose its type. For example, if _wrapit was called for a max() function on a masked array, the max returned would be for the ndarray's max, not the masked_array's max. Let me know if this is a non-issue, I just thought I ought to point it out. Thanks, Ben Root

2 1

build problem on windows: python26.dll not found
by Skipper Seabold July 16, 2010

July 16, 2010

I am way out of my element trying to build numpy on Windows, but we are having a replicable (though somewhat inconsistently on some machines) problem using fmin_cg on Windows that I am trying to debug. I'm on Windows 7 64-bit with Python 2.6.4 First, I try to build numpy. It complains about having no C:\windows\system32\python26.dll, but note C:\Users\skipper>dir %windir%\system32\python26.dll Volume in drive C has no label. Volume Serial Number is 182E-23CE Directory of C:\Windows\system32 10/26/2009 08:36 AM 2,764,288 python26.dll 1 File(s) 2,764,288 bytes 0 Dir(s) 72,496,721,920 bytes free C:\Users\skipper\source\numpy>python setup.py config --compiler=mingw32 build -- compiler=mingw32 bdist_wininst Running from numpy source directory.Forcing DISTUTILS_USE_SDK=1 F2PY Version 2_8480 blas_opt_info: blas_mkl_info: libraries mkl,vml,guide not found in C:\Users\skipper\source\atlaslib32 NOT AVAILABLE atlas_blas_threads_info: Setting PTATLAS=ATLAS libraries ptf77blas,ptcblas,atlas not found in C:\Users\skipper\source\atlasli b32 NOT AVAILABLE atlas_blas_info: FOUND: libraries = ['f77blas', 'cblas', 'atlas'] library_dirs = ['C:\\Users\\skipper\\source\\atlaslib32'] language = c C:\Users\skipper\source\numpy\numpy\distutils\command\config.py:397: Deprecation Warning: +++++++++++++++++++++++++++++++++++++++++++++++++ Usage of get_output is deprecated: please do not use it anymore, and avoid configuration checks involving running executable on the target machine. +++++++++++++++++++++++++++++++++++++++++++++++++ DeprecationWarning) No module named msvccompiler in numpy.distutils; trying from distutils FOUND: libraries = ['f77blas', 'cblas', 'atlas'] library_dirs = ['C:\\Users\\skipper\\source\\atlaslib32'] language = c define_macros = [('ATLAS_INFO', '"\\"?.?.?\\""')] lapack_opt_info: lapack_mkl_info: mkl_info: libraries mkl,vml,guide not found in C:\Users\skipper\source\atlaslib32 NOT AVAILABLE NOT AVAILABLE atlas_threads_info: Setting PTATLAS=ATLAS libraries ptf77blas,ptcblas,atlas not found in C:\Users\skipper\source\atlasli b32 libraries lapack_atlas not found in C:\Users\skipper\source\atlaslib32 numpy.distutils.system_info.atlas_threads_info NOT AVAILABLE atlas_info: libraries lapack_atlas not found in C:\Users\skipper\source\atlaslib32 numpy.distutils.system_info.atlas_info FOUND: libraries = ['lapack', 'f77blas', 'cblas', 'atlas'] library_dirs = ['C:\\Users\\skipper\\source\\atlaslib32'] language = f77 No module named msvccompiler in numpy.distutils; trying from distutils FOUND: libraries = ['lapack', 'f77blas', 'cblas', 'atlas'] library_dirs = ['C:\\Users\\skipper\\source\\atlaslib32'] language = f77 define_macros = [('ATLAS_INFO', '"\\"?.?.?\\""')] non-existing path in '': 'site.cfg.example' running config running build running config_cc unifing config_cc, config, build_clib, build_ext, build commands --compiler opti ons running config_fc unifing config_fc, config, build_clib, build_ext, build commands --fcompiler opt ions running build_src build_src building py_modules sources building library "npymath" sources Looking for python26.dll Building import library (arch=AMD64): "C:\Python26\libs\libpython26.a" (from C:\ Windows\system32\python26.dll) C:\Windows\system32\python26.dll C:\Python26\libs\python26.def objdump.exe: 'C:\Windows\system32\python26.dll': No such file Traceback (most recent call last): File "setup.py", line 210, in <module> setup_package() File "setup.py", line 203, in setup_package configuration=configuration ) File "C:\Users\skipper\source\numpy\numpy\distutils\core.py", line 186, in set up return old_setup(**new_attr) File "C:\Python26\lib\distutils\core.py", line 152, in setup dist.run_commands() File "C:\Python26\lib\distutils\dist.py", line 975, in run_commands self.run_command(cmd) File "C:\Python26\lib\distutils\dist.py", line 995, in run_command cmd_obj.run() File "C:\Users\skipper\source\numpy\numpy\distutils\command\build.py", line 37 , in run old_build.run(self) File "C:\Python26\lib\distutils\command\build.py", line 134, in run self.run_command(cmd_name) File "C:\Python26\lib\distutils\cmd.py", line 333, in run_command self.distribution.run_command(command) File "C:\Python26\lib\distutils\dist.py", line 995, in run_command cmd_obj.run() File "C:\Users\skipper\source\numpy\numpy\distutils\command\build_src.py", lin e 152, in run self.build_sources() File "C:\Users\skipper\source\numpy\numpy\distutils\command\build_src.py", lin e 163, in build_sources self.build_library_sources(*libname_info) File "C:\Users\skipper\source\numpy\numpy\distutils\command\build_src.py", lin e 298, in build_library_sources sources = self.generate_sources(sources, (lib_name, build_info)) File "C:\Users\skipper\source\numpy\numpy\distutils\command\build_src.py", lin e 385, in generate_sources source = func(extension, build_dir) File "numpy\core\setup.py", line 673, in get_mathlib_info st = config_cmd.try_link('int main(void) { return 0;}') File "C:\Python26\lib\distutils\command\config.py", line 257, in try_link self._check_compiler() File "C:\Users\skipper\source\numpy\numpy\distutils\command\config.py", line 4 5, in _check_compiler old_config._check_compiler(self) File "C:\Python26\lib\distutils\command\config.py", line 107, in _check_compil er dry_run=self.dry_run, force=1) File "C:\Users\skipper\source\numpy\numpy\distutils\ccompiler.py", line 561, i n new_compiler compiler = klass(None, dry_run, force) File "C:\Users\skipper\source\numpy\numpy\distutils\mingw32ccompiler.py", line 91, in __init__ build_import_library() File "C:\Users\skipper\source\numpy\numpy\distutils\mingw32ccompiler.py", line 282, in build_import_library return _build_import_library_amd64() File "C:\Users\skipper\source\numpy\numpy\distutils\mingw32ccompiler.py", line 305, in _build_import_library_amd64 generate_def(dll_file, def_file) File "C:\Users\skipper\source\numpy\numpy\distutils\mingw32ccompiler.py", line 252, in generate_def if i == len(dump): UnboundLocalError: local variable 'i' referenced before assignment

1 0

Array concatenation performance
by John Porter July 16, 2010

July 16, 2010

Has anyone got any advice about array creation. I've been using numpy for a long time and have just noticed something unexpected about array concatenation. It seems that using numpy.array([a,b,c]) is around 20 times slower than creating an empty array and adding the individual elements. Other things that don't work well either: numpy.concatenate([a,b,c]).reshape(3,-1) numpy.concatenate([[a],[b],[c])) Is there a better way to efficiently create the array ? See the following snippet: --------------------------------------- import time import numpy as nx print 'numpy version', nx.version.version t = time.time() # test array a = nx.arange(1000*1000) print 'a ',time.time()-t t = time.time() # create array in the normal way.. b0 = nx.array([a,a,a]) print 'b0',time.time()-t t = time.time() # create using empty array b1 = nx.empty((3,)+(a.shape)) b1[0] = a b1[1] = a b1[2] = a print 'b1',time.time()-t print nx.all((b0==b1)) ----------------------------------------- Produces the output: numpy version 1.3.0 a 0.0019519329071 b0 0.286643981934 b1 0.0116579532623 equal True

7 13

How to use decorator numpy.vectorize for methods in class
by Shailendra July 16, 2010

July 16, 2010

Hi All, I tried to use the decorator @numpy.vectorize for the class method without success. Can any one point me to the correct way of using it? I am able to use the numpy.vectorize() sucessfully. Below is the code with did not work ==================== ================ import numpy class test: @numpy.vectorize #<========= Using decorator def add(self,x,y): return x+y def operate(self,operation,x,y): if operation =='add': return self.add(x,y) #<==============trying to call def test_test(): test_instance=test() x=numpy.arange(1,10) y=numpy.arange(11,20) print test_instance.operate('add',x,y) if __name__=='__main__': test_test() ==================== ================ File "test.py", line 10, in operate return self.add(x,y) File "/usr/local/lib/python2.5/site-packages/numpy/lib/function_base.py", line 1854, in __call__ raise ValueError, "mismatch between python function inputs"\ ValueError: mismatch between python function inputs and received arguments --------------------------------------------------------------------------------------------------------- But the following modification works ==================== ================ import numpy class test: def add(self,x,y): return x+y def operate(self,operation,x,y): if operation =='add': newfunc=numpy.vectorize(self.add) #<============ Explicitly vectorizing return newfunc(x,y) #<============ calling the vectorized func def test_test(): test_instance=test() x=numpy.arange(1,10) y=numpy.arange(11,20) print test_instance.operate('add',x,y) if __name__=='__main__': test_test() ==================== ================ What exactly is happening in case of using decorator? Thanks, Shailendra

2 1

isinf raises in inf
by John Hunter July 16, 2010

July 16, 2010

I am seeing a problem on Solaris since I upgraded to svn HEAD. np.isinf does not handle np.inf. See ipython session below. I am not seeing this problem w/ HEAD on an ubuntu linux box I tested on In [1]: import numpy as np In [2]: np.__version__ Out[2]: '2.0.0.dev8480' In [3]: x = np.inf np.inf np.info np.infty In [3]: x = np.inf In [4]: np.isinf(x) Warning: invalid value encountered in isinf Out[4]: True In [5]: np.seter np.seterr np.seterrcall np.seterrobj In [5]: np.seterr(all='raise') Out[5]: {'over': 'print', 'divide': 'print', 'invalid': 'print', 'under': 'ignore'} In [6]: np.isinf(x) --------------------------------------------------------------------------- FloatingPointError Traceback (most recent call last) /home/titan/johnh/<ipython console> FloatingPointError: invalid value encountered in isinf In [7]: !uname -a SunOS udesktop191 5.10 Generic_139556-08 i86pc i386 i86pc In [43]: !gcc --version gcc (GCC) 3.4.3 (csl-sol210-3_4-branch+sol_rpath) Copyright (C) 2004 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Posted on tracker: http://projects.scipy.org/numpy/ticket/1547

4 14

Question on Absoft 11 and f2py
by Sturla Molden July 16, 2010

July 16, 2010

Any good reason CompaqVisualFCompiler should raise an exception when I'm not using it? Using Enthought 6.2-2. Sturla D:\test>f2py -c --fcompiler=absoft --compiler=msvc -m hello hello.f90 Forcing DISTUTILS_USE_SDK=1 Unexpected ValueError in C:\Python26\lib\site-packages\numpy\distutils\fcompiler \compaq.pyc Traceback (most recent call last): File "C:\Python26\Scripts\f2py-script.py", line 24, in <module> main() File "C:\Python26\lib\site-packages\numpy\f2py\f2py2e.py", line 557, in main run_compile() File "C:\Python26\lib\site-packages\numpy\f2py\f2py2e.py", line 443, in run_co mpile fcompiler.load_all_fcompiler_classes() File "C:\Python26\lib\site-packages\numpy\distutils\fcompiler\__init__.py", li ne 715, in load_all_fcompiler_classes __import__ (module_name) File "C:\Python26\lib\site-packages\numpy\distutils\fcompiler\compaq.py", line 55, in <module> class CompaqVisualFCompiler(FCompiler): File "C:\Python26\lib\site-packages\numpy\distutils\fcompiler\compaq.py", line 95, in CompaqVisualFCompiler raise e ValueError: [u'path', u'lib']

1 0

Re: [Numpy-discussion] [SciPy-User] Saving Complex Numbers
by David Warde-Farley July 16, 2010

July 16, 2010

(CCing NumPy-discussion where this really belongs) On 2010-07-08, at 1:34 PM, cfrazer(a)uci.edu wrote: > Need Complex numbers in the saved file. Ack, this has come up several times according to list archives and no one's been able to provide a real answer. It seems that there is nearly no formatting support for complex numbers in Python. for a single value, "{0.real:.18e}{0.imag:+.18e}".format(val) will get the job done, but because of the way numpy.savetxt creates its format string this isn't a trivial fix. Anyone else have ideas on how complex number format strings can be elegantly incorporated in savetxt? David

2 1

Matrix dot product over an axis(for a 3d array/list of matrices)
by Emmanuel Bengio July 15, 2010

July 15, 2010

Hello, I have a list of 4x4 transformation matrices, that I want to "dot with" another list of the same size (elementwise). Making a for loop that calculates the dot product of each is extremely slow, I thought that maybe it's due to the fact that I have thousands of matrices and it's a python for loop and there's a high Python overhead. I do something like this: >> for a,b in izip(Rot,Trans): >> c.append(numpy.dot(a,b)) Is there a way to do this in one instruction? Or is there a way to do this all using weave.inline? -- Emmanuel

5 12