Mailman 3 August 2007 - NumPy-Discussion

Fwd: Request for Use Cases - h5import and text data
by Francesc Altet Aug. 18, 2007

Aug. 18, 2007

Hi, This has been sent to the hdf-forum(a)hdfgroup.org list, but it should of interest to NumPy/SciPy lists too. Remember that you can access most of the HDF5 files from Python by using PyTables. Cheers, -- >0,0< Francesc Altet http://www.carabos.com/ V V Cárabos Coop. V. Enjoy Data "-" ----------- Original message ------------------------------------- Request for Use Cases - h5import and text data h5import is an HDF5 tool that converts floating point or integer data stored in ASCII or binary files into the HDF5 format. Currently h5import only processes numeric data. The HDF Group plans to add support for importing text data into HDF5 using h5import. We are now soliciting use cases that will guide the design of the text to dataset import feature in h5import. Please consider text you might want to import and how you would want to access that text once it is in the HDF5 file, and send your use cases to help(a)hdfgroup.org before September 17, 2007. Thank you for your help as we strive to improve our tools. ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/ _______________________________________________ Pytables-users mailing list Pytables-users(a)lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/pytables-users ------------------------------------------------------- -- >0,0< Francesc Altet http://www.carabos.com/ V V Cárabos Coop. V. Enjoy Data "-"

1 0

Finding a row match within a numpy array
by Andy Cheesman Aug. 18, 2007

Aug. 18, 2007

Dear nice people I'm trying to match a row (b) within a large numpy array (a). My most successful attempt is below hit = equal(b, a) total_hits = add.reduce(hit, 1) max_hit = argmax(total_hits, 0) answer = a[max_hit] where ... a = array([[ 0, 1, 2, 3], [ 4, 5, 6, 7], [ 8, 9, 10, 11], [12, 13, 14, 15]]) b = array([8, 9, 10, 11]) I was wondering if people could suggest a possible more efficient route as there seems to be numerous steps. Thanks Andy

2 2

numpy.array does not take generators
by Geoffrey Zhu Aug. 17, 2007

Aug. 17, 2007

Hi All, I want to construct a numpy array based on Python objects. In the below code, opts is a list of tuples. For example, opts=[ ('C', 100, 3, 'A'), ('K', 200, 5.4, 'B')] If I use a generator like the following: K=numpy.array(o[2]/1000.0 for o in opts) It does not work. I have to use: numpy.array([o[2]/1000.0 for o in opts]) Is this behavior intended? By the way, it is quite inefficient to create numpy array this way, because I have to create a regular python first, and then construct a numpy array. But I do not want to store everything in vector form initially, as it is more natural to store them in objects, and easier to use when organizing the data. Does anyone know any better way? Thanks, Geoffrey

5 5

.transpose() of memmap array fails to close()
by Glen W. Mabey Aug. 17, 2007

Aug. 17, 2007

Hello, When assigning a variable that is the transpose() of a memmap array, the ._mmap member doesn't get copied, I guess: In [1]:import numpy In [2]:amemmap = numpy.memmap( '/tmp/afile', dtype=numpy.float32, shape=(4,5), mode='w+' ) In [3]:bmemmap = amemmap.transpose() In [4]:bmemmap.close() --------------------------------------------------------------------------- <type 'exceptions.AttributeError'> Traceback (most recent call last) /home/gmabey/src/R9619_dev_acqlibweb/Projects/R9619_NChannelDetection/NED/<ipython console> in <module>() /usr/local/stow/numpy-20070605_svn-py2.5/lib/python2.5/site-packages/numpy/core/memmap.py in close(self) 86 87 def close(self): ---> 88 self._mmap.close() 89 90 def __del__(self): <type 'exceptions.AttributeError'>: 'NoneType' object has no attribute 'close' > /usr/local/stow/numpy-20070605_svn-py2.5/lib/python2.5/site-packages/numpy/core/memmap.py(88)close() 87 def close(self): ---> 88 self._mmap.close() 89 This is an issue when the data is accessed in an order that is different from how it is stored on disk, as: bmemmap = numpy.memmap( '/tmp/afile', dtype=numpy.float32, shape=(4,5), mode='w+' ).transpose() So the object that was originally produced not accessible. I imagine there is some better way to indicate order of dimensions, but regardless, doing In [4]:bmemmap._mmap = amemmap._mmap is a hack workaround. Best regards, Glen Mabey

4 11

deleting value from array
by mark Aug. 16, 2007

Aug. 16, 2007

I am trying to delete a value from an array This seems to work as follows >>> a = array([1,2,3,4]) >>> a = delete( a, 1 ) >>> a array([1, 3, 4]) But wouldn't it make more sense to have a function like a.delete(1) ? I now get the feeling the delete command needs to copy the entire array with exception of the deleted item. I guess this is a hard thing to do efficiently? Thanks, Mark

4 4

OS X universal SciPy: success!
by Josh Marshall Aug. 14, 2007

Aug. 14, 2007

I've been trying to easily build an OS X universal SciPy for some time now. [1] My prior efforts consisted of lipo-ing together the PPC and x86 'Superpacks' put together by Chris Fonnesbeck [2], which worked for distributing my image processing app locally. However, this isn't really a good way to do it, specially not for putting up for general use. I came across a successful Universal build of gfortran [4,5], and then shortly found this message [5] claiming it could possibly be used to build a universal SciPy. So, I gave it a shot and it works! (with some tricks...) The build has both ppc and i386 architectures in every .so in the scipy install. The tests run fine on my G4, but I haven't yet had the chance to try it on an Intel Mac. If anyone is keen to do so, please let me know. There will need to be some modifications to numpy distutils, since it presumes that there isn't a universal Fortran. The patch below is just a hack to get it working, and will break any non-universal gfortran. Essentially, all that needs to happen is to have '-arch ppc -arch i386' added to any call to gfortran (both compile and link) , and the '-march' flags removed. What's the best way to add this functionality to numpy distutils? I couldn't think of any way to test for a universal compiler, other than trying to compile a test file and seeing if it dies with the multiple arch flags. Regards, Josh Marshall [1] http://mail.python.org/pipermail/pythonmac-sig/2006-December/ 018556.html [2] http://trichech.us/?page_id=5 [3] http://r.research.att.com/tools/ [4] http://r.research.att.com/gfortran-4.2.1.dmg [5] http://mail.python.org/pipermail/pythonmac-sig/2007-May/018975.html isengard:~/Development/Python/numpy-svn/numpy/distutils/fcompiler Josh $ svn diff gnu.py Index: gnu.py =================================================================== --- gnu.py (revision 3964) +++ gnu.py (working copy) @@ -102,6 +102,7 @@ minor) opt.extend(['-undefined', 'dynamic_lookup', '-bundle']) + opt.extend(['-arch ppc -arch i386']) else: opt.append("-shared") if sys.platform.startswith('sunos'): @@ -183,12 +184,13 @@ # Since Apple doesn't distribute a GNU Fortran compiler, we # can't add -arch ppc or -arch i386, as only their version # of the GNU compilers accepts those. - for a in '601 602 603 603e 604 604e 620 630 740 7400 7450 750'\ - '403 505 801 821 823 860'.split(): - if getattr(cpu,'is_ppc%s'%a)(): - opt.append('-mcpu='+a) - opt.append('-mtune='+a) - break + #for a in '601 602 603 603e 604 604e 620 630 740 7400 7450 750'\ + # '403 505 801 821 823 860'.split(): + # if getattr(cpu,'is_ppc%s'%a)(): + # opt.append('-mcpu='+a) + # opt.append('-mtune='+a) + # break + opt.append('-arch ppc -arch i386') return opt

1 0

Vectorize leaks
by Chris Fonnesbeck Aug. 13, 2007

Aug. 13, 2007

I have narrowed a memory leak in PyMC down to the vectorize() function in numpy. I have a simple inverse logit transformation function: invlogit = lambda x: 1.0 / (1.0 + exp(-1.0 * x)) which runs without leaking when used iteratively during simulations. However, when I try to vectorize it, the process' rsize grows each iteration of the simulation. Using a recent (<2 days old) svn build of numpy on OS X 10.4. C.

2 1

f2py and string arrays
by Mark.Miller Aug. 13, 2007

Aug. 13, 2007

Quick question... I have a Fortran function for f2py declared as follows (just an example). module test1 contains subroutine manip(length, array, a, b) integer :: length, a,b character(length) :: array(0:a-1,0:b-1) !f2py intent(in) length,a,b !f2py intent(inout) array array(0,0) = "1111" end subroutine manip end module test1 It compiles using f2py without issue. The f2py-generated docstring correctly lists the required arguments: manip - Function signature: manip(length,array,[a,b]) Required arguments: length : input int array : in/output rank-2 array('S') with bounds (a,b) However, I'm getting some type conversion errors when using the function in python: >>> from mymodule import test1 >>> import numpy >>> a=numpy.empty((3,3,'S4',order='F') >>> a[:,:]='2222' >>> test1.manip(4,a,3,3) Traceback (most recent call last): File "(stdin)", line 1, in (module) ValueError: failed to initialize intent(inout) array --expected elsize = 1 but got 4 -- input 'S' not compatible to 'c' Does f2py not work with numpy string arrays? I have some excellent alternate implementations for this type of thing, but would prefer an approach similar to that outlined above. Thanks, -Mark

1 0

[f2py] Adding custom code in module initialization code
by David Cournapeau Aug. 13, 2007

Aug. 13, 2007

Hi, I would like to know if it is possible to tell f2py to call some functions inside the initialization function of a module ? I found a mention to add some function to the module function list, but nothing about the initialization function. cheers, David

2 3

SIMD friendly allocators: first patch
by David Cournapeau Aug. 11, 2007

Aug. 11, 2007

Hi, I put a first version of aligned allocators for numpy here: http://projects.scipy.org/scipy/numpy/ticket/568 (sorry for the duplicate, but I had problems connecting with the trac server). I have tested it on linux only for now, but if problem arise, it should not be too difficult to solve (will test it soon on windows and mac os X). It does not provide yet high level interface (eg requestion python arrays with given alignment), but if people agree with the current design, those should not be too difficult to implement. cheers, David

1 0