tinynumpy

tinynumpy copied to clipboard

http://docs.scipy.org/doc/numpy/reference/routines.math.html

Some exist already. Most of these look pretty straightforward to implement.

Hey, I'm trying to use tinynumpy on another project, eppy, where we need to be able to use some numpy functionality, where it's not installed. I need to use the multiply function but it looks like there are some C routine dependencies. I wonder if you have any plans to implement this, or might be willing to help me implement it? I'm eager to do the work but unfamiliar with C.

Cheers

Eric

Hi Eric, sorry about the slow reply. If you can point me at the C routines, I will do my best to decipher them.

Best regards, Wade

Perfect; I think there's only one. I'll take another look later today and get back to you...

Hi Eric, sorry about the slow reply. If you can point me at the C routines, I will do my best to decipher them.

Best regards, Wade

— Reply to this email directly or view it on GitHub https://github.com/wadetb/tinynumpy/issues/10#issuecomment-116706307.

Wade, I've posted the python function multiply() to a gist. I need to dig a little further to determine if it is the only C function, but the most obvious _vec_string(), which is the function returned by multiply().

Interesting, if I read that function’s documentation right:

It takes two arguments, an array of strings, and an array of integers, both ideally having the same length.
The result is an array of strings. For each input string, it is appended to the output array “multiplied" by the corresponding integer value, where multiplication means repeating the string.

So given two input arrays: ‘a’,’b’,’c’,’d' 1,2,1,0

I would expect: ‘a',’bb',’c',''

Regarding its implementation, let’s look at the _vec_string function:

https://github.com/numpy/numpy/blob/38d6f099bf97f46024346112f94881575bb420f0/numpy/core/src/multiarray/multiarraymodule.c#L3857 https://github.com/numpy/numpy/blob/38d6f099bf97f46024346112f94881575bb420f0/numpy/core/src/multiarray/multiarraymodule.c#L3857

From reading through the C code, it chains to _vec_string_with_args since the args tuple is not empty.

The basic operation of _vec_string_with_args is:

1. Create a multi-iterator which iterates the string array + the int array together.
2. Create a result array of the same length as the multi-iterator using the given descriptor; in this case what’s passed in is type of the string array.
3. Iterate over the multi-iterator, calling the given method with the values from the iterator as arguments; in this case “mul” is passed in and called on the string and int.
4. Return the result array.

Once you get past the syntax difference and the refcounting and other syntactic overhead, it’s not too hard to read the C code. Let me know if you need more explanation or if there are other functions you need parsed.

-Wade

On Jun 29, 2015, at 9:49 PM, Eric Youngson [email protected] wrote:

Wade, I've posted the python function multiply https://gist.github.com/eayoungs/a61c7b09ec6f2e4af567 to a gist. I need to dig a little further to determine if it is the only C function, but the most obvious _vec_string() https://gist.github.com/eayoungs/cb0a6896a37008ad11e8, which is the function returned by multiply().

— Reply to this email directly or view it on GitHub https://github.com/wadetb/tinynumpy/issues/10#issuecomment-116910222.

Yeah, it seems strange but that's the only multiply function I've found within numpy.

Hm, the docs imply a more general feature:

http://docs.scipy.org/doc/numpy/reference/generated/numpy.multiply.html http://docs.scipy.org/doc/numpy/reference/generated/numpy.multiply.html

I think it’s because multiply is implemented in pure C in the “np” module, probably for performance. I’m not sure but I think the implementation may be here: https://github.com/numpy/numpy/blob/23e10e18f7951fe040c91134da894f6efdffebe6/numpy/core/src/multiarray/number.c#L370 https://github.com/numpy/numpy/blob/23e10e18f7951fe040c91134da894f6efdffebe6/numpy/core/src/multiarray/number.c#L370

Anyway, this is one of those areas where the architecture of true numpy is way more than what we need for tinynumpy, so what I usually do is carefully read the document, play around with real numpy in a Python shell to see how it behaves, and reimplement the same functionality (to a reasonable extent) in tinynumpy using pure Python.

On Jun 30, 2015, at 12:43 AM, Eric Youngson [email protected] wrote:

Yeah, it seems strange but that's the only multiply function I've found within numpy.

— Reply to this email directly or view it on GitHub https://github.com/wadetb/tinynumpy/issues/10#issuecomment-116954891.

I should mention that in the code where the function is being used, the 'multiply' is actually an asterisk operator (*) rather than a function call but it seems to call the same function when I run it in an IPython notebook.

That implementation seems suspiciously compact. Any idea what the n_ops.multiply function is all about? I keep suspecting I'm going down the rabbit hole with this.

Can you paste a small code snippet that demonstrates the discrepancy? Thanks!

Sent from my Verizon Wireless 4G LTE smartphone

-------- Original message -------- From: Eric Youngson [email protected] Date: 06/30/2015 9:37 PM (GMT-05:00) To: wadetb/tinynumpy [email protected] Cc: Wade Brainerd [email protected] Subject: Re: [tinynumpy] Implement subset of Mathematical functions (#10)

I should mention that in the code where the function is being used, the 'multiply' is actually an asterisk operator (*) rather than a function call but the numpy array works and the tinynumpy array fails.

— Reply to this email directly or view it on GitHub.

I've pulled out the code that's causing the test to fail and posted it here. I've put the unit test and all the functions that it calls. When I run it in an IPython Notebook. I get the following error:

---

NameError Traceback (most recent call last) in () 51 assert almostequal(answer, result, places=3) == True 52 ---> 53 test_tilt()

in test_tilt() 48 ) 49 for poly,answer in data: ---> 50 result = tilt(poly) 51 assert almostequal(answer, result, places=3) == True 52

in tilt(poly) 37 vec_z = array([0,0,1]) 38 # return (90 - angle2vecs(vec_alt, vec_z)) # update by Santosh ---> 39 return (angle2vecs(vec_alt, vec_z)) 40 41 def test_tilt():

in angle2vecs(vec1, vec2) 12 print type(vec1) 13 print type(vec2) ---> 14 vec1_modulus = sqrt((vec1_vec1).sum()) 15 vec2_modulus = sqrt((vec2_vec2).sum()) 16 if (vec1_modulus * vec2_modulus) == 0:

/Users/eayoungs/repo/Code/SimTools/eppy/eppy/geometry/tinynumpy.pyc in mul(self, i) 1094 multiply 1095 """ -> 1096 return array(multiply(self, i)) 1097 1098

/Users/eayoungs/repo/Code/SimTools/eppy/eppy/geometry/tinynumpy.pyc in multiply(a, i) 390 # raise ValueError("Can only multiply by integers") 391 # out_size = _get_num_chars(a_arr) * max(long(i_arr.max()), 0) --> 392 return _vec_string( 393 a_arr, (a_arr.dtype.type, out_size), 'mul', (i_arr,)) 394

NameError: global name '_vec_string' is not defined

Wade, I belive I've implemented all the math functions I need for the other project, eppy. I'm ready to commit those functions back to your project and would like to open a new branch to contain the changes for review before merging so that you can review them. I don't see any other branches in the project, so I'm wondering if I should open a new branch with the issue number and description as it's name?

Thx

Eric

---

FYI: I tried to create a new branch as described above but don't have permissions

Wonderful, thanks Eric. A new branch for the issue sounds fine, feel free to use the naming scheme you proposed.

Thanks Wade, but I don't have permissions yet. Can you grant them to me?

Numpy's structure is such that the determinant function is located in another file called linalg. In order to mask numpy in the import statement, I've implemented a separate linalg file in eppy and I'm wondering if you prefer to have all the functions in the main tinynumpy module with the ndarray class, as they are now, or have the file structure mimic that of numpy.

The thought behind mimicking the numpy structure is so that we can seamlessly use numpy when it's available or use tinynumpy when it's not.

Cheers

Eric

About the branch, the standard way to do this is to Fork tinynumpy using GitHub's interface, commit your branch there, and submit a Pull Request (again using GitHub's interface) to the parent repository.

Then, I can one-click Pull the branch, test, and finally merge it into master.

Regarding files, it seems like the linalg module is part of the numpy package. (I'm just skimming http://stackoverflow.com/questions/5956627/python-import)

So, the least surprising way to import it from tinynumpy would be:

import tinynumpy
import tinynumpy.linalg

Let's try to do whatever arrangement of files and directories enables that syntax.

Regarding testing, I use the python unitest module while almarklein uses something else - I haven't yet been able to run his tests.

Thanks so much for working on this module!

Sounds good, Wade. This is only the second project I've ever contributed to and the other uses a branch and merge process, rather than a fork and pull request. I will fork the code and submit a pull request, probably before Monday.

---

No worries on the contributing; I am using your code for the base ndarray class, so it's the least I can do to contribute back my additions. After all, I'm already creating them for eppy. I'd really appreciate any review or suggestions on the code that you can offer and I'd be happy to make the changes you suggest.

Cheers

Eric