Module wrapping

[hgrecco]

I am cross posting this here from hgrecco/pint#24

It would be nice to transform your umath module in a function plus a function call

import math

def wrap_module(module, wrapped, wrapfun):
    # Here goes most of the code inside umath
    # but replacing 
    # math -> wrapped
    # wraps -> wrapfun

wrap_module(sys.modules[__name__], math, wraps)

In this way, other people using your library for their own classes (like me!) could just call wrap_module with a different wrapping function.

What do you think?

[lebigot]

I see what you mean.

In principle, uncertainties may do something similar. Note, however, that not all functions are wrapped in the same way anymore: functions that are locally constant (umath.ceil(), etc.) now return a Python constant, not a value with uncertainty, so they are wrapped in a specific way.

However, I do not want to add anything that is not necessary, I prefer minimal, "essential" code. I wonder if the procedure that you propose is the best way to go: I guess that you want your module to call wrap_module() with wrapped = your module? But why not do things in reverse: can your code use (wrap, I guess) the functions from uncertainties.umath (if needed) instead of those from math?

[lebigot]

Hi Hermann,

Is the issue that you initially raised still a problem for pint?

[hgrecco]

We now use uncertainties as the numerical class underlying Pint's Measurements (ufloat + units). All simple things work as charm (simple arithmetic, string formatting, etc). I will soon implement string parsing (also using uncertainties functions). I am really happy about this.

We are now looking for a way to have arrays and math operations.

For arrays, my understanding is that the suggested way to use numpy arrays with uncertainties is to create an ndarray of ufloat objects. In Pint, we do the opposite: we embed an array into a Quantity. This difference creates some integration issues .

For math operations, in Pint we chose to use numpy functions (instead of Python's math). So instead of creating a module wrapping math we have added methods and attributes to Quantity so it can deal properly with numpy functions. In this way we support almost every single numpy function without requiring the users to import a different module. We are thinking what is the best way to start providing math support for Measurements.

[lebigot]

Thank you for this very lucid description of the situation: I understand it much better.

I would like to explore that "Pint way" of integrating NumPy. Can you describe in more details how Pint "properly deals with NumPy functions" through the "addition of methods and attributes"? I haven't looked at your code, but I have looked for such a protocol in NumPy's documentation without ever finding any…

[hgrecco]

Just for clarity, let me say that when I say support numpy functions I mean that if o is your object of class O, numpy.cos(o) does the right thing. In addition to monkey patching (which I do not like), there are two ways to do it: embedding and subclassing.

You can embed a numpy array in your class O and implement numpy ndarray attributes and methods (clip, fill, put, etc) and magic methods (__array_wrap__, __array_finalize__, etc).

These magic methods are called at the beginning or end of numpy ufuncs and other numpy functions. One of the arguments that they receive is the ufunc that was called and therefore you can take appropriate action for every calculation. For example you can have dict mapping each ufunc to its derivative and after each ufunc call, calculate the new std_dev and use it in the object.

This is the way we have originally choose in Pint. It has worked remarkably well and we support more numpy functions than any other units package out there. However, we have found a limitation. Any function that calls asanyarray or similar fails as it is not a ndarray subclass and there seems to be no way out. You cannot duck-type. This is the reason for hgrecco/pint#37, hgrecco/pint#39, hgrecco/pint#127

So we are looking now into subclassing. The implementation will be roughly the same but everything should work. I am trying to do it without loosing the ability to use Pint without NumPy, keeping the API simple and a single constructor. This will probably come in Pint 0.6 (we will release 0.5 before the end of this month)

Finally, let me add that this comes with a cost. In Pint, some calculations are done twice. To my knowledge, there is no way to signal NumPy that my pure python code will handle the calculation. This means that if you do sqrt(1. * kilometer / meter) (without simplifying first), Pint actually does sqrt(1000), but NumPy still does sqrt(1) and this value is dropped. This will not be a problem for you because to calculate f(x +/- e) you still need f(x).

[lebigot]

This is interesting. Thank you for the information.

Would it help Pint if uncertainties.unumpy.uarray() created a custom undarray instead, that defined the NumPy magic methods?

[hgrecco]

Pint transfers the task to the underlying numerical type. If the user wants numpy.cos(o), Pint takes care of the units and then calls numpy.cos(<magnitude of o with the right units>). So if you create numerical type (undarray) that supports this type of operations with numpy functions, it will be a big win.

[lebigot]

I will have a look, then. I'm sorry but I'm not sure when, though,… The fact that NumPy always calls the ufunc complicates things for uncertainties, but as you said at least this won't incur a processing time penalty. I need to invest more thoughts in this before deciding what to do. I hope this is not too much of a problem for Pint.

[hgrecco]

Take your time. I am really happy about how the collaboration between Pint and uncertainties is working. We find a common interest, we agree on a few things and then we implement it. We all have other things to do, so features land when we can implement them.

Please keep me posted! And if you find a better way, I might ported to Pint. For 0.6, I will start working on subclassing. I will let you know how that goes.
Maybe you found a better solution that we can take it to Pint.

[lebigot]

Leaving a reference note about handling uncertainties in arrays in a different way than currently done by uncertainties: numpy.cos(obj) returns obj.cos() if it is defined (NumPy 1.8.1). I have not checked whether this is guaranteed by NumPy, but if it is this might be very useful.

[lebigot]

For reference: direct implementation of uncertainties in NumPy ndarrays: astropy/astropy#3715.

[hgrecco]

Thanks for the reference. It is roughly the direction I was proposing, right?

[lebigot]

@hgrecco As far as I can tell, yes, it is similar to what you were proposing, but I haven't checked the details.