support for simple interpolation with new values

[den-run-ai]

something along the lines:

df.reindex(df.index.tolist()+new_values).sort().interpolate().ix[new_values]

[TomAugspurger]

Could you write up a sketch of what the API would look like?

[den-run-ai]

The API is simple: pass new_values as keyword argument to interpolate()
with a default = None.

On Wed, Aug 12, 2015, 10:16 PM Tom Augspurger [email protected]
wrote:

Could you write up a sketch of what the API would look like?

—
Reply to this email directly or view it on GitHub
#10811 (comment).

[TomAugspurger]

What if there are missing values, are they interpolated at too?
Do you return the interpolated values only, or their union with the original Series?
What does new_values look like for a DataFrame?

On Aug 12, 2015, at 10:23 PM, denfromufa [email protected] wrote:

The API is simple: pass new_values as keyword argument to interpolate()
with a default = None.

On Wed, Aug 12, 2015, 10:16 PM Tom Augspurger [email protected]
wrote:

Could you write up a sketch of what the API would look like?

—
Reply to this email directly or view it on GitHub
#10811 (comment).

—
Reply to this email directly or view it on GitHub.

[den-run-ai]

1. Return interpolation for new_values only.
2. new_values is plain 1d list for dataframe index of 1d.
3. This feature can be extended to Multiindex later.

On Thu, Aug 13, 2015, 7:32 AM Tom Augspurger [email protected]
wrote:

What if there are missing values, are they interpolated at too?
Do you return the interpolated values only, or their union with the
original Series?
What does new_values look like for a DataFrame?

On Aug 12, 2015, at 10:23 PM, denfromufa [email protected]
wrote:

The API is simple: pass new_values as keyword argument to interpolate()
with a default = None.

On Wed, Aug 12, 2015, 10:16 PM Tom Augspurger [email protected]
wrote:

Could you write up a sketch of what the API would look like?

—
Reply to this email directly or view it on GitHub
<#10811 (comment)
.

—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub
#10811 (comment).

[den-run-ai]

I actually forgot .sort() before interpolate.

On Thu, Aug 13, 2015, 9:08 AM Denis Akhiyarov [email protected]
wrote:

1. Return interpolation for new_values only.
2. new_values is plain 1d list for dataframe index of 1d.
3. This feature can be extended to Multiindex later.

On Thu, Aug 13, 2015, 7:32 AM Tom Augspurger [email protected]
wrote:

What if there are missing values, are they interpolated at too?
Do you return the interpolated values only, or their union with the
original Series?
What does new_values look like for a DataFrame?

On Aug 12, 2015, at 10:23 PM, denfromufa [email protected]
wrote:

The API is simple: pass new_values as keyword argument to interpolate()
with a default = None.

On Wed, Aug 12, 2015, 10:16 PM Tom Augspurger <[email protected]

wrote:

Could you write up a sketch of what the API would look like?

—
Reply to this email directly or view it on GitHub
<#10811 (comment)
.

—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub
#10811 (comment).

[den-run-ai]

And new_values should really handle 1d numpy arrays and python iterable
objects.

On Thu, Aug 13, 2015, 9:12 AM Denis Akhiyarov [email protected]
wrote:

I actually forgot .sort() before interpolate.

On Thu, Aug 13, 2015, 9:08 AM Denis Akhiyarov [email protected]
wrote:

1. Return interpolation for new_values only.
2. new_values is plain 1d list for dataframe index of 1d.
3. This feature can be extended to Multiindex later.

On Thu, Aug 13, 2015, 7:32 AM Tom Augspurger [email protected]
wrote:

What if there are missing values, are they interpolated at too?
Do you return the interpolated values only, or their union with the
original Series?
What does new_values look like for a DataFrame?

On Aug 12, 2015, at 10:23 PM, denfromufa [email protected]
wrote:

The API is simple: pass new_values as keyword argument to interpolate()
with a default = None.

On Wed, Aug 12, 2015, 10:16 PM Tom Augspurger <
[email protected]>
wrote:

Could you write up a sketch of what the API would look like?

—
Reply to this email directly or view it on GitHub
<
https://github.com/pydata/pandas/issues/10811#issuecomment-130515586>.

—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub
#10811 (comment).

[den-run-ai]

and here is the function to enable this feature:

def interpeasy(df, new_idxs):
    return df.reindex(
        np.concatenate(
        (df.index, np.unique(new_idxs)))
        ).sort().interpolate().ix[new_idxs]

[shoyer]

I agree that something like this would be useful -- in many cases this is exactly what I'm looking for with interpolation. Note that from an implementation perspective, this is also much closer to what scipy's interpolation functions already do.

@denfromufa what would the full new function signature look like? e.g., we currently have this: http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.interpolate.html

[TomAugspurger]

FWIW, I think this might warrant a new method, e.g. df.interpolate_at(values, method='linear'). I wanted to include something like this when I did interpolate on DataFrames originally, but the API got too clunky.

[shoyer]

I do recall some discussion about interpolate_at, possibly related to fillna (which is very similar to interpolate in functionality). Let me see if I can dredge up that discussion.

On Thu, Aug 13, 2015 at 7:40 PM, Tom Augspurger [email protected]
wrote:

FWIW, I think this might warrant a new method, e.g. df.interpolate_at(values, method='linear'). I wanted to include something like this when I did interpolate on DataFrames originally, but the API got too clunky.

Reply to this email directly or view it on GitHub:
#10811 (comment)

[den-run-ai]

#9340

On Thu, Aug 13, 2015, 9:52 PM Stephan Hoyer [email protected]
wrote:

I do recall some discussion about interpolate_at, possibly related to
fillna (which is very similar to interpolate in functionality). Let me see
if I can dredge up that discussion.

On Thu, Aug 13, 2015 at 7:40 PM, Tom Augspurger [email protected]
wrote:

FWIW, I think this might warrant a new method, e.g.
df.interpolate_at(values, method='linear'). I wanted to include something
like this when I did interpolate on DataFrames originally, but the API got

too clunky.

Reply to this email directly or view it on GitHub:
#10811 (comment)

—
Reply to this email directly or view it on GitHub
#10811 (comment).

[TomAugspurger]

Thanks @denfromufa, let's close this one and move over the #9340.