Ability to use rolling computations in charts

[aschonfeld]

Yes, it would be great to have a feature to perform rolling computations. If it's not asking much, it would be nice to be able to specify the computation method.

Please, let me know if you need help implementting or specifying it.

Originally posted by @mindlessbrain in #43 (comment)

[aschonfeld]

@mindlessbrain,

It sounds like you would like to use rolling computations in general and not for applying them to the standard groupby(...).corr output that is displayed in the Correlations popup. This seems like something well suited for the "Charts" popup.

Currently you can specify and x & y axes to your chart as well as an aggregation. So in the case that you have some timeseries data you could do something like this:

• x-axis: date
• y-axis: [some numeric column]
• aggregation: rolling

And then I would add some other options strictly associated with rolling computations, for example:

• computation: sum, mean, etc...
• window
• min_periods
• win_type
• closed

Then once you have made your selections you can click the "Load" button and that will give you a timeseries chart similar to waht you see in the Correlations popup (minus the nice red/yellow/green gradient for the line color)

[aschonfeld]

Also, I'll only make the "rolling" aggregation available if you've selected a column for the x-axis that is of a datetime data type

[drlima]

@aschonfeld,

I agree with you on putting it on the "Charts" popup, following the mentioned logic: Agregation -> Computation. It would be useful if it only became available on the presence of a datetime like column on the "X:" axis.
However, for pairwise rolling calculations like the rolling correlation, I think we'd need to input a third variable, i.e., the x, y, and the datetime variable.

What do you think?

[drlima]

Additionally, I loved the linked behaviour of the "Correlations" popup.

Do you think it's possible to have a tool for building linked charts?
(perhaps this should be on a new issue)

[aschonfeld]

Ok, I forgot about the pairwise stuff. Then my originally suggestion of a rolling toggle on the timeseries correlation chart will be useful and it will simply default to a window of 4, but you'll have to ability to change it if you want.

As for the linked chart builder, thats a great idea. It was pretty straight-forward in the correlations world because its only natural to go from timeseries to a scatter of the individual date's data. I've created a new issue with my thoughts, #46

[aschonfeld]

@mindlessbrain here is a preview of what the correlations popup will look like with data structured like yours

So the way it will work is that in the scenario where you have columns in your dataframe of type datetime and there is only one row of data per date then the data will have this "rolling" capability. Essentially, in this case when you click a cell in the grid it will bring up a rolling pearson correlation (window of 4 by default) on the two columns associated with the cell and it will also bring up the scatter plot as well.

As you can see in the screenshot you will be able to change what date & window you're using for the correlations.

The original functionality will still exist:

1. no columns of type datetime exist: it will display the scatter plot on click.
2. columns of type datetime exist, but they have more than one row associated with each date: the original timeseries correlation (cross-sectional) chart will display and you can click on individual points to bring up the scatter

[drlima]

That's great, @aschonfeld!

When you say

only one row associated with each date

You mean a datetime or a date?
I say that because "2019-12-29 15:46:50" is a different timestamp (datetime) from "2019-12-29 12:00:00", while they both share the same date.
I am only asking this because it could become a future bug, I mean, the user should be able to work with some intraday data as well as daily/monthly etc.

[aschonfeld]

It does a value_count() on the datetime column you have selected so it should be fine. It doesnt set the frequency to ‘D’ or anything like that. Hoping to get this released tonight and then I can work on adding rolling computations to the main chart builder as well as scatter charts.

[aschonfeld]

@mindlessbrain v1.6.5 is available which includes the rolling correlation functionality. Will keep you posted on the other features I mentioned earlier.

[drlima]

Hi, @aschonfeld.
I am testing here, and it looks great!

There are some suggestions and potential bugs that I am finding.
May I post it here, or in a separate issue?

[aschonfeld]

I’d post them in a new issue, thanks

[aschonfeld]

added in v1.6.6