Convert SHAP plots to probability plots

[anilkumarpanda]

Problem Description

SHAP values do a great job of explaining complex ML models. However they are not easily understandable by the model users .The concept of probability is slightly better to grasp (with some technical knowledge ) for decision makers.

The current shap package provide shap values.

However using the additive property of shap values and using interpolation methods we can convert them to probabilities.

The dependency plots thus created :

• Provides more information and are intuitive.
• Are more interpretable as we can quantify effects in terms of probability. e.g we can say that being 60–70 yo leads to a decrease of 27% in probability of survival compared to being 0–10 yo (from +14% to -13%);

A detailed explanation can be found in this blogpost

Desired Outcome
For dependence plots instead of SHAP values, display the probability outputs.

Solution Outline
Currently the probatus.interpret module provides methods for displaying the SHAP plot. .We can implement a new class ShapProbaInterpreter that provide the above mentioned functionality.

[sbjelogr]

Doesn't model_output='probability' in TreeShap kinda do the same thing?

[anilkumarpanda]

In case model_output='probability the shap values explain the predicted probability. However the output are still shap values.
It would be beneficial if we show the dependence plot as feature value vs effect on predicted probability directly rather than on shap values.

[Matgrb]

I have read further here:
shap/shap#963

I suppose the model_output='probability` does the same transform that is presented in the article.

I also think that we should first of all try to use the tools that we already have from SHAP. #103 issue is going to allow passing model_output parameter and we should check whether this already provides similar results as in the article. If it is the case, we could make a HowTo guide, to explain how to use this properly and how to analyse the results in terms of probability. We could also slightly modify the graph labels to indicate that this is the probability for some parameter settings.

If this is not feasible using shap parameter, then we could consider implementing a new class. I would be hesitant to do that, because it would take a lot of work to implement it, and it would take some time to convince the users to trust this one, instead the basic SHAP one.

What do you think about this?

[anilkumarpanda]

The SHAP issue clarifies a lot. I too tried to sum up the values to probability output and that did not match. However I was missing the expected value.

Anyway I think the above charts can be achieved by slightly modifying the graph labels and would be a prudent way as well.

[Matgrb]

I agree that having adjusted labels for probability outputs would be a good modification.

I am going to close this issue now, in order to keep the discussions focused on one topic. Feel free to open a new issue, in which this modification is proposed.