Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add sample_weight to eval_metric #8706

Merged
merged 3 commits into from
Feb 4, 2023

Conversation

BenEfrati
Copy link
Contributor

Hi,
After #6751 provide a custom eval_metric via fit is deprecated.

In our eval_metric function we call to get_weight() from dmatrix.
When passing the metric function via constructor or set_params method we're unable to get sample_weight.

It might be useful for other sklearn metric function that accept sample_weight argument

@idanmoradarthas
Copy link

This is an important ability!

Copy link
Member

@trivialfis trivialfis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the new feature! Let's not add an additional parameter and pass the weight when present.

@BenEfrati
Copy link
Contributor Author

Thank you for the new feature! Let's not add an additional parameter and pass the weight when present.

Agree,
Are you ok with the try except approach?

@trivialfis
Copy link
Member

I think we can patch the _metric_decorator as follow:

def _metric_decorator(func: Callable) -> Metric:
    """Decorate a metric function from sklearn.

    Converts a metric function that uses the typical sklearn metric signature so that
    it is compatible with :py:func:`train`

    """

    def inner(y_score: np.ndarray, dmatrix: DMatrix) -> Tuple[str, float]:
        y_true = dmatrix.get_label()
        weight = dmatrix.get_weight()
        if weight.size == 0:
            return func.__name__, func(y_true, y_score)
        else:
            return func.__name__, func(y_true, y_score, sample_weight=weight)

    return inner

and add a test.

Copy link
Member

@trivialfis trivialfis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@BenEfrati
Copy link
Contributor Author

Could you please help add a test in https://github.com/dmlc/xgboost/blob/master/tests/python/test_with_sklearn.py ?

Sure

Copy link
Member

@trivialfis trivialfis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the nice feature!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants