Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modify files to support distributed mode for XGBoost #553

Merged
merged 4 commits into from
Mar 27, 2023

Conversation

qbc2016
Copy link
Collaborator

@qbc2016 qbc2016 commented Mar 27, 2023

As the title says.

@qbc2016 qbc2016 changed the title Modify files to supprot distributed mode for XGBoost Modify files to support distributed mode for XGBoost Mar 27, 2023
Copy link
Collaborator

@xieyxclack xieyxclack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, please refer to inline comments for suggestions

@@ -62,7 +62,7 @@ def _find_and_send_split(self, split_ref, tree_num, node_num):
self.model[tree_num][node_num].member = client_id

split_ref['feature_idx'] -= accum_dim
split_child = False
split_child = 0
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest preserving True/False here to make it readable, and only transforming the type when split_child is in message.content

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When using distributed mode, we need to support the case where adopted datasets are distributed (load from different files)

Copy link
Collaborator

@xieyxclack xieyxclack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.
TODO: to enable multiple files in distributed mode, which is provided in the previous version but has bugs now. We will fix the bugs ASAP.

@xieyxclack xieyxclack merged commit 13275ba into alibaba:master Mar 27, 2023
@qbc2016 qbc2016 deleted the dev_distributed_xgb branch March 31, 2023 09:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants