Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle special characters in JSON model dump. #9474

Merged
merged 1 commit into from
Aug 14, 2023

Conversation

trivialfis
Copy link
Member

@trivialfis trivialfis commented Aug 11, 2023

Close #9352 .

  • Reuse the string handling logic in Json.

The test coverage for special characters is not quite complete at the moment. Expect some failures with features that need to parse the dump output. On top of my mind for things to test:

  • Dataframe dump on Python and R.
  • Feature importance.

I tested model plot, which uses the dot format and it works fine with special characters.

char number[NumericLimits<float>::kToCharsSize];
auto res = to_chars(number, number + sizeof(number), num->GetNumber());
std::array<char, NumericLimits<float>::kToCharsSize> number;
auto res = to_chars(number.data(), number.data() + number.size(), num->GetNumber());
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to use std::to_chars, now that we are using C++17? Not blocking this release, let's consider using std::to_chars in the future.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The implementation is unspecified by the standard. MSVC can produce different result than GCC.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even for integers?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not for interger. I will do some cleanups after 2.0

@trivialfis trivialfis merged commit 05d7000 into dmlc:master Aug 14, 2023
@trivialfis trivialfis deleted the fix-json-dump branch August 14, 2023 07:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants