We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
NullType
Describe the bug
We allow casting from other datatypes to NullType (see can_cast_types). I suspect that this is a mistake.
can_cast_types
For example, Spark doesn't allow casting from any datatypes to NullType.
I also checked C++ Cast kernels. Mostly the datatypes cannot cast to NullType (e.g. https://github.com/apache/arrow/blob/53752adc6b81166cd4ee7db5a819494042f29197/cpp/src/arrow/compute/kernels/scalar_cast_test.cc#L187-L211). I said "mostly" because I don't see explicit tests for other cases like Decimal, but I suspect that it is not allowed but just lack of test coverage.
I don't see there is cast kernel in C++ for casting to NullType except for a special case (dict<null> -> null): https://github.com/apache/arrow/blob/53752adc6b81166cd4ee7db5a819494042f29197/cpp/src/arrow/compute/kernels/scalar_cast_numeric.cc#L748-L751. Even for that, there is a comment questioning about it.
dict<null> -> null
To Reproduce Steps to reproduce the behavior:
Expected behavior
Cast kernel should not allow casting from any datatypes to NullType.
Additional context Add any other context about the problem here.
The text was updated successfully, but these errors were encountered:
NULL
Decimal
The result of discussion: #1922 (comment)
Sorry, something went wrong.
I will give a follow-up pr to remove casting other data type to Null Data type.
viirya
Successfully merging a pull request may close this issue.
Describe the bug
We allow casting from other datatypes to NullType (see
can_cast_types
). I suspect that this is a mistake.For example, Spark doesn't allow casting from any datatypes to NullType.
I also checked C++ Cast kernels. Mostly the datatypes cannot cast to NullType (e.g. https://github.com/apache/arrow/blob/53752adc6b81166cd4ee7db5a819494042f29197/cpp/src/arrow/compute/kernels/scalar_cast_test.cc#L187-L211). I said "mostly" because I don't see explicit tests for other cases like Decimal, but I suspect that it is not allowed but just lack of test coverage.
I don't see there is cast kernel in C++ for casting to NullType except for a special case (
dict<null> -> null
): https://github.com/apache/arrow/blob/53752adc6b81166cd4ee7db5a819494042f29197/cpp/src/arrow/compute/kernels/scalar_cast_numeric.cc#L748-L751. Even for that, there is a comment questioning about it.To Reproduce
Steps to reproduce the behavior:
Expected behavior
Cast kernel should not allow casting from any datatypes to NullType.
Additional context
Add any other context about the problem here.
The text was updated successfully, but these errors were encountered: