Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[SPARK-39476][SQL] Disable Unwrap cast optimize when casting from Lon…
…g to Float/ Double or from Integer to Float ### What changes were proposed in this pull request? Cast from Integer to Float or from Long to Double/Float may loss precision if the length of Integer/Long beyonds the **significant digits** of a Double(which is 15 or 16 digits) or Float(which is 7 or 8 digits). For example, ```select *, cast(a as int) from (select cast(33554435 as foat) a )``` gives `33554436` instead of `33554435`. When it comes the optimization rule `UnwrapCastInBinaryComparison`, it may result in incorrect (confused) result . We can reproduce it with following script. ``` spark.range(10).map(i => 64707595868612313L).createOrReplaceTempView("tbl") val df = sql("select * from tbl where cast(value as double) = cast('64707595868612313' as double)") df.explain(true) df.show() ``` With we disable this optimization rule , it returns 10 records. But if we enable this optimization rule, it returns empty, since the sql is optimized to ``` select * from tbl where value = 64707595868612312L ``` ### Why are the changes needed? Fix the behavior that may confuse users (or maybe a bug?) ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Add a new UT Closes apache#36873 from WangGuangxin/SPARK-24994-followup. Authored-by: wangguangxin.cn <[email protected]> Signed-off-by: Wenchen Fan <[email protected]>
- Loading branch information