[SPARK-21603][SQL][FOLLOW-UP] Use -1 to disable maxLinesPerFunction #19031
Conversation
|
Test build #81055 has finished for PR 19031 at commit
|
maropu
force-pushed
the
SPARK-21603-FOLLOWUP-2
branch
from
60dc64e to
9438655
Compare
|
I'd prefer using |
| .intConf | ||
| .checkValue(maxLines => maxLines >= -1, "The maximum must not be a negative integer, -1 to " + | ||
| "always activate whole-stage codegen.") |
There was a problem hiding this comment.
The maximum must not be a negative integer, except for -1 using to always activate whole-stage codegen.
| @@ -577,9 +577,11 @@ object SQLConf { | |||
| .doc("The maximum lines of a single Java function generated by whole-stage codegen. " + | |||
| "When the generated function exceeds this threshold, " + | |||
| "the whole-stage codegen is deactivated for this subtree of the current query plan. " + | |||
| "The default value 4000 is the max length of byte code JIT supported " + | |||
| "for a single function(8000) divided by 2.") | |||
| "The default value 2667 is the max length of byte code JIT supported " + | |||
There was a problem hiding this comment.
2667? I don't see you change the default value below.
maropu
force-pushed
the
SPARK-21603-FOLLOWUP-2
branch
from
8a60378 to
a0854ad
Compare
|
LGTM |
|
Test build #81062 has finished for PR 19031 at commit
|
|
Test build #81065 has finished for PR 19031 at commit
|
|
ping @gatorsmile |
|
ping |
|
This is an internal conf. For the advanced users, we do not encourage them to disable it. If they want to disable it, they can simply set it to a number above 8000. Thus, setting |
|
ok, I'll close for now. Thanks! |
What changes were proposed in this pull request?
This pr used
-1to disablemaxLinesPerFunctionadded in #18810 because-1seems to be more natural to disable this too-long-function optimization (e.g. we couldn't useInt.MaxValueinspark-default.conf).How was this patch tested?
Modified the existing
WholeStageCodegenSuite