[SPARK-29656][ML][PYSPARK] ML algs expose aggregationDepth

[zhengruifeng]

What changes were proposed in this pull request?

expose expert param aggregationDepth in algs: GMM/GLR

Why are the changes needed?

SVC/LoR/LiR/AFT had exposed expert param aggregationDepth to end users. It should be nice to expose it in similar algs.

Does this PR introduce any user-facing change?

yes, expose new param

How was this patch tested?

added pytext tests

[SparkQA]

Test build #112920 has finished for PR 26322 at commit c43dc30.

• This patch fails MiMa tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
• class _RobustScalerParams(HasInputCol, HasOutputCol, HasAggregationDepth):
• class _VectorIndexerParams(HasInputCol, HasOutputCol, HasHandleInvalid, HasAggregationDepth):
• class _ALSParams(_ALSModelParams, HasMaxIter, HasRegParam, HasCheckpointInterval, HasSeed,

[huaxingao]

I guess you forgot to put setAggregationDepth in class RobustScaler

[zhengruifeng]

Good catch! thanks!

[SparkQA]

Test build #112988 has finished for PR 26322 at commit 484effd.

• This patch fails to build.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #112991 has finished for PR 26322 at commit 8fc6517.

• This patch fails PySpark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #112996 has finished for PR 26322 at commit 8480a1e.

• This patch fails PySpark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #113017 has finished for PR 26322 at commit caeb609.

• This patch fails SparkR unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[zhengruifeng]

retest this please

[SparkQA]

Test build #113025 has finished for PR 26322 at commit caeb609.

• This patch fails PySpark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[srowen]

Hm, I am really not sure users can meaningfully tune this, so I'm not sure about adding so much complexity for it. In some cases I think we use reduce vs treeReduce on purpose as it can be faster; it really depends on the usage.

[zhengruifeng]

@srowen Thanks for reviewing!
This PR is for consistency, I will review it again and try to avoding adding to much complexity.

[SparkQA]

Test build #113072 has finished for PR 26322 at commit 11dbab4.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
• class _RobustScalerParams(HasInputCol, HasOutputCol, HasRelativeError):
• class _VectorIndexerParams(HasInputCol, HasOutputCol, HasHandleInvalid):
• class _ALSParams(_ALSModelParams, HasMaxIter, HasRegParam, HasCheckpointInterval, HasSeed):

[zhengruifeng]

retest this please

[SparkQA]

Test build #113264 has finished for PR 26322 at commit 11dbab4.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
• class _RobustScalerParams(HasInputCol, HasOutputCol, HasRelativeError):
• class _VectorIndexerParams(HasInputCol, HasOutputCol, HasHandleInvalid):
• class _ALSParams(_ALSModelParams, HasMaxIter, HasRegParam, HasCheckpointInterval, HasSeed):

[zhengruifeng]

merged to master, thanks all

[srowen]

Retroactively: OK I understand the argument was consistency. I might have pinged for a confirmation first though as I wasn't in favor of this at first look.

[zhengruifeng]

@srowen Sorry for my careless merge. Should I revert it first?
I had revert changes in ALS/Features, now there are 3 main changes:
1, GMM add aggregationDepth;
2, GLR add aggregationDepth, to do this, I need to add this new param in IterativelyReweightedLeastSquares && WeightedLeastSquares;
3, For the change reduce => treeReduce in VectorIndexer, I left it alone by mistake, I guess I need to revert it in some way.

[srowen]

I think it's OK if it's for consistency.