Match Spark's NaN handling in collect_set

[thirtiseven]

Fixes #8808

collect_set function currently does not support NaNs in struct[Array(Double)] or struct[Array(Float)] types. It is because Spark handling NaN's equality by different way for non-nested float/double and float/double in nested types.

This PR checks and sets the NaNEquality in collectSet dynamically. If we are doing a collect set on an Float or Double column, then we set NaNEquality to UNEQUAL, but if we are doing a collect set on anything else we set it to ALL_EQUAL.

Changes in tests and gpuOverrides are basically revert of #8783

[thirtiseven]

Oh I didn't remove the xfailed case and it is still failed... Looking into it, convert it to draft
Passed now

[thirtiseven]

build

[abellina]

this check performed a recursive check for floats/doubles at all nested levels, but your changes seem to say that Array[Double] and Array[Float] are UNEQUAL. Should your code handle Array[Array[Double]] (and any level of nesting, including things like Struct[Array[Double]] for example)? Are those cases UNEQUAL or are they somehow compared differently. If so we should add comments clarifying why an array/struct a 2nd+ level is ok.

[thirtiseven]

Oh yes it's a bit confusing. I added a comment.

In GpuCollectBase we have:

def child: Expression
override def dataType: DataType = ArrayType(child.dataType, containsNull = false)

My change wants to make NaNs UNEQUAL only if child.dataType is double or float, otherwise ALL_EQUAL.

But when we call CudfCollectSet or CudfMergeSets we need to pass dataType to it. So when I check if datatype is Array(Double) or Array(Float), what I really want to check is if child.dataType is float or double.

(I hope I made it clear...)

[thirtiseven]

build

[revans2]

The code looks good, I just had the same comments as Jason about not including nans in some places in the tests.

[thirtiseven]

@jlowe @revans2 Thanks for review, I removed those NaN datagens.

[thirtiseven]

build