[SPARK-28699][Core] Cache an indeterminate RDD could lead to incorrect result while stage rerun

[xuanyuanking]

What changes were proposed in this pull request?

It's another case for the indeterminate stage/RDD rerun while stage rerun happened. In the CachedRDDBuilder, we miss tracking the isOrderSensitive characteristic to the newly created MapPartitionsRDD.
This patch just a safeguard, if we need the support for stage rerunning, it should be done after #24892.

How was this patch tested?

Integrated test with an exception, instead of the wrong answer.

import scala.sys.process._
import org.apache.spark.TaskContext

val res = spark.range(0, 10000 * 10000, 1).map{ x => (x % 1000, x)}
// kill an executor in the stage that performs repartition(239)
val df = res.repartition(113).cache.repartition(239).map { x =>
  if (TaskContext.get.attemptNumber == 0 && TaskContext.get.partitionId < 1 && TaskContext.get.stageAttemptNumber == 0) {
    throw new Exception("pkill -f -n java".!!)
  }
  x
}
val r2 = df.distinct.count()

[xuanyuanking]

Ignore this for the behavior change, as the approach now, we need to abort the stage of the current mapStage.
As we will finally support stage rerun, I suggest to skip this behavior, we can directly support the cache scenario after SPARK-25341 merged.

[cloud-fan]

I don't quite get it. The isOrderSensitive flag is used to describe the map function. Why the map function is order sensitive?

[SparkQA]

Test build #108978 has finished for PR 25420 at commit b3ea90f.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[viirya]

I think we only run cached plan once. I think it should be determinate? If so, this map should not be order sensitive effectively?

[dongjoon-hyun]

Retest this please.

[dongjoon-hyun]

@xuanyuanking Can we have a test case to prevent a future regression on this?

[SparkQA]

Test build #108993 has finished for PR 25420 at commit b3ea90f.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[xuanyuanking]

Sorry for the late. Thanks for the comment and review! After taking a further investigation, I found the root cause for this issue is about radix sort in ShuffleExchangeExec, please review it in #25491.
The bug fix of DAGScheduler is still needed, I'll give another follow-up PR and add UT for it. Done in #25498.
After all the fix has done, I'll give detailed explanation in the JIRA description.