Code

Debug Hadoop 3.2 profile#24

Code

Closed

HyukjinKwon wants to merge 17 commits intomasterHyukjinKwon/spark:masterfrom SPARK-29250-debugHyukjinKwon/spark:SPARK-29250-debugCopy head branch name to clipboard

+656-644

Commits

Commits on Oct 6, 2020

• [WIP][SPARK-29250] Upgrade to Hadoop 3.2.1

  

  sunchao

  authored and

  HyukjinKwon

  committed
  749055c

  Copy full SHA for 749055c

• Switch to Python3 for YARN tests & fix a bug

  

  sunchao

  authored and

  HyukjinKwon

  committed
  855bf70

  Copy full SHA for 855bf70

• Fix more tests related to Python3

  

  sunchao

  authored and

  HyukjinKwon

  committed
  77dc194

  Copy full SHA for 77dc194

• Remove more dependencies

  

  sunchao

  authored and

  HyukjinKwon

  committed
  50a031f

  Copy full SHA for 50a031f

• Add Maven profiles to handle Hadoop 2.7

  

  sunchao

  authored and

  HyukjinKwon

  committed
  4c3067f

  Copy full SHA for 4c3067f

• Fix default profile

  

  sunchao

  authored and

  HyukjinKwon

  committed
  6393293

  Copy full SHA for 6393293

• Require explicit Hadoop profile

  

  sunchao

  authored and

  HyukjinKwon

  committed
  8b9f381

  Copy full SHA for 8b9f381

• Debug Hadoop 3.2 profile

  HyukjinKwon

  committed
  59d5242

  Copy full SHA for 59d5242

• Run the test directly

  HyukjinKwon

  committed
  d08c8ff

  Copy full SHA for d08c8ff

• which python

  HyukjinKwon

  committed
  8aba753

  Copy full SHA for 8aba753

• show path in driver

  HyukjinKwon

  committed
  0d383dd

  Copy full SHA for 0d383dd

• send paths

  HyukjinKwon

  committed
  e70c62a

  Copy full SHA for e70c62a

• test

  HyukjinKwon

  committed
  1cd5df5

  Copy full SHA for 1cd5df5

• Check PATH

  HyukjinKwon

  committed
  d40391d

  Copy full SHA for d40391d

• Try to set PATH in both driver and executor

  HyukjinKwon

  committed
  bf4340f

  Copy full SHA for bf4340f

• show path

  HyukjinKwon

  committed
  f7d646f

  Copy full SHA for f7d646f

• minimised fix

  HyukjinKwon

  committed
  51ede03

  Copy full SHA for 51ede03