Code

Beyond the jvm updates#123

Code

Merged

holdenk merged 37 commits intohigh-performance-spark:mainhigh-performance-spark/high-performance-spark-examples:mainfrom holdenk:beyond-the-jvm-updatesholdenk/high-performance-spark-examples:beyond-the-jvm-updatesCopy head branch name to clipboardApr 1, 2024

+552-14

Commits

Commits on Mar 17, 2024

• Update git ignore.

  holdenk

  committed
  4e93385

  Copy full SHA for 4e93385

• Start adding some shims for running accelerators.

  holdenk

  committed
  b0c5480

  Copy full SHA for b0c5480

Commits on Mar 18, 2024

• Try and setup velox

  Show description for 8c547e5
  holdenk

  committed
  8c547e5

  Copy full SHA for 8c547e5

• Update accel stuff

  holdenk

  committed
  a14aa7b

  Copy full SHA for a14aa7b

Commits on Mar 20, 2024

• Get Gluten + Spark3.4 to party (note: this fails because of Gluten segfault)

  Show description for 8c90b72
  holdenk

  committed
  8c90b72

  Copy full SHA for 8c90b72

• Fix comet resolution

  holdenk

  committed
  9e607f7

  Copy full SHA for 9e607f7

• Multiple extensions (Iceberg and Comet)

  holdenk

  committed
  0ac92c7

  Copy full SHA for 0ac92c7

• Turn on Comet shuffle

  holdenk

  committed
  1fe4897

  Copy full SHA for 1fe4897

• Style fixes

  holdenk

  committed
  f4fe039

  Copy full SHA for f4fe039

• Use version 3.4.2 and also use setup rust action for speed

  holdenk

  committed
  d9d4a39

  Copy full SHA for d9d4a39

• Seperate out setup comet so we can debug faster.

  holdenk

  committed
  c18210c

  Copy full SHA for c18210c

• Setup jdk versions for happiness.

  holdenk

  committed
  9f79085

  Copy full SHA for 9f79085

• Change caching to make sense

  holdenk

  committed
  f6e565b

  Copy full SHA for f6e565b

• Work around the classloader issue we found.

  holdenk

  committed
  a11ec96

  Copy full SHA for a11ec96

• shellcheck fix.

  holdenk

  committed
  9f57e26

  Copy full SHA for 9f57e26

• Hmm why no version.

  holdenk

  committed
  44e41b0

  Copy full SHA for 44e41b0

• Fix version pass in for setup

  holdenk

  committed
  4182d89

  Copy full SHA for 4182d89

• Fix comet setup

  holdenk

  committed
  9057522

  Copy full SHA for 9057522

• Try and fix gluten build

  holdenk

  committed
  b7fc797

  Copy full SHA for b7fc797

• Style fix and statically link

  holdenk

  committed
  dbff266

  Copy full SHA for dbff266

Commits on Mar 21, 2024

• vcpkg

  holdenk

  committed
  a9a01d3

  Copy full SHA for a9a01d3

• Try and fix vcpkg

  holdenk

  committed
  5ad5c3b

  Copy full SHA for 5ad5c3b

Commits on Mar 25, 2024

• meh vcpkg is kind of a pain, lets skip it.

  holdenk

  committed
  07229d6

  Copy full SHA for 07229d6

• Huzzah --driver-class-path does the trick.

  holdenk

  committed
  f716ef7

  Copy full SHA for f716ef7

• Make setup_gluten_deps better formated for book inclusion

  holdenk

  committed
  8354fe7

  Copy full SHA for 8354fe7

• Tag the gluten setup

  holdenk

  committed
  769ffeb

  Copy full SHA for 769ffeb

• Disable gluten SQL

  holdenk

  committed
  65f4a98

  Copy full SHA for 65f4a98

• Tag comet example for inclusion.

  holdenk

  committed
  a0fa2ba

  Copy full SHA for a0fa2ba

• Add Python UDF/UDAF examples.

  holdenk

  committed
  a6bd178

  Copy full SHA for a6bd178

• Style fixes

  holdenk

  committed
  50f804c

  Copy full SHA for 50f804c

Commits on Mar 26, 2024

• Move SparkSession builder up

  holdenk

  committed
  e377e9e

  Copy full SHA for e377e9e

• style fix

  holdenk

  committed
  1b8c55c

  Copy full SHA for 1b8c55c

• Fix typing import + pd.DF

  holdenk

  committed
  362e17a

  Copy full SHA for 362e17a

Commits on Apr 1, 2024

• Style fix

  holdenk

  committed
  4cd8622

  Copy full SHA for 4cd8622

• Use axel if present

  holdenk

  committed
  4d23e5e

  Copy full SHA for 4d23e5e

• Add mypy to tox.ini so we don't depend on it being in the system setup.

  holdenk

  committed
  4d3373b

  Copy full SHA for 4d3373b

• Fix axel command.

  holdenk

  committed
  2419a8c

  Copy full SHA for 2419a8c