Code

[RELEASE] dask-cuda v0.13#267

Code

Merged

raydouglass merged 124 commits intomasterrapidsai/dask-cuda:masterfrom branch-0.13rapidsai/dask-cuda:branch-0.13Copy head branch name to clipboardMar 31, 2020

+1,251-211

Commits

Commits on Jan 14, 2020

• Start implementation of explicit comms

  madsbk

  committed
  eb8f3c7

• local_cudf_merge: cleanup output

  madsbk

  committed
  c13d013

• removed debug prints

  madsbk

  committed
  55d42d3

• Clean up

  madsbk

  committed
  b111bad

• local_cudf_merge: clean up of output

  madsbk

  committed
  bb7ae38

• Added test of explicit comms

  madsbk

  committed
  b78e0c8

Commits on Jan 15, 2020

• removed some dead code

  madsbk

  committed
  8754354

• added cudf-merge test

  madsbk

  committed
  7d1f4ac

• test: manually setting index name

  madsbk

  committed
  ed28831

Commits on Jan 16, 2020

• Using Dask Comm object instead of UCX direct

  madsbk

  committed
  9880cb9

Commits on Jan 17, 2020

• DOC Updates for v0.13

  raydouglass

  committed
  c65a5c4

• dataframe_operation(): now take any number of dataframe inputs

  madsbk

  committed
  abbbc95

• test_cudf_merge(): made the input more interesting

  madsbk

  committed
  a77f6c0

• Support of uneven distributed dataframes

  madsbk

  committed
  479a48a

• Testing uneven distributed dataframes

  madsbk

  committed
  fb1c4e4

• overlapping comm of left_table with hashing of right_table

  madsbk

  committed
  abe170c

Commits on Jan 20, 2020

• Fixed barrier()

  madsbk

  committed
  fb7fb6b

• Merge(): Implement support of pandas dataframes

  madsbk

  committed
  e7fa1d7

• rename: cudf_merge => dataframe_merge

  madsbk

  committed
  5323cef

• merged the two dataframe utils

  madsbk

  committed
  005802c

• dataframe_merge(): added on, left_on, and right_on arguments

  madsbk

  committed
  bcc1dfc

Commits on Jan 21, 2020

• Test: added different nworkers

  madsbk

  committed
  914f621

• making cudf-merge benchmark use the new comm API

  madsbk

  committed
  fd95cd4

• Merge pull request #228 from rapidsai/branch-0.12

  Show description for 96a43b7
  GPUtester

  authored
  96a43b7

Commits on Jan 22, 2020

• dataframe_operation(): always submit to all workers

  madsbk

  committed
  b964024

• dataframe_operation(): removed the random input

  Show description for f7a1c53
  madsbk

  committed
  f7a1c53

• Fixed typo

  madsbk

  committed
  648800f

• partition_by_hash(): now uses Dask primitives

  madsbk

  committed
  530f659

Commits on Jan 24, 2020

• dataframe_merge(): added the how argument

  madsbk

  committed
  a8fabf6

• dataframe_operation(): now passes npartition info

  Show description for 7186b95
  madsbk

  committed
  7186b95

• Added some docs

  madsbk

  committed
  72b8e38

• cleanup

  madsbk

  committed
  d257ac5

Commits on Jan 27, 2020

• using isinstance cudf.DataFrame ...

  madsbk

  committed
  3e4ac4e

• doc

  madsbk

  committed
  68d7909

• Clean up

  madsbk

  committed
  e50f8a9

• doc

  madsbk

  committed
  171f1f1

Commits on Jan 30, 2020

• Merge pull request #226 from madsbk/explicit_comms

  Show description for a007e45
  rjzamora

  authored
  a007e45

Commits on Feb 4, 2020

• Merge pull request #234 from rapidsai/branch-0.12

  Show description for 710e6ca
  GPUtester

  authored
  710e6ca

• Use DeviceBuffer to provide to_device function

  Show description for 64846f5
  jakirkham

  committed
  64846f5

• Add a shim copy_to_host function

  Show description for 2d84f2f
  jakirkham

  committed
  2d84f2f

• Use the copy_to_host shim function

  jakirkham

  committed
  b5b2e9c

Commits on Feb 6, 2020

• Add RMMPool util class to be used as a Dask plugin

  pentschev

  committed
  717615b

• Add RMM pool support and a test for LocalCUDACluster

  pentschev

  committed
  7d1ab98

• Add RMM pool support and a test for dask-cuda-worker

  pentschev

  committed
  035875c

• Merge remote-tracking branch 'upstream/branch-0.13' into rmm-pool-support

  pentschev

  committed
  1338d38

• Flatten data and cast as uint8

  Show description for cd77904
  jakirkham

  committed
  cd77904

• Add #pragma no cover to LocalCUDACluster

  pentschev

  committed
  afa9b24

Commits on Feb 7, 2020

• Merge pull request #235 from jakirkham/use_devbuf

  Show description for 869b790
  pentschev

  authored
  869b790

• Improve documentation for RMM pool

  pentschev

  committed
  78f8435

• Merge pull request #236 from pentschev/rmm-pool-support

  Show description for 474108a
  pentschev

  authored
  474108a

Commits on Feb 19, 2020

• pin cupy to 7.1

  quasiben

  committed
  d8ab37c

• restirct to less than 7.2

  quasiben

  committed
  d734ac9

• Update ci/gpu/build.sh

  Show description for d9f7b22
  

  quasiben

  and

  jakirkham

  authored
  d9f7b22

Commits on Feb 20, 2020

• more explicitly define range of acceptable cupy versions

  quasiben

  committed
  7a2c99c

• fix conflict

  quasiben

  committed
  d33b908

• Merge pull request #239 from quasiben/pin-cupy

  Show description for dd18b6f
  quasiben

  authored
  dd18b6f

Commits on Feb 28, 2020

• Moved get_ucx_net_devices to utils

  pentschev

  committed
  12a72bb

• Use get_ucx_net_devices in get_preload_options

  pentschev

  committed
  13e711a

• Add new get_ucx_config utility function

  pentschev

  committed
  d1deb08

• Use Dask configuration to create Nanny objects in dask-cuda-worker

  pentschev

  committed
  1580721

• Raises if no CUDA device is specified in get_ucx_net_devices

  pentschev

  committed
  3f9d50f

• Use get_ucx_net_devices in initialize

  pentschev

  committed
  6b43e17

• Fix get_ucx_net_devices variable names

  pentschev

  committed
  1a0e302

• Remove create_cuda_context from get_ucx_config call

  pentschev

  committed
  bebb9ff

• fix ucx-net things

  

  quasiben

  authored and

  pentschev

  committed
  627432a

• run rmm pool on scheduler

  

  quasiben

  authored and

  pentschev

  committed
  0d5c065

Commits on Feb 29, 2020

• Enable INSTALL_DASK_MASTER for CI

  pentschev

  committed
  7ca2b26

Commits on Mar 1, 2020

• Add myself to the changelog authors' list

  jakirkham

  committed
  e8fb021

• Note DeviceBuffer used for spilling

  Show description for 0e3ea6a
  jakirkham

  committed
  0e3ea6a

• Relax CuPy pin

  jakirkham

  committed
  aefd5a2

• Note CuPy pin is now relaxed

  jakirkham

  committed
  c3abb60

• Respect temporary-directory config for spilling

  Show description for cbcbda8
  jakirkham

  committed
  cbcbda8

• Test DeviceHostFile uses temporary-directory

  Show description for 4cb8618
  jakirkham

  committed
  4cb8618

• Note temporary-directory used in changelog

  jakirkham

  committed
  22a3dfe

• Keep spill path in DeviceHostFile

  Show description for b2ebb8d
  jakirkham

  committed
  b2ebb8d

• Wrap long line

  jakirkham

  committed
  1001c88

• Check the value of self.disk_func_path

  Show description for cc9b998
  jakirkham

  committed
  cc9b998

• Merge pull request #249 from jakirkham/note_pr_235

  Show description for e0d810c
  pentschev

  authored
  e0d810c

• Fix UCX test for Dask configurations

  pentschev

  committed
  8168abc

• Verify that temporary_directory was created

  jakirkham

  committed
  d4ebfdf

• Merge rapidsai/branch-0.13 into jakirkham/use_tmp_dir_config

  jakirkham

  committed
  4aa3007

• Merge rapidsai/branch-0.13 into jakirkham/relax_cupy_pin

  jakirkham

  committed
  1e76e00

• Use makedirs(..., exist_ok=True)

  Show description for 37c5e20
  jakirkham

  committed
  37c5e20

• Merge pull request #247 from jakirkham/use_tmp_dir_config

  Show description for 958c2b4
  pentschev

  authored
  958c2b4

• Update get_ucx_config to respect new distributed format

  pentschev

  committed
  265d927

• Merge rapidsai/branch-0.13 into jakirkham/relax_cupy_pin

  jakirkham

  committed
  6d15a5c

• Fix black formatting

  pentschev

  committed
  a05d2a7

• Add a dask-cuda-worker test for UCX_NET_DEVICES

  pentschev

  committed
  d63a325

Commits on Mar 2, 2020

• Merge pull request #248 from jakirkham/relax_cupy_pin

  Show description for eb68ba1
  quasiben

  authored
  eb68ba1

• Remove wrong "ucx." prefix from get_ucx_config

  pentschev

  committed
  42c1c4c

• Use config= in test_initialize

  pentschev

  committed
  6559784

• Simplify get_ucx_config

  pentschev

  committed
  a7ef5b3

• Fix usage of get_ucx_config in dask_cuda_worker

  pentschev

  committed
  6669bab

• Update LocalCUDACluster to use get_ucx_config

  pentschev

  committed
  bdc4ae3

• Change test ports to avoid TIME_WAIT issues

  pentschev

  committed
  7184bce

Commits on Mar 3, 2020

• Add 'tcp' and 'cuda_copy' flags to ucx_get_config

  pentschev

  committed
  9cdbb14

• Add tcp flag to TCP over UCX test

  pentschev

  committed
  3252aa3

• Remove hanging prints

  pentschev

  committed
  9e1c773

Commits on Mar 4, 2020

• Add tests for UCX util functions

  pentschev

  committed
  1d41aa6

• Fix get_ucx_net_devices test to ensure success without IB

  pentschev

  committed
  b314a73

• Fix get_ucx_options test to ensure success without IB

  pentschev

  committed
  d4340e8

• Add comments on benchmark RMM pool for scheduler

  pentschev

  committed
  05a73b7

• Merge pull request #246 from pentschev/fix-ucx-options

  Show description for 39c10ca
  quasiben

  authored
  39c10ca

Commits on Mar 12, 2020

• partition_by_hash(): added ignore_index=False

  madsbk

  committed
  a17f8be

• dataframe_merge: ignore_index=True

  madsbk

  committed
  a689766

• updated changelog

  madsbk

  committed
  80b88da

• Merge pull request #253 from madsbk/partition_by_hash_ignore_index

  Show description for ba6829e
  quasiben

  authored
  ba6829e

Commits on Mar 19, 2020

• Require Dask 2.9.0+

  Show description for d5d2a8e
  jakirkham

  committed
  d5d2a8e

• Drop Numba DeviceNDArray code for sizeof

  Show description for 3e68459
  jakirkham

  committed
  3e68459

• Note sizeof for Numba DeviceNDArrays dropped

  jakirkham

  committed
  b9bc27e

• Merge pull request #257 from jakirkham/drop_numba_sizeof_code

  Show description for bf6f868
  pentschev

  authored
  bf6f868

Commits on Mar 24, 2020

• Register dict as potential device object

  madsbk

  committed
  0a94220

• Merge pull request #260 from madsbk/spill_dicts

  Show description for 259f36a
  pentschev

  authored
  259f36a

Commits on Mar 25, 2020

• Update changelog for 0.13

  pentschev

  committed
  bffd590

• Merge pull request #263 from pentschev/update-changelog-0.13

  Show description for a51b127
  pentschev

  authored
  a51b127

Commits on Mar 26, 2020

• Require Distributed 2.11.0+

  jakirkham

  committed
  a54004c

• Rerun black on the codebase

  jakirkham

  committed
  c856539

• Avoid _ in function name

  Show description for f7db79b
  jakirkham

  committed
  f7db79b

• Use "dask" serialization to move to/from host

  Show description for e2b2c6b
  jakirkham

  committed
  e2b2c6b

• Coerce memoryviews to NumPy ndarrays

  Show description for 87992a3
  jakirkham

  committed
  87992a3

• Assert the "dask" serializer is used in tests

  jakirkham

  committed
  a18fe5b

• Update docs on DeviceSerialized frames

  Show description for b9d277b
  jakirkham

  committed
  b9d277b

• Note "dask" serialization used for spilling

  jakirkham

  committed
  3f1577c

• Merge pull request #256 from jakirkham/use_dask_serialization_spill_to_host

  Show description for 2dab1b6
  pentschev

  authored
  2dab1b6