a few tweaks to speed up data generation

[rongou]

With the default parameters, the data generation time is reduced from 47.674 seconds to 2.027 seconds on my machine.

But I still run out of memory when converting to DMatrix if I try to generate too big of a dataset. Need to investigate further.

@RAMitchell

[RAMitchell]

What if we just use a completely randomly generated X/y matrix e.g. numpy.random.rand? It will be fast to generate and still generate trees. It will be more or less useless for checking accuracy but for speed it should be fine.

[rongou]

Is that more or less what make_classification() is doing? After these tweaks the time to generate the data seems reasonable, but I still run out of memory on my desktop (32 GB RAM) if I attempt too big of a dataset.

[RAMitchell]

make_classification() is doing something a little more complicated. From memory I think it tries to generate a set of Gaussian clusters and assign points to those clusters. Its not very memory efficient or fast.

[rongou]

I think there is some value in checking the accuracy.

Anyway, with the last commit, now I can reliably crash the training on my laptop (32GB RAM, Quadro P1000 with 4GB):

$ python benchmark_tree.py --rows 13000000 --columns 100 --test_size 0.01
Generating dataset: 13000000 rows * 100 columns
0.01/0.99 test/train split
Generate Time: 48.709909200668335 seconds
DMatrix Start
DMatrix Time: 5.693941593170166 seconds
Training with 'gpu_hist'
terminate called after throwing an instance of 'thrust::system::system_error'
  what():  parallel_for failed: out of memory
Aborted (core dumped)

[rongou]

@RAMitchell it's a good idea to switch to random numpy arrays. :) Please take another look.