Willing to support max_bs argument for Backtest#optimize?

[binarymason]

Would love to be able to pass a max batch size argument to optimize like this:

bt.optimize(..., max_bs=500)

backtesting.py/backtesting/backtesting.py

Lines 1335 to 1338 in 73e1534

	def _batch(seq):
	n = np.clip(len(seq) // (os.cpu_count() or 1), 5, 300)
	for i in range(0, len(seq), n):
	yield seq[i:i + n]

Is that something that you would support? I'm happy to make a PR

[kernc]

I guess you're seeing your subprocess iterations taking too long? How high would you set your max? I strongly prefer intelligent defaults over separate arguments for every little detail. I guess the max (so far 300) could be better set introspecting upon passed data size, so as to assure speedy and responsive updating of the loop regardless of size. A PR for something like this is certainly welcome. 👍

[binarymason]

Exactly, I have a 38 core machine that is only utilitizing a few cores and taking too long because my param_combos is not too terribly long but my dataset is large. In my specific case param_combos had a total length of 60, but that was only creating 12 batches and letting 26 cores sit idle.

After digging into this some more, I think I have a solution we can make without increasing maximum. Wish I could get idle cores completely down to zero, but it is at least an improvement.
#295