Use Cats backpressure to throttle writes (and conversions to HTTP requests)

[thorkildcognite]

No description provided.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 83.33%. Comparing base (28a3425) to head (54431c0).

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #1000   +/-   ##
=======================================
  Coverage   83.33%   83.33%           
=======================================
  Files          47       47           
  Lines        3156     3156           
  Branches      452      455    +3     
=======================================
  Hits         2630     2630           
  Misses        526      526           

Files with missing lines	Coverage Δ
...main/scala/cognite/spark/v1/RawTableRelation.scala	94.87% <100.00%> (ø)

[dmivankov]

compared to _.grouped.toSeq.grouped.foreach it will now create a vector of all batches, they will reference all Row items from iterator which potentially could be not a lot of mem, less than request bodies, more than constant mem

but compared to .grouped.toVector it is about the same apart from having semaphore

[dmivankov]

doesn't look like there's a out-of-the box Seq.parTraverse_, or even better the one that would not take more items when semaphore is full, so for now we can try the .toVector.parTraverse_