Spark 1271 (1320) cogroup and groupby should pass iterator[x]#242

Closed

holdenk wants to merge 39 commits intoapache:masterfrom holdenk:spark-1320-cogroupandgroupshouldpassiterator

+252-153

Commits on Apr 8, 2014

This is the it compiles point of replacing Seq with Iterator and JList with JIterator in the groupby and cogroup signatures
holdenk
committed
Calling size on an iterator is not so good if we want to use it after
holdenk
committed
Fix some tests
holdenk
committed
Fix groupbykey to actually give back an iterator
holdenk
committed
I think this might be a bad rabbit hole. Started work to make CoGroupedRDD use iterator and then went crazy
holdenk
committed
Fix Java API suite
holdenk
committed
hmmm try and fix up basic operation suite
holdenk
committed
Revert "I think this might be a bad rabbit hole. Started work to make CoGroupedRDD use iterator and then went crazy"
holdenk
committed
org.apache.spark.rdd.PairRDDFunctionsSuite passes
holdenk
committed
core/tests now pass
holdenk
committed
Fix some of the types in the streaming JavaAPI suite. Probably still needs more work
holdenk
committed
I really need to stop calling size on iterators, it is the path of sadness.
holdenk
committed
Fix style issues
holdenk
committed
Revert this but for now put things in list pandas
holdenk
committed
Add a join based on the problem in SVD
holdenk
committed
Revert me, but we have some stuff to debug
holdenk
committed
Fix the bug
holdenk
committed
Revert
holdenk
committed
Revert "Revert this but for now put things in list pandas"
holdenk
committed
Revert "Add a join based on the problem in SVD"
holdenk
committed
Switch cast in PairDStreamFunctions
holdenk
committed
Fix test issues\!
holdenk
committed
Fix groupbykey test in JavaAPISuite of streaming
holdenk
committed
oops, was checking wrong iterator
holdenk
committed
Fix cogroup test in JavaAPISuite for streaming
holdenk
committed
Start investigating moving to iterators for python API like the Java/Scala one. tl;dr: We will have to write our own iterator since the default one doesn't pickle well
holdenk
committed
Try and make pickling list iterators work
holdenk
committed
Revert "Try and make pickling list iterators work"
holdenk
committed
Python tests now pass with iterator pandas
holdenk
committed
Refactor from iterator to iterable
holdenk
committed
Fix the core test suite to compile
holdenk
committed
Fix streaming tests
holdenk
committed
Fix Java 8 cogroup tests for the new API
holdenk
committed
If guava 14 had iterables
holdenk
committed
maybe I should have had a coffee first. use correct import for guava iterables
holdenk
committed
switch resultitr to resultiterable
holdenk
committed
use toSeq instead
holdenk
committed
Fix merge up to master
holdenk
committed
Fix bad merge, should have been Iterable rather than Iterator
holdenk
committed