Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-7691] [SQL] Refactor CatalystTypeConverter to use type-specific row accessors #6222

Closed
wants to merge 13 commits into from
Closed

[SPARK-7691] [SQL] Refactor CatalystTypeConverter to use type-specific row accessors #6222

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
1e87a45
WIP refactoring of CatalystTypeConverters
JoshRosen May 17, 2015
7ca7fcb
Comments and cleanup
JoshRosen May 17, 2015
ae3278d
Throw ClassCastException errors during inbound conversions.
JoshRosen May 25, 2015
9c0e4e1
Remove last use of convertToScala().
JoshRosen May 25, 2015
85bba9d
Fix wrong input data in InMemoryColumnarQuerySuite
JoshRosen May 25, 2015
8033d4c
Fix serialization error in UserDefinedGenerator.
JoshRosen May 25, 2015
677ff27
Fix null handling bug; add tests.
JoshRosen May 25, 2015
6ad0ebb
Fix JavaHashingTFSuite ClassCastException
JoshRosen May 25, 2015
3f7b2d8
Initialize converters lazily so that the attributes are resolved first
JoshRosen May 26, 2015
6edf7f8
Re-add convertToScala(), since a Hive test still needs it
JoshRosen May 27, 2015
5989593
Use new SparkFunSuite base in CatalystTypeConvertersSuite
JoshRosen May 29, 2015
befc613
Add tests to document Option-handling behavior.
JoshRosen Jun 1, 2015
740341b
Optimize method dispatch for primitive type conversions
JoshRosen Jun 1, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions mllib/src/test/java/org/apache/spark/ml/feature/JavaHashingTFSuite.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,9 +55,9 @@ public void tearDown() {
@Test
public void hashingTF() {
JavaRDD<Row> jrdd = jsc.parallelize(Lists.newArrayList(
RowFactory.create(0, "Hi I heard about Spark"),
RowFactory.create(0, "I wish Java could use case classes"),
RowFactory.create(1, "Logistic regression models are neat")
RowFactory.create(0.0, "Hi I heard about Spark"),
RowFactory.create(0.0, "I wish Java could use case classes"),
RowFactory.create(1.0, "Logistic regression models are neat")
));
StructType schema = new StructType(new StructField[]{
new StructField("label", DataTypes.DoubleType, false, Metadata.empty()),
Expand Down
Loading