Skip to content

Commit

Permalink
[SPARK-22002][SQL][FOLLOWUP][TEST] Add a test to check if the origina…
Browse files Browse the repository at this point in the history
…l schema doesn't have metadata.

## What changes were proposed in this pull request?

This is a follow-up pr of apache#19231 which modified the behavior to remove metadata from JDBC table schema.
This pr adds a test to check if the schema doesn't have metadata.

## How was this patch tested?

Added a test and existing tests.

Author: Takuya UESHIN <[email protected]>

Closes apache#20585 from ueshin/issues/SPARK-22002/fup1.
  • Loading branch information
ueshin authored and gatorsmile committed Feb 12, 2018
1 parent 5bb1141 commit 0c66fe4
Showing 1 changed file with 22 additions and 0 deletions.
22 changes: 22 additions & 0 deletions sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCSuite.scala
Original file line number Diff line number Diff line change
Expand Up @@ -1168,4 +1168,26 @@ class JDBCSuite extends SparkFunSuite
val df3 = sql("SELECT * FROM test_sessionInitStatement")
assert(df3.collect() === Array(Row(21519, 1234)))
}

test("jdbc data source shouldn't have unnecessary metadata in its schema") {
val schema = StructType(Seq(
StructField("NAME", StringType, true), StructField("THEID", IntegerType, true)))

val df = spark.read.format("jdbc")
.option("Url", urlWithUserAndPass)
.option("DbTaBle", "TEST.PEOPLE")
.load()
assert(df.schema === schema)

withTempView("people_view") {
sql(
s"""
|CREATE TEMPORARY VIEW people_view
|USING org.apache.spark.sql.jdbc
|OPTIONS (uRl '$url', DbTaBlE 'TEST.PEOPLE', User 'testUser', PassWord 'testPass')
""".stripMargin.replaceAll("\n", " "))

assert(sql("select * from people_view").schema === schema)
}
}
}

0 comments on commit 0c66fe4

Please sign in to comment.