Skip to content

Commit

Permalink
[SPARK-14204][SQL] register driverClass rather than user-specified class
Browse files Browse the repository at this point in the history
This pull request fixes an issue in which cluster-mode executors fail to properly register a JDBC driver when the driver is provided in a jar by the user, but the driver class name is derived from a JDBC URL (rather than specified by the user).  The consequence of this is that all JDBC accesses under the described circumstances fail with an `IllegalStateException`. I reported the issue here: https://issues.apache.org/jira/browse/SPARK-14204

My proposed solution is to have the executors register the JDBC driver class under all circumstances, not only when the driver is specified by the user.

This patch was tested manually.  I built an assembly jar, deployed it to a cluster, and confirmed that the problem was fixed.

Author: Kevin McHale <[email protected]>

Closes apache#12000 from mchalek/jdbc-driver-registration.

(cherry picked from commit 0a13e4c)
  • Loading branch information
mchalek authored and zzcclp committed Jun 3, 2016
1 parent 23395c8 commit 9dda59e
Showing 1 changed file with 1 addition and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ object JdbcUtils extends Logging {
DriverManager.getDriver(url).getClass.getCanonicalName
}
() => {
userSpecifiedDriverClass.foreach(DriverRegistry.register)
DriverRegistry.register(driverClass)
val driver: Driver = DriverManager.getDrivers.asScala.collectFirst {
case d: DriverWrapper if d.wrapped.getClass.getCanonicalName == driverClass => d
case d if d.getClass.getCanonicalName == driverClass => d
Expand Down

0 comments on commit 9dda59e

Please sign in to comment.