The attempt to upgrade Spark version to 2.3.0 is not successful

[TitusAn]

Describe the bug
The attempt to upgrade Spark version to 2.3.0 result in no success. The executable no longer passes some of the unit tests, with error

FAILURE! - in io.archivesunleashed.SimpleDfTest
io.archivesunleashed.SimpleDfTest  Time elapsed: 3.968 sec  <<< ERROR!
java.lang.IllegalAccessError: tried to access method com.google.common.base.Stopwatch.<init>()V from class org.apache.hadoop.mapreduce.lib.input.FileInputFormat
	at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:379)

FAILURE! - in io.archivesunleashed.ArchiveRecordTest
io.archivesunleashed.ArchiveRecordTest  Time elapsed: 0 sec  <<< ERROR!
org.apache.spark.SparkException: Only one SparkContext may be running in this JVM (see SPARK-2243). To ignore this error, set spark.driver.allowMultipleContexts = true. The currently running SparkContext was created at:
org.apache.spark.SparkContext.<init>(SparkContext.scala:75)
io.archivesunleashed.SimpleDfTest$$anonfun$1.apply$mcV$sp(SimpleDfTest.scala:40)
io.archivesunleashed.SimpleDfTest$$anonfun$1.apply(SimpleDfTest.scala:36)

Tests run: 50, Failures: 0, Errors: 11, Skipped: 0

To Reproduce
Steps to reproduce the behavior (e.g.):

1. Open pom.xml in the root directory of the project
2. Change <spark.version>2.1.1</spark.version> to <spark.version>2.3.0</spark.version>
3. Build the project

Expected behavior
The project can be built successfully, and passes all unit tests.

Desktop/Laptop (please complete the following information):

• OS: Ubuntu 16.04 LTS
• Java: 1.8.0
• Scala: 2.11.6

Additional context
Change logs of Spark 2.2 and 2.3 may provide insights into the problem.
Spark 2.2 Change Log
Spark 2.3 Change Log

[ruebot]

@lintool do you have that branch laying around from the camp when we upgraded to 2.3.0, when we were working on the broken twitter stuff?

[ruebot]

nvm, we got it here.

I'll take care of this one.