Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-12692][BUILD][STREAMING] Scala style: Fix the style violation (Space before "," or ":") #10685

Closed
wants to merge 2 commits into from

Conversation

sarutak
Copy link
Member

@sarutak sarutak commented Jan 10, 2016

Fix the style violation (space before , and :).
This PR is a followup for #10643.

@@ -26,20 +26,20 @@ import org.slf4j.{Logger, LoggerFactory}
private[sink] trait Logging {
// Make the log field transient so that objects with Logging can
// be serialized and used on another machine
@transient private var log_ : Logger = null
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't write like log_: Logger. So, can we rename those vars to which starts with _?

@SparkQA
Copy link

SparkQA commented Jan 10, 2016

Test build #49066 has finished for PR 10685 at commit 39ef2b8.

  • This patch fails RAT tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • class PageView(val url: String, val status: Int, val zipCode: Int, val userID: Int)
    • class FlumeEventServer(receiver: FlumeReceiver) extends AvroSourceProtocol
    • class ObjectInputStreamWithLoader(_inputStream: InputStream, loader: ClassLoader)
    • abstract class InputDStream[T: ClassTag] (_ssc: StreamingContext)
    • abstract class ReceiverInputDStream[T: ClassTag](_ssc: StreamingContext)

@sarutak sarutak force-pushed the SPARK-12692-followup-streaming branch 3 times, most recently from 2a184e8 to 15e958d Compare January 10, 2016 20:22
@sarutak sarutak force-pushed the SPARK-12692-followup-streaming branch from 5733a35 to 708f934 Compare January 10, 2016 20:24
@SparkQA
Copy link

SparkQA commented Jan 10, 2016

Test build #49067 has finished for PR 10685 at commit 4190930.

  • This patch fails MiMa tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • class PageView(val url: String, val status: Int, val zipCode: Int, val userID: Int)
    • class FlumeEventServer(receiver: FlumeReceiver) extends AvroSourceProtocol
    • class ObjectInputStreamWithLoader(_inputStream: InputStream, loader: ClassLoader)
    • abstract class InputDStream[T: ClassTag] (_ssc: StreamingContext)
    • abstract class ReceiverInputDStream[T: ClassTag](_ssc: StreamingContext)

@rxin
Copy link
Contributor

rxin commented Jan 10, 2016

LGTM - the mima check is related to the logging trait change.

@SparkQA
Copy link

SparkQA commented Jan 10, 2016

Test build #49069 has finished for PR 10685 at commit 708f934.

  • This patch fails MiMa tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • class PageView(val url: String, val status: Int, val zipCode: Int, val userID: Int)
    • class FlumeEventServer(receiver: FlumeReceiver) extends AvroSourceProtocol
    • class ObjectInputStreamWithLoader(_inputStream: InputStream, loader: ClassLoader)
    • class ConstantInputDStream[T: ClassTag](_ssc: StreamingContext, rdd: RDD[T])
    • abstract class InputDStream[T: ClassTag] (_ssc: StreamingContext)
    • abstract class ReceiverInputDStream[T: ClassTag](_ssc: StreamingContext)

@SparkQA
Copy link

SparkQA commented Jan 10, 2016

Test build #49072 has finished for PR 10685 at commit 0055761.

  • This patch fails MiMa tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@sarutak
Copy link
Member Author

sarutak commented Jan 10, 2016

The reason of the MiMa failure is Maven Central doesn't provide mvttv3;1.0.1 while streaming-mqtt_2.10;1.6.0 depends on mqttv3;1.0.1 right?

[info]  [SUCCESSFUL ] org.apache.spark#spark-streaming-mqtt_2.10;1.6.0!spark-streaming-mqtt_2.10.jar (21ms)
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  ::          UNRESOLVED DEPENDENCIES         ::
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  :: org.eclipse.paho#org.eclipse.paho.client.mqttv3;1.0.1: not found
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::

@sarutak
Copy link
Member Author

sarutak commented Jan 10, 2016

#10683 and #10678 may hit the same MiMa problem

@sarutak
Copy link
Member Author

sarutak commented Jan 10, 2016

retest this please.

@SparkQA
Copy link

SparkQA commented Jan 11, 2016

Test build #49074 has finished for PR 10685 at commit 0055761.

  • This patch fails from timeout after a configured wait of 250m.
  • This patch merges cleanly.
  • This patch adds no public classes.

@sarutak
Copy link
Member Author

sarutak commented Jan 11, 2016

retest this please.

@SparkQA
Copy link

SparkQA commented Jan 11, 2016

Test build #49085 has finished for PR 10685 at commit 0055761.

  • This patch fails MiMa tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@JoshRosen
Copy link
Contributor

Jenkins, retest this please.

@SparkQA
Copy link

SparkQA commented Jan 11, 2016

Test build #49119 has finished for PR 10685 at commit 0055761.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@sarutak
Copy link
Member Author

sarutak commented Jan 11, 2016

retest this please.

@SparkQA
Copy link

SparkQA commented Jan 11, 2016

Test build #49138 has finished for PR 10685 at commit 0055761.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@sarutak
Copy link
Member Author

sarutak commented Jan 11, 2016

retest this please.

@SparkQA
Copy link

SparkQA commented Jan 11, 2016

Test build #49144 has finished for PR 10685 at commit 0055761.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@sarutak
Copy link
Member Author

sarutak commented Jan 11, 2016

retest this please.

@SparkQA
Copy link

SparkQA commented Jan 11, 2016

Test build #49153 has finished for PR 10685 at commit 0055761.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@sarutak
Copy link
Member Author

sarutak commented Jan 12, 2016

I think the PySpark unit test failures above are related to #10704 and already fixed.

@sarutak
Copy link
Member Author

sarutak commented Jan 12, 2016

retest this please.

@SparkQA
Copy link

SparkQA commented Jan 12, 2016

Test build #49199 has finished for PR 10685 at commit 0055761.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@rxin
Copy link
Contributor

rxin commented Jan 12, 2016

Thanks - I'm going to merge it.

@asfgit asfgit closed this in 39ae04e Jan 12, 2016
@rxin
Copy link
Contributor

rxin commented Jan 12, 2016

@sarutak I actually reverted this change after talking more with Michael and looking at the diffs. The problem is that for colon, in some cases it is really useful to have a space before colon. In particular:

  1. In many cases for type bounds, ti is good to have a space before the colon so it is more clear.
  2. For type descriptions (e.g. variable : Int), it is also often useful to have a space.

I don't know whether it is feasible to actually implement the rules for the general colon, but exclude type description and type bounds. If yes, we should add this enforcement. Otherwise I'd prefer not to have it at all.

For comma, we should always have it (at least I haven't seen a case in which we don't want it yet).

@sarutak
Copy link
Member Author

sarutak commented Jan 13, 2016

1. In many cases for type bounds, ti is good to have a space before the colon so it is more clear.
2. For type descriptions (e.g. variable : Int), it is also often useful to have a space.

It exactly make sense. O.K, let's check style violation only for comma.
Only the change for SQL is reverted for now so would you like me to revert the rest of the changes?

@rxin
Copy link
Contributor

rxin commented Jan 13, 2016

The rest are ok since they are small. I reverted the sql one because it was large.

asfgit pushed a commit that referenced this pull request Jan 13, 2016
… before ",")

Fix the style violation (space before , and :).
This PR is a followup for #10643 and rework of #10685 .

Author: Kousuke Saruta <[email protected]>

Closes #10732 from sarutak/SPARK-12692-followup-sql.
@sarutak sarutak deleted the SPARK-12692-followup-streaming branch April 12, 2016 13:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants