[SPARK-23727][SQL] Support for pushing down filters for DateType in parquet #20851

yucai · 2018-03-18T03:54:50Z

What changes were proposed in this pull request?

This PR supports for pushing down filters for DateType in parquet

How was this patch tested?

Added UT and tested in local.

gatorsmile · 2018-03-18T03:57:50Z

ok to test

gatorsmile · 2018-03-18T04:04:32Z

...e/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala

+      def d: Date = new Date(Date.valueOf("2018-03-01").getTime + 24 * 60 * 60 * 1000 * (int - 1))
+    }
+
+    withParquetDataFrame((1 to 4).map(i => Tuple1(i.d))) { implicit df =>


These test cases only cover the limited cases. We need to check the boundary values

Could you kindly give me some examples about what kind of boundary tests? I checked parquet integer push down and ORC date type push down, seems like have covered all their tests.

gatorsmile · 2018-03-18T04:05:14Z

sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala

@@ -148,6 +193,15 @@ private[parquet] object ParquetFilters {
    case BinaryType =>
      (n: String, v: Any) =>
        FilterApi.gtEq(binaryColumn(n), Binary.fromReusedByteArray(v.asInstanceOf[Array[Byte]]))
+    case DateType =>


Add a new internal SQLConf to make it configurable. Users can turn it off if they hit a bug introduced in this PR

Have added, kindly help review.

SparkQA · 2018-03-18T07:05:01Z

Test build #88344 has finished for PR 20851 at commit 079af71.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

HyukjinKwon · 2018-03-18T10:19:04Z

sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala

+      (n: String, v: Any) => {
+        FilterApi.eq(
+          intColumn(n),
+          Option(v).map{ date =>


nit: p{ -> p {

SparkQA · 2018-03-18T18:54:01Z

Test build #88353 has finished for PR 20851 at commit 15bd28d.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2018-03-18T20:08:08Z

Test build #88356 has finished for PR 20851 at commit 1f2b450.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

viirya · 2018-03-19T02:02:03Z

sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala

+          intColumn(n),
+          Option(v).map { date =>
+            val days = date.asInstanceOf[java.sql.Date].getTime / (24 * 60 * 60 * 1000)
+            days.toInt.asInstanceOf[Integer]


Use DateTimeUtils.fromJavaDate here and below?

viirya · 2018-03-19T02:50:07Z

sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala

@@ -353,6 +353,12 @@ object SQLConf {
    .booleanConf
    .createWithDefault(true)

+  val PARQUET_FILTER_PUSHDOWN_DATE_ENABLED = buildConf("spark.sql.parquet.filterPushdown.date")
+    .doc("If true, enables Parquet filter push-down optimization for Date. " +


Should be an internal conf, i.e., .internal()?

@gatorsmile any idea?

Yes, it should be an internal conf. In Spark 3.0 release, we will revisit all the internal confs and remove all the unnecessary confs.

gatorsmile · 2018-03-19T03:53:38Z

cc @michal-databricks @cloud-fan

ghost · 2018-03-19T04:11:20Z

sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala

+      (n: String, v: Any) => {
+        FilterApi.notEq(
+          intColumn(n),
+          Option(v).map { date =>


These 3 lines are repeated several times in this change. Please refactor to a new method.

SparkQA · 2018-03-19T07:05:01Z

Test build #88367 has finished for PR 20851 at commit 8f36d1e.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

HyukjinKwon · 2018-03-20T03:54:12Z

...e/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala

-    withSQLConf(SQLConf.PARQUET_FILTER_PUSHDOWN_ENABLED.key -> "true") {
+    withSQLConf(
+      SQLConf.PARQUET_FILTER_PUSHDOWN_ENABLED.key -> "true",
+      SQLConf.PARQUET_FILTER_PUSHDOWN_DATE_ENABLED.key -> "true") {


nit:

withSQLConf( SQLConf.PARQUET_FILTER_PUSHDOWN_ENABLED.key -> "true", SQLConf.PARQUET_FILTER_PUSHDOWN_DATE_ENABLED.key -> "true", SQLConf.PARQUET_VECTORIZED_READER_ENABLED.key -> "false") {

HyukjinKwon · 2018-03-20T03:59:53Z

...e/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala

@@ -313,6 +316,36 @@ class ParquetFilterSuite extends QueryTest with ParquetTest with SharedSQLContex
    }
  }

+  test("filter pushdown - date") {
+    implicit class IntToDate(int: Int) {


Shall we pass a string here and convert it into a date?

How about this way? It is from ORC's date push down.

test("filter pushdown - date") { val dates = Seq("2017-08-18", "2017-08-19", "2017-08-20", "2017-08-21").map { day => Date.valueOf(day) } withOrcDataFrame(dates.map(Tuple1(_))) { implicit df => checkFilterPredicate('_1.isNull, PredicateLeaf.Operator.IS_NULL) checkFilterPredicate('_1 === dates(0), PredicateLeaf.Operator.EQUALS) checkFilterPredicate('_1 <=> dates(0), PredicateLeaf.Operator.NULL_SAFE_EQUALS) checkFilterPredicate('_1 < dates(1), PredicateLeaf.Operator.LESS_THAN) checkFilterPredicate('_1 > dates(2), PredicateLeaf.Operator.LESS_THAN_EQUALS) checkFilterPredicate('_1 <= dates(0), PredicateLeaf.Operator.LESS_THAN_EQUALS) checkFilterPredicate('_1 >= dates(3), PredicateLeaf.Operator.LESS_THAN) checkFilterPredicate(Literal(dates(0)) === '_1, PredicateLeaf.Operator.EQUALS) checkFilterPredicate(Literal(dates(0)) <=> '_1, PredicateLeaf.Operator.NULL_SAFE_EQUALS) checkFilterPredicate(Literal(dates(1)) > '_1, PredicateLeaf.Operator.LESS_THAN) checkFilterPredicate(Literal(dates(2)) < '_1, PredicateLeaf.Operator.LESS_THAN_EQUALS) checkFilterPredicate(Literal(dates(0)) >= '_1, PredicateLeaf.Operator.LESS_THAN_EQUALS) checkFilterPredicate(Literal(dates(3)) <= '_1, PredicateLeaf.Operator.LESS_THAN) } }

Either way looks fine but I usually stick to what other codes do around the fix though ..

Ok, do you mean this way? Looks like we need more words :).

implicit class StringToDate(s: String) { def d: Date = Date.valueOf(s) } withParquetDataFrame( Seq("2017-08-18", "2017-08-19", "2017-08-20", "2017-08-21").map(i => Tuple1(i.d))) { implicit df => checkFilterPredicate('_1.isNull, classOf[Eq[_]], Seq.empty[Row]) checkFilterPredicate('_1.isNotNull, classOf[NotEq[_]], Seq("2017-08-18", "2017-08-19", "2017-08-20", "2017-08-21").map(i => Row.apply(i.d))) checkFilterPredicate('_1 === "2017-08-18".d, classOf[Eq[_]], "2017-08-18".d) checkFilterPredicate('_1 =!= "2017-08-18".d, classOf[NotEq[_]], Seq("2017-08-19", "2017-08-20", "2017-08-21").map(i => Row.apply(i.d)))

I think "2017-08-19".d is at least better than 1.d.

Yup, I think this one is better than the current one.

As per @cloud-fan 's suggestion, I have changed 1.d to 1.date, which one is perferred?

"2017-08-18".d

1.date

"2017-08-18".date

the string one is more readable to me

HyukjinKwon · 2018-03-20T04:15:09Z

sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala

@@ -72,6 +82,14 @@ private[parquet] object ParquetFilters {
      (n: String, v: Any) => FilterApi.notEq(
        binaryColumn(n),
        Option(v).map(b => Binary.fromReusedByteArray(v.asInstanceOf[Array[Byte]])).orNull)
+    case DateType if SQLConf.get.parquetFilterPushDownDate =>
+      (n: String, v: Any) => {


nit:

(n: String, v: Any) => FilterApi.notEq( intColumn(n), Option(v).map { d => DateTimeUtils.fromJavaDate(d.asInstanceOf[java.sql.Date]).asInstanceOf[Integer] }.orNull)

cloud-fan · 2018-03-20T17:15:55Z

...e/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala

+      checkFilterPredicate('_1.isNull, classOf[Eq[_]], Seq.empty[Row])
+      checkFilterPredicate('_1.isNotNull, classOf[NotEq[_]], (1 to 4).map(i => Row.apply(i.d)))
+
+      checkFilterPredicate('_1 === 1.d, classOf[Eq[_]], 1.d)


1.d is weird, can we name it 1.date?

I agree 1.date is better, but binary is using "1.b", shall we keep the same pattern with it?

test("filter pushdown - binary") { implicit class IntToBinary(int: Int) { def b: Array[Byte] = int.toString.getBytes(StandardCharsets.UTF_8) }

b might be OK, but d is confusing as it's also a postfix for double.

Got it, thanks very much for explanation.

cloud-fan · 2018-03-20T17:21:29Z

sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala

+      (n: String, v: Any) => FilterApi.eq(
+          intColumn(n),
+          Option(v).map { d =>
+            DateTimeUtils.fromJavaDate(d.asInstanceOf[java.sql.Date]).asInstanceOf[Integer]


shall we respect session local timezone here?

I think we should do DateTimeUtils.fromJavaDate(d.asInstanceOf[java.sql.Date], SQLConf.get.sessionLocalTimeZone).asInstanceOf[Integer]

SparkQA · 2018-03-20T20:24:00Z

Test build #88426 has finished for PR 20851 at commit 82c5a73.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2018-03-21T07:05:02Z

Test build #88454 has finished for PR 20851 at commit 7946bea.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

dongjoon-hyun · 2018-03-22T03:51:30Z

sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala

+      "This configuration only has an effect when 'spark.sql.parquet.filterPushdown' is enabled.")
+    .internal()
+    .booleanConf
+    .createWithDefault(true)


For this kind of thing, we had better start with false.

dongjoon-hyun · 2018-03-22T03:54:26Z

sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala

+    case DateType if SQLConf.get.parquetFilterPushDownDate =>
+      (n: String, v: Any) => FilterApi.eq(
+        intColumn(n),
+        Option(v).map(date => dateToDays(date.asInstanceOf[java.sql.Date]).asInstanceOf[Integer])


Since we imported Date at line 20, we can remove java.sql. There are 6 occurrences.

- Option(v).map(date => dateToDays(date.asInstanceOf[java.sql.Date]).asInstanceOf[Integer]) - .orNull) + Option(v).map(date => dateToDays(date.asInstanceOf[Date]).asInstanceOf[Integer]).orNull)

dongjoon-hyun · 2018-03-22T04:08:49Z

...e/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala

+            '_1 < "2018-03-19".date || '_1 > "2018-03-20".date,
+            classOf[Operators.Or],
+            Seq(Row("2018-03-18".date), Row("2018-03-21".date)))
+        }


nit. Indentation.

Could you kindly show me how to improve? Thanks!

test("filter pushdown - date") { implicit class StringToDate(s: String) { def date: Date = Date.valueOf(s) } val data = Seq("2018-03-18", "2018-03-19", "2018-03-20", "2018-03-21") withParquetDataFrame(data.map(i => Tuple1(i.date))) { implicit df => checkFilterPredicate('_1.isNull, classOf[Eq[_]], Seq.empty[Row]) checkFilterPredicate('_1.isNotNull, classOf[NotEq[_]], data.map(i => Row.apply(i.date))) checkFilterPredicate('_1 === "2018-03-18".date, classOf[Eq[_]], "2018-03-18".date) checkFilterPredicate('_1 <=> "2018-03-18".date, classOf[Eq[_]], "2018-03-18".date) checkFilterPredicate('_1 =!= "2018-03-18".date, classOf[NotEq[_]], Seq("2018-03-19", "2018-03-20", "2018-03-21").map(i => Row.apply(i.date))) checkFilterPredicate('_1 < "2018-03-19".date, classOf[Lt[_]], "2018-03-18".date) checkFilterPredicate('_1 > "2018-03-20".date, classOf[Gt[_]], "2018-03-21".date) checkFilterPredicate('_1 <= "2018-03-18".date, classOf[LtEq[_]], "2018-03-18".date) checkFilterPredicate('_1 >= "2018-03-21".date, classOf[GtEq[_]], "2018-03-21".date) checkFilterPredicate( Literal("2018-03-18".date) === '_1, classOf[Eq[_]], "2018-03-18".date) checkFilterPredicate( Literal("2018-03-18".date) <=> '_1, classOf[Eq[_]], "2018-03-18".date) checkFilterPredicate( Literal("2018-03-19".date) > '_1, classOf[Lt[_]], "2018-03-18".date) checkFilterPredicate( Literal("2018-03-20".date) < '_1, classOf[Gt[_]], "2018-03-21".date) checkFilterPredicate( Literal("2018-03-18".date) >= '_1, classOf[LtEq[_]], "2018-03-18".date) checkFilterPredicate( Literal("2018-03-21".date) <= '_1, classOf[GtEq[_]], "2018-03-21".date) checkFilterPredicate(!('_1 < "2018-03-21".date), classOf[GtEq[_]], "2018-03-21".date) checkFilterPredicate( '_1 < "2018-03-19".date || '_1 > "2018-03-20".date, classOf[Operators.Or], Seq(Row("2018-03-18".date), Row("2018-03-21".date))) } }

Oops, I edited it.

Thanks, @HyukjinKwon ! That's better what I thought.

HyukjinKwon · 2018-03-22T04:55:18Z

Seems fine otherwise.

dongjoon-hyun · 2018-03-22T06:15:07Z

...e/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala

+        '_1 < "2018-03-19".date || '_1 > "2018-03-20".date,
+        classOf[Operators.Or],
+        Seq(Row("2018-03-18".date), Row("2018-03-21".date)))
+        }


This is different from what @HyukjinKwon gave you. :)

yucai · 2018-03-22T06:22:23Z

Sorry for mistake! is it the same now? @HyukjinKwon @dongjoon-hyun

SparkQA · 2018-03-22T07:05:02Z

Test build #88498 has finished for PR 20851 at commit 423a938.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds the following public classes (experimental):
implicit class StringToDate(s: String)

SparkQA · 2018-03-22T07:05:02Z

Test build #88505 has finished for PR 20851 at commit a58eec8.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2018-03-22T07:05:02Z

Test build #88506 has finished for PR 20851 at commit b43e36c.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

HyukjinKwon · 2018-03-22T07:26:16Z

retest this please

SparkQA · 2018-03-22T10:47:01Z

Test build #88512 has finished for PR 20851 at commit b43e36c.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

cloud-fan · 2018-03-23T04:26:47Z

sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala

+      "This configuration only has an effect when 'spark.sql.parquet.filterPushdown' is enabled.")
+    .internal()
+    .booleanConf
+    .createWithDefault(false)


an internal by-default-false conf usually means it's not available for users...

@dongjoon-hyun @HyukjinKwon suggested such configuration could be better to start with "false", but actually ORC does support Date push down by default, any idea?

I am fine either way.

@yucai . The reason is that spark.sql.orc.filterPushdown is still false in Spark 2.3 while spark.sql.parquet.filterPushdown is true. We don't know this is safe or not.

Anyway, we have 6 or more months for Apache Spark 2.4. We may enable this in master branch temporarily for testing purpose only, and are able to disable this at the last moment of 2.4 release like we did about ORC conf if there is some issue.

BTW, did you use this in your company a lot in production?

I think it's common that we turn on new feature by default, if there is no known regression. And turn it off if we find regression later.

If you think so, +1. :)

BTW, based on Apache Spark way, I assume that this will not land on branch-2.3 with spark.sql.parquet.filterPushdown.date=true.

This is not a bug fix. We will not backport it to branch-2.3

Great! Thank you for confirmation.

@dongjoon-hyun, we are investigating to use this feature in some kind of eBay's queries, if its performance is good, will benefit a lot.
As per our discussion, I will turn it on by default, thanks very much!

cloud-fan · 2018-03-26T03:04:11Z

sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala

+    case DateType if SQLConf.get.parquetFilterPushDownDate =>
+      (n: String, v: Any) => FilterApi.eq(
+        intColumn(n),
+        Option(v).map(date => dateToDays(date.asInstanceOf[Date]).asInstanceOf[Integer]).orNull)


sorry I was wrong. I took a look at how these dates get created, in DataSourceStrategy.translateFilter. Actually they are created via DateTimeUtils.toJavaDate without timezone, which means here we should not use timezone either.

cloud-fan · 2018-03-26T03:05:13Z

sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala

@@ -129,6 +154,10 @@ private[parquet] object ParquetFilters {
    case BinaryType =>
      (n: String, v: Any) =>
        FilterApi.gt(binaryColumn(n), Binary.fromReusedByteArray(v.asInstanceOf[Array[Byte]]))
+    case DateType if SQLConf.get.parquetFilterPushDownDate =>


we should refactor it, so that adding a new data type doesn't need to touch so many places. This can be done later.

SparkQA · 2018-03-26T05:34:57Z

Test build #88575 has finished for PR 20851 at commit b70def0.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2018-03-29T12:17:35Z

Test build #88700 has finished for PR 20851 at commit 759f468.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

cloud-fan · 2018-03-30T07:07:49Z

thanks, merging to master!

…arquet ## What changes were proposed in this pull request? This PR supports for pushing down filters for DateType in parquet ## How was this patch tested? Added UT and tested in local. Author: yucai <[email protected]> Closes apache#20851 from yucai/SPARK-23727.

[SPARK-23727][SQL] Support DATE predict push down in parquet

079af71

gatorsmile reviewed Mar 18, 2018

View reviewed changes

HyukjinKwon reviewed Mar 18, 2018

View reviewed changes

yucai changed the title ~~[SPARK-23727][SQL] Support DATE predict push down in parquet~~ [SPARK-23727][SQL] Support for pushing down filters for DateType in parquet Mar 18, 2018

yucai added 2 commits March 19, 2018 00:05

minor, code style

59d8483

Add spark.sql.parquet.filterPushdown.date

1f2b450

yucai force-pushed the SPARK-23727 branch from 15bd28d to 1f2b450 Compare March 18, 2018 16:40

viirya reviewed Mar 19, 2018

View reviewed changes

ghost reviewed Mar 19, 2018

View reviewed changes

yucai added 2 commits March 19, 2018 13:17

Make spark.sql.parquet.filterPushdown.date internal

4df29df

Use DateTimeUtils.fromJavaDate

8f36d1e

HyukjinKwon reviewed Mar 20, 2018

View reviewed changes

code style

82c5a73

cloud-fan reviewed Mar 20, 2018

View reviewed changes

yucai added 2 commits March 21, 2018 12:45

Use 1.date instead of 1.d

784b625

Should respect session local timezone

7946bea

Make suite more readable

423a938

dongjoon-hyun reviewed Mar 22, 2018

View reviewed changes

code style

a58eec8

dongjoon-hyun reviewed Mar 22, 2018

View reviewed changes

minor

b43e36c

cloud-fan reviewed Mar 23, 2018

View reviewed changes

Set spark.sql.parquet.filterPushdown.date=true by default

b70def0

cloud-fan reviewed Mar 26, 2018

View reviewed changes

convert date without timezone

759f468

asfgit closed this in b02e76c Mar 30, 2018

[SPARK-23727][SQL] Support for pushing down filters for DateType in parquet #20851

[SPARK-23727][SQL] Support for pushing down filters for DateType in parquet #20851

Conversation

yucai commented Mar 18, 2018 • edited Loading

What changes were proposed in this pull request?

How was this patch tested?

gatorsmile commented Mar 18, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gatorsmile Mar 18, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SparkQA commented Mar 18, 2018

Choose a reason for hiding this comment

SparkQA commented Mar 18, 2018

SparkQA commented Mar 18, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gatorsmile commented Mar 19, 2018

Choose a reason for hiding this comment

SparkQA commented Mar 19, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yucai Mar 21, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SparkQA commented Mar 20, 2018

SparkQA commented Mar 21, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dongjoon-hyun Mar 22, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

HyukjinKwon Mar 22, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

HyukjinKwon commented Mar 22, 2018

Choose a reason for hiding this comment

yucai commented Mar 22, 2018

SparkQA commented Mar 22, 2018

SparkQA commented Mar 22, 2018

SparkQA commented Mar 22, 2018

HyukjinKwon commented Mar 22, 2018

SparkQA commented Mar 22, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SparkQA commented Mar 26, 2018

SparkQA commented Mar 29, 2018

cloud-fan commented Mar 30, 2018

yucai commented Mar 18, 2018 •

edited

Loading

gatorsmile Mar 18, 2018 •

edited

Loading

yucai Mar 21, 2018 •

edited

Loading

dongjoon-hyun Mar 22, 2018 •

edited

Loading

HyukjinKwon Mar 22, 2018 •

edited

Loading