[GLUTEN-1712] Create folder if not exits on HDFS write #267

JkSelf · 2023-05-22T06:30:09Z

No description provided.

zhouyuan · 2023-05-23T13:32:03Z

velox/connectors/hive/storage_adapters/hdfs/HdfsWriteFile.cpp

+  // Check whether the parentDir exist, create it if not exist.
+  if (hdfsExists(hdfsClient_, parentDir.c_str()) == -1) {
+    hdfsCreateDirectory(hdfsClient_, parentDir.c_str());
+  }


does this work for "/example.parquet" ?

Yes. This ut can test this case.

relative pr: add decimal column reader support oap-project#254 Add utility method MemoryUsageTracker::highUsage() oap-project#227 Support parquet read case sensitive option oap-project#126 Make varchar and varbinary compatible oap-project#115 Create folder if not exits on HDFS write oap-project#267

write parquet data into temp dir

31caa33

zhouyuan reviewed May 23, 2023

View reviewed changes

zhouyuan changed the title ~~[GLUTEN-1712]Write parquet data into temp dir~~ [GLUTEN-1712] Create folder if not exits on HDFS write May 24, 2023

zhouyuan merged commit 3e2b6f5 into oap-project:main May 24, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[GLUTEN-1712] Create folder if not exits on HDFS write #267

[GLUTEN-1712] Create folder if not exits on HDFS write #267

JkSelf commented May 22, 2023

zhouyuan May 23, 2023

JkSelf May 24, 2023

[GLUTEN-1712] Create folder if not exits on HDFS write #267

[GLUTEN-1712] Create folder if not exits on HDFS write #267

Conversation

JkSelf commented May 22, 2023

zhouyuan May 23, 2023

Choose a reason for hiding this comment

JkSelf May 24, 2023

Choose a reason for hiding this comment