Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[OPPRO-77] adding hdfs support #152

Closed
wants to merge 1 commit into from
Closed

Conversation

zhouyuan
Copy link
Contributor

@zhouyuan zhouyuan commented May 11, 2022

What changes were proposed in this pull request?

This patch adds HDFS support

Signed-off-by: Yuan Zhou [email protected]

How was this patch tested?

locally verified

@github-actions
Copy link

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/oap-project/gluten/issues

Then could you also rename commit message and pull request title in the following format?

[Gluten-${ISSUES_ID}] ${detailed message}

See also:

@zhouyuan
Copy link
Contributor Author

depends on: oap-project/velox#11

@zhztheplayer zhztheplayer force-pushed the main branch 2 times, most recently from c6de69a to df3ea7a Compare May 24, 2022 02:35
@zhouyuan zhouyuan force-pushed the wip_hdfs branch 2 times, most recently from a3f8cb8 to f6fc466 Compare July 6, 2022 04:10
@jinchengchenghh jinchengchenghh force-pushed the main branch 2 times, most recently from 8c08f38 to b850a55 Compare July 8, 2022 02:57
@zhouyuan zhouyuan force-pushed the wip_hdfs branch 7 times, most recently from 1d5fe07 to 8ae8625 Compare July 13, 2022 15:00
@rui-mo rui-mo mentioned this pull request Jul 20, 2022
@zhouyuan zhouyuan force-pushed the wip_hdfs branch 2 times, most recently from f52e4aa to 28002f9 Compare September 6, 2022 03:58
@zhouyuan zhouyuan marked this pull request as ready for review September 6, 2022 03:59
Signed-off-by: Yuan Zhou <[email protected]>
@KevinyhZou
Copy link
Contributor

@zhouyuan hello, I use this pr to support read dwrf file from hdfs, but encounter the error in DwrfDatasource#InspectSchema, which do not support hdfs. and I have make another pr: #389, please help to review.

@FelixYBW
Copy link
Contributor

FelixYBW commented Oct 8, 2022

@KevinyhZou FYI, Velox redesigned their parquet reader and we have add it to Gluten. You can test parquet file format now.

@zhouyuan
Copy link
Contributor Author

zhouyuan commented Oct 8, 2022

@KevinyhZou Thanks for working on this! let's switch to your new PR

@zhouyuan
Copy link
Contributor Author

zhouyuan commented Oct 8, 2022

replaced by #389

@zhouyuan zhouyuan closed this Oct 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants