Add docs and examples for object_store_opendal

[Xuanwo]

object_store_opendal has been used by some users but we don't have a doc page for it. We need to improve this part.

related to: apache/arrow-rs#5752 (comment)

Tasks

• Add a README for this project, including
  • What's this project for
  • How to build/test/use this project
• Examples
  • Example for object_store
  • Example for datafusion
  • Example for polars
  • Example for delta-rs

[hanxuanliang]

This seems like a good opportunity to understand object_store, and I would like to give it a try.

[Xuanwo]

Hi, @hanxuanliang, are you interested in adding more examples?

[meteorgan]

I'm currently working on tasks related to object_store_opendal. and plan to address some legacy issues.
Do we still need additional examples ?
Also, could you clarify the task examples for datafusion ? As far as I know, Datafusion is already using object_store.

[Xuanwo]

Hi, @meteorgan, it’s greatly appreciated if you’re willing to take them. Yes, I believe we still need more examples.

Also, could you clarify the task examples for datafusion ? As far as I know, Datafusion is already using object_store.

I would like a complete end-to-end example that integrates all of them: DataFusion, OpenDAL, and object_store_opendal.

[meteorgan]

Hi, @meteorgan, it’s greatly appreciated if you’re willing to take them. Yes, I believe we still need more examples.

Also, could you clarify the task examples for datafusion ? As far as I know, Datafusion is already using object_store.

I would like a complete end-to-end example that integrates all of them: DataFusion, OpenDAL, and object_store_opendal.

I think i roughly understand your point. like this example in DataFusion: https://github.com/apache/datafusion/blob/30660e02eb2f1e730d7711685bee46fddfb23c5a/datafusion-examples/examples/external_dependency/query-aws-s3.rs ?

[Xuanwo]

I think i roughly understand your point. like this example in DataFusion: apache/datafusion@30660e0/datafusion-examples/examples/external_dependency/query-aws-s3.rs ?

Yep!

[meteorgan]

I've written an example of querying data from S3 using datafusion and object_store_opendal. However, when running the example, the following error was output:
Error: ObjectStore(NotSupported { source: Unsupported (permanent) at => get_opts is not implemented so far }).
so, It seems I need to address this issue: #5115 first. 😅