Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC-2758: Merge Append Into Write #2758

Merged
merged 6 commits into from
Aug 2, 2023
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
79 changes: 79 additions & 0 deletions core/src/docs/rfcs/2758_merge_append_into_write.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
- Proposal Name: `merge_append_into_write`
- Start Date: 2023-08-02
- RFC PR: [apache/incubator-opendal#2758](https://github.com/apache/incubator-opendal/pull/2758)
- Tracking Issue: [apache/incubator-opendal#0000](https://github.com/apache/incubator-opendal/issues/0000)
Xuanwo marked this conversation as resolved.
Show resolved Hide resolved

# Summary

Merge the `appender` API into `writer` by introducing a new `writer_with.append(true)` method to enable append mode.

# Motivation

Currently OpenDAL has separate `appender` and `writer` APIs:

```rust
let mut appender = op.appender_with("file.txt").await?;

appender.append(bs).await?;
appender.append(bs).await?;
```

This duplication forces users to learn two different APIs for writing data.

By adding this change, we can:

- Simpler API surface - users only need to learn one writing API.
- Reduce code duplication between append and write implementations.
- Atomic append semantics are handled internally in `writer`.
- Reuse the `sink` api for both `overwrite` and `append` mode.

# Guide-level explanation

The new approach is to enable append mode on `writer`:

```rust
let mut writer = op.writer_with("file.txt").append(true).await?;

writer.write(bs).await?; // appends to file
writer.write(bs).await?; // appends again
```

Calling `writer_with.append(true)` will start the writer in append mode. Subsequent `write()` calls will append rather than overwrite.

There is no longer a separate `appender` API.

# Reference-level explanation

We will add an `append` flag into `OpWrite`:

```rust
impl OpWrite {
pub fn with_append(mut self, append: bool) -> Self {
self.append = append;
self
}
}
```

All services need to check `append` flag and handle append mode accordingly. Services that not support append should return an `Unsupported` error instead.

# Drawbacks

- `writer` API is more complex with the append mode flag.
- Internal implementation must handle both overwrite and append logic.

# Rationale and alternatives

None

# Prior art

Python's file open() supports an `"a"` mode flag to enable append-only writing.

# Unresolved questions

None

# Future possibilities

None
3 changes: 3 additions & 0 deletions core/src/docs/rfcs/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -145,3 +145,6 @@ pub mod rfc_2299_chain_based_operator_api {}

#[doc = include_str!("2602_object_versioning.md")]
pub mod rfc_2602_object_versioning {}

#[doc = include_str!("2758_merge_append_into_write.md")]
pub mod rfc_2758_merge_append_into_write {}