Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(bigtable): Allow GC updates on emulated aggregate column family #11499

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

return nil, status.Errorf(codes.InvalidArgument, "Immutable fields 'value_type.aggregate_type' cannot be updated")
}

if newcf.valueType == nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This condition needs to be changed.

Right now, column family has only two fields that can be updated. So, it makes sense to assume that when a field is nil, it is not getting updated and the other one is. This is not future compatible in the scenario where more fields get added to column family.

// A set of columns within a table which share a common configuration.
type ColumnFamily struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
// Garbage collection rule specified as a protobuf.
// Must serialize to at most 500 bytes.
//
// NOTE: Garbage collection executes opportunistically in the background, and
// so it's possible for reads to return a cell even if it matches the active
// GC expression for its family.
GcRule *GcRule `protobuf:"bytes,1,opt,name=gc_rule,json=gcRule,proto3" json:"gc_rule,omitempty"`
// The type of data stored in each of this family's cell values, including its
// full encoding. If omitted, the family only serves raw untyped bytes.
//
// For now, only the `Aggregate` type is supported.
//
// `Aggregate` can only be set at family creation and is immutable afterwards.
//
// If `value_type` is `Aggregate`, written data must be compatible with:
// - `value_type.input_type` for `AddInput` mutations
ValueType *Type `protobuf:"bytes,3,opt,name=value_type,json=valueType,proto3" json:"value_type,omitempty"`
}

Instead, use the UpdateMask field. As documented here, read the updatemask from the req. Then, check if it is nil. If it is nil, assume it is updating gcrule.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, this was extremely helpful! I've just pushed some changes that will allow for partial updates of the column family based on the UpdateMask, as well as some additional tests to validate the expected behavior of the UpdateMask in this case. Please let me know if there's anything else I need to change.

@wesnel wesnel force-pushed the wesnel/fix-aggregate-cf-gc-policy-update branch from 2bef391 to 087ebb2 Compare January 25, 2025 01:39
@wesnel wesnel requested a review from bhshkh January 25, 2025 01:42
@wesnel wesnel force-pushed the wesnel/fix-aggregate-cf-gc-policy-update branch from 087ebb2 to e52fc23 Compare January 25, 2025 01:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigtable Issues related to the Bigtable API.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

bigtable: Emulator does not allow GC policy changes on existing aggregate column family
2 participants