You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
segment filenames are limited to 8 characters and are currently a base 10 number that is incremented every time a new segment is created. Although sufficient for most use cases it does mean that for very high throughput Ra clusters it is possible that the number will wrap around after a few years when creating a new segment every second or so.
A wrap around event would cause the segments to be sorted in the wrong order and would cause incorrect log reads.
Expected behavior
by adding a prefix to the segment name we can ensure it is much harder to cause a wrap around event.
Alternative 1: Add the expansion of "~YYMMDD" to the segment name as a prefix. This would allow the creation of over 1000 segment files per second which is extremely unlikely. This approach would also retain the sorting requirements even for servers with pre-existing segment files with the old format.
This alternative creates quite odd looking file names
Alternative 2:
upgrade to a 16 or 20 character name and rename existing segments. this can be done in the segment writer's init function and a marker file can be written to avoid re-entering the upgrade flow once done.
This also should be done for the WAL.
The text was updated successfully, but these errors were encountered:
Describe the bug
segment filenames are limited to 8 characters and are currently a base 10 number that is incremented every time a new segment is created. Although sufficient for most use cases it does mean that for very high throughput Ra clusters it is possible that the number will wrap around after a few years when creating a new segment every second or so.
A wrap around event would cause the segments to be sorted in the wrong order and would cause incorrect log reads.
Expected behavior
by adding a prefix to the segment name we can ensure it is much harder to cause a wrap around event.
Alternative 1: Add the expansion of "~YYMMDD" to the segment name as a prefix. This would allow the creation of over 1000 segment files per second which is extremely unlikely. This approach would also retain the sorting requirements even for servers with pre-existing segment files with the old format.
This alternative creates quite odd looking file names
Alternative 2:
upgrade to a 16 or 20 character name and rename existing segments. this can be done in the segment writer's init function and a marker file can be written to avoid re-entering the upgrade flow once done.
This also should be done for the WAL.
The text was updated successfully, but these errors were encountered: