Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support version tagging #588

Closed
changhiskhan opened this issue Feb 15, 2023 · 0 comments · Fixed by #2482
Closed

Support version tagging #588

changhiskhan opened this issue Feb 15, 2023 · 0 comments · Fixed by #2482
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed python rust Rust related tasks

Comments

@changhiskhan
Copy link
Contributor

Git-like version tags make it easier to manage lots of versions and quickly know which versions are released for example.

Version tags should be written to the version Manifest (protos: https://github.com/eto-ai/lance/blob/main/protos/format.proto#L74, rust: https://github.com/eto-ai/lance/blob/main/rust/src/format/manifest.rs#L55)

Write

  1. Manifest file is written here when writing a dataset.
  2. The version tag should be passed down from python -> pyo3 -> core lance (see step 1).
  3. Currently only WriteParams are passed down to core lance. The API should distinguish writeparams and metadata so it's probably best to create a new struct that can be extended to hold (1) tags, (2) schema metadata, and (3) version-specific metadata

Read

The tag needs to be added to the PyDict in the pyo3 code: https://github.com/eto-ai/lance/blob/main/python/src/dataset.rs#L183

Checkout

On the python side, add a convenience feature to checkout a tag instead of a specific version number

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed python rust Rust related tasks
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant