New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make API for importing dataset less verbose #341

Closed

jaichopra opened this issue Dec 1, 2022 · 1 comment · Fixed by #342

Assignees

Contributor

jaichopra commented Dec 1, 2022

Currently we require users to proxy through arrow APIs to create a dataset from a DataFrame or Parquet file.

i.e.

import pyarrow as pa

# Import a pandas DataFrame to Lance
tbl = pa.Table.from_pandas(my_dataframe)
lance.write_dataset(tbl, '/tmp/my_dataframe.lance')

# Import a Parquet file to Lance
parquet_dataset = ds.dataset('/tmp/hello.parquet')
lance.write_dataset(parquet_dataset, '/tmp/hello.lance')

It's preferred to hide this away and make it as simple as possible

jaichopra assigned eddyxu and changhiskhan

Contributor

eddyxu commented Dec 1, 2022

Do we want to get this in before updating README?

eddyxu mentioned this issue

[Python] lance.write_dataset takes pandas DataFrame #342

Merged

eddyxu closed this as completed in #342

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment