Skip to content
This repository has been archived by the owner on Apr 10, 2024. It is now read-only.

dtype precision / conversions #20

Open
jreback opened this issue Sep 8, 2016 · 1 comment
Open

dtype precision / conversions #20

jreback opened this issue Sep 8, 2016 · 1 comment
Labels

Comments

@jreback
Copy link

jreback commented Sep 8, 2016

this may not actually be an issue as we aren't using float np.nan as our missing marker, but
we tend to have some subtle issues when int64 are downcast to float64, IOW we have missing values in an integer array. We end up storing them as object to avoid this precision loss.

Just a reminder to test for things like this.

xref pandas-dev/pandas#14020 as an example

@wesm
Copy link
Owner

wesm commented Sep 8, 2016

Missing data uniformity and removing all the implicit type casting is definitely a top 5 priority from my POV. Not being able to exchange data with file formats and databases with high fidelity (e.g. integer->float casting with values over 2^53 actually loses data) is a serious problem for production use as an ETL / data engineering tool.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

2 participants