Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Npy file support #17

Merged
merged 1 commit into from
Jan 3, 2025
Merged

Npy file support #17

merged 1 commit into from
Jan 3, 2025

Conversation

seanmcl
Copy link
Collaborator

@seanmcl seanmcl commented Jan 2, 2025

NumPy uses a custom binary format to store ndarrays. This file implements that format.

structure Dtype where
name : TensorLib.Dtype.Name
order : ByteOrder
deriving BEq, Repr, Inhabited
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this defined already in Dtype.lean?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, but this one supports "native" which I want to avoid but need to support parsing. There is some duplicate constructors but it's not too awful I think. I thought about trying to use Lean to get the byte ordering of the processor but decided against it. We can revisit later?

descr : Dtype
dataOrder : DataOrder := DataOrder.C
shape : Shape
deriving Repr, Inhabited
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Side comment, I have a "tensor type" in NKL, which is basically this structure. However, I need BEq.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's not use Header for that. This is supposed to be an annoying implementation detail. You don't need the version number for example.

@seanmcl seanmcl force-pushed the sm/02-dtype branch 2 times, most recently from 85c35c4 to 629ebe3 Compare January 3, 2025 23:00
Base automatically changed from sm/02-dtype to main January 3, 2025 23:01
NumPy uses a custom binary format to store ndarrays.
This file implements that format.
@seanmcl seanmcl merged commit e997936 into main Jan 3, 2025
1 check passed
@seanmcl seanmcl deleted the sm/03-npy branch January 3, 2025 23:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants