Support viewing JLD2 files #46
Comments
|
Do you have a sample file available for testing? |
|
EDIT: I also opened an issue here: silx-kit/h5web#1699 |
|
We will definitely strive to support JLD2 files. Thanks for opening an issue in the H5Web repo. As explained, the problem comes from h5wasm not currently supporting committed datatypes. I've opened an issue on the h5wasm repo too: usnistgov/h5wasm#80 |
|
Thanks to @bmaranville and @axelboc work, we made good progress towards this. I could read this simple JLD2 file without issue using the main branch of H5Web: example.zip Generation script# Inspired by https://github.com/JuliaIO/JLD2.jl?tab=readme-ov-file#jld2 using JLD2 |
|
I can confirm! julia> using JLD2
julia> struct InnerStruct
x::String
y::Int
end
julia> struct OuterStruct
a::Int
b::InnerStruct
c::NTuple{3,Int}
end
julia> jldsave("nested_compound.jld2"; data=OuterStruct(1, InnerStruct("two",3),(4,5,6))) |
|
Thanks for double-checking 🙂 I figured that the file was too simple so I hope to get a more complex example from people there: julia-vscode/julia-vscode#2863 (comment) |
|
Here are two more files which are conceptually interesting: H5Web currently gives you no way to view / de-reference linked datasets in this way. In my eyes, this is good enough for a release at this stage. Future feature ideas might be enhanced pretty printing for compound types and enabling the manual loading of referenced datasets. julia> using JLD2
julia> mutable struct RecursiveStruct
x::Float64
y::RecursiveStruct
RecursiveStruct(x)=new(x)
RecursiveStruct(x,y)=new(x,y)
end
julia> jldsave("recursive.jld2"; r=^C
julia> r = RecursiveStruct(1)
RecursiveStruct(1.0, #undef)
julia> r2 = RecursiveStruct(2, r)
RecursiveStruct(2.0, RecursiveStruct(1.0, #undef))
julia> r.y = r2
RecursiveStruct(2.0, RecursiveStruct(1.0, RecursiveStruct(#= circular reference @-2 =#)))
julia> jldsave("recursive.jld2"; r)
julia> load("recursive.jld2")
Dict{String, Any} with 1 entry:
"r" => RecursiveStruct(1.0, RecursiveStruct(2.0, RecursiveStruct(#= circular … =#)))
julia> struct ObjIDPreservation
arr1::Vector{Int}
arr2::Vector{Int}
end
julia> arr = [1,2,3,4,5,6]
6-element Vector{Int64}:
1
2
3
4
5
6
julia> obj = ObjIDPreservation(arr, arr)
ObjIDPreservation([1, 2, 3, 4, 5, 6], [1, 2, 3, 4, 5, 6])
julia> obj.arr1 === obj.arr2 # references the same memory
true
julia> jldsave("objidpreservation.jld2"; obj)
julia> data = load("objidpreservation.jld2", "obj")
ObjIDPreservation([1, 2, 3, 4, 5, 6], [1, 2, 3, 4, 5, 6])
julia> data.arr1 === data.arr2
true |
|
Sorry I don't really know how to use the main branch version of h5web, but I think tests above is good for most jld2 file case I will use |
|
#47 improves support for committed datatypes and #48 allows JLD2 files to open directly in H5Web. This should be sufficient to bring basic support for most JLD2 files and resolve this issue. I've started a discussion thread in the H5Web repo with improvement ideas mentioned in this issue so as to not lose track of them. Feel free to continue the discussion and/or create proper feature requests for these ideas over there. |
Is your feature request related to a problem?
JLD2is a file type natively created by Julia programming language, and it is heavily used in high performance scientific programs, JLD2JLD2is designed as comprising a subset of HDF5, though I'm non-expert in this repository, I think it might be reachable to supportJLD2with likewise interface.Alternatives you've considered
The authors of
JLD2.jlare eagering for a vscode-extension to visualizeJLD2files, see discussions belowhttps://discourse.julialang.org/t/jld2-preview-in-vscode/80050/6
julia-vscode/julia-vscode#2863
Additional context
Currently opening a
JLD2using H5web gives error message like below, any hint to solve it?The text was updated successfully, but these errors were encountered: