get_variable: Unsupported variable types

[maximilianpreisinger]

Hello Guys! I run the following piece of code:

using MATLAB
                   
mf = MatFile(filename)
vars = variable_names(mf)
my_var = get_variable(mf,vars[4])
close(mf)   

And I get the error:

ERROR: ArgumentError: Unsupported kind of variable.
Stacktrace:
 [1] jvalue(::MATLAB.MxArray) at /home/maximilian/.julia/v0.6/MATLAB/src/mxarray.jl:612
 [2] get_variable(::MATLAB.MatFile, ::String) at /home/maximilian/.julia/v0.6/MATLAB/src/matfile.jl:49
 [3] macro expansion at /home/maximilian/.julia/v0.6/Atom/src/repl.jl:118 [inlined]
 [4] anonymous at ./<missing>:?

The wanted variable shows the type 1x1 SimulationOutput in Matlab.

Are you intending to add support for SimulationOutput variables, which you get when you run a Simulink .slx Simulation? Do you have an overview of the variable types, which you support, or which you don't support?

Furthermore, when opening nested 1x1 struct variables (I have a struct that is 4 times nested, which looks in matlab something like this: parameter.a.b.c.d = [1 0 0] for example. I get the following error:

ERROR: NULL pointer for MxArray.
Stacktrace:
 [1] MATLAB.MxArray(::Ptr{Void}, ::Bool) at /home/maximilian/.julia/v0.6/MATLAB/src/mxarray.jl:8
 [2] Dict(::MATLAB.MxArray) at /home/maximilian/.julia/v0.6/MATLAB/src/mxarray.jl:589
 [3] jvalue(::MATLAB.MxArray) at /home/maximilian/.julia/v0.6/MATLAB/src/mxarray.jl:610
 [4] Dict(::MATLAB.MxArray) at /home/maximilian/.julia/v0.6/MATLAB/src/mxarray.jl:590
 [5] jvalue(::MATLAB.MxArray) at /home/maximilian/.julia/v0.6/MATLAB/src/mxarray.jl:610
 [6] get_variable(::MATLAB.MatFile, ::String) at /home/maximilian/.julia/v0.6/MATLAB/src/matfile.jl:49
 [7] macro expansion at /home/maximilian/.julia/v0.6/Atom/src/repl.jl:118 [inlined]
 [8] anonymous at ./<missing>:?

So I suppose, you also do not support nested struct variables at the moment. Will this change?

[MaxandreJ]

I agree, no support for nested structs. I would really appreciate if there was support for it.

[MaxandreJ]

Here is a workaround using PyCall and scipy.io. First you need to make sure that your nested structs are really structs and not user-defined classes, in which case it won't work. To transform a Matlab class into struct before exporting, run in MATLAB the command my_struct = struct(my_class);. This will trigger a warning, ignore it.

Then this is the code I used to import this nested struct into MATLAB (with Julia v1.0)

using PyCall

function get_matlab_struct(filepath::String, variable_name::String) :: PyObject

    sio = pyimport("scipy.io")

    matfile = sio[:loadmat](filepath)
    my_struct_array = matfile[variable_name]

    return my_struct_array
end

function get_struct_fieldnames(my_struct_element::PyObject) :: Array{String, 1}
    fieldnames_tuple = my_struct_element[:dtype][:names]

    fieldnames_array = collect(fieldnames_tuple)

    return fieldnames_array
end

function get_matlab_struct_as_dict(my_struct::PyObject) :: Dict{String, Any}

    my_struct_element = my_struct[1,1]

    fieldnames = get_struct_fieldnames(my_struct_element)

    my_dict = Dict()
    for fieldname in fieldnames
        my_dict[fieldname] =  get(my_struct_element,fieldname)
    end

    return my_dict
end

I call the functions in that code snippet using

my_struct = get_matlab_struct(filepath, variable_name)
dict = get_matlab_struct_as_dict(my_struct)
#dict[fieldname_containing_struct] itself contains another matlab struct
dict[fieldname_containing_struct] = get_matlab_struct_as_dict(dict[fieldname_containing_struct])

the variables (filepath, variable_name and fieldname_containing_struct) that are used may be defined as

filepath = "../../Data/simulations.mat"
variable_name = "simulations_export"
fieldname_containing_struct = "values"

Hope that helps!

[musm]

can you provide a MWE ?

[MaxandreJ]

Thank you for answering @musm !

Okay, I've got it! The problem isn't to do with nested structs, actually. The existence of a struct makes the bug less easy to track down though.

What I've found to be the gist of the bug is that string arrays, introduced in MATLAB R2016b, are not supported by MATLAB.jl.

This can be seen by running in MATLAB

my_string_array = ["This is", "a test"];

then saving the variable calling it string_array.mat (I used the GUI), and then running in Julia:

using MATLAB

my_matfile_contents = read_matfile("../../Data/TestNestedStruct/string_array.mat")

my_variable_MxArray = my_matfile_contents["my_string_array"]

my_variable_julia = jvalue(my_variable_MxArray)

which results in the error

ERROR: ArgumentError: Unsupported kind of variable.

Now if you nest a string array in a struct, by running the MATLAB code

my_string_array = ["This is", "a test"];

my_struct = struct;

my_struct.my_string_array = my_string_array;

save it in as struct_string_array.mat and try to import the matfile in Julia using the code

my_matfile_contents = read_matfile("../../Data/TestNestedStruct/struct_string_array.mat")

my_struct_MxArray = my_matfile_contents["my_struct"]

my_struct_julia = jvalue(my_struct_MxArray)

you get the error

ERROR: NULL pointer for MxArray.

I suggest, listing first the things that should be done in priority and that are simpler to implement:

1. Indicate on the github README file that MATLAB string arrays are not supported, and that they can cause the error ERROR: NULL pointer for MxArray. if they are nested within a struct.
2. Have string arrays be imported as a type that cannot be accessed within Julia, without returning an error (but just a warning), so that other fields of structs can still be accessed in Julia. For example, scipy.io returns a class MatlabOpaque for MATLAB string arrays, but doesn't return an error and lets me access the other fields of the struct that were imported.
3. Allow the import of MATLAB string arrays into Julia.

(As an aside, I think, but I'm not entirely sure, that a similar bug exists for importing in Julia user-defined classes in MATLAB, but since scipy.io doesn't work for that either, it's not too bad... It should probably be mentionned in the README too nevertheless. I don't have any more time tonight to provide a MWE for this last point about classes so I may be wrong on that...)

[musm]

@MaxandreJ thanks for the details, indeed you are right. I was exploring the issue and basically it seems like some of the matlab api cannot deal with string arrays, which is why we get null pointers.

[musm]

Basically the current workaround is to use the matlab function convertContainedStringsToChars directly from matlab to create a compatible .dat file that then can be loaded using MATLAB.jl

[MaxandreJ]

Thank you for finding this function, yes that works perfectly! If the variable you want to export to Julia has a string array, this function should be called on the variable before saving it, either to a .dat file or a .mat file.

[musm]

Basically this is impossible with the C engine for MATLAB, we would have to use the C++ engine instead, which is a big project, but a long term goal.

[MaxandreJ]

Thanks for digging into this @musm ! Although a complete fix doesn't seem possible in the short term, I think the package could be improved by (as I said in a previous post on this thread):

• Indicate on the github README file that MATLAB string arrays are not supported, and that they can cause the error ERROR: NULL pointer for MxArray. if they are nested within a struct.

or better,

• Have string arrays be imported as a type that cannot be accessed within Julia, without returning an error (but just a warning), so that other fields of structs can still be accessed in Julia. For example, scipy.io returns a class MatlabOpaque for MATLAB string arrays, but doesn't return an error and lets me access the other fields of the struct that were imported.

Best,
Max

[musm]

Main issue addressed by updating the readme. Thanks @maximilianpreisinger

[MaxandreJ]

Thank you @musm and @maximilianpreisinger !