DWARF variant metadata for compressed enums with niche is ambiguous

[Kobzol]

After the change of DWARF metadata generation for enums (#54004), compressed enums with niches are not distinguishable from tagged or empty enums in GDB pretty printing (not in the actual Rust pretty printers, as they do not support the new format yet).

In the new enum debug metadata format, enums have a variant field that should serve as a discriminant to pretty print the correct variant of the enum. However, when the Rust enum has a niche discriminant, the variant contains some integer value that has to be interpreted in a special way. Before #54004, compressed enums were marked (https://github.com/rust-lang/rust/blob/master/src/etc/debugger_pretty_printers_common.py#L230 ) and could be distinguished from normal enums. But currently, the variant field has the same name for all kinds of enums and therefore it is not possible to distinguish (and correctly pretty print) compressed enums.

Example:

enum MyEnum1<T> { A(T), B }

fn main()
{
    let a1 = MyEnum1::A(String::from("111"));
    let a2: MyEnum1<String> = MyEnum1::B;
}

This is a niche enum that gets its debug niche value here (https://github.com/rust-lang/rust/blob/master/src/librustc_codegen_llvm/debuginfo/metadata.rs#L1516).

Printing this in GDB:

# valobj is the debugged variable
content = valobj[valobj.type.fields()[0]]
print(valobj.type.fields()[0].name, content[content.type.fields()[0]])

outputs:

<<variant>> 94022087334720 // a1
<<variant>> 0 // a2

As opposed to a tagged enum:

let a1 = MyEnum1::A(5);
let a2: MyEnum1<u32> = MyEnum1::B;

which outputs:

<<variant>> 0 // a1
<<variant>> 1 // a2

I don't see any way how can I distinguish that this is not a normal enum and that the discriminant has to be handled in a special way. And even if there was such a way, how am I supposed to interpret the discriminant? In the old version the name of the discriminant encoded some metadata that helped to interpret the value (https://github.com/rust-lang/rust/blob/master/src/etc/debugger_pretty_printers_common.py#L289), but there's nothing like that in the new version. Am I missing something?

I noticed this while modifying enum pretty printers in the IntelliJ Rust plugin (intellij-rust/intellij-rust#4155).

Related issues:
#54004
#55701
#59509

[philipc]

This is the debuginfo I'm seeing for your example:

< 2><0x000003cf>      DW_TAG_structure_type
                        DW_AT_name                  MyEnum1<alloc::string::String>
                        DW_AT_byte_size             0x00000018
                        DW_AT_alignment             0x00000008
< 3><0x000003d6>        DW_TAG_variant_part
                          DW_AT_discr                 <0x000003db>
< 4><0x000003db>          DW_TAG_member
                            DW_AT_type                  <0x000004bb>
                            DW_AT_alignment             0x00000008
                            DW_AT_data_member_location  0
                            DW_AT_artificial            yes(1)
< 4><0x000003e2>          DW_TAG_variant
< 5><0x000003e3>            DW_TAG_member
                              DW_AT_name                  A
                              DW_AT_type                  <0x000003fe>
                              DW_AT_alignment             0x00000008
                              DW_AT_data_member_location  0
< 4><0x000003ef>          DW_TAG_variant
                            DW_AT_discr_value           0
< 5><0x000003f1>            DW_TAG_member
                              DW_AT_name                  B
                              DW_AT_type                  <0x0000041a>
                              DW_AT_alignment             0x00000008
                              DW_AT_data_member_location  0
< 3><0x000003fe>        DW_TAG_structure_type
                          DW_AT_name                  A
                          DW_AT_byte_size             0x00000018
                          DW_AT_alignment             0x00000008
< 4><0x00000405>          DW_TAG_member
                            DW_AT_name                  __0
                            DW_AT_type                  <0x00000080>
                            DW_AT_alignment             0x00000008
                            DW_AT_data_member_location  0
< 4><0x00000410>          DW_TAG_template_type_parameter
                            DW_AT_type                  <0x00000080>
                            DW_AT_name                  T
< 3><0x0000041a>        DW_TAG_structure_type
                          DW_AT_name                  B
                          DW_AT_byte_size             0x00000018
                          DW_AT_alignment             0x00000008
< 4><0x00000421>          DW_TAG_template_type_parameter
                            DW_AT_type                  <0x00000080>
                            DW_AT_name                  T

I don't see any way how can I distinguish that this is not a normal enum and that the discriminant has to be handled in a special way.

If you had to, I think you could do this by checking if the discriminant overlaps any members in any of the variants. So for the example, the discriminant at 0x000003db overlaps with the A::__0 member at 0x00000405. But I'm not sure there is a need for this.

And even if there was such a way, how am I supposed to interpret the discriminant?

I'm not sure that there is a need to handle this differently. The only difference is that one variant does not have a DW_AT_discr_value attribute, which means it is the default variant. So if no other discriminant values match then it should be used. For the example, variant 0x000003e2 is the default.

[Kobzol]

Thank you for your response, may I ask what you used to pretty print the debug info? 🙂
The problem that I'm facing is that I need to find the active variant of an enum from the GDB Python API (I need to read the value of the discriminant and be able to distinguish what variant does the enum hold).

If there is a "classic" discriminant, then I read the value of the discriminant, if it's 0, it's the first variant, it it's 1, it's the second variant etc. By knowing the variant, I can pretty print it correctly inside GDB.

However, if the niche is used, the value of the discriminant is some arbitrary value (the memory contents of the niche variant) or 0 (that actually means that the enum contains the second variant because the first one cannot contain zero as a valid value, e.g. it's a reference). If I knew that the enum is a niche enum, I could assume that zero in discriminant means it's the second variant and everything else means that it's the first, niche variant (however I think that there may be also more complicated cases, I'm not sure about that).

The problem is that I don't currently see a way how to extract the information that the first member doesn't have the DW_AT_discr_value attribute in GDB. In the old debuginfo format, the information about the niche enum was encoded in the enum discriminant name (https://github.com/rust-lang/rust/blob/master/src/etc/debugger_pretty_printers_common.py#L289). Theferore it was possible to pretty print the niche enums. Is there something similar in the current debug info version?

[philipc]

I used dwarfdump, but readelf, objdump and llvm-dwarfdump can print it as well.

However, if the niche is used, the value of the discriminant is some arbitrary value (the memory contents of the niche variant) or 0 (that actually means that the enum contains the second variant because the first one cannot contain zero as a valid value, e.g. it's a reference).

Right.

If I knew that the enum is a niche enum, I could assume that zero in discriminant means it's the second variant and everything else means that it's the first, niche variant (however I think that there may be also more complicated cases, I'm not sure about that).

You can tell it is a niche enum by looking at the DW_AT_discr_value attributes, and these give you the values, you don't need to assume anything.

The problem is that I don't currently see a way how to extract the information that the first member doesn't have the DW_AT_discr_value attribute in GDB.

I don't know anything about how to extract that information in gdb, but if that isn't possible in gdb, then the solution is to fix gdb. As far as I can tell, the DWARF contains all the required information, and the old method of encoding information in the names was a hack that is no longer needed. I'm pretty sure @tromey had this all working in gdb though.

[Kobzol]

I don't know anything about how to extract that information in gdb, but if that isn't possible in gdb, then the solution is to fix gdb. As far as I can tell, the DWARF contains all the required information, and the old method of encoding information in the names was a hack that is no longer needed.

Thank you, this is what I wanted to confirm. @tromey do you have any suggestions on how to extract this information from gdb.Value or gdb.Type?

[tromey]

A few things to say on this topic.

First, if this is just about the display of Rust enums on the gdb command line, then you should not need a pretty-printer at all, any more. gdb understands how to handle enums directly.

Using the example given above, I get:

(gdb) p/r a2
$1 = q::MyEnum1<alloc::string::String>::B

Second, note that the state before #54004 was not ideal -- there were variants that weren't represented by the encoding scheme. I would be opposed to an attempt to go back to that approach.

That said, I think there are two things that are reasonable to want that gdb doesn't implement:

1. Support for Rust enums in MI varobjs. See bug 20163. I don't know if IntelliJ uses these or not.
2. Support for variants in gdb's Python layer. As above, this isn't needed for ordinary printing; but maybe it's useful for other purposes. I don't think there's a bug open for this yet. I'm reasonably likely to do something in this area in the medium term as part of my work on Ada; but if someone wants to start it sooner, I'm happy to provide advice etc.

[Kobzol]

Thank you for the details! With GDB 8.1, it only prints
$1 = example_project::MyEnum2<alloc::string::String>, i.e. it doesn't specify the variant correctly. With GDB 8.3 (which is bundled with IntelliJ), I see the output correctly, so clearly there was some change that added support for reading this DWARF format. The print itself is fine, as you said.

However, in IntelliJ we need support for this in the Python API layer. The IDE doesn't just print the variable, it shows it in a hierarchical tree, to display the elements of a Vec, attributes of a struct etc. So from a gdb.Value we would like to get the current variant of the enum to get it's fields and contents. We have a problem with the optimized enums though, because right now I don't see any way of getting the enum's variant from the Python API (while there existed a way before using the encoded enum, even though it was clearly a hack).

If you think that adding this functionality to the Python API is feasible, I'd like to try to add it. Where would this have to go, would GDB have to be modified? Before it was possible to read the enum variant even though GDB didn't support it directly, because Rust generated debug info in a way that allowed this. The new enum debug generation is much cleaner, but it would be nice to get this little feature back :)

[tromey]

Where would this have to go, would GDB have to be modified?

Yeah, gdb would have to be modified. It turns out there is a gdb bug for this.

My current thought (and I'll update that bug as well) is to mark the gdb.Type's fields as "dynamic" in some way, and then have some method on gdb.Value that can be used to see which field (or fields in the Ada case) are actually available. This is more generic than what Rust needs, but it would be good for my purposes to have a single approach that can also work for Ada (which allows much more variety here).

The idea then would be that a Rust enum type would list all the variants, but given a value of that type, it would be easy to determine which branch was active. Would this work for your purposes? I don't recall offhand if access to the discriminant is also available, but if you need it, I can't think of a reason that would prevent this as well.

[Kobzol]

Accessing the discriminant was just a means to an end to select the correct variant of the enum. The real functionality that we need (and that I guess any other Rust debugger or pretty printer would like to have) is exactly what you describe, given a gdb.Value, get the active enum variant so that we can query and display its fields (and also the variant name, which is useful for debugging).

[tromey]

FWIW the gdb patch landed a while back. It is in the latest release.

[Kobzol]

Thanks for the info, it works with newer GDB (>= 10.1) and this issue can be thus closed.