Architecture-dependent Page Table Entry (PTE) flags for x86_64 and aarch64

[kevinaboos]

For purposes of comparison with #696

Remaining TODO: add #[test] cases for the four conversion routines. Done

[NathanRoyer]

Looks good to me, other than the two other comments I left on specific lines.

Not so important: PteFlags has the flags that best match x86_64 & aarch64; if we want to support risc-v in the future, would we change it based on flags common to aarch64, x86_64 & risc-v? I guess we'll see in the future

[kevinaboos]

ok this should be completely ready now. @NathanRoyer please re-review and also see how I did the cool cfg stuff with docs

[NathanRoyer]

ok this should be completely ready now. @NathanRoyer please re-review

Your solution for ACCESSED will work. However, regarding PAGE_DESCRIPTOR, I'm not sure I understand why we shouldn't put it by default like ACCESSED because it's also a requirement for things to work... Without PAGE_DESCRIPTOR, all the other flags will be interpreted as if they were in a block descriptor

edit: we solved this during our meeting

also see how I did the cool cfg stuff with docs

Very nice, didn't know about this trick

[NathanRoyer]

Also, should we add a dummy PteFlags::is_huge(self) -> bool method? one that would always returns false?
(For the moment I commented out the calls to that method in memory)

[NathanRoyer]

By manually setting the bit, the descriptor was accepted into the TLB and I encountered no page fault. Maybe this was because of my configuration of translation control registers though? I'll check. It could also be qemu not implementing FEAT_HAFDBS.

Something I forgot: TCR_EL1 (the translation control register) has two flags that control the behaviour of the CPU regarding the ACCESSED and DIRTY flags: HA (for ACCESSED) and HD (for DIRTY). However, even with these enabled, qemu triggers an access fault when the bit is initially cleared.

Regarding the output address size: I was using 48-bit OA mode in my experimental code; I assumed we could only address up to 48 bits (4 levels * 9 bits + 12 bits).

However, actually one of the following configurations can be selected (using TCR_ELn::IPS and TCR_ELn::DS - weirdly, the latter is not exposed by cortex_a... I assumed the DS field was just not present in TCR_EL1):

As we can see, the 52-bit output address mode is a bit more complex as some bits of the OA are shifted to the middle of lower attributes. This allows the dirty bit to always occupy the 51st bit. In 48-bit OA mode, bits 48 & 49 are reserved and bit 50 is the Guarded Page bit - implementation is optional.

In 52-bit OA mode, the shareability attributes are stored in the TCR_EL1 register, specifcially in field SH0 for TTBR0 page table walks and in field SH1 for TTBR1 page table walks. This limits us to two shareability profiles and also complexifies page table management a bit since that requires us to maintain two page tables at all time.

Apparently long mode x86_64 also supports 52-bit OA mode; I don't know how this is configured however, but maybe you know.

[kevinaboos]

Also, should we add a dummy PteFlags::is_huge(self) -> bool method? one that would always returns false? (For the moment I commented out the calls to that method in memory)

whoa, definitely not. That would be invalid and could lead to us completely misinterpreting what PTE values mean. For a concrete example of why, in the initial bootstrap assembly code (boot.asm) we actually do use huge pages to efficiently and easily map large regions of the very lowest and very highest addresses before jumping to Rust code. If we attempted to call translate() on one of those addresses early on, it would completely fail because it would unconditionally consider it a non-huge page PTE.

In any case, why do something like that which could confuse the caller? It also acts as a sanity check if we ever do come across a mapping that uses huge pages, or in the future when we add proper support for them.

[kevinaboos]

By manually setting the bit, the descriptor was accepted into the TLB and I encountered no page fault. Maybe this was because of my configuration of translation control registers though? I'll check. It could also be qemu not implementing FEAT_HAFDBS.

Something I forgot: TCR_EL1 (the translation control register) has two flags that control the behaviour of the CPU regarding the ACCESSED and DIRTY flags: HA (for ACCESSED) and HD (for DIRTY). However, even with these enabled, qemu triggers an access fault when the bit is initially cleared.

That's fine, it seems that those bits just allow hardware to set the access flag (as is the default behavior on x86) instead of the OS always doing it (?). Looks like the Access Flag Fault always occurs regardless of HA/HD value, so let's just always set the ACCESSED bit for now, but also implement an Access Flag Fault handler such that we know when it happens (and can handle those faults in the future if/when we support swapping pages to disk).

[kevinaboos]

Regarding the output address size: I was using 48-bit OA mode in my experimental code; I assumed we could only address up to 48 bits (4 levels * 9 bits + 12 bits).

48-bit is fine with me, we can use that for now and if it presents problems we can move to 52-bit.

As we can see, the 52-bit output address mode is a bit more complex as some bits of the OA are shifted to the middle of lower attributes. This allows the dirty bit to always occupy the 51st bit. In 48-bit OA mode, bits 48 & 49 are reserved and bit 50 is the Guarded Page bit - implementation is optional.

I was wondering about the DIRTY bit, that makes sense. Glad we resolved that.

In 52-bit OA mode, the shareability attributes are stored in the TCR_EL1 register, specifcially in field SH0 for TTBR0 page table walks and in field SH1 for TTBR1 page table walks. This limits us to two shareability profiles and also complexifies page table management a bit since that requires us to maintain two page tables at all time.

We'd only ever use OUTER_SHAREABLE anyway, so that's not a problem. We could just use the same page tables for both TTBRs, i think.

Apparently long mode x86_64 also supports 52-bit OA mode; I don't know how this is configured however, but maybe you know.

We don't use it, but it's much simpler on x86 -- the 52-bit physical address can pretty much always be used.

[kevinaboos]

@NathanRoyer I have addressed all of our comments, please double-check everything again.

Important Note: in the memory_structs crate, you will need to implement aarch64-specific canonicalization for 48-bit physical addresses, since they now differ from the 52-bit physical addresses we use on x86_64.
Also, I assume that virtual addresses on aarch64 are also canonicalized in a similar fashion to x86; if not, you'll need to handle that too.
This stuff is on L129-159 of memory_structs/src/lib.rs

[NathanRoyer]

@NathanRoyer I have addressed all of our comments, please double-check everything again.

About PteFlagsAarch64::_CONTIGUOUS:

If not set, this translation table is not contiguous with the previous one in memory.

I think it's not necessarily the previous one; from the DDI0487:

The Contiguous bit identifies a descriptor as belonging to a group of adjacent translation table entries that point to a contiguous OA range.

But anyway, the nuance is small and we don't use it so I'll leave it as is.

Important Note: in the memory_structs crate, you will need to implement aarch64-specific canonicalization for 48-bit physical addresses, since they now differ from the 52-bit physical addresses we use on x86_64. Also, I assume that virtual addresses on aarch64 are also canonicalized in a similar fashion to x86; if not, you'll need to handle that too. This stuff is on L129-159 of memory_structs/src/lib.rs

I responded to this on Discord: the most significant bits of an address are typically used for tagged addresses (ASID) or ignored, when that mechanism is disabled.

I will approve this, you can merge it in.

[kevinaboos]

About PteFlagsAarch64::_CONTIGUOUS:

If not set, this translation table is not contiguous with the previous one in memory.

I think it's not necessarily the previous one; from the DDI0487:

The Contiguous bit identifies a descriptor as belonging to a group of adjacent translation table entries that point to a contiguous OA range.

Right, thanks for pointing that out. I'll fix it.