Is there a mechanism to emulate only CAS operations?

[romancardenas]

I'm working with the SparkFun RED-V RedBoard, which contains an E310g002 RISC-V microcontroller. This microcontroller is supposed to support atomic operations, and therefore I can compile projects targeting riscv32imac-unknown-none-elf. However, this microcontroller always raises an exception for CAS instructions (i.e., in practice only amoxxx instructions are allowed).

Is there any mechanism to configure portable-atomic to only emulate CAS instructions while natively executing the other atomic operations? Or am I doomed to compiling for riscv32imc-unknown-none-elf and forgetting about native atomic support?

Thanks in advance!

[taiki-e]

However, this microcontroller always raises an exception for CAS instructions (i.e., in practice only amoxxx instructions are allowed).

Is this an officially recognized bug? If not, it is not very clear whether it is a bug in the chip or a bug in your environment.

For example, I have seen several cases where there is a bug in the linker script, resulting in misalignment (mvdnes/spin-rs#154 (comment), rust-lang/rust#86693, etc.). Atomic instructions require proper alignment, so I would not be surprised if some atomic instructions caused exceptions in such cases. Could you check to see if the pointer is aligned correctly?

If this is really a bug in the chip, I can consider providing a way to work around it in some way.

[romancardenas]

It is not a bug, it is a (very annoying) feature.

The microcontroller datasheet explicitly mentions that it supports atomic operations except CAS, which raise a LoadFault/StoreFault exception.

[taiki-e]

The microcontroller datasheet explicitly mentions that it supports atomic operations except CAS, which raise a LoadFault/StoreFault exception.

Thanks for the clarification.

I can compile projects targeting riscv32imac-unknown-none-elf

As far as I know, at least both LLVM and GCC treat RISC-V with the A extension as if all instructions in the A extension are available (godbolt). So, compiling as riscv32imac for this CPU should usually be considered unsound because it is impossible to safely run a binary that is compiled as riscv32imac, even though it claims to be riscv32imac (unless you can ensure that the instructions causing the problem are not used by auditing the standard library, all dependencies, and the C library to which it is linked).

So, we should compile as riscv32imc for this CPU, which supports some instructions of the A extension.

only emulate CAS instructions while natively executing the other atomic operations

RISC-V A extension does not actually have CAS instructions and has LR/SC and AMO (CAS is implemented by using LR/SC), but the actual situation here is more complicated:

• RISC-V does not have {8,16}-bit AMOs, so some of the {8,16}-bit atomic RMWs are implemented by using LR/SC, which isn't supported in this CPU. (godbolt)
• There is not an AMO for all the operations we provide, CAS is probably not the only operation for which we actually need to provide emulation. (e.g., fetch_neg)
• If the A extension is not enabled, the compiler will not generate code that uses AMO.

---

Considering the above, the implementation would be in the form of:

• Add Cargo feature to use AMO on RISC-V without A extension (such as riscv32imc).
  • This feature requires unsafe-assume-single-core feature.
    • critical-section implementation may not be lock-free, so it cannot be used together with critical-section feature for now.
• Use AMO for 32-bit RMWs that have corresponding AMO instructions and {8,16}-bit RMWs that can be emulated by 32-bit AMOs (e.g., fetch_and/fetch_or).
  • Other RMWs will be emulated as before.
• Use inline assembly for implementation.
  • I have code that implements atomics in AMO in another repository, so it would basically be porting and extending that code.
    • Instructions in the A extension are not available on RISC-V without A extension by default, but can be made available by using the .option arch, +a directive.

Any thoughts?

[romancardenas]

Sorry for not being precise. The instructions that are missing in E310x are LR and SC.

I agree with your approach:

• We compile for riscv32imc targets.
• We add a feature to use LR/SC-free AMO instructions when available.
  • We use the unsafe-assume-single-core feature for the other atomic functions.

[taiki-e]

Filed #124 to implement this.

[taiki-e]

#124 has been published in 1.5.0.

[taiki-e]

This feature seems to have been ratified as Zaamo a few month ago.

https://github.com/riscv/riscv-zaamo-zalrsc/blob/v1.0.0/zaamo-zalrsc.adoc#1-atomic-memory-operations-extension-zaamo

The Zaamo extension comprises the atomic memory operation (AMO) subset of the A extension cite:[unpriv].

I plan to make force-amo feature available via -C target-feature=+zaamo in portable-atomic 1.8.

[taiki-e]

Support for Zaamo feature (and various improvements) are published in 1.8.0.

[romancardenas]

Hi @taiki-e

I have been playing around with the -C target-feature=+zaamo in plain Rust but it seems that it still does not work (even in nightly) for the core atomic crate. Do you know where should I ask for this feature to be available?

[taiki-e]

zaamo only covers some atomic operations. so there is no way to support this in core::sync::atomic (which supports load store only or all atomic operations). (a, zalrsc, or zacas is needed)

portable-atomic unsafe-assume-single-core can support this because it can use alternative implementation that compatible with zaamo for operations that not covered by zaamo.

[romancardenas]

I thought core would support all atomic operations covered by zaamo and leave the others undefined. In this way, portable-atomic would only be needed for non-zaamo operations. I guess this is not possible (yet), but future versions of Rust might implement it.

[taiki-e]

I think it's relatively easy in portable-atomic with unsafe-assume-single-core/critical-section disabled, but I think it's (technically possible but) much harder in core. In addition to the fact that it basically always requires -Zbuild-std, in RISC-V without a extension there are various patterns for atomic support based on target features: the operations that zaamo can support depend on its width (not only its kind), it also depends on the presence or absence of zabha or zacas (or zalrsc).

[romancardenas]

Now I'm trying to compile a project for a Zaamo target with portable-atomic and using -C target-feature=+zaamo. However, I am missing something, as portable-atomic complains about not having default support.

In Cargo.toml I have:

[dependencies]
# portable-atomic = { version = "1.8", default-features = false, features = ["unsafe-assume-single-core", "force-amo"] }
portable-atomic = { version = "1.8", default-features = false }

I compile my toy project with:

RUSTFLAGS="-C target-feature=+zaamo" cargo build --target riscv32imc-unknown-none-elf

But compilation fails:

error[E0277]: `fetch_xor` requires atomic CAS but not available on this target by default
    --> e310x-hal/src/gpio.rs:103:11
     |
103  |         r.fetch_xor(mask, Ordering::SeqCst);
     |           ^^^^^^^^^ this associated function is not available on this target by default
     |
     = help: the trait `portable_atomic::diagnostic_helper::HasFetchXor` is not implemented for `&portable_atomic::AtomicU32`
     = note: consider enabling one of the `unsafe-assume-single-core` or `critical-section` Cargo features
     = note: see <https://docs.rs/portable-atomic/latest/portable_atomic/#optional-features> for more.

How can I use the new Zaamo support properly?

PS:

the compilation also raises a warning about the Zaamo feature:

warning: unknown and unstable feature specified for `-Ctarget-feature`: `zaamo`
  |
  = note: it is still passed through to the codegen backend, but use of this feature might be unsound and the behavior of this feature can change in the future
  = help: consider filing a feature request

Sounds like Rust is still not 100% compatible with this target feature, even in Nightly.

[taiki-e]

unsafe-assume-single-core + zaamo is supported, but no unsafe-assume-single-core + zaamo is not yet supported (internally supported but not exposed). That said, it should be relatively easy to support as mentioned above by adjusting the cfg.

UPDATE: will be supported by #185