-
Notifications
You must be signed in to change notification settings - Fork 10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Start of Kobj support: Semaphores and Threads #12
Conversation
Move the module wrapper into `lib.rs` so that we can add documentation to the module itself. Signed-off-by: David Brown <[email protected]>
Document a handful of functions so that we can enforce documentation. Signed-off-by: David Brown <[email protected]>
Add a directive to consider missing documentation on public symbols to be a compliation error. Signed-off-by: David Brown <[email protected]>
Create a simple error type to handle Zephyr errors. This can be expanded in the future to better be able to distinguish the errors, but this allows us to properly return values from wrapped functions. Signed-off-by: David Brown <[email protected]>
Provide the atomic types from portable-atomic in `zephyr::sync::atomic`, and, if allocation is enabled, provide `zephyr::sync::Arc`. The `alloc::arc` will only be available on Zephyr targets that have atomic intrinsics, and the portable one can be supported on any Zephyr target. There are some limitations described in the documentation. Signed-off-by: David Brown <[email protected]>
Create wrappers for static kernel objects. This is done through a few wrapper types. Generally, a StaticKernelObject which contains a 'value' which is an unsafe cell containing the actual zephyr-typed kernel object, and then some traits to use these, specifically `KobjGet` to fetch the underlying pointer, and `KobjInit` which supports initialization and getting a higher-level wrapper around the underlying object pointer. Signed-off-by: David Brown <[email protected]>
The atomic operations in Rust's core crate only work on platforms that have atomic intrinsics. Zephyr supports atomics on other platforms as well, generally implementing them with spinlocks. Use the portable-atomics library, and configure it so use the spinlock provided by the critical section crate.
The `sys::Semaphore` is a wrapper for Zephyr's `k_sem` type. This adds rules to `kobj_define!` to statically declare these, which will have a `.take()` method, giving an instance that can be used to coordinate between threads. Signed-off-by: David Brown <[email protected]>
Add a dependent type to the Wrapped trait to indicate the arguments that 'take' and hence init needs. These will typically be either `()`, or a tuple with the Zephyr object's parameters. Signed-off-by: David Brown <[email protected]>
Implement a general critical section handler, using spinlocks, for Rust users to be able to use the `critical-section` crate. This crate is used by other crates as well. Signed-off-by: David Brown <[email protected]>
The Rust attribute that indicates structure alignment only supports alignment values that are numeric constants. This makes it difficult to set alignment based on a value coming from the build environment, which is commonly done. Implement a bit of a workaround with an `AlignAs` type that is parameterized with the alignment. Parameters can be compile-time constants not just numeric constants. This works by having instances for each of the desired alignments that each have the numeric constant as the alignment. The instances can be expanded if necessary. Signed-off-by: David Brown <[email protected]>
Create a config `CONFIG_RUST_ALLOC` that will hook Rust's allocation system into the `malloc`/`free` allocator provided on Zephyr. This will allow the `alloc` crate in rust to be used. Signed-off-by: David Brown <[email protected]>
Bindgen only exports defined values that are clearly numeric constants, which is thrown off when the defined value has a cast. Work around this by defining wrapper constants that bindgen is able to expose. We use the `ZR_` prefix (Zephyr Rust) to avoid conflicts with any other symbols. Signed-off-by: David Brown <[email protected]>
These are wrappers that alloc Zephyr threads to be statically declared as well as started, from safe code. Signed-off-by: David Brown <[email protected]>
The dining philosophers example, in Rust. This is generalized, and provides multiple implementations, exercising various different synchronization primivites in Zephyr Signed-off-by: David Brown <[email protected]>
Upgrade the versions of all crates in this module to match the current Zephyr version of 3.7.0. It isn't completely clear if this is the right thing to do, as Zephyr doesn't maintain semantic versioning. But having the numbers match will help make it clear which version of Zephyr goes with which version of this crate. Signed-off-by: David Brown <[email protected]>
This function is safe, so add a simple wrapper around the unsafe version. Signed-off-by: David Brown <[email protected]>
It just prints the type, but allows other structs containing Semaphores to be printed. Signed-off-by: David Brown <[email protected]>
This is required to allow it to be a static. Signed-off-by: David Brown <[email protected]>
The semaphore implementation is safe to use in multiple places, so mut is not necessary and prevents their use in most cases. Signed-off-by: David Brown <[email protected]>
This is not intended to be visible, but needed for macros. Signed-off-by: David Brown <[email protected]>
These need to be initialized by macros, so expose them publicly. It would probably be better to export a constructor. Signed-off-by: David Brown <[email protected]>
This is needed to initialize this value into a static. Signed-off-by: David Brown <[email protected]>
Depend upon the 3.7.0 version of Zephyr. Signed-off-by: David Brown <[email protected]>
Use of Box depends on Alloc being enabled, so also conditionalize the extern and use declarations. Signed-off-by: David Brown <[email protected]>
Update the yaml to remove not-yet-implemented variants, and make sure the regexp matches things the test actually prints. Signed-off-by: David Brown <[email protected]>
Fix the version dependency to match the Zephyr release version. Signed-off-by: David Brown <[email protected]>
zephyr/src/object.rs
Outdated
/// Construct an empty of these objects, with the zephyr data zero-filled. This is safe in the | ||
/// sense that Zephyr we track the initialization, and they start in the uninitialized state. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I appreciate the attention to good docs.
Not sure if these suggestions are the same as your original intent.
/// Construct an empty of these objects, with the zephyr data zero-filled. This is safe in the | |
/// sense that Zephyr we track the initialization, and they start in the uninitialized state. | |
/// Construct an empty instance of these objects, with the zephyr data zero-filled. This is safe in the | |
/// sense that in Zephyr we track the initialization, and they start in the uninitialized state. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe this should be addressed, namely by rewriting the whole doc section.
/// permitted count. | ||
pub fn give(&self) { | ||
unsafe { | ||
k_sem_give(self.item) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I needed to remind myself that omitting the semicolon with the last expression of a function returns that value 🦀
Does the documentation engine publish return type information for these functions?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would have to be declared in the code, it is never inferred. In this case, give returns nothing, and the zephyr function is void.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks reasonable to me - just a few comments.
A couple of things I’d like to do are to add some more comments justifying the safety of code that uses unsafe, see if I can make a few pub fields not pub with constructors, and try to figure out how to allow declaring an array of stacks. |
The bindgen code copies the doc strings directly from the C headers. These have a different syntax than the Rust doc headers. Until we can find or create a tool to convert these, disable these warnings. Signed-off-by: David Brown <[email protected]>
Add comments to the object module describing the safety goals, and how we achieve them. Signed-off-by: David Brown <[email protected]>
The docs reference `alloc::format`. Include the extern for this crate so that this will work. Signed-off-by: David Brown <[email protected]>
Fix a few broken doc links, by filling in full references. Signed-off-by: David Brown <[email protected]>
This had an `extern crate alloc` for a link in the docs, but there is no actual dependency on this. Replace this with a resolved link to the standard documentation for formatting. Signed-off-by: David Brown <[email protected]>
Although the user is not intended to actually create these static thread stack objects, make an alias for the type in `_export` so that the macro is a little easier to read. This will become more important as this type picks up some constructors. Signed-off-by: David Brown <[email protected]>
Instead of fully expanding the thread stack initializer in the kobj_define macro, create a hidden constructor as a const fn. Signed-off-by: David Brown <[email protected]>
Create a const constructor function for building the stack arrays. This requires unsafe, and can't use MaybeUninit, because the array version of this is still unsable. Just use zero initialization, and make sure the loop initializes everything. Use this function in the kobj_define macro in order to allow apps to define arrays of thread stacks, in addition to arrays of threads. Signed-off-by: David Brown <[email protected]>
Now that kboj_define supports arrays of thread stacks, change the expanded definitions into array definitions. The demo now properly supports changing `NUM_PHIL` to use a different number of philosophers. Signed-off-by: David Brown <[email protected]>
Add a small docgen crate. To help facilitate checking the docs, this project can be built, and then `cargo doc` run there. This includes the symlink for the config file, because that is needed in order for this to be able to build. Signed-off-by: David Brown <[email protected]>
Ensure that the links actually resolve correctly, eliminating all of the warnings from a doc build. Signed-off-by: David Brown <[email protected]>
I have implemented I've also added a docgen, which can be built, and then allows |
Don't just run hello world, but all samples that are under samples. Signed-off-by: David Brown <[email protected]>
Duration::millis_at_least(((delay + 1) * period) as Tick) | ||
} | ||
|
||
kobj_define! { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks good - certainly an improvement
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Cool stuff! 🎉
I did not manage to get through all commits today. But here are some comments already :).
There is a decent chance that some of my confusion is due to my limited knowledge about Zephyr internals!
It is mostly the different types for "StaticFoo" and "Foo" that confuse me a bit... My naive feeling would be that there should be ways to do a 1:1 mapping of C type to Rust type without needing the two variants? But again, chances are I miss things! 😇
writeln!(&mut f, "pub mod kconfig {{").unwrap(); | ||
|
||
let file = File::open(&dotconfig).expect("Unable to open dotconfig"); | ||
for line in BufReader::new(file).lines() { | ||
let line = line.expect("reading line from dotconfig"); | ||
if let Some(caps) = config_hex.captures(&line) { | ||
writeln!(&mut f, " #[allow(dead_code)]").unwrap(); | ||
writeln!(&mut f, " pub const {}: usize = {};", | ||
writeln!(&mut f, "#[allow(dead_code)]").unwrap(); | ||
writeln!(&mut f, "pub const {}: usize = {};", | ||
&caps[1], &caps[2]).unwrap(); | ||
} else if let Some(caps) = config_int.captures(&line) { | ||
writeln!(&mut f, " #[allow(dead_code)]").unwrap(); | ||
writeln!(&mut f, " pub const {}: isize = {};", | ||
writeln!(&mut f, "#[allow(dead_code)]").unwrap(); | ||
writeln!(&mut f, "pub const {}: isize = {};", | ||
&caps[1], &caps[2]).unwrap(); | ||
} else if let Some(caps) = config_str.captures(&line) { | ||
writeln!(&mut f, " #[allow(dead_code)]").unwrap(); | ||
writeln!(&mut f, " pub const {}: &'static str = {};", | ||
writeln!(&mut f, "#[allow(dead_code)]").unwrap(); | ||
writeln!(&mut f, "pub const {}: &'static str = {};", | ||
&caps[1], &caps[2]).unwrap(); | ||
} | ||
} | ||
writeln!(&mut f, "}}").unwrap(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This reads like it snuck into an unrelated commit? It does not seem to have anything to do with documentation?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It didn't sneak, it is very intentional. I needed to move the pub mod kconfig {
up into lib.rs
so that I could add both doc comments and directives to not fail on missing docs on the entries. To do that, I had to remove that from the generated code, and I unindented it as well.
/// Map a return result from Zephyr into an Result. | ||
/// | ||
/// Negative return results being considered errors. | ||
pub fn to_result(code: c_int) -> Result<c_int> { | ||
if code < 0 { | ||
Err(Error(-code as u32)) | ||
} else { | ||
Ok(code) | ||
} | ||
} | ||
|
||
/// Map a return result, with a void result. | ||
pub fn to_result_void(code: c_int) -> Result<()> { | ||
to_result(code).map(|_| ()) | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wonder: Do these have to be pub? They seem to be mostly internal helpers? I find it to be slightly awkward API to be exposed to the outside world. So should this maybe be pub(crate)
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Definitely a consideration. I still think they are useful for an app user that wants to use a binding that we haven't wrapped yet.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reducing the API may make it easier for people to use. This extra API seems to be a convenience API, but, at least for me, it just adds extra API for people who will use the crate to learn.
One possibility here would be to add a ZephyrCode
type that could implement those methods. I think using the type Error
is misleading because we are using it for error and also for normal returning code.
/// Map a return result from Zephyr into an Result. | ||
/// | ||
/// Negative return results being considered errors. | ||
pub fn to_result(code: c_int) -> Result<c_int> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is the c_int in the return needed for compatibility here? As this function already checks for negative values, I would find a uint
more intuitive from an API usability perspective.
I would have also expected a convert to non FFI types, given that we are using the Rust Result, too?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The idea is to just keep the type as much like the underlying type, and any conversions can happen by the wrapper that is using this.
zephyr/src/object.rs
Outdated
/// Each can be wrapped appropriately. The wrapped type is the instance that holds the raw pointer. | ||
pub trait Wrapped { | ||
/// The wrapped type. This is what `take()` on the StaticKernelObject will return after | ||
/// initialization. | ||
type T; | ||
|
||
/// Initialize this kernel object, and return the wrapped pointer. | ||
fn get_wrapped(&self) -> Self::T; | ||
} | ||
|
||
/// A state indicating an uninitialized kernel object. | ||
/// | ||
/// This must be zero, as kernel objects will | ||
/// be represetned as zero-initialized memory. | ||
pub const KOBJ_UNINITIALIZED: usize = 0; | ||
|
||
/// A state indicating a kernel object that is being initialized. | ||
pub const KOBJ_INITING: usize = 1; | ||
|
||
/// A state indicating a kernel object that has completed initialization. This also means that the | ||
/// take has been called. And shouldn't be allowed additional times. | ||
pub const KOBJ_INITIALIZED: usize = 2; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I typically see people keeping the struct definition + impl's together, so I would consider moving this block before the struct definition?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll look through it, it gets messy when there are multiple structs and traits that interact with each other.
/// sense that Zephyr we track the initialization, and they start in the uninitialized state. | ||
pub const fn new() -> StaticKernelObject<T> { | ||
StaticKernelObject { | ||
value: unsafe { mem::zeroed() }, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I may lack context here, but I like how most projects put // SAFETY comments before unsafe blocks. That makes reviewing a lot easier IMHO. Here one could probably add a comment about how later code ensures that this is fully initialized at some point.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Honestly, I think this whole method should probably just be unsafe.
Ordering::Acquire) | ||
{ | ||
return None; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you already have a Result<> type, you can also do: .ok()?
on it [1]
[1] https://doc.rust-lang.org/std/result/enum.Result.html#method.ok
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll look at it.
zephyr/src/object.rs
Outdated
type T; | ||
|
||
/// Initialize this kernel object, and return the wrapped pointer. | ||
fn get_wrapped(&self) -> Self::T; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am a bit confused about the "pointer" documentation here while the trait really takes any type.
Overall, I am a bit confused about this being abstracted with pointers. Couldn't this also be abstracted with references on the Rust side?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
They don't follow the semantics of Rust references, they are never dereferenced from Rust, and are really just part of the C api. Also, this has been remove anyway (but it is still a wrapped pointer).
zephyr/src/object.rs
Outdated
pub struct StaticKernelObject<T> { | ||
#[allow(dead_code)] | ||
/// The underlying zephyr kernel object. | ||
value: UnsafeCell<T>, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I may not have grasped the full context yet, but to me this mostly seems to be used as a MaybeUninit
? 🤔 Could that be used instead here? It reads like it will do the same job, but provide some more explicit meaning.
Also: Nothing prevents somebody from moving such an object, right? Are those C counterparts fine with being moved? I would assume synchronization primitives keep all kind of self referential pointers internally. So I am a bit surprised that this can safely be wrapped without some sort of Pinning / modeling things are references to ensure things cannot be moved around.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The UnsafeCell
is absolutely critical to this working. It is treated as magical by the Rust compiler and basically tells it that even though the container isn't mut
, this T
inside of it might change. Since the C code is likely to change these, we need to declare this like this.
} | ||
let result = self.get_wrapped(); | ||
self.init.store(KOBJ_INITIALIZED, Ordering::Release); | ||
Some(result) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there an example that uses this (maybe my grep skills fail me :P)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Look in samples. It is used by any user code using statically declared kernel objects.
The reason to have these two types is that most kernel objects in Zephyr need to be declared statically. But static globals aren't very useful in Rust (without unsafe), so we have this init_once
method that is called on the global static to return a single instance that wraps the pointer to the static, and can be used more freely in Rust.
/// A zephyr `k_sem` usable from safe Rust code. | ||
#[derive(Clone)] | ||
pub struct Semaphore { | ||
/// The raw Zephyr `k_sem`. | ||
item: *mut k_sem, | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This seems to work fine as other code carefully avoids constructing this for non-static semahpores. Does zephyr also have dynamic ones? I found these internal pointers made it harder to reason about safety eventually.
I wonder: Have you considered something like:
[repr(transparent)]
pub struct Semaphore {
value: UnsafeCell<MaybeUninit<k_sem>>,
_pin: PhantomPinned,
}
?
(see https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/rust/kernel/types.rs#n261 / rust-lang/rust#43467 for context)
Then of course most code would work with a &Semaphore
, but that seems like a more direct mapping of C -> Rust to me?
If all you ever deal with are static objects, then I guess this is fine too. But I think the direct wrapping is going to be a pattern elsewhere eventually anyway? It seems to me like that may also allow to unify this together with the "static object" that currently needs to be defined separately?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So Semaphore
is not a static semaphore, it is a singleton reference to some kind of semaphore. The StaticSemaphore is one way of declaring the semaphores (and currently the only way).
I intend to add dynamic variants of many of these, but as the Zephyr dynamic object support is rather poor at the moment, these will likely just have a pool of the objects that they take from. The Semaphore
type will be the same, but there will lookly be a Semaphore::new
that returns one from the pool. When it is dropped, it will be returned to the pool.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems ok to me, gotta start somewhere with this stuff and I think my major concerns are mostly covered. Time will tell what ends up working/not working more definitively.
@Ablu I think I'm going to move forward with merging this, and address some of your concerns in the next PR. The ones about code organization and safety comments don't really affect the code itself. Hopefully, the concerns about the static stuff will make more sense in the next PR as I add |
This PR pulls out some of what is implemented in #5, to try to reduce how much code needs to be reviewed.
In addition, this also incorporates @teburd suggestions around the safety of
Thread
and related.The thread initialization is now safe in the sense that a given thread can only be initialized once, and a given stack can also only be used once.
The
sys::Semaphore
is safe to get and use, based on the documentation of the zephyr API.A disadvantage here is that the initialization is more complicated,
and currently stacks cannot be declared in(fixed).kobj_define!
as an array. A possible solution to this is with either a proc-macro, or to find a supporting proc macro package to help with this