Introduce p99

[carlescufi]

Proposal to introduce p99, a library for preprocessor functions, as a separate module in Zephyr or, alternatively, in the main tree given the criticality of the code it contains (i.e. it would be used by nearly every single subsystem in Zephyr).

Background:
#26299 tried to introduce a new implementation of the IS_EMPTY macro, but the one that was found was not suitable licensed, since it was part of p99 which, until very recently, was being released under the Zephyr-incompatible QPL license.
The main author of p99, @gustedt, offered to move the p99 codebase to a new location and relicense it under the Apache 2.0 license, and proceeded to do so.

Options available:

1. Replace the code in sys/util.h with p99 and include p99 as a Zephyr module, removing (via deprecation) any functionality in sys/util.h that is already part of p99. This will have implications since p99 will then become critical to build any Zephyr application
2. Replace the code in sys/util.h with p99 and include p99 in-tree, removing (via deprecation) any functionality in sys/util.h that is already part of p99. Since p99 is now Apache-2.0 licensed, this is an option
3. Take portions of p99 into the tree, either by placing them in sys/util.h or in separate, original files

I'd like to discuss the approach to be taken here in order to make sure that we both ensure that Zephyr is robust even without modules, and we are able to contribute back changes to p99 upstream.

[pabigot]

To clarify: the difference between 1 and 2 is whether p99 is provided via a west module, or copied directly into Zephyr's master branch?

And if it is a module a question is whether it provides all of p99, or just the subdirectory with the headers we need?

Some thoughts:

• As I noted here I'm not in favor of copying code from other projects into Zephyr because it increases the maintenance burden.
• I'm also not a fan of Zephyr modules that are significantly different from their upstream (only a part, or "imported" from a release into a new tree that can't be directly compared with the corresponding upstream tree), as it complicates maintaining the module when upstream fixes bugs or adds functionality.
• The specific function in <sys/util.h> that triggered this investigation is not the only piece of code in that header that has dubious provenance and licensing. This is a prime motivation for replacing the entire thing with a solution that somebody else confirms is properly licensed.
• So providing this material in any location other than an external module that can be easily compared with its origin adds complexity in continuing to confirm that what Zephyr is providing truly complies with its licensing claims.

My original proposal suggested adding P99 (in its entirety) as an external module; converting the implementation of corresponding parts of <sys/util.h> to shallow wrappers of the equivalent P99 macros; marking those wrapper macros deprecated; and replacing their use in-tree with direct use of the P99 macros.

I would like that to be one of the options on the table.

[carlescufi]

To clarify: the difference between 1 and 2 is whether p99 is provided via a west module, or copied directly into Zephyr's master branch?

Correct

My original proposal suggested adding P99 (in its entirety) as an external module; converting the implementation of corresponding parts of <sys/util.h> to shallow wrappers of the equivalent P99 macros; marking those wrapper macros deprecated; and replacing their use in-tree with direct use of the P99 macros.

That is really option number 1, albeit a bit more detailed. I will try to make it clearer in the issue description. EDIT: I've now done so.

[mbolivar-nordic]

Making this a module would make p99 the first "mandatory" module in all Zephyr configurations, since util.h provides core macros like IS_ENABLED().

(I know things like vendor HALs are mandatory for driver shims around them, but that's not mandatory for all targets.)

Do we want to have such a thing? I obviously use west for everything, so I don't object from a personal perspective, but I would be curious if @aescolar or others who don't use west have any concerns.

I also confess I am a bit underwhelmed by this response:

#26299 (comment)

How can we be a bit more confident we are not replacing one piece of code with dubious provenance with another?

[carlescufi]

@galak @MaureenHelm @nashif would very much appreciate your input on this matter.

[galak]

What's the minimal set of files we need from p99?

[pabigot]

What's the minimal set of files we need from p99?

I don't think that can be answered without going through <sys/util.h> and identifying what parts are redundant with material from P99 and are things we might want to replace. An initial survey suggests these to start:

• p99_generated for p99_paste
• p99_paste for p99_args
• p99_args for P99_ISEMPTY()
• p99_for P99_FOR
• p99_if for conditionals
• p99_logical for logical operations
• p99_paste for token pasting

There are others that don't match existing capability but might be useful, e.g. p99_block, but we can ignore them for now.

[mbolivar-nordic]

Still relevant?

[pabigot]

Depends how much we care about problematic licensing of in-tree content taken from other sources.

But the solution path I prefer didn't get support, and I don't think anybody else will be doing anything about it. So we could just close this.

[carlescufi]

Let's close this then but then I suggest that @nordic-krch picks #26299 up again and we extract what we need from p99 into util.h instead.