Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor ST Mbed targets to be CMake buildsystem targets #14199

Merged

Conversation

rwalton-arm
Copy link
Contributor

@rwalton-arm rwalton-arm commented Jan 27, 2021

Summary of changes

Refactor all ST targets to be CMake buildsystem targets. This removes
the need for checking MBED_TARGET_LABELS repeatedly and allows us to be
more flexible in the way we include MBED_TARGET source in the build.

A side effect of this is it will allow us to support custom targets
without breaking the build for 'standard' targets, as we use CMake's
standard mechanism for adding build rules to the build system, rather
than implementing our own layer of logic to exclude files not needed for
the target being built. Using this approach, if an MBED_TARGET is not
linked to using target_link_libraries its source files will not be
added to the build. This means custom target source can be added to the
user's application CMakeLists.txt without polluting the build system
when trying to compile for a standard MBED_TARGET.

Impact of changes

You will need to remove any calls to mbed_set_mbed_target_linker_script
in your application level CMakeLists.txt as this function has been removed.

Migration actions required

This PR depends on PRs to fix the examples' CMakeList files. I'll link the
PRs here after they've been raised.

Documentation

None


Pull request type

[] Patch update (Bug fix / Target update / Docs update / Test update / Refactor)
[x] Feature update (New feature / Functionality change / New API)
[] Major update (Breaking change E.g. Return code change / API behaviour change)

Test results

[] No Tests required for this change (E.g docs only update)
[x] Covered by existing mbed-os tests (Greentea or Unittest)
[] Tests / results supplied as part of this PR

Reviewers

@Patater @hugueskamba @0xc0170 @urutva


@ciarmcom ciarmcom added the release-type: patch Indentifies a PR as containing just a patch label Jan 27, 2021
@ciarmcom ciarmcom requested review from 0xc0170, hugueskamba, Patater, urutva and a team January 27, 2021 15:00
@ciarmcom
Copy link
Member

@rwalton-arm, thank you for your changes.
@urutva @hugueskamba @Patater @0xc0170 @ARMmbed/team-st-mcd @ARMmbed/mbed-os-tools @ARMmbed/mbed-os-maintainers please review.

@jeromecoutant
Copy link
Collaborator

Got an error :
CMake Error at CMakeLists.txt:18 (mbed_set_mbed_target_linker_script):
Unknown CMake command "mbed_set_mbed_target_linker_script".

@rwalton-arm
Copy link
Contributor Author

rwalton-arm commented Jan 27, 2021

Got an error :
CMake Error at CMakeLists.txt:18 (mbed_set_mbed_target_linker_script):
Unknown CMake command "mbed_set_mbed_target_linker_script".

Thanks for taking a look. I need to raise PRs removing the function call from the examples. If you delete mbed_set_mbed_target_linker_script for now from your top level CMakeLists.txt it should work. This PR should also be a "major" change as users will need to edit their app level CMakeLists.txt and I've just noticed I set it to "patch" and didn't mention it in the commit message. I'll fix that now.

@jeromecoutant
Copy link
Collaborator

mbedtools compile -m NUCLEO_H743ZI2 -t GCC_ARM -f

This works now!

Copy link
Collaborator

@jeromecoutant jeromecoutant left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Compilation is OK with 3 random boards.

targets/CMakeLists.txt is not updated as well ?

Copy link
Contributor

@0xc0170 0xc0170 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome! We are getting slowly there, CMake targets in targets directory ! More to go!

[x] Major update (Breaking change E.g. Return code change / API behaviour change)

Can we make this nonbreaking ? It would simplify things, I don't think we are ready to release a new major version the next month.

what if we keep the old function there, and print a warning that the function disappears in the next major? We will accumulate more things like this. Or do we need to have the linker in this PR ?

another one: this is porting the targets keeping the name. If we are creating something new, can we fix the naming to follow the components names?
to illustrate: NUCLEO_F072RB would be nucleo-f072rb ? Either this or we rename all the components however the application ones use mbed-os/mbed-os-baremetal so I guess rather to follow them than otherway around. To translate this from old names to new should be fairly straightforward (find and replace function in CMake?).

Shall we also create mcu targets ? Previous tools had the limitation there to use TARGET_ as no MCU_ prefix was implemented. Thus target MCU_STM32L4R9xI was using actually TARGET_STM32L4R9xI. This is a base target, shouldn't it be named as such? The custom target would inherit from mcutargets. The tree would look likenucleo-f32l4inherits frommcu-stm32l4`. I had so many discussions about this problem in the tree, we started adding mcu too late and never had an opportunity to fix it. We might have now.

What about naming scheme for targets, would it make sense to have mbed-target prefix? these are specific to mbed, they are targets. What naming should custom targets use? For instance, we used to have target-frmd-k64f-gcc in Mbed 3.

INTERFACE
system_clock.c
${STARTUP_FILE}
)

target_include_directories(mbed-core
target_include_directories(STM32F091xC
Copy link
Contributor

@0xc0170 0xc0170 Jan 27, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume this follows the naming convection we have in targets.json? do we need to keep this as mixed uppercase with lower case?
I would use lower case in all targets if possible (the translation should be straightforward from old to new). Or not?

Copy link
Contributor Author

@rwalton-arm rwalton-arm Jan 28, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the convention from the folder structure and targets.json. Generally CMake style is to use camel case for targets, which I see we don't follow in Mbed, so I just went with the folder naming convention. Is the Mbed style to use all lowercase?

tools/cmake/SetLinkerScript.cmake Outdated Show resolved Hide resolved
@rwalton-arm
Copy link
Contributor Author

rwalton-arm commented Jan 28, 2021

Awesome! We are getting slowly there, CMake targets in targets directory ! More to go!

[x] Major update (Breaking change E.g. Return code change / API behaviour change)

Can we make this nonbreaking ? It would simplify things, I don't think we are ready to release a new major version the next month.
what if we keep the old function there, and print a warning that the function disappears in the next major? We will accumulate more things like this. Or do we need to have the linker in this PR ?

Yeah, I'll look into making this non-breaking. I can keep the old function around and add a warning. We do need the linker script changes in this PR for the new stuff to work correctly (it's also a more flexible architecture going forward if the target provides the linker script rather than setting a global property).

another one: this is porting the targets keeping the name. If we are creating something new, can we fix the naming to follow the components names?
to illustrate: NUCLEO_F072RB would be nucleo-f072rb ? Either this or we rename all the components however the application ones use mbed-os/mbed-os-baremetal so I guess rather to follow them than otherway around. To translate this from old names to new should be fairly straightforward (find and replace function in CMake?).

Shall we also create mcu targets ? Previous tools had the limitation there to use TARGET_ as no MCU_ prefix was implemented. Thus target MCU_STM32L4R9xI was using actually TARGET_STM32L4R9xI. This is a base target, shouldn't it be named as such? The custom target would inherit from mcutargets. The tree would look likenucleo-f32l4inherits frommcu-stm32l4`. I had so many discussions about this problem in the tree, we started adding mcu too late and never had an opportunity to fix it. We might have now.

What about naming scheme for targets, would it make sense to have mbed-target prefix? these are specific to mbed, they are targets. What naming should custom targets use? For instance, we used to have target-frmd-k64f-gcc in Mbed 3.

We can't change the target names, they need to match what we have in targets.json for now as we pass the target name from the tools straight to target_link_libraries so the names have to be identical for now. However, for the MCU targets I think we can change them as they aren't "leaf targets" we can build for (they aren't public in targets.json), so we can name them whatever we want. I'll take a look at adding an MCU prefix.

@0xc0170
Copy link
Contributor

0xc0170 commented Jan 28, 2021

We can't change the target names, they need to match what we have in targets.json for now as we pass the target name from the tools straight to target_link_libraries so the names have to be identical for now.

target.json stays as it is, we just do postprocessing in CMake to translate old naming to the new one. I'll create a draft to illustrate (based on this PR, to do the minimal effort to see if its worth doing).

@ladislas
Copy link
Contributor

Regarding the breaking change, do you know how many users are using it? Can you get the pip install stats?

As the new tools are still in beta, and based on the issues/feedback on Github and the forum, it doesn't seem like too many people are using it in their daily work.

I would not worry about a breaking change on something that's not widely used and that will only make the life of the users better in the short term.

IMHO, more generally, all things related to CMake should be treated as non-breaking as the user base actively using them is quite small (because of the current limitatations) and people can always pin the version they need. Plus, I'd say that if you want to cmake, updating your main CMakeLists.txt is really not an issue. :)

@rwalton-arm rwalton-arm force-pushed the dev/rwalton-arm/refactor-st-mbed-targets branch from 99614a0 to 7e50d55 Compare January 28, 2021 12:27
@ladislas
Copy link
Contributor

a quick search for mbed_set_mbed_target_linker_script only returns 44 mbed repositories and 11 user repositories:

https://github.com/search?p=6&q=mbed_set_mbed_target_linker_script&type=Code

@rwalton-arm
Copy link
Contributor Author

a quick search for mbed_set_mbed_target_linker_script only returns 44 mbed repositories and 11 user repositories:

https://github.com/search?p=6&q=mbed_set_mbed_target_linker_script&type=Code

I agree that we shouldn't make this a "breaking" change. I've changed the PR type to "feature". The confusion here was my fault for marking the PR as a "major" incorrectly.

tools/cmake/SetLinkerScript.cmake Outdated Show resolved Hide resolved
tools/cmake/SetLinkerScript.cmake Outdated Show resolved Hide resolved
targets/TARGET_STM/CMakeLists.txt Show resolved Hide resolved
@rwalton-arm rwalton-arm force-pushed the dev/rwalton-arm/refactor-st-mbed-targets branch from 7e50d55 to 222a556 Compare February 1, 2021 12:24
@mergify mergify bot dismissed Patater’s stale review February 1, 2021 12:24

Pull request has been modified.

0xc0170
0xc0170 previously requested changes Feb 1, 2021
Copy link
Contributor

@0xc0170 0xc0170 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The outstanding questions are:

  • can we be consistent in CMake to use lower cases with - delimeter (translation in CMake via a simple string and replace), no change in tagets.json or tools, just we keep CMake target naming same

Future wish: I wish to have mcu, vendor or similar in the target names to describe better what is it - and also better hierarchy of targets. This might wait as it needs to refactor targets.json and much more thinking how to structure them (it's not trivial task). we live what we have for now, targets.json

CMakeLists.txt Outdated
@@ -6,6 +6,7 @@
cmake_minimum_required(VERSION 3.19.0 FATAL_ERROR)

include(${MBED_CONFIG_PATH}/mbed_config.cmake)
include(tools/cmake/SetLinkerScript.cmake)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we rename to set_linker_script ? we do not use camel case naming just CMakeLists does.

@@ -67,7 +67,7 @@ target_compile_definitions(mbed-core

# Add compile definitions for backward compatibility with the toolchain
# supported. New source files should instead check for __GNUC__ and __clang__
# for the GCC_ARM and ARM toolchains respectively.
# for the GCC_ARM and ARM toolchains respectively.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how this spaces got in... Jamie proposed to have formatter for CMake, we should review that one to avoid these in future.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've used https://github.com/cheshirekow/cmake_format before and it worked well. There are pre-commit hooks available, so we could add it to CI quite easily. It's very easy to miss trailing whitespace if your editor isn't set up correctly to flag it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found the PR adding format: #14011 ,if you have a moment, please review

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

besides that, I can make a PR to add .editorconfig -- it's almost a standard and it's super useful for this kind of case.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

besides that, I can make a PR to add .editorconfig -- it's almost a standard and it's super useful for this kind of case.

That might be useful.

Refactor all ST targets to be CMake buildsystem targets. This removes
the need for checking MBED_TARGET_LABELS repeatedly and allows us to be
more flexible in the way we include MBED_TARGET source in the build.

A side effect of this is it will allow us to support custom targets
without breaking the build for 'standard' targets, as we use CMake's
standard mechanism for adding build rules to the build system, rather
than implementing our own layer of logic to exclude files not needed for
the target being built. Using this approach, if an MBED_TARGET is not
linked to using `target_link_libraries` its source files will not be
added to the build. This means custom target source can be added to the
user's application CMakeLists.txt without polluting the build system
when trying to compile for a standard MBED_TARGET.
@Patater Patater force-pushed the dev/rwalton-arm/refactor-st-mbed-targets branch from a156131 to d9e184b Compare February 4, 2021 15:28
@mergify mergify bot dismissed stale reviews from Patater and 0xc0170 February 4, 2021 15:29

Pull request has been modified.

@Patater
Copy link
Contributor

Patater commented Feb 4, 2021

Rebased on latest master branch. Made sure to update mbed_set_linker_script() to include the ${CMAKE_EXECUTABLE_SUFFIX} in the name of the map file output by the linker, as that change was made upstream and conflicted with moving the linkerscript associating function to set_linker_script.cmake.

@0xc0170
Copy link
Contributor

0xc0170 commented Feb 4, 2021

I merged all examples, started CI.

@mergify mergify bot added needs: work and removed needs: CI labels Feb 4, 2021
@mbed-ci
Copy link

mbed-ci commented Feb 4, 2021

Jenkins CI Test : ❌ FAILED

Build Number: 1 | 🔒 Jenkins CI Job | 🌐 Logs & Artifacts

CLICK for Detailed Summary

jobs Status
jenkins-ci/mbed-os-ci_unittests ✔️
jenkins-ci/mbed-os-ci_cmake-example-GCC_ARM
jenkins-ci/mbed-os-ci_cmake-example-ARM
jenkins-ci/mbed-os-ci_build-greentea-GCC_ARM ✔️
jenkins-ci/mbed-os-ci_build-greentea-ARM ✔️
jenkins-ci/mbed-os-ci_build-cloud-example-GCC_ARM ✔️
jenkins-ci/mbed-os-ci_build-example-GCC_ARM ✔️
jenkins-ci/mbed-os-ci_build-cloud-example-ARM ✔️
jenkins-ci/mbed-os-ci_build-example-ARM ✔️

@0xc0170
Copy link
Contributor

0xc0170 commented Feb 4, 2021

CI restarted

@adbridge
Copy link
Contributor

adbridge commented Feb 4, 2021

Ci restarted

@mbed-ci
Copy link

mbed-ci commented Feb 4, 2021

Jenkins CI Test : ❌ FAILED

Build Number: 2 | 🔒 Jenkins CI Job | 🌐 Logs & Artifacts

CLICK for Detailed Summary

jobs Status
jenkins-ci/mbed-os-ci_cmake-example-GCC_ARM
jenkins-ci/mbed-os-ci_cmake-example-ARM
jenkins-ci/mbed-os-ci_unittests ✔️
jenkins-ci/mbed-os-ci_build-greentea-GCC_ARM ✔️
jenkins-ci/mbed-os-ci_build-greentea-ARM ✔️
jenkins-ci/mbed-os-ci_build-cloud-example-ARM ✔️
jenkins-ci/mbed-os-ci_build-cloud-example-GCC_ARM
jenkins-ci/mbed-os-ci_build-example-ARM
jenkins-ci/mbed-os-ci_build-example-GCC_ARM

@mbed-ci
Copy link

mbed-ci commented Feb 4, 2021

Jenkins CI Test : ✔️ SUCCESS

Build Number: 3 | 🔒 Jenkins CI Job | 🌐 Logs & Artifacts

CLICK for Detailed Summary

jobs Status
jenkins-ci/mbed-os-ci_cmake-example-GCC_ARM ✔️
jenkins-ci/mbed-os-ci_unittests ✔️
jenkins-ci/mbed-os-ci_cmake-example-ARM ✔️
jenkins-ci/mbed-os-ci_build-cloud-example-GCC_ARM ✔️
jenkins-ci/mbed-os-ci_build-greentea-ARM ✔️
jenkins-ci/mbed-os-ci_build-greentea-GCC_ARM ✔️
jenkins-ci/mbed-os-ci_build-cloud-example-ARM ✔️
jenkins-ci/mbed-os-ci_build-example-GCC_ARM ✔️
jenkins-ci/mbed-os-ci_build-example-ARM ✔️
jenkins-ci/mbed-os-ci_cmake-example-test ✔️
jenkins-ci/mbed-os-ci_greentea-test ✔️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.