-
Notifications
You must be signed in to change notification settings - Fork 13.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Enable convenient unit testing using googletest on POSIX #11573
Conversation
############################################################################ | ||
|
||
# Download and unpack googletest at configure time | ||
configure_file(${CMAKE_CURRENT_LIST_DIR}/CMakeLists.txt.in googletest-download/CMakeLists.txt) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think I'm missing why this has to be done at configure time like this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Because otherwise you don't have the CMakeLists-tree of gtest at configure time and can't configure the dependencies correctly.
I see two options around it:
- You have the source anyways because you add it as a submodule but since all submodules have to be present at anytime I prefered the download only if it's actually needed.
- You hack around and achieve that cmake only needs to configure the dependencies to gtest on build time but I didn't achieve that because you need to wrap it and make assumptions about the structure of gtest. It's easy to screw up the include directories and other things this way.
We can still revisit, this is the first state I find acceptable.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
btw the method for the download at configure time is the one from the docs, see last point and following description here https://github.com/google/googletest/blob/master/googletest/README.md#incorporating-into-an-existing-cmake-project
CMakeLists.txt
Outdated
# Testing - Automatic unit and integration testing with CTest | ||
# | ||
|
||
option(unit_testing "Configure unit test targets" OFF) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can the existing cmake testing facilities be used directly?
enable_testing()
- https://cmake.org/cmake/help/v3.8/command/enable_testing.html?highlight=i
Then you can check BUILD_TESTING
.https://github.com/PX4/Firmware/blob/d42bb01e0ca83813f1d8a96576c6fb9244bb7a19/platforms/posix/CMakeLists.txt#L117
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I found out about the concept and wanted to do that but they are used already for the SITL integration tests. I'd like to be able to run the tests separately since the SITL tests take so much longer. I planned to have a structured test name and use ctest -R
to filter the correct ones. So you think distinguishing build time doesn't make sense? I could agree with that. Then gtest is always needed for tests in general and it all depends on BUILD_TESTING
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah and testing is always enabled for sitl... https://github.com/PX4/Firmware/blob/fc7c7ac20630951b17e8644ccc175b9c4e743409/CMakeLists.txt#L303-L305
Can I make it optional?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
since the SITL tests take so much longer
Are you talking about the SITL unit tests? Each is roughly 1-3 seconds.
Example: http://ci.px4.io:8080/blue/organizations/jenkins/PX4%2FFirmware/detail/master/722/pipeline/21
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, I expected the basic tests to run two order of magnitudes faster. Is that impatient? 😇 I think it's just starting the simulator binary that takes the big part of that time, otherwise the smallest tests would not all have around the same duration.
Anyways I think we should allow to run the unit tests separately since I presume they'll finish a lot faster until we write a crazy amount of tests or huge loops.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll just add all tests and the gtest library fetching with enable_testing()
and hence BUILD_TESTING
but make enabling tests optional for posix. Then I can still filter the tests if I only want to run the quick unit tests e.g. with make tests unit
.
16cb44b
to
4556184
Compare
c287a96
to
a503ca1
Compare
a503ca1
to
8da3163
Compare
rebased after merging #11308 |
ee9e99e
to
701d7d5
Compare
Rebased and started to enable the unit test framework with camkes |
82e88cd
to
c1f685f
Compare
@mcsauder Sorry the latest commit at the time you wrote your comment was an unfinished temporary dump of ongoing tests that's why the commit message started with |
581d8f1
to
105909a
Compare
I rebased again. Since I would now start writing useful unit tests I would say this pr is ready for review. One thing I already noticed is the license text is not in every file, I'll fix that quickly. |
@dagar Any chance to get the test framework in? I'm willing to continue to work on how it's embedded in cmake but would appreciate if I was able to start to write unit tests without blowing up this PR. The tests are now included in |
This is quite useful! |
Nice work @MaEtUgR ! |
visibility.h is included globally in PX4 via cmake compile flags. It contains poisoning the exit() command which is used by gtest to close the test application. Removing the flag for gtest compilation fixes the compile error: gtest.cc:4757:7: error: attempt to use poisoned "exit"
Example: Before when you passed "make tests TESTFILTER=Attitude" and subsequently "make tests TESTFILTER=Atti" it found the string "TESTFILTER=Atti" in "TESTFILTER=Attitude" and hence the check if the configuration is already correct passed. The fix checks for the configuration parameter including the subsequent space separator and after that strips the space away again such that the list VERIFIED_CMAKE_OPTIONS doesn't contain trailing spaces.
378ec31
to
715a9ba
Compare
@dagar rebased on rc1 to run CI again |
Describe problem solved by the proposed pull request
Part of the description I wrote in #11308
The idea is to enable unit testing for any C++ class of the project with very low overhead. For example if you want to test the class
AttitudeControl
you:AttitudeControlTest.cpp
next to the classes source filespx4_add_gtest(SRC AttitudeControlTest.cpp LINKLIBS AttitudeControl)
to the folder'sCMakeLists.txt
When you run
make unit_test
you get the following output (runs all the unit tests):And it neither needs to rebuild all the libraries you want to test if you're using
px4_sitl_default
anyways nor does gtest need to be loaded when you're not unit testing with it thanks to #11533Describe your preferred solution
A clear and concise description of what you have implemented.
Additional context
Test data / coverage
Developed and tested on Cygwin Windows toolchain, I will also test on Ubuntu.