Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gz_TEST: improve initial sim time test reliability #1916

Merged
merged 2 commits into from
Mar 31, 2023

Conversation

scpeters
Copy link
Member

@scpeters scpeters commented Mar 5, 2023

🦟 Bug fix

Improves reliability of a test added in #1801.

Summary

The test added in #1801 is a little flaky:

I've added delays, which wasn't enough on its own. For some reason the test subscriber on macOS doesn't get any messages for the first 100 iterations or so. Increasing to 200 iterations was enough in most cases on my machine, so I increased to 500 just to be safe. To reproduce the issue, reduce the iterations to 100 and view the console output.

Checklist

  • Signed all commits for DCO
  • Added tests
  • Updated documentation (as needed)
  • Updated migration guide (as needed)
  • Consider updating Python bindings (if the library has them)
  • codecheck passed (See contributing)
  • All tests passed (See test coverage)
  • While waiting for a review on your PR, please help review another open pull request to support the maintainers

Note to maintainers: Remember to use Squash-Merge and edit the commit message to match the pull request summary while retaining Signed-off-by messages.

Add some sleep statements and increase test iterations.

Signed-off-by: Steve Peters <[email protected]>
@scpeters scpeters requested a review from mjcarroll as a code owner March 5, 2023 03:17
@github-actions github-actions bot added the 🌱 garden Ignition Garden label Mar 5, 2023
@codecov
Copy link

codecov bot commented Mar 5, 2023

Codecov Report

Merging #1916 (e569d3f) into gz-sim7 (753ec8d) will increase coverage by 0.06%.
The diff coverage is 85.29%.

❗ Current head e569d3f differs from pull request most recent head d238274. Consider uploading reports for the commit d238274 to get more accurate results

@@             Coverage Diff             @@
##           gz-sim7    #1916      +/-   ##
===========================================
+ Coverage    64.54%   64.61%   +0.06%     
===========================================
  Files          347      348       +1     
  Lines        27777    27853      +76     
===========================================
+ Hits         17930    17997      +67     
- Misses        9847     9856       +9     
Impacted Files Coverage Δ
include/gz/sim/components/Component.hh 100.00% <ø> (ø)
src/gui/plugins/spawn/Spawn.cc 9.88% <0.00%> (-0.40%) ⬇️
include/gz/sim/components/Factory.hh 97.16% <95.74%> (-0.37%) ⬇️
src/ComponentFactory.cc 100.00% <100.00%> (ø)
src/systems/physics/Physics.cc 67.22% <100.00%> (+0.87%) ⬆️

... and 2 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@scpeters
Copy link
Member Author

scpeters commented Mar 6, 2023

still flaky; it failed in one of the jammy actions:

@@ -135,7 +135,7 @@ TEST(CmdLine, GazeboServer)
TEST(CmdLine, SimtimeArgument)
{
std::string cmd =
kGzCommand + " -r -v 4 --iterations 100 --initial-sim-time 1000.5 " +
kGzCommand + " -r -v 4 --iterations 500 --initial-sim-time 1000.5 " +
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the loaded sdf file has some plugins, none of the more time consuming plugins such as Physics are loaded. Also, the plugins.sdf world is set to run as fast as possible, so this command may finish running before transport is done initializing. My suggestion is to slow it down using -z 1000 or even -z 100.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Signed-off-by: Addisu Z. Taddese <[email protected]>
@azeey
Copy link
Contributor

azeey commented Mar 31, 2023

The ABI failure is because there are new changes in gz-sim7 since this was last updated. Since it's pretty clear this doesn't actually break ABI, I'll go ahead and merge.

@azeey azeey merged commit 3d8d41b into gz-sim7 Mar 31, 2023
@azeey azeey deleted the scpeters/initial_sim_time_test_reliability branch March 31, 2023 16:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🌱 garden Ignition Garden
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

3 participants