Update sweep logic to re-schedule unchanged jobs when SWEEPER_ENABLED is toggled

[qreshi]

Signed-off-by: Mohammad Qureshi [email protected]

Issue #, if available: #89

Description of changes:

• Fixed the case where if an enabled job is de-scheduled when SWEEPER_ENABLED is set to false, jobs that were unchanged were not re-scheduled when SWEEPER_ENABLED was set to back to true
• Added a manual sweep of all shards when initBackgroundSweep() is called since the runnable being scheduled in the threadpool is only executed for the first time after the SWEEPER_PERIOD (which defaults to 5 minutes) has passed and this could lead to delays in jobs that should run more frequently than that
• Added a test to capture the changes
• Confirmed with an anti-test that the written test failed as expected without the fix

CheckList:
[x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov-commenter]

Codecov Report

Merging #243 (4106377) into main (15bb70b) will decrease coverage by 0.08%.
The diff coverage is 75.00%.

@@             Coverage Diff              @@
##               main     #243      +/-   ##
============================================
- Coverage     78.66%   78.57%   -0.09%     
- Complexity      215      218       +3     
============================================
  Files           173      173              
  Lines          6959     6968       +9     
  Branches        912      915       +3     
============================================
+ Hits           5474     5475       +1     
- Misses         1002     1003       +1     
- Partials        483      490       +7     

Impacted Files	Coverage Δ
.../kotlin/org/opensearch/alerting/core/JobSweeper.kt	71.50% <75.00%> (-0.76%)	⬇️
...ain/kotlin/org/opensearch/alerting/AlertService.kt	78.04% <0.00%> (-0.98%)	⬇️
...in/kotlin/org/opensearch/alerting/MonitorRunner.kt	71.38% <0.00%> (-0.68%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 15bb70b...4106377. Read the comment docs.