Significant improvements in noise modeling

[ChristopherRabotin]

Effects

Lots of breaking changes in this work.

• Add Gauss Markov noise models: first order GM (bias only) and coupled bias and drift model (where the bias is a second order GM and the drift is first order). This is a recommended best practice for modeling biases according to D'Souza in "NASA Best Practices for Navigation Filters"
• Change EstimateFrom trait to support many different sensitivity matrix sizes (this was not possible beforehand)
• Replace noise model of Ground Stations from Gaussian normal to Gauss Markov
• Allow initialization of ground stations and their GM noise characteristics from YAML
• Allow simulation of Gauss Markov models only and from Python (with Parquet export) to verify whether the model seems to match the characteristics of the ground station
• Allow for Gauss Markov noise on the ground station clock to model
• Disable noise in OD process when building the expected measurement
• Change all OD seeds to use the PCG 64 Fast algorithm. This ensures reproducibility of all the runs.
• Allow generating a dispersed state estimate from its covariance and a truth state
• Significantly expand the tests: this is greatly enhanced by the new ability to fully reproduce a run since all random number generators are seedable

If this is a new feature or a bug fix ...

• Yes, the branch I'm proposing to merge is called issue-xyz where xyz is the number of the issue.

If this change adds or modifies a validation case

• Yes and:
  • I have coded up, or updated, an existing test case; and
  • I have provided a GMAT script which confirms the result(s); and
  • I have updated the VALIDATION.md file with the new error data between nyx and GMAT.
  • I will specify the version of GMAT used for the validation.

[ChristopherRabotin]

Results of the "Robustness" test

Dispersions

• Inclination dispersed by 2.5e-3 degrees
• RAAN by 0.022 degrees
• AoP by 0.02 degrees

// Disperse the initial state based on some orbital elements errors given from ULA Atlas 5 user guide, table 2.3.3-1 <https://www.ulalaunch.com/docs/default-source/rockets/atlasvusersguide2010a.pdf>
    // This assumes that the errors are ONE TENTH of the values given in the table. It assumes that the launch provider has provided an initial state vector, whose error is lower than the injection errors.
    let generator = GaussianGenerator::from_std_devs(
        initial_state,
        &[
            (StateParameter::Inclination, 0.0025),
            (StateParameter::RAAN, 0.022),
            (StateParameter::AoP, 0.02),
        ],
    )
    .unwrap();

Truth initial state:
[Earth J2000] 2019-12-31T23:59:23 UTC   position = [-9042.862234, 18536.333069, 6999.957069] km velocity = [-3.288789, -2.226285, 1.646738] km/s
[Earth J2000] 2019-12-31T23:59:23 UTC   sma = 22000.000000 km   ecc = 0.010000  inc = 30.000000 deg     raan = 80.000000 deg    aop = 40.000000 deg     ta = 359.999999 deg
Filter initial state:
[Earth J2000] 2019-12-31T23:59:23 UTC   position = [-9050.309259, 18530.208096, 7006.546242] km velocity = [-3.287950, -2.228182, 1.645847] km/s
[Earth J2000] 2019-12-31T23:59:23 UTC   sma = 22000.000000 km   ecc = 0.010000  inc = 30.002369 deg     raan = 79.986313 deg    aop = 40.041822 deg     ta = 0.000001 deg
Initial state dev:      11.679 km       0.002 km/s
[Earth J2000] 2019-12-31T23:59:23 UTC   position = [7.447026, 6.124973, -6.589172] km   velocity = [-0.000838, 0.001897, 0.000891] km/s

Tracking arc set up

• Measurements every 60 seconds with Madrid and Canberra.
• Noise modeled as Gauss markov process, high precision
• Range and Doppler modeling in sync

Filter setup

• SNC set to 1e-4 km^2/s^2 in EME2000 frame (all axes), constant, disabled after 2 minutes of gap in measurements
• Iterate once on the first three hours of tracking data
• Start the CKF with the rest of the data, which transition into an EKF after 1000 measurements, and disabled if there aren't any measurements for three minutes

Results

RMAG error = 9.95e1 m   VMAG error = 2.180e1 mm/s

State deviation

(Ad hoc plot so the units are km and km/s)

Covariance plots

[ChristopherRabotin]

The previous EKF trigger was never triggered because there were only 996 measurements in the remaining set.

Filter setup

Trigger EKF after 300 measurements.

Results

RMAG error = 7.44e0 m   VMAG error = 4.731e0 mm/s

There are warnings during the run because the EKF is enabled even when the state deviation is not within the three sigmas. This can probably be fixed by toying with the SNC.

 INFO  nyx_space::od::process           > Navigation propagating for a total of 20 h 22 min with step size 1 min
 INFO  nyx_space::od::process           > Processing 994 measurements with covariance mapping
 INFO  nyx_space::od::process           >   0% done (0 measurements processed)
 INFO  nyx_space::od::process           >  10% done (100 measurements processed)
 INFO  nyx_space::od::process           >  20% done (199 measurements processed)
 WARN  nyx_space::od::process           > EKF enabled @ 2020-01-01T07:42:23 UTC but filter DIVERGING
 INFO  nyx_space::od::process           >  30% done (299 measurements processed)
 INFO  nyx_space::od::process           > EKF disabled @ 2020-01-01T10:06:23 UTC
 INFO  nyx_space::od::process           >  40% done (398 measurements processed)
 INFO  nyx_space::od::process           >  50% done (497 measurements processed)
 INFO  nyx_space::od::process           >  60% done (597 measurements processed)
 WARN  nyx_space::od::process           > EKF enabled @ 2020-01-01T16:46:23 UTC but filter DIVERGING
 INFO  nyx_space::od::process           >  70% done (696 measurements processed)
 INFO  nyx_space::od::process           >  80% done (796 measurements processed)
 INFO  nyx_space::od::process           >  90% done (895 measurements processed)
 INFO  nyx_space::od::process           > 100% done (994 measurements processed)

State deviation

Residuals

These look extremely good, the noise may be too low.

Covariance plots (EME2000 frame)

[ChristopherRabotin]

Results from 19ace2b

One way results for a MEO spacecraft

Initial state deviation

Initial estimate:
=== Prediction @ 2019-12-31T23:59:23 UTC -- within 3 sigma: true ===
state [Earth J2000] 2019-12-31T23:59:23 UTC     position = [-9045.344786, 18534.292006, 7002.153779] km velocity = [-3.288510, -2.226917, 1.646441] km/s
sigmas [5.5467591943091556e1,3.7493455598629886e1,4.3429805205314224e1,7.02724001508767e-7,3.5982340051832854e-6,7.935577188194398e-7]

Truth initial state:
[Earth J2000] 2019-12-31T23:59:23 UTC   position = [-9042.862234, 18536.333069, 6999.957069] km velocity = [-3.288789, -2.226285, 1.646738] km/s
[Earth J2000] 2019-12-31T23:59:23 UTC   sma = 22000.000000 km   ecc = 0.010000  inc = 30.000000 deg     raan = 80.000000 deg    aop = 40.000000 deg     ta = 359.999999 deg
Filter initial state:
[Earth J2000] 2019-12-31T23:59:23 UTC   position = [-9045.344786, 18534.292006, 7002.153779] km velocity = [-3.288510, -2.226917, 1.646441] km/s
[Earth J2000] 2019-12-31T23:59:23 UTC   sma = 22000.000000 km   ecc = 0.010000  inc = 30.000790 deg     raan = 79.995438 deg    aop = 40.013941 deg     ta = 360.000000 deg
Initial state dev:      3892.883 km     0.752 km/s
[Earth J2000] 2019-12-31T23:59:23 UTC   position = [2.482552, 2.041063, -2.196710] km   velocity = [-0.000279, 0.000632, 0.000297] km/s

Results

RMAG error = 6.924949 m VMAG error = 0.000999 m/s

Trajectory error and estimated covariance

Zoom on the first six hours

Prefit residuals

The residuals drop after the EKF turns on (300 measurements without a gap in time).

Postfit residuals

Two way results for a MEO spacecraft

Same initial state deviation and noise levels. SNC is slightly different, but tuning of that may lead to better results.

Results

RMAG error = 7.157629 m VMAG error = 0.001309 m/s

Trajectory error and estimated

Zoom on first six hours

covariance

Prefit residuals

Postfit residuals

[codecov-commenter]

Codecov Report

❗ No coverage uploaded for pull request base (master@bbd36a1). Click here to learn what that means.
Patch has no changes to coverable lines.

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

Additional details and impacted files

@@            Coverage Diff            @@
##             master     #129   +/-   ##
=========================================
  Coverage          ?   29.16%           
=========================================
  Files             ?      126           
  Lines             ?    33301           
  Branches          ?        0           
=========================================
  Hits              ?     9712           
  Misses            ?    23589           
  Partials          ?        0           

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.