some failing tests #99

michaelkirk · 2022-02-01T00:28:35Z

We're seeing some failures in CI for #98 e.g.

https://github.com/georust/proj/runs/5014199807?check_suite_focus=true

Test failure look unrelated?

Indeed! I'm looking into that.

The failures seem eerily similar to the ones I found in: #97 (comment)

I opened this PR just as a way to gather more data, to see if I could find a pattern.

CI Run from this branch: https://github.com/georust/proj/runs/5013682778?check_suite_focus=true

✅ rust 1.52
⭕️ rust 1.53

❌ default features

failures:

---- proj::test::test_inverse_projection stdout ----
thread 'proj::test::test_inverse_projection' panicked at 'assert_relative_eq!(t.x(), 0.43633200013698786)

left  = 0.43636512759931073
right = 0.43633200013698786

', src/proj.rs:1069:9
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace

---- proj::test::test_london_inverse stdout ----
thread 'proj::test::test_london_inverse' panicked at 'assert_relative_eq!(t.x(), 0.0023755864830313977)

left  = 0.002407322568220306
right = 0.0023755864830313977

', src/proj.rs:1086:9

---- proj::test::test_projection stdout ----
thread 'proj::test::test_projection' panicked at 'assert_relative_eq!(t.x(), 500119.7035366755, epsilon = 1e-5)
left = 499972.70746607287
right = 500119.7035366755

', src/proj.rs:1054:9

failures:
proj::test::test_inverse_projection
proj::test::test_london_inverse
proj::test::test_projection

❌ --features=network

failures:

---- proj::test::test_inverse_projection stdout ----
thread 'proj::test::test_inverse_projection' panicked at 'assert_relative_eq!(t.x(), 0.43633200013698786)

left  = 0.43636512759931073
right = 0.43633200013698786

', src/proj.rs:1069:9
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace

---- proj::test::test_london_inverse stdout ----
thread 'proj::test::test_london_inverse' panicked at 'assert_relative_eq!(t.x(), 0.0023755864830313977)

left  = 0.002407322568220306
right = 0.0023755864830313977

', src/proj.rs:1086:9

---- proj::test::test_projection stdout ----
thread 'proj::test::test_projection' panicked at 'assert_relative_eq!(t.x(), 500119.7035366755, epsilon = 1e-5)

left  = 499972.70746607287
right = 500119.7035366755

', src/proj.rs:1054:9

failures:
proj::test::test_inverse_projection
proj::test::test_london_inverse
proj::test::test_projection

✅ --features=bundled_proj

❌ --no-default-features

failures:

---- proj::test::test_inverse_projection stdout ----
thread 'proj::test::test_inverse_projection' panicked at 'assert_relative_eq!(t.x(), 0.43633200013698786)

left  = 0.43636512759931073
right = 0.43633200013698786

', src/proj.rs:1069:9
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace

---- proj::test::test_london_inverse stdout ----
thread 'proj::test::test_london_inverse' panicked at 'assert_relative_eq!(t.x(), 0.0023755864830313977)

left  = 0.002407322568220306
right = 0.0023755864830313977

', src/proj.rs:1086:9

---- proj::test::test_projection stdout ----
thread 'proj::test::test_projection' panicked at 'assert_relative_eq!(t.x(), 500119.7035366755, epsilon = 1e-5)

left  = 499972.70746607287
right = 500119.7035366755

', src/proj.rs:1054:9

failures:
proj::test::test_inverse_projection
error: test failed, to rerun pass '--lib'
proj::test::test_london_inverse
proj::test::test_projection

test result: FAILED. 12 passed; 3 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.33s

✅ --features=network bundled_proj

❌ --features=network geo-types

failures:

---- proj::test::test_inverse_projection stdout ----
thread 'proj::test::test_inverse_projection' panicked at 'assert_relative_eq!(t.x(), 0.43633200013698786)

left  = 0.43636512759931073
right = 0.43633200013698786

', src/proj.rs:1069:9
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace

---- proj::test::test_london_inverse stdout ----
thread 'proj::test::test_london_inverse' panicked at 'assert_relative_eq!(t.x(), 0.0023755864830313977)

left  = 0.002407322568220306
right = 0.0023755864830313977

', src/proj.rs:1086:9

---- proj::test::test_projection stdout ----
thread 'proj::test::test_projection' panicked at 'assert_relative_eq!(t.x(), 500119.7035366755, epsilon = 1e-5)

left  = 499972.70746607287
right = 500119.7035366755

', src/proj.rs:1054:9

failures:
proj::test::test_inverse_projection
proj::test::test_london_inverse
proj::test::test_projection

test result: FAILED. 13 passed; 3 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.34s

✅ --features=bundled_proj geo-types
✅ --features=network bundled_proj geo-types

✅ rust 1.54
✅ rust 1.55
✅ rust 1.56
✅ rust 1.57
✅ rust 1.58

Curiously in #97, all the rust-1.53 tests were passing and it was the rust-1.52 tests that were failing. I'm not opposed to raising the MSRV, but it doesn't seem like the failures are as simple as "broken on old rust".

… are

michaelkirk · 2022-02-01T16:50:03Z

I reran with no changes and now see all the rust-1.53 tests as passing, but two from rust-1.54 are now failing... there doesn't seem to be any pattern to the failures that I can see.

CI output

tldr;

✅ rust 1.52
✅ rust 1.53

✅ rust 1.54

✅default features
✅ --features=network

❌ --features=bundled_proj

failures:
---- src/lib.rs - (line 56) stdout ----

Test executable failed (exit code 101).
stderr:

thread 'main' panicked at 'assert_relative_eq!(result.0, 1450880.29, epsilon = 1e-2)
left  = NaN
right = 1450880.29

', src/lib.rs:20:1

note: run with RUST_BACKTRACE=1 environment variable to display a backtrace
failures:

src/lib.rs - (line 56)

✅ --no-default-features

❌ --features=network bundled_proj

failures:
---- src/lib.rs - (line 56) stdout ----

Test executable failed (exit code 101).
stderr:

thread 'main' panicked at 'assert_relative_eq!(result.0, 1450880.29, epsilon = 1e-2)
left  = 21993543.96732857
right = 1450880.29

', src/lib.rs:20:1

✅ --features=network geo-types
✅ --features=bundled_proj geo-types
✅ --features=network bundled_proj geo-types

✅ rust 1.55
✅ rust 1.56
✅ rust 1.57
✅ rust 1.58

lnicola · 2022-02-04T11:38:01Z

I get this one in a doc test on master, but only in release builds:

running 11 tests
test src/lib.rs - (line 56) ... ok
test src/proj.rs - proj::Proj::convert (line 714) ... ok
test src/geo_types.rs - geo_types::geo_types::Coordinate<T> (line 1) ... ok
test src/proj.rs - proj::ProjBuilder::proj_known_crs (line 419) ... ok
test src/lib.rs - (line 31) ... ok
test src/geo_types.rs - geo_types::geo_types::Point<T> (line 28) ... ok
test src/proj.rs - proj::Proj::new_known_crs (line 502) ... ok
test src/lib.rs - (line 179) ... ok
test src/lib.rs - (line 142) ... FAILED
test src/proj.rs - proj::Proj::project_array (line 804) ... ok
test src/proj.rs - proj::Proj::convert_array (line 770) ... ok

failures:

---- src/lib.rs - (line 142) stdout ----
Test executable failed (exit code 101).

stderr:
thread 'main' panicked at 'assert_relative_eq!(result.x(), 158458.67, epsilon = 1e-2)

    left  = -14339492.078978319
    right = 158458.67

lnicola · 2022-02-04T11:47:27Z

It's strange. The test still fails (with a NaN) as a non-doc test. If I remove the proj_errno_reset, it starts to pass, but test_error_recovery now fails, which I guess invalidates my hopes about functions resetting errno on success.

…k/fix-tests

lnicola · 2022-02-10T18:51:58Z

Looks like CI passed now 🎉.

104: Initialize `zt` before calling `proj_trans` r=michaelkirk a=lnicola CC #99 r? `@michaelkirk` Co-authored-by: Laurențiu Nicola <[email protected]>

michaelkirk · 2022-02-11T23:16:41Z

Fixed in #104

michaelkirk added 2 commits January 31, 2022 10:36

test CI

6e0bce2

DO NOT MERGE: added a lot of different versions to see where failures…

3d4640b

… are

michaelkirk mentioned this pull request Feb 1, 2022

Update proj::Proj constructors to return Result instead of Option. #98

Merged

Initialize ZT before calling proj_trans

beed043

lnicola mentioned this pull request Feb 4, 2022

Initialize zt before calling proj_trans #104

Merged

lnicola and others added 2 commits February 4, 2022 18:51

Initialize more union members

ad404a0

Merge remote-tracking branch 'origin/proj_trans-init-input' into mkir…

d1eaaee

…k/fix-tests

bors bot added a commit that referenced this pull request Feb 10, 2022

Merge #104

dea916e

104: Initialize `zt` before calling `proj_trans` r=michaelkirk a=lnicola CC #99 r? `@michaelkirk` Co-authored-by: Laurențiu Nicola <[email protected]>

michaelkirk closed this Feb 11, 2022

michaelkirk deleted the mkirk/fix-tests branch February 23, 2022 19:41

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

some failing tests #99

some failing tests #99

michaelkirk commented Feb 1, 2022 •

edited

Loading

michaelkirk commented Feb 1, 2022

lnicola commented Feb 4, 2022 •

edited

Loading

lnicola commented Feb 4, 2022 •

edited

Loading

lnicola commented Feb 10, 2022

michaelkirk commented Feb 11, 2022

some failing tests #99

some failing tests #99

Conversation

michaelkirk commented Feb 1, 2022 • edited Loading

michaelkirk commented Feb 1, 2022

lnicola commented Feb 4, 2022 • edited Loading

lnicola commented Feb 4, 2022 • edited Loading

lnicola commented Feb 10, 2022

michaelkirk commented Feb 11, 2022

michaelkirk commented Feb 1, 2022 •

edited

Loading

lnicola commented Feb 4, 2022 •

edited

Loading

lnicola commented Feb 4, 2022 •

edited

Loading