Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve "perimeter" classification #94

Merged

Conversation

michaelkirk
Copy link
Contributor

@michaelkirk michaelkirk commented Jan 22, 2025

IMPROVES #67 (There are still some known issues, but I think we should start with this and see how much of a priority the remaining issues are / gather more test cases).

In LTNv1, neighborhoods were assembled by tracing an outline of roads. Apparently there were some problems with this - in particular around dealing with bridges/tunnels into a neighborhood.

So in LTNv2, we now have the user draw a polygon, and use topology checks against that polygon to determine if a road is interior/exterior/crossing.

Firstly, checking if a road is topologically inside, outside, or crossing a boundary isn't sufficient for determining when it's a "perimeter" road. A better metric discussed in #67 for defining perimeter was "if a road is parallel and near the boundary" — I've attempted to implement that metric in this PR.

Another big wrinkle: We were using Relate::is_within/is_intersects topology to check if a road is Interior/Exterior/Perimeter. However, due to floating point rounding, there are some perimeter roads which are partially/mostly/entirely outside of the neighborhood (albeit by only a very small distance). To address this I'm checking the perimeter relationship first. And only if that fails, checking on topology. So now, so long as a segment is near and parallel to the boundary, it will be considered a Perimeter - regardless of whether it is inside or outside the boundary.

Overall, it seems to give better results, but it is (mostly) a slower implementation (though not drastically so):

build neighbourhood: bristol_east
                        time:   [8.5696 ms 8.5968 ms 8.6279 ms]
                        change: [+20.926% +21.448% +21.956%] (p = 0.00 < 0.05)
build neighbourhood: bristol_west
                        time:   [12.717 ms 12.775 ms 12.844 ms]
                        change: [+6.8103% +7.9397% +9.0261%] (p = 0.00 < 0.05)
build neighbourhood: strasbourg
                        time:   [11.598 ms 11.925 ms 12.276 ms]
                        change: [-12.421% -8.9352% -5.1781%] (p = 0.00 < 0.05)

Note: I've removed the Crosses { ratio } enum variant and added a Perimeter. If there's a reason to keep Crosses { ratio } around at run time, I can restore it and we can have { Interior, Exterior, Perimeter, Crosses }

Some other avenues I explored but backed out of:

  1. Do Relate topology checks to quickly filter out "external" roads, but use a buffered boundary polygon (via the geo_buffer crate) to be sure any nearby perimeter roads are within the boundary. This actually ended up being a bit slower, primarily because buffering the neighborhood polygon is expensive.

  2. Use i_overlay for linestring trimming in line_in_polygon: This ended up being slower than your existing impl in clip_linestring_to_polygon. I suspect it might be more robust to use i_overlay, but without seeing an issue I'm hesitant to slow this down further.

The remaining case I know of that we don't handle well is when a substantial portion of the road crosses out of the boundary. The portion inside the boundary should be classified as Interior or Perimeter, but we don't have a good metric for it. I naively tried to split these Roads, but some of them exit and re-enter the polygon, which I believe should be thought of as two separate roads, but at this point all the RoadIds have already been assigned. Addressing that seemed like a bigger change, so I thought I'd start with a simpler metric and see how much of a problem it is in practice.

One way of addressing this would look like splitting all the roads by the neighborhood boundaries up front, before assigning RoadIDs, but I suspect that might be complicated because currently RoadId's are assigned before any neighborhoods are created (kind of guessing, but I think it's true?), so I'm going to leave it here for now. Let me know what you think?

Copy link
Collaborator

@dabreegster dabreegster left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome! I'll run locally and look at some example cases, but I maybe do need your local geo to be pushed to a GH branch somewhere first.

One way of addressing this would look like splitting all the roads by the neighborhood boundaries up front, before assigning RoadIDs, but I suspect that might be complicated because currently RoadId's are assigned before any neighborhoods are created (kind of guessing, but I think it's true?), so I'm going to leave it here for now. Let me know what you think?

The user can draw neighbourhood boundaries at any time in the workflow right now, and even change existing ones later. Also, two boundaries are allowed to overlap right now. (Need to discuss any requirements upstream for preventing this.) We could maybe just cache per session the resulting Neighbourhood after a user opens it, but it might not be worth the effort, depending how many people tend to use in one session.

backend/Cargo.toml Outdated Show resolved Hide resolved
backend/benches/build_neighbourhood.rs Outdated Show resolved Hide resolved
backend/src/geo_helpers/slice_nearest_boundary.rs Outdated Show resolved Hide resolved
&self,
closest_to: &LineString,
) -> (LineString, LineString) {
use geo::{coord, HasDimensions};
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style question: Any preference not to lift this to the top of this module?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

backend/src/geo_helpers/slice_nearest_boundary.rs Outdated Show resolved Hide resolved
backend/src/lib.rs Outdated Show resolved Hide resolved
backend/src/neighbourhood.rs Show resolved Hide resolved
backend/src/neighbourhood.rs Outdated Show resolved Hide resolved
backend/src/neighbourhood.rs Show resolved Hide resolved
backend/src/neighbourhood.rs Show resolved Hide resolved
Also: extract logic for setting up neighborhood to re-use between unit
tests and benches.
I'm not sure about this fix - it basically just reverts 8bc4f2f which
was added for a reason, but I don't know what exactly 8bc4f2f was
intended to fix.

With this change however, I notice a lot of new (spurious?) entrances in
the portage_bay map.
It's a lot (~50%) slower:

    build neighbourhood: bristol_east
                            time:   [11.548 ms 11.558 ms 11.572 ms]
                            change: [+62.834% +63.286% +63.646%] (p = 0.00 < 0.05)
                            Performance has regressed.
    Found 10 outliers among 100 measurements (10.00%)
      1 (1.00%) low severe
      2 (2.00%) low mild
      3 (3.00%) high mild
      4 (4.00%) high severe

    build neighbourhood: bristol_west
                            time:   [17.219 ms 17.258 ms 17.299 ms]
                            change: [+44.468% +45.814% +47.127%] (p = 0.00 < 0.05)
                            Performance has regressed.
    Found 2 outliers among 100 measurements (2.00%)
      2 (2.00%) high mild

    build neighbourhood: strasbourg
                            time:   [14.421 ms 14.811 ms 15.230 ms]
                            change: [+8.7470% +13.108% +17.707%] (p = 0.00 < 0.05)
                            Performance has regressed.
    Found 5 outliers among 100 measurements (5.00%)
      5 (5.00%) high mild
Speeds things up a bit - still slower than `main`.

```
$ cargo bench --baseline=main

build neighbourhood: bristol_east
                        time:   [8.6624 ms 8.7156 ms 8.7718 ms]
                        change: [+22.329% +23.125% +23.936%] (p = 0.00 < 0.05)
build neighbourhood: bristol_west
                        time:   [12.923 ms 13.002 ms 13.086 ms]
                        change: [+8.6468% +9.8544% +11.051%] (p = 0.00 < 0.05)
build neighbourhood: strasbourg
                        time:   [12.255 ms 12.559 ms 12.877 ms]
                        change: [-7.7712% -4.0888% -0.3678%] (p = 0.03 < 0.05)
```
This didn't work out - it costs more time to buffer the boundary polygon than we gain by skipping perimeter checks.

bench --baseline=main

build neighbourhood: bristol_east
                        time:   [12.801 ms 12.886 ms 12.974 ms]
                        change: [+80.836% +82.047% +83.353%] (p = 0.00 < 0.05)
                        Performance has regressed.

build neighbourhood: bristol_west
                        time:   [18.777 ms 18.910 ms 19.045 ms]
                        change: [+57.915% +59.776% +61.624%] (p = 0.00 < 0.05)
                        Performance has regressed.

build neighbourhood: strasbourg
                        time:   [15.052 ms 15.450 ms 15.867 ms]
                        change: [+13.615% +17.984% +22.772%] (p = 0.00 < 0.05)
                        Performance has regressed.
This reverts commit e21aca4e5022b729b92fe29be1047bf084936006.
Not much effect on performance.

build neighbourhood: bristol_east
                        time:   [8.5745 ms 8.5931 ms 8.6163 ms]
                        change: [-3.0349% -2.2135% -1.4374%] (p = 0.00 < 0.05)
build neighbourhood: bristol_west
                        time:   [12.724 ms 12.747 ms 12.777 ms]
                        change: [-1.6700% -1.1600% -0.6716%] (p = 0.00 < 0.05)
build neighbourhood: strasbourg
                        time:   [11.806 ms 12.173 ms 12.569 ms]
                        change: [+1.8521% +6.1709% +10.481%] (p = 0.00 < 0.05)
Elements that cross the border get there own designation, and are
currently ignored. In theory we could trim the "crossing" pieces into
Internals/External/Perimeter pieces, but there are some issues:

TODO:

- CORRECTNESS: Lines that exit and re-enter the boundary need to have
  separate road ids if we are going to treat them as separate roads.
- PERF: more efficiently trim *everything* up front.

$ cargo bench --baseline=main

build neighbourhood: bristol_east
                        time:   [16.266 ms 16.322 ms 16.386 ms]
                        change: [+129.58% +130.58% +131.71%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 17 outliers among 100 measurements (17.00%)
  4 (4.00%) high mild
  13 (13.00%) high severe

build neighbourhood: bristol_west
                        time:   [25.452 ms 25.528 ms 25.612 ms]
                        change: [+113.63% +115.69% +117.66%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 10 outliers among 100 measurements (10.00%)
  8 (8.00%) high mild
  2 (2.00%) high severe

build neighbourhood: strasbourg
                        time:   [17.427 ms 17.700 ms 18.011 ms]
                        change: [+30.810% +35.168% +39.741%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 20 outliers among 100 measurements (20.00%)
…mount.

Mixed perf reviews - I wouldn't expect it to affect much.

$ cargo bench

build neighbourhood: bristol_east
                        time:   [8.6568 ms 8.6976 ms 8.7427 ms]
                        change: [-0.8167% -0.1032% +0.5630%] (p = 0.77 > 0.05)
build neighbourhood: bristol_west
                        time:   [13.485 ms 13.600 ms 13.714 ms]
                        change: [+4.8756% +5.7908% +6.8186%] (p = 0.00 < 0.05)
build neighbourhood: strasbourg
                        time:   [12.509 ms 12.846 ms 13.196 ms]
                        change: [-3.7309% +0.5815% +4.9240%] (p = 0.79 > 0.05)
@michaelkirk michaelkirk force-pushed the mkirk/improve-roads-in-boundary branch from 863f3a7 to 6df939f Compare January 22, 2025 21:26
In theory this should be a little faster than inserting at 0, but
benches were mixed:

build neighbourhood: bristol_east
                        time:   [8.6401 ms 8.6833 ms 8.7309 ms]
                        change: [-3.2046% -2.0667% -1.0198%] (p = 0.00 < 0.05)
build neighbourhood: bristol_west
                        time:   [12.744 ms 12.774 ms 12.813 ms]
                        change: [-1.4889% -0.8343% -0.2018%] (p = 0.01 < 0.05)
build neighbourhood: strasbourg
                        time:   [11.933 ms 12.211 ms 12.503 ms]
                        change: [-0.2716% +2.9797% +6.3002%] (p = 0.08 > 0.05)
Copy link
Contributor Author

@michaelkirk michaelkirk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I think I've addressed all your concerns (and rebased). Please take another lookg @dabreegster

https://github.com/a-b-street/ltn/pull/94/files/98bdaa92aa562d8eb18da420fd260736a67b12d5..334f94cd6cd336fc8c753670c313d23b0b6152f9 should be a diff since your last review.

Copy link
Collaborator

@dabreegster dabreegster left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fantastic, thanks so much for the PR! I just spot-checked a few random places, and the perimeter road detection seems great everywhere, and the slight perf degradation isn't noticeable. Really really awesome work

@dabreegster dabreegster merged commit 0923a69 into a-b-street:main Jan 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants