Extraneous requests for already-known events during client pagination (SYN-560)

[matrixbot]

A sytest test observes that synapse makes extraneous /event/:event_id requests for events it is known at that time to already have, when the client makes a pagination request.

More specifically, when the client calls /messages at
https://github.com/matrix-org/sytest/blob/8511ea309840ee03403f3d73f5178d80362f6610/tests/50federation/34room-backfill.pl#L78

the Synapse-Under-Test has already been told the m.room.create and m.room.member event for the creator, as part of the /send_join response. Additionally, by inspecting its sqlite database just before it makes that call, those events do indeed appear in the file. However, by running this test, the SUT is observed to make two /event/ requests over federation with the event IDs of these very two events anyway.

(Imported from https://matrix.org/jira/browse/SYN-560)

(Reported by @leonerd)

[matrixbot]

Jira watchers: @leonerd

[matrixbot]

After some analysis, it seems to be getting slightly confused around synapse/handlers/federation.py lines 310 or so. It computes the list of "dangling edges" in the backfilled chunk response (which in our case is an empty set, because the backfill returns the entire history of the room, as it's so small). As this list of edges is empty, it doesn't manages to look up any existing auth events for the room. It then notices that it needs two, and makes outbound requests for them.

-- @leonerd

[matrixbot]

An initial fix for this bug is now pushed to the paul/SYN-560 branch. Still to be answered is what might happen if the backfill result doesn't contain the required event, but the local event database still did. I'm unclear currently whether that would therefore appear in the edges list and be picked up, or not.

-- @leonerd

[richvdh]

paul/SYN-560 was never merged but appears to have been superceded by #987, which I think fixed this bug.

This code has now been completely rewritten by #6526.