Drop similarity-response messages with duplicate identifiers

[egbertbouman]

Fixes Tribler/tribler#1524.

[synctext]

class PingRequestCache(RandomNumberCache)
Polite RequestCache reminder, please do not forget about the None return value for the pop and/or get. Shocking state-of-the-code.

[egbertbouman]

@synctext That wasn't the problem. I believe the issue was due to two messages arriving with the same message id (which is possible if one of them was delayed for more then 10s). Then, when both messsages get processed in the same batch, the check function did not complain about an invalid message id because for a batch of messages we first do all the checks, and then all the callbacks.

[synctext]

which is possible if one of them was delayed for more then 10s

Sadly that is not very realistic for network propagation times. I've had this bug on our university Gbit/s network. Network latency there is sub-ms. Can Dispersy be that overloaded that the message queue blocks it for 10+ seconds? Or is this another 16-bit identifier clash, only 65536 unique IDs possible, so birthday paradox kicks in?

[egbertbouman]

@synctext Such a delay would most likely be due to Dispersy, yes, and this would only happen very rarely. The issue occurs if the message response takes more the 10s, so that's 2 messages (perhaps I wasn't clear on that).

[egbertbouman]

Example of delays while waiting for similarity responses during ~4 hours. Measured from request_cache.add (in create_similarity_request) until request_cache.pop (in on_similarity_response). There are some gaps at the end of the graph due to internet connection troubles, but you get the idea.

[synctext]

wow, point proven with real data. End-to-end latency is epic slow.