Improve thread contention around diagnostics

[pepeiborra]

These shared locks introduce contention and resulting in decreasing performance as the core count increases.

1. Debouncer

The async debouncer that we use in ghcide to deduplicate diagnostics is backed by a single lock containing a Hashmap indexed by rule key used to store all the pending actions. We then grab this lock for every rule execution, for rules with diagnostics. In total, a rule execution grabs the lock twice:

1. To enqueue the diagnostics update in the debouncer
2. When the update fires, to clean up

We then hold the lock for as long as it takes to update the map and cancel the previous async, if any. There is a bit of room for improvement here, by combining HashMap accesses as well as executing the cancel outside the lock.

Moreover, I have added an option to install a different Debouncer, e.g. the noopDebouncer, if desired.

2. Diagnostics lock

We store all the published diagnostics in a HashMap indexed by NFP behind a single lock. We then grab this lock once for every rule execution, for rules with diagnostics, and hold it for the whole publish loop. We can do better by executing the publish loop out of the lock

Fixes #1545

[wz1000]

Could we use lock-free concurrent maps? For example I found https://hackage.haskell.org/package/ttrie and https://hackage.haskell.org/package/ctrie-0.2

[wz1000]

There is also stm-containers: https://hackage.haskell.org/package/stm-containers

[pepeiborra]

I have never used those packages, so I have no idea how performant, reliable, etc they are

[wz1000]

stm-containers seems to have a decent number of users: https://packdeps.haskellers.com/reverse/stm-containers

Apart from that, have you considered something like TVar (Map k (TVar v))? That way there is no contention if multiple threads are acting on different keys.

[wz1000]

Some recent benchmarks I found comparing these approaches: https://lowerbound.io/blog/2019-10-24_concurrent_hash_table_performance.html

[pepeiborra]

@wz1000 I think these lock-free / finer-grain approaches are very worth discussing, perhaps in a new issue rather than in this PR, since there are other aspects to consider e.g. stability and code complexity.

In the meantime, I have generalised the approach used in this PR and raised a follow-up: #1553

[pepeiborra]

Some recent benchmarks I found comparing these approaches: https://lowerbound.io/blog/2019-10-24_concurrent_hash_table_performance.html

The bad performance of MVar is very worrying. I'm going to make a quick experiment using atomicModifyCAS just for kicks