Send document-version-annotated diagnostics to avoid staleness

[radeksimko]

Context

As part of #1346 we implemented validation jobs which run asynchronously.

As mentioned in #1346 (comment) this means that - under pressure (with enough changes/jobs in queue) - users could potentially observe outdated diagnostics as the jobs are cleared from the queue in the order they were submitted. These stale diagnostics should always eventually be overwritten by the up-to-date ones though.

Implementation Notes

LSP offers a way of associating document version numbers with published diagnostics, as per https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#publishDiagnosticsParams This would at least (in theory) mean that users are never presented with irrelevant/stale diagnostics as the client should discard diagnostics for earlier document versions (this is to my best understanding of the protocol spec, it may not work that way in practice - needs to be verified!).

Proposal

• Look into whether we could avoid computing diagnostics entirely for outdated document versions (i.e. avoid wasting CPU/memory) - e.g. returning from jobs early if we know the job was submitted for version X and the latest known version Y is higher
• Publish diagnostics with document version attribute (as per above)