Rewrite hls-graph to not use the Shake code

[ndmitchell]

This PR reimplements the key algorithms from Shake directly. Most of the tests still work, but there are some failures relating to progress messages. Unfortunately, those failures also stop the benchmark from working, and thus I have no idea of the relative performance. In order to simplify relative to Shake, and make it more suitable for HLS:

• No serialisation logic at all.
• A lot less focus on error handling and fast exception propagation, since we never have exceptions.
• No continuation monad or thread pool, everything is separate threads (managed by async) and direct ReaderT IO.
• The Ids and Intern modules are lifted from Shake unchanged.

All the interesting code is in the Database module. Everything else is just wrapping and implementing required type classes on the underlying types.

Things that don't work:

• The benchmarks, which seem to have race conditions around progress messages.
• Most tests work, but the "fix syntax error" ones don't. Unfortunately, they also fail with race conditions around progress messages, but sometimes I won those races and they still failed.
• All profiling is stubbed out. If we consider that important, I think we can get it back. I think reusing the Shake progress reporting HTML is even feasible, with a bit of modularisation in Shake. It wouldn't be unreasonable for Shake to make that a separate package.

How we move forward is probably heavily influenced by what the benchmarks say. If this new Shake is either within the ballpark of same speed or faster, I suggest we move entirely to this new one, at which point things like @pepeiborra's reverse dependencies are way easier to integrate.

[jneira]

Sorry if it is not related at all but would be #796 relevant? this could enable the change suggested there?

[ndmitchell]

The work already merged enables #796. This would be going in a different direction with our own implementation

[pepeiborra]

Very impressive! I read through the code and was able to understand most of it. Planning to play with it this weekend and make it work with the benchmarks.

[pepeiborra]

Suggested change

	-- Hold the lock while mutating Ids/Values
	databaseLock :: !Lock,
	databaseIds :: !(IORef (Intern Key)),
	databaseValues :: !(Ids (Key, Status))
	-- TODO (someday) Replace the 3 fields below by a concurrent (mutable) Hashtable
	-- Hold the lock while mutating Ids/Values
	databaseLock :: !Lock,
	databaseIds :: !(IORef (Intern Key)),
	databaseValues :: !(Ids (Key, Status))

[pepeiborra]

are these fields lazy on purpose?

[pepeiborra]

Suggested change

	= Clean Result
	| Dirty (Maybe Result)
	| Running (IO Result) (Maybe Result)
	= Clean !Result
	| Dirty !(Maybe Result)
	| Running !(IO Result) !(Maybe Result)

[pepeiborra]

Suggested change

	resultData :: BS.ByteString
	resultData :: !BS.ByteString

[pepeiborra]

I'm not familiar with this trick. What is the Box for?

[pepeiborra]

Suggested change

	actionBracket :: IO a -> (a -> IO b) -> (a -> Action c) -> Action c
	-- TODO Generalize type signatures and implement exception classes
	actionBracket :: IO a -> (a -> IO b) -> (a -> Action c) -> Action c

[pepeiborra]

Suggested change

	shakeRunDatabase (ShakeDatabase threads lenAs1 as1 db) as2 = withNumCapabilities threads $ do
	-- TODO move capability bits out of hls-graph
	shakeRunDatabase (ShakeDatabase threads lenAs1 as1 db) as2 = withNumCapabilities threads $ do

[pepeiborra]

Suggested change

	actionDeps :: !(IORef (Maybe [Id])) -- Nothing means always rerun
	actionDeps :: !(IORef (Maybe [Id])) -- ^ Nothing means always rerun

[pepeiborra]

After rebasing on top of #1754 benchmarks are failing with out of range errors:

╰─>$ tail -n 20 ghcide/bench-results/unprofiled/Cabal-3.0.0.0/HEAD/completions.output.log
                "method": "workspace/didChangeWatchedFiles",
                "id": "globalFileWatches"
            }
        ]
    },
    "method": "client/registerCapability",
    "id": 0
}
ghcide: Ix{Int}.index: Index (43) out of range ((0,0))
[DEBUG] Set files of
--> {
    "jsonrpc": "2.0",
    "params": null,
    "method": "shutdown",
    "id": 0
}
fd:12: hPutBuf: resource vanished (Broken pipe)
name        | success | samples | startup | setup | userTime | delayedTime | totalTime
----------- | ------- | ------- | ------- | ----- | -------- | ----------- | ---------
completions | False   | 50      | 0.00s   | 0.00s | 0.00s    | 0.00s       | 0.00s

[ndmitchell]

That's a different error to what I was seeing. I'll double check tomorrow morning that I didn't forget to send up any of my changes.

[wz1000]

On IRC, we were discussing if it was possible to make all the book-keeping contention free, by eliminating the locks and running in STM (and using a concurrent map like stm-containers?).

This bit seems to be the problematic part, the rest of the computation seem like it can easily be moved into STM. But it might just be possible to make something work using judicious use of unsafePerformIO, unsafeIOToSTM and the proper care. Do you think this would be possible?

[pepeiborra]

Continued in #2060