Graylog: Improve message formatting

[DominicOram]

Multiline messages do not format nicely on graylog, instead showing /n chars. See https://graylog.diamond.ac.uk/messages/graylog-daq-mx-hyperion_7/5018d5d6-06ef-11ef-9f8e-223dcd8d7ad4 we should try and work out if we can format them better

Acceptance Criteria

• Investigate better formatting in graylog
• Fix if we can
• If this is a limitation of graylog document it somewhere as a known issue

[DominicOram]

@dperl-dls thinks this may be fixed, check

[d-perl]

Not limited to graylog, it is in all logs: DiamondLightSource/hyperion#634 - should fix this at the same time

[d-perl]

oops, closed by accident sorry

[DominicOram]

• We should make the log formatter take care of this and replace /n with newlines properly
• Timebox to 1 day, if we make no progress document on here why we had to stop and stop

[d-perl]

It does look like with the most recent version (I think this is probably a python 3.11 thing) we do get e.g. tracebacks formatted correctly:

https://graylog.diamond.ac.uk/messages/graylog-daq-mx-hyperion_17/6069fe10-0ea0-11ef-a69c-fae0dce5bd43

Meanwhile the json is generally formatted compactly, this is because we insert it in strings ourselves as the repr of a dict. If we want this to be formatted the solution is to do it ourselves, with json.dumps(..., indent=n):

https://graylog.diamond.ac.uk/messages/graylog-daq-mx-hyperion_17/741f64c5-0ea2-11ef-a69c-fae0dce5bd43

[DominicOram]

Ah, if it's just the json then I think we should format it ourselves nicely for the logs

[d-perl]

Well, we should probably add some solution for tracebacks in 3.10 if we are supporting it, but I don't think it's very urgent