Drivers: Make InternalCapabilities.DisableLogCollection Public

[endocrimes]

Proposal

The current set of Internal Driver Capabilities have been relatively stable since the introduction of the plugin interface, and some would greatly benefit 3rd party users of Nomad and Custom Task Drivers. This proposal suggests moving the DisableLogCollection feature to the public capabilities API.

While not perfect in isolation (Nomad offers no way to disable log collection for individual tasks, which means when scheduling to a driver with log collection disabled, the task still has an associated set of resources allocated for log storage), this goes a long way towards reducing overhead in a range of cases.

Use-cases

There are many use-cases where a driver author might want to give operators the ability to save on the overhead of deploying an unnecessary logmon for every task. Similarly to the Nomad Docker Driver's usage, the most obvious is where a runtime offers the ability to manage logs externally to Nomad.

This is particularly important when building high-density runtimes (and task drivers) - where no-op logmon overhead can quickly become an issue when deploying X000 allocations/node and logging is handled externally.

Attempted Solutions

We currently deploy a fork of Nomad that has a slightly jankier approach 😅.

[tgross]

Hi @endocrimes! Exposing this capability makes sense to me.

A little while back I spent a few days hacking on some thoughts for something like this (and #14636 #14056 #9211 and a variety of logging-related issues). I've got a disorganized branch with a couple different experiements at main...hack-logging-plugins which I'm going to carve out some time on in the next major release cycle. The larger "logging plugins" concept may not land but we definitely want to support the simpler lower-resource use case you're talking about here.

[endocrimes]

@tgross Awesome - always happy to chat if you wanna bounce ideas around - from that branch I love the new logmon prospect (and the logging plugin concept in general).

It'd go a long way to alleviating one of the harder things to do cleanly with non-Docker drivers 😅 too, which is pretty good for everyone.

[tgross]

Done in #17196 which will ship in Nomad 1.6.0! I'll have a PR up (probably later today) adding the capability to the other internal drivers.

[github-actions]

I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.