Backport preset and plugins functionality plus graceful shutdown to 0.13.0

[benjie]

Description

Some people are hesitant to update to 0.14 due to the breaking schema changes involved. The plugin system gives them more ways to deal with this. On top of this, the shutdown in 0.13 is more of a controlled crash, so this addresses that properly by reapplying the fixes from 0.16 back to 0.13.

This is a very significant backport of a lot of functionality, but none of the database changes have been backported and that means that we're still using the old (slow on large tables) queue strategy, batch jobs are not supported (upgrade for that!), and neither are the crontab changes (including job keys on crontabs) or force unlocking of workers.

One major breaking change is that the experimental --watch flag has been removed. Backporting the features from 0.16 whilst maintaining support for --watch was too much, and since it was always experimental I've opted to remove it.

Performance impact

Marginal. For better performance, upgrade to 0.14 or even better: 0.16!

Security impact

None known.

Checklist

• My code matches the project's code style and yarn lint:fix passes.
• I've added tests for the new feature, and yarn test passes.
• I have detailed the new feature in the relevant documentation.
• I have added this feature to 'Pending' in the RELEASE_NOTES.md file (if one exists).
• If this is a breaking change I've explained why.