-
Notifications
You must be signed in to change notification settings - Fork 2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
networking: Inject implicit constraints on CNI plugins when using bridge mode #15473
Conversation
057a868
to
b64914f
Compare
Looks good to me. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Neat! It could be helpful to have a comment next to nomadCNIConfigTemplate
with a reminder to add or remove these implicit constraints if the template changes.
This also makes me thing about mixed version clusters 🤔
If you happen to have clients with different nomadCNIConfigTemplate
, these constraints may not necessarily be true since they are applied in the servers. But that's a very edge case.
It does make me wonder if we could generate these constraints dynamically from nomadCNIConfigTemplate
😅
Ooh the upgrade path, yeah that gets interesting. When we have added implicit constraints in the past it was for stuff like Consul version, where clients had long already been fingerprinted, or native service discovery, which required new clients anyway. This is a case where we need to maintain compatibility with old clients not yet being fingerprinted. Not sure we can merge this until like 1.5.x, or if we backport #15473 first and wait a while. |
Kinda missed the boat on 1.7; pushing to 1.8 |
…dge mode This PR adds a job mutator which injects constraints on the job taskgroups that make use of bridge networking. Creating a bridge network makes use of the CNI plugins: bridge, firewall, host-local, loopback, and portmap. Starting with Nomad 1.5 these plugins are fingerprinted on each node, and as such we can ensure jobs are correctly scheduled only on nodes where they are available, when needed.
b64914f
to
0be5eab
Compare
0be5eab
to
f33980f
Compare
I've rebased this PR on main, added handling of the early exit we introduced for NUMA, and added an upgrade note. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! Maybe worth adding a note to the bridge config that these constraints may need to be updated if the bridge spec is modified.
nomad/client/allocrunner/networking_bridge_linux.go
Lines 150 to 152 in c3e7b13
// Update website/content/docs/networking/cni.mdx when the bridge configuration | |
// is modified. | |
const nomadCNIConfigTemplate = `{ |
Done! |
…dge mode (#15473) This PR adds a job mutator which injects constraints on the job taskgroups that make use of bridge networking. Creating a bridge network makes use of the CNI plugins: bridge, firewall, host-local, loopback, and portmap. Starting with Nomad 1.5 these plugins are fingerprinted on each node, and as such we can ensure jobs are correctly scheduled only on nodes where they are available, when needed.
I'm going to lock this pull request because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active contributions. |
This PR adds a job mutator which injects constraints on the job taskgroups
that make use of bridge networking. Creating a bridge network makes use of the
CNI plugins: bridge, firewall, host-local, loopback, and portmap. Starting
with Nomad 1.5 these plugins are fingerprinted on each node, and as such we
can ensure jobs are correctly scheduled only on nodes where they are available,
when needed.
Targets v1.5.0
Closes #14022