Add to agent exponential backoff for fetching new tasks #3206
Conversation
| @@ -59,10 +98,10 @@ func (r *Runner) Run(runnerCtx context.Context) error { | |||
| // get the next workflow from the queue | |||
| work, err := r.client.Next(runnerCtx, r.filter) | |||
There was a problem hiding this comment.
The server is blocking Next requests until it has a job, so there should be no need to backoff here.
| } | ||
| } | ||
|
|
||
| func (r *Runner) Run(runnerCtx context.Context) error { | ||
| retry := backoff.NewExponentialBackOff() |
There was a problem hiding this comment.
Why do you want to backoff exponential instead of linear here?
There was a problem hiding this comment.
Because if you fetched a task last time its more likely that there are some next to come
Else in the worst case wait 10s
We also had a 1.2Tb log an other day ... It just did not kill the whole server as we had ~2Tb of free storage before |
|
With this in the worstcase we have ~1.7Mb per day |
Did you enabled debug logs? |
|
no ... that would be no bug then
|
|
☝️ the mentioned bug should already be fixed ... so that's why i labled it enhancement not bug |
|
and no the server should and do not block the agent on a next() call. so just enable debug and then disable the agent on the server ... you can see that the agent trys to fetch in a loop nonstop. |
|
Superseded by #3378 |
other wise we DOS the server
Sponsored by Kithara Software GmbH