Fix agent polling #3378

anbraten · 2024-02-12T17:09:21Z

Supersedes #3206
closes #1990
closes woodpecker-ci/autoscaler#50

This solves a bug that NoSchedule wasn't considered if the agent already had workers waiting for tasks in the queue by kicking those workers from the queue when updating the agents no-schedule flag via api. We now also kick the agents workers when deleting the agent.

In addition a timeout prevents agents with NoSchedule from spamming the server.

Removes errored tasks from the database queue.

I also removed the queue info completed property as it is never used.

codecov · 2024-02-12T17:39:32Z

Codecov Report

Attention: 45 lines in your changes are missing coverage. Please review.

Comparison is base (65d88be) 36.67% compared to head (a48ded3) 36.60%.
Report is 4 commits behind head on main.

❗ Current head a48ded3 differs from pull request most recent head 5defdde. Consider uploading reports for the commit 5defdde to get more accurate results

Files	Patch %	Lines
server/queue/persistent.go	0.00%	14 Missing ⚠️
server/grpc/rpc.go	0.00%	13 Missing ⚠️
server/queue/fifo.go	18.75%	13 Missing ⚠️
server/api/agent.go	0.00%	5 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3378      +/-   ##
==========================================
- Coverage   36.67%   36.60%   -0.07%     
==========================================
  Files         225      225              
  Lines       14788    14819      +31     
==========================================
+ Hits         5423     5425       +2     
- Misses       8972     9001      +29     
  Partials      393      393

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

server/grpc/rpc.go

woodpecker-bot · 2024-02-12T18:57:15Z

Deployment of preview was successful: https://woodpecker-ci-woodpecker-pr-3378.surge.sh

xoxys · 2024-02-16T08:30:48Z

How does this PR supersede #3206?

anbraten · 2024-02-16T08:32:35Z

It fixes the long-polling loop

xoxys · 2024-02-16T08:46:38Z

That was clear already, but wouldn't it still make sense to add a backoff to the agent polling?

anbraten · 2024-02-16T08:50:43Z

That was clear already, but wouldn't it still make sense to add a backoff to the agent polling?

I think we can consider that later on. Atm the Poll call is blocking and therefore an agent would request with x workers once and only does new requests as soon as one worker needs a new task. That shouldn't be a lot of requests.

## [2.4.0](https://github.com/woodpecker-ci/woodpecker/releases/tag/2.4.0) - 2024-03-19 ### 🔒 Security - Improve security context handling [[#3482](#3482)] - fix(deps): update module github.com/moby/moby to v24.0.9+incompatible [[#3323](#3323)] ### ✨ Features - Cli setup command [[#3384](#3384)] - Add bitbucket datacenter (server) support [[#2503](#2503)] - Cli updater [[#3382](#3382)] ### 📚 Documentation - Delete docs for v0.15.x [[#3508](#3508)] - Add deployment plugin [[#3495](#3495)] - Bump follow-redirects and fix broken anchors [[#3488](#3488)] - fix: plugin doc page not found [[#3480](#3480)] - Documentation improvements [[#3376](#3376)] - fix(deps): update docs npm deps non-major [[#3455](#3455)] - Add "Sonatype Nexus" plugin [[#3446](#3446)] - Add blog post [[#3439](#3439)] - Add "Gradle Wrapper Validation" plugin [[#3435](#3435)] - Add blog post [[#3410](#3410)] - Extend core ideas documentation [[#3405](#3405)] - docs: fix contributions link [[#3363](#3363)] - Update/fix some docs [[#3359](#3359)] - chore(deps): update dependency marked to v12 [[#3325](#3325)] ### 🐛 Bug Fixes - Fix skip setup for some general cli commands [[#3498](#3498)] - Move generic agent flags to cmd/agent/core [[#3484](#3484)] - Fix usage of WOODPECKER_DATABASE_DATASOURCE_FILE [[#3404](#3404)] - Set pull-request id and labels on pr-closed event [[#3442](#3442)] - Update org name on login [[#3409](#3409)] - Do not alter secret key upper-/lowercase [[#3375](#3375)] - fix: can't run multiple services on k8s [[#3395](#3395)] - Fix agent polling [[#3378](#3378)] - Remove empty strings from slice before parsing agent config [[#3387](#3387)] - Set correct link for commit [[#3368](#3368)] - Fix schema links [[#3369](#3369)] - Fix correctly handle gitlab pr closed events [[#3362](#3362)] - fix: update schema event_enum to remove error warning when.event [[#3357](#3357)] - Fix version check on next [[#3340](#3340)] - Ignore gitlab merge request events without code changes [[#3338](#3338)] - Ignore gitlab push events without commits [[#3339](#3339)] - Consider gitlab inherited permissions [[#3308](#3308)] - fix: agent panic when node is terminated during step execution [[#3331](#3331)] ### 📈 Enhancement - Enable golangci linter gomnd [[#3171](#3171)] - Apply "grpcnotrace" go build tag [[#3448](#3448)] - Simplify store interfaces [[#3437](#3437)] - Deprecate alternative names on secrets [[#3406](#3406)] - Store workflows/steps for blocked pipeline [[#2757](#2757)] - Parse email from Gitea webhook [[#3420](#3420)] - Replace http types on forge interface [[#3374](#3374)] - Prevent agent deletion when it's still running tasks [[#3377](#3377)] - Refactor internal services [[#915](#915)] - Lint for event filter and deprecate `exclude` [[#3222](#3222)] - Allow editing all environment variables in pipeline popups [[#3314](#3314)] - Parse backend options in backend [[#3227](#3227)] - Make agent usable for external backends [[#3270](#3270)] - Add no branches text [[#3312](#3312)] - Add loading spinner to repo list [[#3310](#3310)] ### Misc - Post on mastodon when releasing a new version [[#3509](#3509)] - chore(deps): update dependency alpine_3_18/ca-certificates to v20240226 [[#3501](#3501)] - fix(deps): update module github.com/google/go-github/v59 to v60 [[#3493](#3493)] - fix(deps): update dependency @intlify/unplugin-vue-i18n to v3 [[#3492](#3492)] - chore(deps): update dependency vue-tsc to v2 [[#3491](#3491)] - chore(deps): update dependency eslint-config-airbnb-typescript to v18 [[#3490](#3490)] - chore(deps): update web npm deps non-major [[#3489](#3489)] - fix(deps): update golang (packages) [[#3486](#3486)] - fix(deps): update module google.golang.org/protobuf to v1.33.0 [security] [[#3487](#3487)] - chore(deps): update docker.io/techknowlogick/xgo docker tag to go-1.22.1 [[#3476](#3476)] - chore(deps): update docker.io/golang docker tag to v1.22.1 [[#3475](#3475)] - Update prettier version [[#3471](#3471)] - chore(deps): update woodpeckerci/plugin-ready-release-go docker tag to v1.1.0 [[#3464](#3464)] - chore(deps): lock file maintenance [[#3465](#3465)] - chore(deps): update postgres docker tag to v16.2 [[#3461](#3461)] - chore(deps): update lycheeverse/lychee docker tag to v0.14.3 [[#3429](#3429)] - fix(deps): update golang (packages) [[#3430](#3430)] - More `when` filters [[#3407](#3407)] - Apply `documentation`/`ui` label to corresponding renovate updates [[#3400](#3400)] - chore(deps): update dependency eslint-plugin-simple-import-sort to v12 [[#3396](#3396)] - chore(deps): update typescript-eslint monorepo to v7 (major) [[#3397](#3397)] - fix(deps): update module github.com/google/go-github/v58 to v59 [[#3398](#3398)] - chore(deps): update docker.io/techknowlogick/xgo docker tag to go-1.22.0 [[#3392](#3392)] - chore(deps): update docker.io/golang docker tag [[#3391](#3391)] - fix(deps): update golang (packages) [[#3393](#3393)] - chore(deps): update docker.io/woodpeckerci/plugin-docker-buildx docker tag to v3.1.0 [[#3394](#3394)] - Add link checking [[#3371](#3371)] - Apply `dependencies` label to all PRs [[#3358](#3358)] - chore(deps): update docker.io/woodpeckerci/plugin-docker-buildx docker tag to v3.0.1 [[#3324](#3324)] --------- Co-authored-by: 6543 <[email protected]> Co-authored-by: Anbraten <[email protected]>

This property was removed from woodpecker in woodpecker-ci/woodpecker#3378 and made it to the 2.4.0 release

fix agent polling

cda806e

anbraten added bug Something isn't working server labels Feb 12, 2024

anbraten added 2 commits February 12, 2024 18:21

lower timeout

b6c8165

adjust comments

e573513

anbraten requested a review from a team February 12, 2024 17:31

fix agent was set to no-schedule meanwhile

33fd684

anbraten commented Feb 12, 2024

View reviewed changes

server/grpc/rpc.go Outdated Show resolved Hide resolved

anbraten added 2 commits February 12, 2024 19:40

Merge remote-tracking branch 'upstream/main' into fix-agent-poll

a3938e7

use context to kick workers

42db9b0

anbraten added 3 commits February 12, 2024 19:57

improve comments

b7c3937

cleanup

2afb499

fix lint

a48ded3

qwerty287 added this to the 2.4.0 milestone Feb 15, 2024

xoxys approved these changes Feb 16, 2024

View reviewed changes

Merge branch 'main' into fix-agent-poll

5defdde

anbraten enabled auto-merge (squash) February 16, 2024 08:55

anbraten merged commit 0e0d018 into woodpecker-ci:main Feb 16, 2024
7 checks passed

anbraten deleted the fix-agent-poll branch February 16, 2024 12:43

woodpecker-bot mentioned this pull request Feb 16, 2024

🎉 Release 2.4.0 #3307

Merged

1 task

qwerty287 mentioned this pull request Feb 17, 2024

Add to agent exponential backoff for fetching new tasks #3206

Closed

BrewTestBot mentioned this pull request Mar 19, 2024

woodpecker-cli 2.4.0 Homebrew/homebrew-core#166593

Merged

Nothing4You added a commit to Nothing4You/picus that referenced this pull request Apr 5, 2024

Remove completed_count from WpQueueInfo

1773358

This property was removed from woodpecker in woodpecker-ci/woodpecker#3378 and made it to the 2.4.0 release

Nothing4You added a commit to Nothing4You/picus that referenced this pull request Apr 5, 2024

Remove completed_count from WpQueueInfo

c1d99d5

This property was removed from woodpecker in woodpecker-ci/woodpecker#3378 and made it to the 2.4.0 release

Nothing4You mentioned this pull request Apr 5, 2024

Remove completed_count from WpQueueInfo windsource/picus#11

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix agent polling #3378

Fix agent polling #3378

anbraten commented Feb 12, 2024 •

edited

Loading

codecov bot commented Feb 12, 2024 •

edited

Loading

woodpecker-bot commented Feb 12, 2024 •

edited

Loading

xoxys commented Feb 16, 2024

anbraten commented Feb 16, 2024

xoxys commented Feb 16, 2024

anbraten commented Feb 16, 2024

Fix agent polling #3378

Fix agent polling #3378

Conversation

anbraten commented Feb 12, 2024 • edited Loading

codecov bot commented Feb 12, 2024 • edited Loading

Codecov Report

woodpecker-bot commented Feb 12, 2024 • edited Loading

xoxys commented Feb 16, 2024

anbraten commented Feb 16, 2024

xoxys commented Feb 16, 2024

anbraten commented Feb 16, 2024

anbraten commented Feb 12, 2024 •

edited

Loading

codecov bot commented Feb 12, 2024 •

edited

Loading

woodpecker-bot commented Feb 12, 2024 •

edited

Loading