Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Create a pool of builders that have more storage #3268

Closed
xsuchy opened this issue May 20, 2024 · 7 comments · Fixed by #3408
Closed

Create a pool of builders that have more storage #3268

xsuchy opened this issue May 20, 2024 · 7 comments · Fixed by #3408
Assignees
@nikromen

Comments

@xsuchy
Copy link
Member

xsuchy commented May 20, 2024

Follow up of #3267

We have powerful builders, but this adds "only" CPU and RAM. But not storage.
We have a hardcoded option to mock config to use 140GB for storage using tmpfs
https://pagure.io/fedora-infra/ansible/blob/main/f/roles/copr/backend/files/provision/files/mock/site-defaults.cfg#_18
We must pass an option allowing powerful builders to use more storage.
@github-project-automation github-project-automation bot moved this to Needs triage in CPT Kanban May 20, 2024
@xsuchy xsuchy changed the title Create a pool of builders that requires more storage Create a pool of builders that have more storage May 20, 2024
@praiskup
Copy link
Member

There's the on_demand_powerful tag configured in Ansible, which is coming from Frontend's configuration. Then, we need to have a tool that would be able to modify the Mock configuration when on_demand_powerul tag is used.

@praiskup
Copy link
Member

Alternatively, we could bump the storage +10G everywhere.

@praiskup praiskup moved this from Needs triage to In 3 months in CPT Kanban May 22, 2024
@xsuchy xsuchy mentioned this issue May 22, 2024
Closed
@praiskup praiskup moved this from In 3 months to In Progress in CPT Kanban Aug 26, 2024
nikromen added a commit to nikromen/copr that referenced this issue Sep 9, 2024
@nikromen
rpmbuilds: dynamically specify tmp_fs size
Loading
Loading status checks…
a9e931f 
This moves specifying tmp fs size to rpmbuild from [1] in order to be
able to automatically generate its size for performance builders.

[1] - https://pagure.io/fedora-infra/ansible/blob/main/f/roles/copr/backend/files/provision/files/mock/site-defaults.cfg#_18

See fedora-copr#3268
@nikromen nikromen mentioned this issue Sep 9, 2024
Merged
2 tasks
nikromen added a commit to nikromen/copr that referenced this issue Sep 9, 2024
@nikromen
rpmbuilds: dynamically specify tmp_fs size
Loading
Loading status checks…
be206aa 
This moves specifying tmp fs size to rpmbuild from [1] in order to be
able to automatically generate its size for performance builders.

[1] - https://pagure.io/fedora-infra/ansible/blob/main/f/roles/copr/backend/files/provision/files/mock/site-defaults.cfg#_18

See fedora-copr#3268
@nikromen nikromen mentioned this issue Sep 9, 2024
Merged
nikromen added a commit to nikromen/copr that referenced this issue Sep 9, 2024
@nikromen
rpmbuilds: dynamically specify tmp_fs size
Loading
Loading status checks…
7b54bab 
This moves specifying tmp fs size to rpmbuild from [1] in order to be
able to automatically generate its size for performance builders.

[1] - https://pagure.io/fedora-infra/ansible/blob/main/f/roles/copr/backend/files/provision/files/mock/site-defaults.cfg#_18

See fedora-copr#3268
nikromen added a commit to nikromen/copr that referenced this issue Sep 16, 2024
@nikromen
rpmbuilds: dynamically specify tmp_fs size
Loading
Loading status checks…
befd48a 
This moves specifying tmp fs size to rpmbuild from [1] in order to be
able to automatically generate its size for performance builders.

[1] - https://pagure.io/fedora-infra/ansible/blob/main/f/roles/copr/backend/files/provision/files/mock/site-defaults.cfg#_18

See fedora-copr#3268
nikromen added a commit to nikromen/copr that referenced this issue Sep 17, 2024
@nikromen
rpmbuilds: dynamically specify tmp_fs size
Loading
Loading status checks…
133840f 
This moves specifying tmp fs size to rpmbuild from [1] in order to be
able to automatically generate its size for performance builders.

[1] - https://pagure.io/fedora-infra/ansible/blob/main/f/roles/copr/backend/files/provision/files/mock/site-defaults.cfg#_18

See fedora-copr#3268
nikromen added a commit to nikromen/copr that referenced this issue Sep 17, 2024
@nikromen
rpmbuilds: dynamically specify tmp_fs size
Loading
Loading status checks…
b45b43a 
This moves specifying tmp fs size to rpmbuild from [1] in order to be
able to automatically generate its size for performance builders.

[1] - https://pagure.io/fedora-infra/ansible/blob/main/f/roles/copr/backend/files/provision/files/mock/site-defaults.cfg#_18

See fedora-copr#3268
nikromen added a commit to nikromen/copr that referenced this issue Sep 17, 2024
@nikromen
rpmbuilds: dynamically specify tmp_fs size
Loading
Loading status checks…
34455a8 
This moves specifying tmp fs size to rpmbuild from [1] in order to be
able to automatically generate its size for performance builders.

[1] - https://pagure.io/fedora-infra/ansible/blob/main/f/roles/copr/backend/files/provision/files/mock/site-defaults.cfg#_18

See fedora-copr#3268
nikromen added a commit to nikromen/copr that referenced this issue Sep 17, 2024
@nikromen
rpmbuilds: dynamically specify tmp_fs size
Loading
Loading status checks…
317b113 
This moves specifying tmp fs size to rpmbuild from [1] in order to be
able to automatically generate its size for performance builders.

[1] - https://pagure.io/fedora-infra/ansible/blob/main/f/roles/copr/backend/files/provision/files/mock/site-defaults.cfg#_18

See fedora-copr#3268
nikromen added a commit to nikromen/copr that referenced this issue Sep 17, 2024
@nikromen
rpmbuild: specify mock snippets to mock config via copr-rpmbuild conf…
Loading
Loading status checks…
bb25935 
…ig file

This allows us to specify tpm fs size to rpmbuild in order to be able to
automatically generate its size for performance builders.

See fedora-copr#3268
nikromen added a commit to nikromen/copr that referenced this issue Sep 17, 2024
@nikromen
rpmbuild: specify snippets to mock config via copr-rpmbuild config file
Loading
Loading status checks…
8578e57 
This allows us to specify tpm fs size to rpmbuild in order to be able to
automatically generate its size for performance builders.

See fedora-copr#3268
nikromen added a commit to nikromen/copr that referenced this issue Sep 17, 2024
@nikromen
rpmbuild: specify snippets to mock config via copr-rpmbuild config file
Loading
Loading status checks…
2d5ca8a 
This allows us to specify tpm fs size to rpmbuild in order to be able to
automatically generate its size for performance builders.

See fedora-copr#3268
nikromen added a commit to nikromen/copr that referenced this issue Sep 17, 2024
@nikromen
rpmbuild: specify snippets to mock config via copr-rpmbuild config file
Loading
Loading status checks…
e07b957 
This allows us to specify tpm fs size to rpmbuild in order to be able to
automatically generate its size for performance builders.

See fedora-copr#3268
nikromen added a commit to nikromen/copr that referenced this issue Sep 17, 2024
@nikromen
rpmbuild: specify snippets to mock config via copr-rpmbuild config file
Loading
Loading status checks…
2b5d21c 
This allows us to specify tpm fs size to rpmbuild in order to be able to
automatically generate its size for performance builders.

See fedora-copr#3268
nikromen added a commit to nikromen/copr that referenced this issue Sep 17, 2024
@nikromen
rpmbuild: specify snippets to mock config via copr-rpmbuild config file
Loading
Loading status checks…
3d2e928 
This allows us to specify tpm fs size to rpmbuild in order to be able to
automatically generate its size for performance builders.

See fedora-copr#3268
nikromen added a commit to nikromen/copr that referenced this issue Sep 17, 2024
@nikromen
rpmbuild: specify snippets to mock config via copr-rpmbuild config file
Loading
Loading status checks…
48fd69e 
This allows us to specify tpm fs size to rpmbuild in order to be able to
automatically generate its size for performance builders.

See fedora-copr#3268
nikromen added a commit to nikromen/copr that referenced this issue Sep 18, 2024
@nikromen
rpmbuild: specify snippets to mock config via copr-rpmbuild config file
Loading
Loading status checks…
ffa70c7 
This allows us to specify tpm fs size to rpmbuild in order to be able to
automatically generate its size for performance builders.

See fedora-copr#3268
nikromen added a commit to nikromen/copr that referenced this issue Sep 20, 2024
@nikromen
rpmbuild: specify snippets to mock config via copr-rpmbuild config file
Loading
Loading status checks…
d555403 
This allows us to specify tpm fs size to rpmbuild in order to be able to
automatically generate its size for performance builders.

See fedora-copr#3268
nikromen added a commit to nikromen/copr that referenced this issue Sep 23, 2024
@nikromen
rpmbuild: specify snippets to mock config via copr-rpmbuild config file
Loading
Loading status checks…
c52dc98 
This allows us to specify tpm fs size to rpmbuild in order to be able to
automatically generate its size for performance builders.

See fedora-copr#3268
nikromen added a commit to nikromen/copr that referenced this issue Sep 24, 2024
@nikromen
rpmbuild: specify snippets to mock config via copr-rpmbuild config file
Loading
Loading status checks…
e869d2a 
This allows us to specify tpm fs size to rpmbuild in order to be able to
automatically generate its size for performance builders.

See fedora-copr#3268
nikromen added a commit to nikromen/copr that referenced this issue Sep 24, 2024
@nikromen
rpmbuild: specify snippets to mock config via copr-rpmbuild config file
Loading
Loading status checks…
fb5ff90 
This allows us to specify tpm fs size to rpmbuild in order to be able to
automatically generate its size for performance builders.

See fedora-copr#3268
nikromen added a commit that referenced this issue Sep 25, 2024
@nikromen
rpmbuild: specify snippets to mock config via copr-rpmbuild config file
5d77d36 
This allows us to specify tpm fs size to rpmbuild in order to be able to
automatically generate its size for performance builders.

See #3268
jaitjacob added a commit to jaitjacob/copr that referenced this issue Sep 25, 2024
@jaitjacob
Squashed commit of the following:
a6d1127 
commit 4b1576f
Merge: 57a303e 8b0977a
Author: jait <[email protected]>
Date:   Wed Sep 25 22:59:20 2024 +0530

    Merge branch 'fedora-copr:main' into Webhook-History-UI

commit 8b0977a
Author: Pavel Raiskup <[email protected]>
Date:   Tue Sep 24 21:09:24 2024 +0200

    rpmbuild: unblock testsuite

commit 5d77d36
Author: Jiri Kyjovsky <[email protected]>
Date:   Tue Sep 17 14:35:28 2024 +0200

    rpmbuild: specify snippets to mock config via copr-rpmbuild config file

    This allows us to specify tpm fs size to rpmbuild in order to be able to
    automatically generate its size for performance builders.

    See fedora-copr#3268

commit 57a303e
Author: Jait Jacob <[email protected]>
Date:   Wed Sep 25 13:26:34 2024 +0530

    remove unused import & revert localized_time filter method

commit 658230d
Author: Pavel Raiskup <[email protected]>
Date:   Mon Sep 23 15:20:09 2024 +0200

    backend: unknown resalloc tickets helper cleanup

    If no tickets are taken (which often happens in the staging
    environment), this script encountered corner case issues.

commit d6a3472
Author: Miroslav Suchý <[email protected]>
Date:   Mon Sep 23 22:12:43 2024 +0200

    rpmbuild: do not require rpkg,pyp2rpm,pyp2spec,gem2rpm and fedora-review on rhel

    Resolves: RHBZ#2313878

commit 6ae7c6c
Author: Miroslav Suchý <[email protected]>
Date:   Mon Sep 23 22:04:11 2024 +0200

    rpmbuild: do not require qemu-user-static on rhel

    Resolves: RHBZ#2313879

commit 1559b85
Author: Aurélien Bompard <[email protected]>
Date:   Mon Jul 29 10:54:02 2024 +0200

    Use `super()` without argument to make pylint happy

    Signed-off-by: Aurélien Bompard <[email protected]>

commit f3b1643
Author: Aurélien Bompard <[email protected]>
Date:   Mon Jul 29 10:40:13 2024 +0200

    Message schemas: set chroot message severity to DEBUG

    Signed-off-by: Aurélien Bompard <[email protected]>

commit c192726
Author: Aurélien Bompard <[email protected]>
Date:   Mon Jul 29 10:38:34 2024 +0200

    Message schemas: one-line descriptions should be the summary

    Signed-off-by: Aurélien Bompard <[email protected]>

commit 609d369
Author: Pavel Raiskup <[email protected]>
Date:   Fri Sep 20 14:14:31 2024 +0200

    frontend: fix the 500 for racy creation attempts

    This is TOCTOU issue.  The other checks for duplications (on so many
    places) seem kinda redundant because nothing but try/except for commit()
    may catch these concurrency problems.

    Fixes: fedora-copr#3372

commit 446dcb3
Author: Jiri Kyjovsky <[email protected]>
Date:   Wed Sep 18 16:49:52 2024 +0200

    beaker: use podman for testing inside container if installed

commit 030740a
Author: Jait Jacob <[email protected]>
Date:   Sun Sep 22 23:17:17 2024 +0530

    remove unaccessed import

commit cf1cb81
Author: Jait Jacob <[email protected]>
Date:   Fri Sep 20 02:36:09 2024 +0530

    webhook_history stores UNIX timestamps instead of DateTime

commit 6aeb686
Author: Jiri Kyjovsky <[email protected]>
Date:   Mon Sep 16 10:15:37 2024 +0200

    docker: set hard ulimits for docker container

    Because of bug in python3-daemon [1] we need to set ulimits inside
    docker container, otherwise backend and dist-git ooms.

    [1] - https://bugzilla.redhat.com/show_bug.cgi?id=2307635

commit 6aa1d5e
Author: Jait Jacob <[email protected]>
Date:   Sun Sep 15 19:55:11 2024 +0530

    resolve pylint warnings

commit a2adf81
Author: Jait Jacob <[email protected]>
Date:   Sun Sep 15 19:44:07 2024 +0530

    remove unused code

commit 14faa50
Author: Jakub Kadlcik <[email protected]>
Date:   Thu Sep 5 11:10:15 2024 +0200

    frontend, python, cli: allow admins to set storage for new projects

    See fedora-copr#2533

    This will be useful for beaker tests where we can now add basic tests
    for every supported storage.

commit ad36b8b
Author: Jakub Kadlcik <[email protected]>
Date:   Thu Sep 5 10:36:46 2024 +0200

    frontend: make the default storage for new projects configurable

    See fedora-copr#2533

commit 286a913
Author: Jakub Kadlcik <[email protected]>
Date:   Tue Sep 10 08:36:23 2024 +0200

    backend: add a timeout for waiting until a Pulp task finishes

commit 23a2fa3
Author: Jakub Kadlcik <[email protected]>
Date:   Mon Jul 22 11:23:53 2024 +0200

    backend: actions don't call uses_devel_repo function anymore

commit c5166a1
Author: Jakub Kadlcik <[email protected]>
Date:   Thu Sep 5 09:43:09 2024 +0200

    backend, frontend: implement project and build deletion in Pulp

    Fix fedora-copr#3318
    Fix fedora-copr#3319

commit 19eff0c
Author: Jakub Kadlcik <[email protected]>
Date:   Sun Sep 8 12:03:15 2024 +0200

    copr: wait until Pulp publication is finished

commit ff5288d
Author: Miroslav Suchý <[email protected]>
Date:   Wed Sep 11 08:19:41 2024 +0200

    common: cleanup - remove six dependency

commit 6c71993
Author: Jait Jacob <[email protected]>
Date:   Wed Sep 11 16:21:34 2024 +0530

    fix pylint complaint

commit 649fe51
Author: Jait Jacob <[email protected]>
Date:   Wed Sep 11 16:15:14 2024 +0530

    optimize db calls, remove client side js, resolve review comments

commit 31880b4
Author: Jait Jacob <[email protected]>
Date:   Sun Sep 8 22:32:14 2024 +0530

    cleanup

commit 593a13f
Author: Jait Jacob <[email protected]>
Date:   Sun Sep 8 22:17:58 2024 +0530

    add pagination

commit 99e142e
Merge: 11d7fb1 2bdec45
Author: jait <[email protected]>
Date:   Sun Sep 8 15:03:36 2024 +0530

    Merge branch 'fedora-copr:main' into Webhook-History-UI

commit 11d7fb1
Author: Jait Jacob <[email protected]>
Date:   Sun Sep 8 15:03:29 2024 +0530

    return webhook history in descending order & without any duplicates

commit b7f2b34
Author: Jait Jacob <[email protected]>
Date:   Sat Sep 7 00:22:06 2024 +0530

    frontend: show webhook history table under Setting->Integration
@nikromen nikromen moved this from In Progress to Done in CPT Kanban Sep 30, 2024
@praiskup praiskup reopened this Nov 19, 2024
@praiskup
Copy link
Member

I am reopening because the aarch64 builders don't seem to have this done:

nvme1n1     259:1    0  160G  0 disk 
├─nvme1n1p1 259:5    0   16G  0 part /var/lib/copr-rpmbuild
└─nvme1n1p2 259:6    0  144G  0 part [SWAP]

@praiskup
Copy link
Member

s390x seems better:

vda    252:0    0  320G  0 disk 
├─vda1 252:1    0   16G  0 part /var/lib/copr-rpmbuild
└─vda2 252:2    0  304G  0 part [SWAP]

@praiskup praiskup moved this from Done to Needs triage in CPT Kanban Nov 19, 2024
@nikromen
Copy link
Member

It seems like I forgot to define it for aarch64 in pools.yml or the config file in copr-rpmbuild (but probably the pools.yml)

@nikromen nikromen moved this from Needs triage to In 3 months in CPT Kanban Nov 20, 2024
ryanlerch pushed a commit to fedora-infra/ansible that referenced this issue Nov 27, 2024
@nikromen
copr: dynamically specify builder volume size
612e427 
See fedora-copr/copr#3268
@nikromen nikromen moved this from In 3 months to In Progress in CPT Kanban Dec 2, 2024
@nikromen
Copy link
Member

hm, after the holidays and sick leave it fixed itself? :D

aarch64

NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
nvme0n1 259:0 0 18G 0 disk
├─nvme0n1p1 259:2 0 100M 0 part /boot/efi
├─nvme0n1p2 259:3 0 1000M 0 part /boot
└─nvme0n1p3 259:4 0 16.9G 0 part /var
/home
/
nvme1n1 259:1 0 320G 0 disk
├─nvme1n1p1 259:5 0 16G 0 part /var/lib/copr-rpmbuild
└─nvme1n1p2 259:6 0 304G 0 part [SWAP]

@nikromen nikromen moved this from In Progress to Done in CPT Kanban Jan 20, 2025
@praiskup praiskup reopened this Jan 20, 2025
@github-project-automation github-project-automation bot moved this from Done to Needs triage in CPT Kanban Jan 20, 2025
@praiskup praiskup moved this from Needs triage to In Progress in CPT Kanban Jan 20, 2025
@nikromen
Copy link
Member

ok I missed the non-spot powerful pools. This fixes it: https://pagure.io/fedora-infra/ansible/c/5f10a45efb2f1f03f485ab72fd77950667df08b7?branch=main

@nikromen nikromen moved this from In Progress to Done in CPT Kanban Jan 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@nikromen nikromen

Labels
None yet
Projects
CPT Kanban
Archived in project
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

rpmbuilds: dynamically specify tmp_fs size nikromen/copr
3 participants
@xsuchy @praiskup @nikromen