Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Increase the default of pool_size from 1 to something reasonable #1038

Closed
dhollinger opened this issue Mar 11, 2020 · 2 comments
Closed

Increase the default of pool_size from 1 to something reasonable #1038

dhollinger opened this issue Mar 11, 2020 · 2 comments
Labels

Comments

@dhollinger
Copy link

No description provided.

@baurmatt
Copy link
Contributor

I've did some testing a couple of months ago with our setup:

cat > r10k.yaml << EOF
---
:cachedir: ${cachedir}
:sources:
  syseleven:
    basedir: ${basedir}
    remote: [email protected]:puppet/control.git

:deploy:
  purge_whitelist:
  - resource_types

:forge:
  baseurl: https://forgeapi.example.org

:pool_size: xxxx
EOF

for i in {1..15}; do
  rm -rf ${cachedir}/* ${basedir}/*
  echo -n "Testing pool_size ${i}: "
  gsed -Ei "s,:pool_size:.*,:pool_size: ${i},g" "${PWD}/r10k.yaml"
  { time r10k deploy environment -p production >/dev/null; } 2>&1 | awk '/real/{print $2}'
done
$ bash test_pool_size_performance.sh
Testing pool_size 1: 1m21.416s
Testing pool_size 2: 0m45.172s
Testing pool_size 3: 0m33.896s
Testing pool_size 4: 0m27.042s
Testing pool_size 5: 0m24.344s
Testing pool_size 6: 0m26.082s
Testing pool_size 7: 0m25.882s
Testing pool_size 8: 0m25.277s
Testing pool_size 9: 0m28.250s
Testing pool_size 10: 0m31.092s
Testing pool_size 11: 0m34.072s
Testing pool_size 12: 0m35.007s
Testing pool_size 13: 0m34.627s
Testing pool_size 14: 0m30.623s
Testing pool_size 15: 0m27.948s

This was run on a MacBook Pro 2017 with 2,3 GHz Dual-Core Intel Core i5 and 16GB RAM.

We go with pool_size: 5 for now because there doesn't seem a real performance win for higher values.

npwalker added a commit to npwalker/r10k that referenced this issue Apr 15, 2020
Solves puppetlabs#1038.  pool_size was added as an optional setting
a while ago and people have been using it successfully
in the wild.  This commit increases the default to 4
which seems like a generally useful improvement.
Magisus added a commit to Magisus/r10k that referenced this issue Jul 27, 2020
This commit changes the default number of threads used when downloading
modules from 1 to 4.
Magisus added a commit to Magisus/r10k that referenced this issue Jul 27, 2020
This commit changes the default number of threads used when downloading
modules from 1 to 4.
Magisus added a commit to Magisus/r10k that referenced this issue Jul 28, 2020
This commit changes the default number of threads used when downloading
modules from 1 to 4.
@github-actions
Copy link

This issue has been marked stale because it has had no activity for 60 days. The Puppet Team is actively prioritizing existing bugs and new features, if this issue is still important to you please comment and we will add this to our backlog to complete. Otherwise, it will be closed in 7 days.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants