Skip to content
This repository has been archived by the owner on Jul 30, 2020. It is now read-only.

Lockfile Support #173

Closed
pgassmann opened this issue Nov 21, 2018 · 4 comments · Fixed by #271
Closed

Lockfile Support #173

pgassmann opened this issue Nov 21, 2018 · 4 comments · Fixed by #271

Comments

@pgassmann
Copy link

Original issue is abandonded in archived galaxy-issues project
The related issue about versioning of roles is migrated: ansible/galaxy#83
I recreated this issue in ansible-galaxy and now here.

Original issue by @geerlingguy

It would be helpful if Ansible Galaxy supported a 'lock' file.

When running ansible-galaxy install -r requirements.yml, Ansible Galaxy could create a requirements.lock file (or some other name) that stores the following metadata for each downloaded role:

  • Role name
  • Role version or branch identifier used
  • Specific git tag or hash downloaded

Then, if running the command ansible-galaxy install -r requirements.yml again, it would install based on the exact identifiers in the lock file.

The details could be a little different, of course, and we'd need a command like ansible-galaxy update -r requirements.yml to update an existing lockfile... but this would make idempotent playbook configurations much easier for many projects without requiring all downloaded roles be stored in the project repository (e.g. like Drupal VM currently does).

Other examples of lock file support:

Original Request

For Drupal VM, we've decided to pin our requirements.yml file's role requirements to specific versions (e.g. 3.0.1, 2.5.0, etc.).

It would be nice if we could support some method of semver with role versions (instead of just requiring a branch or tag... and maybe this is related to/duplicate of #70), where you could specify a dependency in requirements.yml like:

- src: geerlingguy.apache
  version: 1.7.*

When galaxy picks the tag to pull, it would check for any in the 1.7.x range, and choose the latest release. That way I don't have to spend so much time maintaining all my requirements files for various projects.

I would love to see something as involved as Composer's Version support, but I know that's a bit of a moonshot. It would be nice to have a lock file that would set the versions currently installed, too, as that would even allow me to set master but lock in the versions at a specific point in time.

Anyways, I was just thinking about how awesome it is to point at tagged role releases in terms of keeping one's playbook completely idempotent and stable—yet how annoying it is to have to manually tinker with the requirements.yml file so often.

@pgassmann
Copy link
Author

pgassmann commented Nov 21, 2018

For Puppet and Chef, there is similar tooling available and there's already support in Pulp for puppet modules:

A useful function is librarian-puppet outdated which shows the difference between the versions in the lock file and the latest version on the forge (would be galaxy)

$ librarian-puppet outdated outdated
puppet-extlib (2.3.1 -> 3.0.0)
puppet-gitlab (2.1.0 -> 3.0.2)
puppet-logrotate (3.3.0 -> 3.4.0)
puppet-mongodb (2.3.0 -> 2.4.1)
puppet-puppetboard (4.1.0 -> 5.0.0)
puppetlabs-apache (3.3.0 -> 3.4.0)
puppetlabs-apt (2.4.0 -> 6.2.1)
puppetlabs-concat (4.2.1 -> 5.1.0)
puppetlabs-inifile (2.3.0 -> 2.4.0)
puppetlabs-mysql (6.1.0 -> 7.0.0)
puppetlabs-ntp (7.2.0 -> 7.3.0)
puppetlabs-postgresql (5.10.0 -> 5.11.0)
puppetlabs-puppetdb (7.0.1 -> 7.1.0)
puppetlabs-stdlib (4.25.1 -> 5.1.0)
puppetlabs-tomcat (2.3.0 -> 2.4.0)
puppetlabs-vcsrepo (2.3.0 -> 2.4.0)
thbe-ssmtp (0.5.12 -> 0.6.0)
theforeman-dhcp (4.1.1 -> 4.2.0)
theforeman-dns (5.3.0 -> 5.3.1)
theforeman-foreman (9.2.0 -> 10.0.0)
theforeman-foreman_proxy (8.0.2 -> 9.0.0)
theforeman-tftp (4.2.0 -> 4.3.0)
unibet-forge_server (0.4.0 -> 0.5.0)

Then you can update individual modules with
$ librarian-puppet update thbe-ssmtp theforeman-tftp --verbose

@geerlingguy
Copy link

Just noting that of all the questions I get in my day-to-day Ansible work, this is one that comes up quite often ("does Ansible Galaxy have any concept of a lock file?"). I'm assuming with mazer and the new depsolver, it would be a lot easier to implement this sort of functionality.

@tima
Copy link
Contributor

tima commented Jan 23, 2019

You are correct @geerlingguy. It does not. ansible-galaxy was never designed to be a package manager so it is certain to be lacking when compared to tools in that space. ansible-galaxy is not being further developed -- we have no intention of adding new features to it -- just critical bug fixes and basic maintenance that we have no intention of adding that to ansible-galaxy.

Mazer is intended to be a content package manager and will need a lock file mechanism amongst other features.

@alikins
Copy link
Contributor

alikins commented May 21, 2019

There may be a very limited subset of this added for 1.0.

Planning on adding something similar to ansible-galaxy's -r/--role-file support.

Probably something along the lines of a cli flag to point to a yaml file that has a dictionary
like the 'dependencies' field of galaxy.yml. ie, a dict mapping collection name to a version spec,
{'alikins.collection_inspect': '>=1.0.0', 'alikins.stuff': '2.3.4'}

But I think that should cover a lot of the use cases.

@alikins alikins added status/in-progress and removed status/new Triage required labels May 21, 2019
@alikins alikins added this to the Mazer 1.0.0 milestone May 21, 2019
@alikins alikins self-assigned this May 21, 2019
alikins added a commit to alikins/mazer that referenced this issue May 28, 2019
Add support for 'collection lockfiles'.
A collections lock specifies a set of collections to install
and the version specs to match.

Fixes ansible#173

Add collections_lock and collections_lockfile models.

Add ansible_galaxy.collections_lockfile with a yaml loader
for a collections lockfile.

Update cli and install action to use the collections lockfile.

The contents of  collections lock file is a yaml file, containing a dictionary.
The dictitionary is the same format as the 'dependencies' dict in
galaxy.yml.

ie, The keys are collection labels (the namespace and name
dot separated ala 'alikins.collection_inspect').

The values is a version spec string. ie "*" or "==1.0.0".

Example contents of a collections lock file:

alikins.collection_inspect: "1.0.0"
alikins.collection_ntp: ">0.0.1,!=0.0.2"
alikins added a commit to alikins/mazer that referenced this issue May 28, 2019
Add support for 'collection lockfiles'.
A collections lock specifies a set of collections to install
and the version specs to match.

Fixes ansible#173

Add collections_lock and collections_lockfile models.

Add ansible_galaxy.collections_lockfile with a yaml loader
for a collections lockfile.

Update cli and install action to use the collections lockfile.

The contents of  collections lock file is a yaml file, containing a dictionary.
The dictitionary is the same format as the 'dependencies' dict in
galaxy.yml.

ie, The keys are collection labels (the namespace and name
dot separated ala 'alikins.collection_inspect').

The values is a version spec string. ie "*" or "==1.0.0".

Example contents of a collections lock file:

alikins.collection_inspect: "1.0.0"
alikins.collection_ntp: ">0.0.1,!=0.0.2"
alikins added a commit to alikins/mazer that referenced this issue May 30, 2019
Add support for 'collection lockfiles'.
Add 'list --lockfile' to output a collections lockfile to stdout.

A collections lock specifies a set of collections to install
and the version specs to match.

Fixes ansible#173

Add collections_lock and collections_lockfile models.

Add ansible_galaxy.collections_lockfile with a yaml loader
for a collections lockfile.

Update cli and install action to use the collections lockfile.

Update README.md with lockfile info

To reproduce an existing installed collection path, use like:

  mazer list --lockfile  > collections_lockfile.yml
  mazer install --collections-path /tmp/somenewplace --collections-lock collections_lockfile.yml

To create a lockfile that matches current versions exactly, add
the '--frozen' flag:

  mazer list --lockfile --frozen

Example output for `mazer list --lockfile`:

alikins.collection_inspect: '*'
example.foobar: '*'

Example output for `mazer list --lockfile --frozen`:

alikins.collection_inspect: ==1.2.3
example.foobar: ==0.0.9

The contents of  collections lock file is a yaml file,
containing a dictionary. The dictitionary is the same
format as the 'dependencies' dict in galaxy.yml.

ie, The keys are collection labels (the namespace and name
dot separated ala 'alikins.collection_inspect').

The values is a version spec string. ie "*" or "==1.0.0".

Example contents of a collections lock file:

alikins.collection_inspect: "1.0.0"
alikins.collection_ntp: ">0.0.1,!=0.0.2"
alikins added a commit that referenced this issue May 30, 2019
Add support for 'collection lockfiles'.
Add 'list --lockfile' to output a collections lockfile to stdout.

A collections lock specifies a set of collections to install
and the version specs to match.

Fixes #173

Add collections_lock and collections_lockfile models.

Add ansible_galaxy.collections_lockfile with a yaml loader
for a collections lockfile.

Update cli and install action to use the collections lockfile.

Update README.md with lockfile info

To reproduce an existing installed collection path, use like:

  mazer list --lockfile  > collections_lockfile.yml
  mazer install --collections-path /tmp/somenewplace --collections-lock collections_lockfile.yml

To create a lockfile that matches current versions exactly, add
the '--frozen' flag:

  mazer list --lockfile --frozen

Example output for `mazer list --lockfile`:

alikins.collection_inspect: '*'
example.foobar: '*'

Example output for `mazer list --lockfile --frozen`:

alikins.collection_inspect: ==1.2.3
example.foobar: ==0.0.9

The contents of  collections lock file is a yaml file,
containing a dictionary. The dictitionary is the same
format as the 'dependencies' dict in galaxy.yml.

ie, The keys are collection labels (the namespace and name
dot separated ala 'alikins.collection_inspect').

The values is a version spec string. ie "*" or "==1.0.0".

Example contents of a collections lock file:

alikins.collection_inspect: "1.0.0"
alikins.collection_ntp: ">0.0.1,!=0.0.2"
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants