Inventory module site and prefix data extended

[Eric-Sim]

This PR satisfies (Issue #640) without causing any changes to default functionality.

Both the extension of Site Data and Prefix Data are enabled by additional parameters passed to the module.
site_data: (boolean)
prefixes: (boolean)
These changes follow the same methodology as the interfaces option.

No changes in the default behavior of the module in order to guarantee legacy applications will continue working as intended.

The site_data parameter will cause the entire data structure returned by the /api/dcim/sites API call to be captured and inserted into hostvars appropriately for each device in inventory, instead of just the slug that is collected currently.

The prefixes option extends this section of the hostvars further by nesting the datastructure of the prefix assigned to the site as well. This is done by a calling an additional endpoint /api/ipam/prefixes.
Current optimizations are maintained with the "fetch_all" option.
Only ACTIVE prefixes, that are assigned to a site are pulled; and only for the sites that have at least one queried device assigned to them.

[Eric-Sim]

Please link to issue #640

[sc68cal]

Some thoughts

[sc68cal]

approved a run through CI

[Eric-Sim]

Looks like 2 of the tasks are issues with the CI pipeline itself, but the other 2 are black complaining.
I allowed this formatting in 9ff30f9, but ended up undoing it when reviewing with sc68cal.

Please let me know if I need to allow black to reformat.
(the changes it wants to make do not help readability)

[sc68cal]

I think we need to tweak our black settings, because I too do not care for that formatting that black is insisting on

[sc68cal]

@rodvand Can you take a look at this? The problem is that black was complaining about a line that was not originally part of his patchset, and in the process of doing this PR he changed this line but frankly I don't think it was a related change.

Does he need to rebase?

[rodvand]

Not quite sure why this happens. @Eric-Sim Do you run black locally to format the code? But yeah, try to rebase based on devel. @sc68cal The latest black requires python 3.6.2 so we need to up our requirements before we upgrade our black version. When that's done it shouldn't be a big deal upgrading. Did a test run with a newer black and it only changed a few of the files, so not a massive re-format.

[Eric-Sim]

I had run black locally to format the code. And it did apply the same formatting that the project CI/CD pipeline is trying to apply. The problem was what @sc68cal had pointed out, it ends up changing things that are outside the scope of what I revised; so I reverted those.

I will rebase from devel tonight.

[Eric-Sim]

My branch is up to date. No new commits in Devel that I can rebase into my branch.

I have re-run black locally and committed its formatting changes.

[sc68cal]

OK, something must be going on with your local black because it is failing on those lines in CI.

[Eric-Sim]

Sorry for the slow update.
I went ahead and manually matched the changes the the project's black wanted, and pushed.

[sc68cal]

This looks good to me, probably this needs to be a squash & merge operation

[bluikko]

What about the functionality for virtualization app? This only does it for dcim?

For users who have both physical and virtual infrastructure a feature parity between the 2 apps would really be needed - or everything must be done for the lowest common denominator (i.e. if some feature is only for dcim but not virtualization then it could not be used).

[Eric-Sim]

What about the functionality for virtualization app? This only does it for dcim?

For users who have both physical and virtual infrastructure a feature parity between the 2 apps would really be needed - or everything must be done for the lowest common denominator (i.e. if some feature is only for dcim but not virtualization then it could not be used).

Hey @bluikko, I can look back through when I get a chance.
Off the top of my head, I don't remember seeing anything that would cause a virtual device to operate differently than a physical one.

[k-y]

I've got the site object getting populated correctly into hostvars and see a single prefix in site, however, only the 1st prefix for each site seems to be getting populated. All of our sites have prefix_count > 1 but I don't see the rest of the prefixes.

e.g.

        "site": {                                                                 
...
            "prefix": {    
                "created": "2021-09-16",
...
            },
            "prefix_count": 23,

[rodvand]

@k-y Could you open an issue with enough data to reproduce?

[Eric-Sim]

I've spoken with them and I'll open the ticket. ...and fix it. :)

…

On Sun, Feb 6, 2022, 13:31 Martin Rødvand ***@***.***> wrote: @k-y <https://github.com/k-y> Could you open an issue with enough data to reproduce? — Reply to this email directly, view it on GitHub <#646 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AWKJUMY2VD2L5P5QG4SBRJLUZ3SDHANCNFSM5HT6VKTA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.***>