Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

InventoryCollection definition #316

Merged

Conversation

slemrmartin
Copy link
Contributor

Issue ManageIQ/manageiq#17396

Description of ManagerRefresh::Inventory::Persister#add_collection() and builder classes for definition of InventoryCollection

Cc @agrare

@@ -264,6 +264,15 @@ The logic in this class has
quite strong assumptions on the data being stored. It assumes that it has the tree structure and each entity contains its kids as a nested hash.
If you are able to achieve that structure in the `refresh_parser.rb`, you are halfway done. Otherwise, good luck :]

### Defining Inventory

Basically, refresh process consists of Collector, Persister and Parser classes (in most providers).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't need Basically, and I would suggest instead of (in most providers) how about Providers which use graph refresh consist of Collector ...

### Defining Inventory

Basically, refresh process consists of Collector, Persister and Parser classes (in most providers).
When Collector loads data from external provider (like AWS), Persister defines structure of Inventory by InventoryCollection objects.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be When the Collector, not sure (like AWS) adds anything I think any developer reading this knows what a provider is, how about the Persister defines the inventory structure based on InventoryCollection objects

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, I wrote like AWS because it seems there is unclear terminology, what provider is.
One time provider is EMS in our database, another time AWS/Azure/etc. is meant.


Basically, refresh process consists of Collector, Persister and Parser classes (in most providers).
When Collector loads data from external provider (like AWS), Persister defines structure of Inventory by InventoryCollection objects.
Inventory is then saved to VMDB (ManageIQ (also CFME) app database). Each inventory belons to certain ActiveRecord model.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe just Inventory is then saved to the database?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Each inventory belons

typo should be belongs

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about Every inventory collection is mapped to an ActiveRecord model?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wrote VMDB intentionally because all documentations uses VMDB when talking about our database.
So I rather write both / VMDB and app database) because at the beginning I was confused what vmdb is.
I think there is no other place in this doc where VMDB term is explained

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also relation between ManageIQ and CFME is explained nowhere (this is not the best place but at least something for new developer)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

others 👍

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also relation between ManageIQ and CFME is explained nowhere

Actually this is intentional, these are upstream docs/guides and should make no mention of CFME

Basically, refresh process consists of Collector, Persister and Parser classes (in most providers).
When Collector loads data from external provider (like AWS), Persister defines structure of Inventory by InventoryCollection objects.
Inventory is then saved to VMDB (ManageIQ (also CFME) app database). Each inventory belons to certain ActiveRecord model.
Parser maps collected data to these inventories.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about Parser maps data collected from the provider to the common format defined by the ActiveRecord model or something like that?


[back](../dev-guide.md)

**Inventory Collection** (```ManagerRefresh::InventoryCollection``` - next called **"IC"**) is an object used by Refresh process.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

next called "IC"

Subsequently called "IC" sounds a bit better, next sounds like just one time after

[back](../dev-guide.md)

**Inventory Collection** (```ManagerRefresh::InventoryCollection``` - next called **"IC"**) is an object used by Refresh process.
It represents properties of inventory like **VM**, **Hardware**, **Operating system**, **Host** etc.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It represents properties of inventory like

How about It represents types of inventory such as ...? When I think of properties of inventory I think of e.g. cpus/memory not whole classes of objects

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean IC defines properties of inventory. As an inventory I mean VM, HW etc.
--> my lng skills :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haha no worries :)

**Inventory Collection** (```ManagerRefresh::InventoryCollection``` - next called **"IC"**) is an object used by Refresh process.
It represents properties of inventory like **VM**, **Hardware**, **Operating system**, **Host** etc.

Each IC is defined by persister (```ManagerRefresh::Inventory::Persister```), stored in it (in hash ```@collections```) and accessible as persister's method. Its name equals IC's association property (will be described later). One persister usually contains many IC definitions.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

stored in it (in hash

Think they are stored in a hash called @collections is better

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its name equals IC's

should be It's and is equal to the IC's


Each IC is defined by persister (```ManagerRefresh::Inventory::Persister```), stored in it (in hash ```@collections```) and accessible as persister's method. Its name equals IC's association property (will be described later). One persister usually contains many IC definitions.

## Defining IC
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about Defining an Inventory Collection

- Common definitions
- Definitions of ICs shared accross providers divided by provider's type (cloud/infra etc.)
- Provider specific definitions
- Recommendations how to structure definitions specifical to provider and type of refresh
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 nice summary


It creates ```ManagerRefresh::InventoryCollection``` and assigns it to **persister's @collections[:vms]**

Method can throw exception **`ManagerRefresh::InventoryCollection::Builder::MissingModelClassError`**
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe throw in a quick explanation about why you might get a missing model class error and how to fix it?


#### - builder_class [*mandatory*]

There are 5 builder classes which groups ICs of the same manager(**TODO**, right?). Because they're used as parameter of ```add_collection()``` they are wrapped to methods with short name (persister's methods):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe be better to say "there is a builder class for every manager type, e.g. cloud, infra, network, ...

@slemrmartin
Copy link
Contributor Author

Cc @martinpovolny please merge after @agrare approves


## Conclusion

Definition of InventoryCollection objects in persisters ensures ```add_collection()``` method.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't tell what this sentence means, it sounds like the inventory collection definition does something to the add_collection method but I'm pretty sure it is the other way around right? add_collection adds the ICs to the persister.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I mean what you wrote, of course :))

@martinpovolny
Copy link
Member

@slemrmartin : waiting for you to resolve latest @agrare 's comments.

👍 And THANK YOU for writing the docs 👍 !

Copy link
Member

@agrare agrare left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍 thanks @slemrmartin this is awesome!

@martinpovolny martinpovolny merged commit b733567 into ManageIQ:master Jun 29, 2018
@slemrmartin slemrmartin deleted the inventory-collection-builder branch July 24, 2018 08:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants