Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

deis doctor includes lots of k8s data #81

Merged
merged 4 commits into from
Aug 2, 2016

Conversation

jackfrancis
Copy link
Member

@jackfrancis jackfrancis commented Jul 12, 2016

This PR delivers the following feature additions:

  • Deis doctor handler introspects the configured "deis" kubernetes namespace and generates a JSON-body POST request to the configured deis doctor API endpoint
  • Kuberenetes data gathered and reported by deis doctor includes:
    • nodes
    • pods (namespaced to "deis")
    • daemonSets (namespaced to "deis")
    • deployments (namespaced to "deis")
    • events (namespaced to "deis")
    • replicaSets (namespaced to "deis")
    • replicationControllers (namespaced to "deis")
    • services (namespaced to "deis")
  • Kubernetes data is gathered au totale by way of a swagger model property that eagerly JSON-encodes the entire source kubernetes resource data inside a data property. This makes for a predictably heavy-weight data payload, and in the future it may make sense to include some filtering. That, and any additional explicit properties data modeling is TODO.

Included in the implementation to support these features are the following under-the-hood items worth emphasizing:

  • A new k8s sub-package was created to accommodate all new business logic responsible for gathering and preparing kubernetes cluster data; pre-existing kubernetes-specific implementations that were formerly in the data package were moved to k8s, as appropriate (e.g., the data.getRCItems() helper function becomes the exportable k8s.GetReplicationControllers().
  • To support the new kubernetes-specific interfaces needed, the handy kubeapp dependency was updated, and in fact github.com/deis absorbed what was formerly @arschles's personal creation.
  • glide.yaml and glide.lock updated to reflect updated kubeapp dependencies
  • Zero-value k8s data mocks added to support very basic contract-abiding test interfaces, but actual mock data is TODO in a follow-up PR.
  • Resource-specific interfaces added as light wrappers to aid the passing around of data and data specifications. In particular, the k8s.ResourceInterface is meant, with the help of its constructor, to conveniently provide a re-usable kubernetes-namespace-specific, per-resource data retrieval singleton.
  • New, swagger-generated client stubs to support deis doctor k8s data.

@jackfrancis jackfrancis added this to the v2.2 milestone Jul 12, 2016
@jackfrancis jackfrancis self-assigned this Jul 12, 2016
@jackfrancis jackfrancis force-pushed the deis-doctor-k8s-data branch 3 times, most recently from ebbc6ba to 2c73dff Compare July 14, 2016 01:41
@jackfrancis jackfrancis changed the title node implementation complete deis doctor includes lots of k8s data Jul 14, 2016
@jackfrancis jackfrancis force-pushed the deis-doctor-k8s-data branch 10 times, most recently from e27ace9 to c838ebe Compare July 15, 2016 23:25
@arschles
Copy link
Member

moving out of the 2.2 milestone

@arschles arschles removed this from the v2.2 milestone Jul 19, 2016
@jackfrancis jackfrancis force-pushed the deis-doctor-k8s-data branch from c838ebe to 34399f0 Compare July 19, 2016 17:43
@codecov-io
Copy link

codecov-io commented Jul 19, 2016

Current coverage is 31.19% (diff: 38.15%)

Merging #81 into master will increase coverage by 5.25%

@@             master        #81   diff @@
==========================================
  Files             9          8     -1   
  Lines           343        359    +16   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits             89        112    +23   
+ Misses          235        220    -15   
- Partials         19         27     +8   

Powered by Codecov. Last update e010f7a...73bd54f

@jackfrancis jackfrancis force-pushed the deis-doctor-k8s-data branch 5 times, most recently from bb4d580 to 8bfad59 Compare July 20, 2016 00:43
@@ -31,7 +32,7 @@ func GetCluster(
}
err = AddUpdateData(&cluster, v, secretGetterCreator)
if err != nil {
return models.Cluster{}, err
log.Println("unable to decorate cluster data with available updates data")
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When we're unable to add definitive "available component update" data we'll log the event, but we don't want to short-circuit the entire version data-gathering process.

@jackfrancis jackfrancis force-pushed the deis-doctor-k8s-data branch from 8bfad59 to 8045f8c Compare July 20, 2016 01:15
@@ -0,0 +1 @@
package data
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: delete this placeholder file (or actually add tests).

@jackfrancis jackfrancis force-pushed the deis-doctor-k8s-data branch from 8045f8c to f2bb03a Compare July 22, 2016 22:46
}
]
}]`))
_ = json.Unmarshal(data, &componentVersions)
Copy link
Member

@arschles arschles Jul 29, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you should either return this error or just construct the []models.ComponentVersion directly (instead of going through the json decoder) and return it

@jackfrancis jackfrancis added this to the v2.3 milestone Aug 1, 2016
@@ -20,7 +20,10 @@ type testAvailableVersions struct{}
func (a testAvailableVersions) Refresh(cluster models.Cluster) ([]models.ComponentVersion, error) {
data := getMockComponentVersions()
var componentVersions []models.ComponentVersion
_ = json.Unmarshal(data, &componentVersions)
err := json.Unmarshal(data, &componentVersions)
if err != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

combine these lines. if err := json.Unmarshal(...); err != nil { ...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same with the below combinations

err = AddUpdateData(&cluster, v)
if err != nil {
log.Printf("unable to decorate cluster data with available updates data: %#v\n", err)
if err = AddUpdateData(&cluster, v); err != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if err := AddUpdateData.... note the : in the if

@arschles arschles added the LGTM1 label Aug 2, 2016
@mboersma
Copy link
Member

mboersma commented Aug 2, 2016

I tested this by putting it into the workflow-dev chart. Resulted in a huge JSON blob full of the above information--good stuff!

@jackfrancis jackfrancis merged commit 84206b2 into deis:master Aug 2, 2016
@jackfrancis jackfrancis deleted the deis-doctor-k8s-data branch August 2, 2016 19:14
@jackfrancis
Copy link
Member Author

closes #79

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants