Proposal: Alternate API representations for resources #123

smarterclayton · 2016-12-01T18:11:57Z

Supports server side handling of transformation of resources.

0xmichalis · 2016-12-23T13:03:56Z

contributors/design-proposals/alternate-api-representations.md

+    which would authorization
+* We wish to support retrieving object representations in multiple schemas - JSON for
+  simple clients and Protobuf for clients concerned with efficiency.
+* Most clients will wish to retrieve the more efficient / simpler object, but for


Is "more efficient" == "simpler"? This seems to contradict with the bullet point above.

0xmichalis · 2016-12-23T13:04:51Z

contributors/design-proposals/alternate-api-representations.md

+there are at least 6 known web-ui or CLI implementations that need to display some
+information about third party resources or additional API groups registered with a server
+without requiring each of them to change.  Providing a server side implementation will
+allow clietns to retrieve meaningful information for the `get` and `describe` style


0xmichalis · 2016-12-23T13:05:23Z

contributors/design-proposals/alternate-api-representations.md

+    to a particular resource in a given namespace should be limited regardless of
+    the representation in use.
+  * Allowing "all namespaces" to be listed would require us to create "fake" resources
+    which would authorization


0xmichalis · 2016-12-23T13:16:39Z

contributors/design-proposals/alternate-api-representations.md

+      }
+    }
+
+    200 OK


Is this section forgotten copy/paste from the GET example above? Or is this supposed to be the reply to the PUT (in which case "replicas" should be 2)?

0xmichalis · 2016-12-23T13:24:10Z

contributors/design-proposals/alternate-api-representations.md

+
+| Name | Value | Default | Description |
+| ---- | ----- | ------- | ----------- |
+| kind | The kind of parameters being sent | `ListOptions` (GET), `DeleteOptions` (DELETE) | The kind of the serialized struct, defaults to ListOptions on GET and DeleteOptions on DELETE. |


We now also have GetOptions. Have we considered merging with ListOptions? Why do we need two separate *Options for listing?

We don't allow filtering of a Get via labels or fields today. It's possible we might in the future. Not required here.

smarterclayton · 2017-02-06T02:41:10Z

Updated with all outstanding feedback and one additional suggestion from @liggitt to have a "shortcut" for common media types, which would show up in swagger (making it easier to discover new types).

Prototype of server side GET is in kubernetes/kubernetes#40848 and helped refine some of the options here. A proposal for server side GET will follow shortly.

soltysh · 2017-02-13T13:36:25Z

contributors/design-proposals/alternate-api-representations.md

+  metadata needed by the generic Garbage Collector or the Namespace Lifecycle Controller
+* Dealing with generic operations like `Scale` correctly from a client across multiple API
+  groups, versions, or servers
+* Return a simple tabular representation of an object or list of objects for naive


From all the use cases described here, this one seems wrong to me. I'm not gonna block this proposal on it, but I find it hard to reason about it.

Maybe we could suggest clients which fields are mandatory and which contains extended data as a sort of annotation in the openapi spec (if possible), instead doing this? I'm worried this will cause us some headaches in the long run.

Another benefit of the approach I'm proposing is the ease of using that information with TPR.

None of our fields are simple. If you look at get describers, the location of that code is arbitrarily in kubectl, but it really belongs on the server. You don't have to use that format, but most clients want it. OpenAPI isn't enough for the use cases we have today, but yes, simple implementations could exist. But having them on the client isn't really helping.

Anyway, comment on the GET proposal.

soltysh · 2017-02-13T13:42:54Z

contributors/design-proposals/alternate-api-representations.md

+we introduce the `api.k8s.io` group with version `v1` that grandfathers all existing resources
+currently described as "unversioned".  A generic client may request that responses be applied
+in this version. The contents of a particular API group version would continue to be bound into
+other group versions (`status.v1.api.k8s.io` would be bound as `Status` into all existing


Why status.v1.api.k8s.io and not status.api.k8s.io/v1? Seems odd to have the version number being part of the group name, or am I missing something.

Just arbitrary representation.

derekwaynecarr · 2017-02-17T03:43:11Z

Is it worth calling out any special behavior if you do a watch?

For example, do tabular formats not repeat column headers?

For things like kubectl --output=wide, is that an entirely different representation and media type I assume?

smarterclayton · 2017-02-17T03:59:30Z

See the get proposal.

soltysh · 2017-02-22T12:19:18Z

I'll move my comments to get proposal, this LGTM.

bgrant0607 · 2017-03-03T20:11:34Z

cc @pwittrock

lavalamp

This seems really close to being ready, sorry it took me so long to look at it.

lavalamp · 2017-03-04T00:10:25Z

contributors/design-proposals/alternate-api-representations.md

+* Return a simple tabular representation of an object or list of objects for naive
+  web or command-line clients to display (for `kubectl get`)
+* Return a simple description of an object that can be displayed in a wide range of clients
+  (for `kubectl describe`)


Do you intend to allow server-side fan out? Or just express links in a way easy for clients to follow.

For the foreseeable future, client side fan out. Maybe someone could write a describe-o-lator that aggregates.

might be good to directly state that.

lavalamp · 2017-03-04T00:11:18Z

contributors/design-proposals/alternate-api-representations.md

+  web or command-line clients to display (for `kubectl get`)
+* Return a simple description of an object that can be displayed in a wide range of clients
+  (for `kubectl describe`)
+* Return the object with fields set by the server cleared (as `kubectl export`)


Defaults are easy, but what about fields set by other system components? Knowing the list of fields to clear seems hard.

Export was described in kubernetes/kubernetes#24855 and initial implementation was done in kubernetes/kubernetes#17483

List of field has to be on the server side and vary by type. We have lots of cases where base export is insufficient (the context of what scope to extract - namespace, cluster, version of Kube needs to be also provided).

lavalamp · 2017-03-04T00:13:07Z

contributors/design-proposals/alternate-api-representations.md

+parameters or media type parameters.
+
+In order to ensure that generic clients can properly deal with many different group versions,
+we introduce the `api.k8s.io` group with version `v1` that grandfathers all existing resources


did this evolve into the meta group?

Yes, will update

lavalamp · 2017-03-04T00:15:58Z

contributors/design-proposals/alternate-api-representations.md

+    which would complicate authorization
+* We wish to support retrieving object representations in multiple schemas - JSON for
+  simple clients and Protobuf for clients concerned with efficiency.
+* Most clients will wish to retrieve a newer format, but for older servers will desire


Just for clarity, in particular, you're not suggesting we remove the version from the URL path for resources, nor that we add it for subresources.

Are you saying that a resource with the version in the path should actually still accept/return multiple versions? It would be really hard to explain that to people.

I think the challenge is something like Scale:

Scale has a server wide version that we version independent of a resource like a pod

Adding v2 pod does not mean that v2 scale is added

The version number in the path applies to the primary object, but all sub resources have no mechanism to control version

I think it would be unusual to say GET /api/v1/namespace/foo/pods/bar -H "Accept: ... as v2". But not unusual to say GET /api/v1/namespace/foo/pods/bar/resources -H "Accept: as resourcesv1.Scale"

OK, we're in violent agreement.

lavalamp · 2017-03-04T00:18:20Z

contributors/design-proposals/alternate-api-representations.md

+| as | Kind name | None | If specified, transform the resource into the following kind (including the group and version parameters). |
+| sv | The server group (`api.k8s.io`) version that should be applied to generic resources returned by this endpoint | Matching server version for the current group and version | If specified, the server should transform generic responses into this version of the server API group. |
+| export | `1` | None | If specified, transform the resource prior to returning to omit defaulted fields. Additional arguments allowed in the query parameter. For legacy reasons, `?export=1` will continue to be supported on the request |
+| pretty | `0`/`1` | `1` | If specified, apply formatting to the returned response that makes the serialization readable (for JSON, use indentation) |


Can you illustrate more combinations of these in the list below?

Or any combinations, actually, since the list below is for shortcuts.

lavalamp · 2017-03-04T00:23:47Z

contributors/design-proposals/alternate-api-representations.md

+| Name | Value | Default | Description |
+| ---- | ----- | ------- | ----------- |
+| kind | The kind of parameters being sent | `ListOptions` (GET), `DeleteOptions` (DELETE) | The kind of the serialized struct, defaults to ListOptions on GET and DeleteOptions on DELETE. |
+| apiVersion | The API version of the parameter struct | `api.k8s.io/v1` | May be altered to match the expected version.  Because we have not yet versioned ListOptions, this is safe to alter. |


These need clearer names. No one is going to expect that these refer to the options rather than the data. I suggest optionsVersion. It's not obvious that we'll need to overload the kind? Maybe we can reserve the appropriate name and say we'll implement it in the future if a use case appears.

Reservation sounds good - I'd like to reserve both to prevent use. This isn't strictly necessary now, but design of body = query equivalence is well established, and i would prefer not to clutter our endpoints with flags and arguments that mutate returned object for as long as we can.

lavalamp · 2017-03-04T00:25:38Z

contributors/design-proposals/alternate-api-representations.md

+   a few limited subsets, and thus retrieving an object into a few known schemas can be made
+   much more efficient. In addition, arbitrary transformation of the object provides
+   opportunities for supporting forward "partial" migration - for instance, returning a
+   ReplicationController as a ReplicaSet to simplify a transition across resource types.


Also it's a bad idea to generalize without a few concrete implementations. We can always add this later, but adding it now would sign us up for a lot of maintenance costs.

Yeah not proposing to add now, just examples.

bgrant0607 · 2017-03-16T20:04:00Z

@erictune Had a suggestion that maybe we could put info into OpenAPI or our discovery API that would enable generic get+describe rendering in kubectl. Computations currently done in the client, such as printPodBase, would still have to move to API fields, of course.

erictune · 2017-03-16T20:27:14Z

We could also have conventions that we document, and clients can implement, like:

for all objects, show metadata.name
if showing multiple namespaces, show metadata.namespace
if the object has a /scale subresource then show replicas as a column.

A couple of things I like about this approach

If we add subresource support to TPRs, then people will get useful columns for their subresources without needing to think about implementing a server side getter.
We should add more subresources like /readiness and /completion, which would partly address Proposal: Introduce Available Pods (MinReadySeconds in the PodSpec) #194. As we add those, then AAs or TPRs that have that subresource get useful columns without writing server side getter code.

bgrant0607 · 2017-03-17T06:05:04Z

Related: kubernetes/kubernetes#34363

In general, many generic clients need to reason about the K8s API at a meta level, not just kubectl.

smarterclayton · 2017-06-03T21:19:30Z

Need to merge this.

lavalamp · 2017-06-04T16:10:52Z

I think I was waiting for a couple updates promised above? :)

smarterclayton · 2017-10-03T00:04:15Z

Updated with all outstanding feedback (mostly just clarifications).

bgrant0607 · 2017-10-03T02:18:01Z

cc @kubernetes/sig-cli-proposals @kubernetes/sig-api-machinery-proposals

castrojo · 2017-10-10T15:53:56Z

This change is

bgrant0607 · 2017-10-11T17:04:46Z

contributors/design-proposals/api-machinery/alternate-api-representations.md

+* ???
+
+
+<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->


The url is wrong.

I think we've given up on analytics for these docs and could just delete this.

bgrant0607 · 2017-10-11T17:05:42Z

contributors/design-proposals/api-machinery/alternate-api-representations.md

+### Old servers
+
+Because old Kubernetes servers are not selective about the content type parameters they
+accept, we may wish to patch the 1.3 and 1.4 server versions to explicitly bypass content


Are we still considering patching older releases? Anyway, we should delete the concrete numbers.

bgrant0607 · 2017-10-11T17:06:58Z

I made a couple minor comments, but think we should merge ASAP and iterate if necessary, since this is already implemented.

Supports server side handling of transformation of resources.

smarterclayton · 2017-10-12T15:30:50Z

Updated

bgrant0607 · 2017-10-12T15:48:17Z

thanks
/lgtm

k8s-github-robot · 2017-10-12T15:53:19Z

Automatic merge from submit-queue.

…on. (kubernetes#123) New draft created per discussion with Fabian and David. Update per feedback from Dan. Removed inactive maintainers per Fabian. Added new maintainers per Fabian and Roman. Signed-off-by: Josh Berkus <[email protected]>

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Dec 1, 2016

smarterclayton mentioned this pull request Dec 1, 2016

Proposal for alternate API representations of resources kubernetes/kubernetes#33900

Closed

liggitt mentioned this pull request Dec 19, 2016

Proposal: kubectl extension #122

Merged

0xmichalis reviewed Dec 23, 2016

View reviewed changes

smarterclayton mentioned this pull request Jan 1, 2017

Implement export functionality server-side for all resources kubernetes/kubernetes#24855

Closed

3 tasks

bgrant0607 assigned lavalamp Jan 18, 2017

smarterclayton force-pushed the serverside branch from d676d40 to 4c03655 Compare February 6, 2017 02:39

This was referenced Feb 13, 2017

Proposal: Server-side kubectl get #363

Merged

Support list/get/watch ObjectMeta in api server kubernetes/kubernetes#30180

Closed

soltysh reviewed Feb 13, 2017

View reviewed changes

lavalamp reviewed Mar 4, 2017

View reviewed changes

smarterclayton mentioned this pull request Jul 25, 2017

kubectl scale implementation for core workload controllers kubernetes/kubernetes#49504

Closed

ironcladlou mentioned this pull request Aug 10, 2017

Use a PartialObjectMetadata based CodecFactory for GC client kubernetes/kubernetes#49718

Closed

k8s-github-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Aug 15, 2017

bgrant0607 mentioned this pull request Aug 22, 2017

Figure out how to handle code in multiple repos kubernetes/kubernetes#24343

Closed

ironcladlou mentioned this pull request Sep 28, 2017

Promote alternate API representations to beta kubernetes/kubernetes#53224

Closed

smarterclayton force-pushed the serverside branch from 4c03655 to 070b680 Compare October 3, 2017 00:11

k8s-ci-robot added sig/cli Categorizes an issue or PR as relevant to SIG CLI. kind/design Categorizes issue or PR as related to design. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. labels Oct 3, 2017

bgrant0607 mentioned this pull request Oct 7, 2017

kubectl plan proposal #1142

Closed

bgrant0607 reviewed Oct 11, 2017

View reviewed changes

Proposal: Alternate API representations for resources

6786b34

Supports server side handling of transformation of resources.

smarterclayton force-pushed the serverside branch from 070b680 to 6786b34 Compare October 12, 2017 15:30

k8s-ci-robot assigned bgrant0607 Oct 12, 2017

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 12, 2017

k8s-github-robot merged commit 65ef42f into kubernetes:master Oct 12, 2017

danehans pushed a commit to danehans/community that referenced this pull request Jul 18, 2023

Updated wrong layout for working-groups table (kubernetes#123)

b351323

+                    }
+                  }
+OK

Proposal: Alternate API representations for resources #123

Proposal: Alternate API representations for resources #123

Conversation

smarterclayton commented Dec 1, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

smarterclayton commented Feb 6, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

derekwaynecarr commented Feb 17, 2017

smarterclayton commented Feb 17, 2017

soltysh commented Feb 22, 2017

bgrant0607 commented Mar 3, 2017

lavalamp left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bgrant0607 commented Mar 16, 2017

erictune commented Mar 16, 2017 • edited Loading

bgrant0607 commented Mar 17, 2017

smarterclayton commented Jun 3, 2017

lavalamp commented Jun 4, 2017

smarterclayton commented Oct 3, 2017

bgrant0607 commented Oct 3, 2017

castrojo commented Oct 10, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bgrant0607 commented Oct 11, 2017

smarterclayton commented Oct 12, 2017

bgrant0607 commented Oct 12, 2017

k8s-github-robot commented Oct 12, 2017

erictune commented Mar 16, 2017 •

edited

Loading