URI templates support

[mpodwysocki]

Uri Templates in TypeSpec

Uri Template Spec

Proposal is to use Uri Templates spec to define encoding, optional parameters, and validation of the input and output of the API.
TypeSpec of course does have its own way of defining part of those (optionality, if a param is a path or query param, etc.) so goal is to unify those.

Reserved expensions Spec

Skipping encoding of certain characters can be done by using + in the param interpolation

@route("{+path}/here") op a(@path path: string): void; // path: /foo/bar ->  route: /foo/bar/here

Equivalent to passing allowReserved: true to @path or @query

Multiple segments

Multiple segments can be specified with the * suffix. By default it should be joined with a comma but a different prefix can be used to specify the separator

@route("blobs/{path*}") op a(@path path: string[]): void; // /blobs/foo,bar
@route("blobs{/path*}") op a(@path path: string[]): void; // /blobs/foo/bar

When using * we should error if the param type is not an array.

The equivalent option would be passing expode: true(same name as openapi3) to @path or @query

Other expansions:

The uri template allows you to specify other ways to expand path and query parameters. Part of this proposal is we support uri template fully. So it means we need equivalent config in TypeSpec.

Path expansion

Style	Explode	Uri Template	Primitive value id = 5	Array id = [3, 4, 5]	Object id = {"role": "admin", "firstName": "Alex"}
simple	false	/users/{id}	/users/5	/users/3,4,5	/users/role,admin,firstName,Alex
simple	true	/users/{id*}	/users/5	/users/3,4,5	/users/role=admin,firstName=Alex
label	false	/users/{.id}	/users/.5	/users/.3,4,5	/users/.role,admin,firstName,Alex
label	true	/users/{.id*}	/users/.5	/users/.3.4.5	/users/.role=admin.firstName=Alex
matrix(path)	true	/users/{;id}	/users/;id=5	/users/;id=3,4,5	/users/;id=role,admin,firstName,Alex
matrix(path)	true	/users/{;id*}	/users/;id=5	/users/;id=3;id=4;id=5	/users/;role=admin;firstName=Alex

Query expansion

Style	Explode	Uri Template	Primitive value id = 5	Array id = [3, 4, 5]	Object id = {"role": "admin", "firstName": "Alex"}
simple	false	/users{?id}	/users?id=5	/users?id=3,4,5	/users?id=role,admin,firstName,Alex
simple	true	/users{?id*}	/users?id=5	/users?id=3&id=4&id=5	/users?role=admin&firstName=Alex

Change to the Http library API

Currently each operation as a path: string property which reference the path. This will remain as it is but a new uriTemplate: string that represent the exact template uri that should be able to be used to generate the uri given all teh path and query parameters.

Example you should be able to do the following given uriTemplateExpander is a spec compliant function that takes a uri template and a map of values and returns the uri

uriTemplateExpander(route.uritemplate, {
  ...pathParametersValues,
  ...queryParametersValues,
});

Examples

TypeSpec	Uri Template
@route("blobs/{path*}") op a(@path path: string[]): void;	/blobs/foo,bar
@route("blobs{/path*}") op a(@path path: string[]): void;	/blobs/foo/bar
@route("blobs{/path*}") op a(@path path: string): void;	/blobs/foo

Things that uri Template don't cover

In the case of explode: false when dealing with arrays or object, openapi2 and openapi3 had some additional styles to serialize those:

• pipeDelimited for arrays ?foo=bar|baz
• spaceDelimited for arrays ?foo=bar baz

In the case of explode: true for query parameters there is also

• deepObject which is /users?id[role]=admin&id[firstName]=Alex

OpenAPI2 also had things that were removed in openapi3:

• tsv tab separated format

Proposal on that

1. Deperecate format: on @query and @header
2. Migrate to some @encode

-op list(@query({format: "ssv"}) id: string[])
+op list(@query @encode(ArrayEncoding.spaceDelimited) id: string[])

@route("blobs{/path*}") op a(@path path: string): void;
@route("blobs") op a(@path({explode: true, style: "path"}) path: string): void;

@route("blobs{/path}") op a(@path({explode: true, style: "path"}) path: string): void;
   ^ error using both uri template and options

Uri template modifier	Typespec option
*	explode: true
+	allowReserved: true
;	style: "matrix"
/	style: "path"
.	style: "label"

@route("blobs{?filter*}") op a(@query filter: string[]): void;
@route("blobs") op a(@query({explode: true}) filter: string[]): void;
// uriTemplate: blobs{?filter*}

[timotheeguerin]

From the original design we had here we seemed to have decided the following but it might be good to verify this is the way to go as it was only partially implemented (url part)

Revised Proposal

• Per RFC 3986 model a type similar to relative-reference that represents fully-encoded url paths in cadl/http.

@format("uri")
model urlPath is string;

Relative and absolute paths

• The spec describes 3 types of relative-references:
  • Absolute paths, starting with '/'
  • Relative paths, not starting with '/'
  • Network paths (disused), starting with '//'

The proposal is to use a single type to represent the common absolute and relative paths, as this provides all necessary information about encoding.

An alternate idea is to provide individual types for relative and absolute paths, but this would largely be for documentary purposes, as we would expect any processor to correctly handle leading and trailing slashes.

Emitters would represent relative and absolute url paths, consistent with the relative-reference concept, producing the following default behavior

Case	Code	Behavior
1. string in path	op simple(@path foo: string)	Encode foo
2. urlPath in path	op simple(@path foo: urlPath)	Don't encode
3. uri in path	op simple(@path foo: uri)	encode reserved characters
4. string in query	op simple(@query foo: string)	Encode foo
5. urlPath in query	op simple(@query foo: urlPath)	Don't encode
6. uri in query	op simple(@query foo: uri)	Don't encode*
7. Path Param is an array of string	op simple(@path foos: string[])	Encode each foo and join with /.

• for backward compatibility with older processors, it may be desirable to encode '/' and '?' in query parameters

Note that here, we are discounting use of the 'network-path' option for relative-path (paths starting with //), as the spec indicates that this is a disused pattern, and we have never seen it. However, we could add a networkPath type in the future, if needed.

But with the following changes

uri -> url
pathUrl -> relativeUrl

[lmazuel]

Feels like the same than Azure/typespec-azure#1022

[timotheeguerin]

yeah that seems to be an additional use for it

[timotheeguerin]

Summarizing the previous proposal with the new names and what we already have for clarity

Add 2 new types:

• url - a string that represents a URL (e.g. https://example.com/foo/bar)
• relativeUrl - a string that represents a relative URL (e.g. foo/bar) Spec

Case	Code	Behavior	Status
1. string in path	op simple(@path foo: string)	Encode foo	✅
2. relativeUrl in path	op simple(@path foo: relativeUrl)	Don't encode	❌
3. url in path	op simple(@path foo: url)	encode reserved characters	✅
4. string in query	op simple(@query foo: string)	Encode foo	✅
5. relativeUrl in query	op simple(@query foo: relativeUrl)	Don't encode	❌
6. url in query	op simple(@query foo: url)	Don't encode*	?
7. Path Param is an array of string	op simple(@path foos: string[])	Encode each foo and join	?

[timotheeguerin]

Concern back from this original proposal that we a mixing the type with the encoding here. There is also the issue that union of encoded and non encoded type are ambigious

op read(@path param: string | relativeUrl): void;

do we encode or not above?

[timotheeguerin]

Uri Templates in TypeSpec

Uri Template Spec

Proposal is to use Uri Templates spec to define encoding, optional parameters, and validation of the input and output of the API.
TypeSpec of course does have its own way of defining part of those (optionality, if a param is a path or query param, etc.) so goal is to unify those.

Reserved expensions Spec

Skipping encoding of certain characters can be done by using + in the param interpolation

@route("{+path}/here") op a(@path path: string): void; // path: /foo/bar ->  route: /foo/bar/here

Equivalent to passing allowReserved: true to @path or @query

Multiple segments

Multiple segments can be specified with the * suffix. By default it should be joined with a comma but a different prefix can be used to specify the separator

@route("blobs{path*}") op a(@path path: string[]): void; // /blobs/foo,bar
@route("blobs{/path*}") op a(@path path: string[]): void; // /blobs/foo/bar

When using * we should error if the param type is not an array.

The equivalent option would be passing expode: true(same name as openapi3) to @path or @query

Other expansions:

The uri template allows you to specify other ways to expand path and query parameters. Part of this proposal is we support uri template fully. So it means we need equivalent config in TypeSpec.

Path expansion

Style	Explode	Uri Template	Primitive value id = 5	Array id = [3, 4, 5]	Object id = {"role": "admin", "firstName": "Alex"}
simple	false	/users/{id}	/users/5	/users/3,4,5
simple	true	/users/{id*}	n/a (error)	/users/3,4,5
label	false	/users/{id}	/users/.5	/users/.3,4,5
label	true	/users/{.id*}	n/a (error)	/users/.3.4.5
matrix(path)	true	/users/{;id}	/users/;id=5	/users/;id=3,4,5
matrix(path)	true	/users/{;id*}	n/a (error)	/users/;id=3;id=4;id=5

Query expansion

Style	Explode	Uri Template	Primitive value id = 5	Array id = [3, 4, 5]	Object id = {"role": "admin", "firstName": "Alex"}
simple	false	/users{?id}	/users?id=5	/users?id=3,4,5
simple	true	/users{?id*}	n/a (error)	/users?id=3&id=4&id=5

Change to the Http library API

Currently each operation as a path: string property which reference the path. This will remain as it is but a new uriTemplate: string that represent the exact template uri that should be able to be used to generate the uri given all teh path and query parameters.

Example you should be able to do the following given uriTemplateExpander is a spec compliant function that takes a uri template and a map of values and returns the uri

uriTemplateExpander(route.uritemplate, {
  ...pathParametersValues,
  ...queryParametersValues,
});

Examples

TypeSpec	Uri Template
@route("blobs{path*}") op a(@path path: string[]): void;	/blobs/foo,bar
@route("blobs{/path*}") op a(@path path: string[]): void;	/blobs/foo/bar
@route("blobs{/path*}") op a(@path path: string): void;	error: expect an array

[qiaozha]

I wonder if the above design can also resolve this issue #2476 ? though the last column is empty as shown in the table.

[markcowl]

@markcowl discuss in DPG meeting 7/18

[ArthurMa1978]

Loop @archerzz to verify if System.Net.Url can handle this.

[archerzz]

Loop @archerzz to verify if System.Net.Url can handle this.

System.Net package doesn't support URI template spec (neither Uri or UriBuilder). However, there are a few 3rd party libraries. The most popular and actively maintained one is https://github.com/std-uritemplate/std-uritemplate

There is an old System.UriTemplate class which only exists in .NET framework and it's more like a string template specific for URI, instead of a full implementation of RFC 6570: https://learn.microsoft.com/en-us/dotnet/api/system.uritemplate?view=netframework-4.8.1

[tadelesh]

For Go, the build in modules does not support URI template proposed in this issue and Go also has quite a high bar to use a 3rd-party module. cc: @ArthurMa1978 it may take some time for Go to fully support this.

[timotheeguerin]

For some info I implemented a uri parser here and looking at other libraries in JS it is quite simple to implement yourself with regex. If we were to define a good test suite I don't think this is something that would be hard to have every generator be able to do themself.
Also there is various level of support you could do that I think will have various amount of priority. The {+path} one is the highest and you could definitely treat that one specially.