From 612cd0ab086acf0fb65d4f55c1848b0542aa28d7 Mon Sep 17 00:00:00 2001 From: Richard van der Hoff Date: Tue, 26 Jul 2022 19:18:15 +0100 Subject: [PATCH 1/4] Clarify the use of JSON in requests and responses --- .../newsfragments/1185.clarification | 1 + .../newsfragments/1185.clarification | 1 + .../newsfragments/1185.clarification | 1 + content/client-server-api/_index.md | 36 ++++++++---- content/identity-service-api.md | 22 +++++-- content/server-server-api.md | 58 +++++++++++-------- 6 files changed, 81 insertions(+), 38 deletions(-) create mode 100644 changelogs/client_server/newsfragments/1185.clarification create mode 100644 changelogs/identity_service/newsfragments/1185.clarification create mode 100644 changelogs/server_server/newsfragments/1185.clarification diff --git a/changelogs/client_server/newsfragments/1185.clarification b/changelogs/client_server/newsfragments/1185.clarification new file mode 100644 index 000000000..0865507a9 --- /dev/null +++ b/changelogs/client_server/newsfragments/1185.clarification @@ -0,0 +1 @@ +Update "API Standards" section to clarify how JSON is used. diff --git a/changelogs/identity_service/newsfragments/1185.clarification b/changelogs/identity_service/newsfragments/1185.clarification new file mode 100644 index 000000000..0865507a9 --- /dev/null +++ b/changelogs/identity_service/newsfragments/1185.clarification @@ -0,0 +1 @@ +Update "API Standards" section to clarify how JSON is used. diff --git a/changelogs/server_server/newsfragments/1185.clarification b/changelogs/server_server/newsfragments/1185.clarification new file mode 100644 index 000000000..0865507a9 --- /dev/null +++ b/changelogs/server_server/newsfragments/1185.clarification @@ -0,0 +1 @@ +Update "API Standards" section to clarify how JSON is used. diff --git a/content/client-server-api/_index.md b/content/client-server-api/_index.md index ca2b3d97d..191aeaacf 100644 --- a/content/client-server-api/_index.md +++ b/content/client-server-api/_index.md @@ -13,20 +13,32 @@ clients which maintain a full local persistent copy of server state. ## API Standards The mandatory baseline for client-server communication in Matrix is -exchanging JSON objects over HTTP APIs. HTTPS is recommended for -communication, although HTTP may be supported as a fallback to support -basic HTTP clients. More efficient optional transports will in future be -supported as optional extensions - e.g. a packed binary encoding over -stream-cipher encrypted TCP socket for low-bandwidth/low-roundtrip -mobile usage. For the default HTTP transport, all API calls use a -Content-Type of `application/json`. In addition, all strings MUST be -encoded as UTF-8. +exchanging JSON objects over HTTP APIs. More efficient transports may be +specified in future as optional extensions. + +HTTPS is recommended for communication. The use of plain HTTP is not +recommended outside test environments. Clients are authenticated using opaque `access_token` strings (see [Client Authentication](#client-authentication) for details). +All `POST` and `PUT` endpoints, with the exception of +[`POST /_matrix/media/v3/upload`](#post_matrixmediav3upload), require the +client to supply a request body containing a (potentially empty) JSON object. +Clients are *not* required to supply a `Content-Type` header. + +Similarly, all endpoints require the server to return a JSON object, +with the exception of 200 responses to +[`GET /_matrix/media/v3/download/{serverName}/{mediaId}`](#get_matrixmediav3downloadservernamemediaid) +and [`GET /_matrix/media/v3/thumbnail/{serverName}/{mediaId}`](#get_matrixmediav3thumbnailservernamemediaid). +Servers should include a `Content-Type` header of `application/json` for all JSON responses. + +All JSON data, in requests or responses, must be encoded using UTF-8. + See also [Conventions for Matrix APIs](/appendices#conventions-for-matrix-apis) -in the Appendices for conventions which all Matrix APIs are expected to follow. +in the Appendices for conventions which all Matrix APIs are expected to follow, +and [Web Browser Clients](#web-browser-clients) below for additional +requirements for server responses. ### Standard error response @@ -196,6 +208,8 @@ only read state (e.g.: `/sync`, get account data, etc). The user is unable to reject an invite to join the server notices room. See the [Server Notices](#server-notices) module for more information. +### Transaction identifiers + The client-server API typically uses `HTTP PUT` to submit requests with a client-generated transaction identifier. This means that these requests are idempotent. The scope of a transaction identifier is a @@ -208,8 +222,6 @@ Some API endpoints may allow or require the use of `POST` requests without a transaction ID. Where this is optional, the use of a `PUT` request is strongly recommended. -{{% http-api spec="client-server" api="versions" %}} - ## Web Browser Clients It is realistic to expect that some clients will be written to be run @@ -308,6 +320,8 @@ specify parameter values. The flow for this method is as follows: {{% http-api spec="client-server" api="wellknown" %}} +{{% http-api spec="client-server" api="versions" %}} + ## Client Authentication Most API endpoints require the user to identify themselves by presenting diff --git a/content/identity-service-api.md b/content/identity-service-api.md index 65369b3e4..d9dfa9c39 100644 --- a/content/identity-service-api.md +++ b/content/identity-service-api.md @@ -38,8 +38,20 @@ Appendix. The mandatory baseline for identity server communication in Matrix is exchanging JSON objects over HTTP APIs. HTTPS is required for -communication, and all API calls use a Content-Type of -`application/json`. In addition, strings MUST be encoded as UTF-8. +communication. + +All `POST` and `PUT` endpoints, with the exception of [`POST +/_matrix/identity/v2/account/logout`](#post_matrixidentityv2accountlogout), +require the client to supply a request body containing a (potentially empty) +JSON object. Clients are *not* required to supply a `Content-Type` header. + +Similarly, all endpoints require the server to return a JSON object. Servers +should include a `Content-Type` header of `application/json` for all JSON +responses. + +All JSON data, in requests or responses, must be encoded using UTF-8. + +### Standard error response Any errors which occur at the Matrix API level MUST return a "standard error response". This is a JSON object which looks like: @@ -103,8 +115,6 @@ the third party identifier. `M_UNKNOWN` An unknown error has occurred. -{{% http-api spec="identity" api="versions" %}} - ## Privacy Identity is a privacy-sensitive issue. While the identity server exists @@ -131,6 +141,10 @@ recommended CORS headers to be returned by servers on all requests are: Access-Control-Allow-Methods: GET, POST, PUT, DELETE, OPTIONS Access-Control-Allow-Headers: Origin, X-Requested-With, Content-Type, Accept, Authorization +## API Version check + +{{% http-api spec="identity" api="versions" %}} + ## Authentication Most endpoints in the Identity Service API require authentication diff --git a/content/server-server-api.md b/content/server-server-api.md index 12d4bc45a..fa485d173 100644 --- a/content/server-server-api.md +++ b/content/server-server-api.md @@ -47,13 +47,41 @@ an HTTPS PUT request. ## API standards -The mandatory baseline for client-server communication in Matrix is -exchanging JSON objects over HTTP APIs. More efficient optional -transports will in future be supported as optional extensions - e.g. a -packed binary encoding over stream-cipher encrypted TCP socket for -low-bandwidth/low-roundtrip mobile usage. For the default HTTP -transport, all API calls use a Content-Type of `application/json`. In -addition, all strings MUST be encoded as UTF-8. +The mandatory baseline for server-server communication in Matrix is +exchanging JSON objects over HTTPS APIs. More efficient transports may be +specified in future as optional extensions. + +All `POST` and `PUT` endpoints require the requesting server to supply a +request body containing a (potentially empty) JSON object. Requesting servers +are *not* required to supply a `Content-Type` header. + +Similarly, all endpoints in this specification require the destination server +to return a JSON object. Servers should include a `Content-Type` header of +`application/json` for all JSON responses. + +All JSON data, in requests or responses, must be encoded using UTF-8. + +### TLS + +Server-server communication must take place over HTTPS. + +The destination server must provide a TLS certificate signed by a known +Certificate Authority. + +Requesting servers are ultimately responsible for determining the trusted Certificate +Authorities, however are strongly encouraged to rely on the operating system's +judgement. Servers can offer administrators a means to override the trusted +authorities list. Servers can additionally skip the certificate validation for +a given whitelist of domains or netmasks for the purposes of testing or in +networks where verification is done elsewhere, such as with `.onion` +addresses. + +Servers should respect SNI when making requests where possible: a SNI should be +sent for the certificate which is expected, unless that certificate is expected +to be an IP address in which case SNI is not supported and should not be sent. + +Servers are encouraged to make use of the [Certificate +Transparency](https://www.certificate-transparency.org/) project. ## Server discovery @@ -143,22 +171,6 @@ delegation are: and other applications using SRV records such [XMPP](https://datatracker.ietf.org/doc/html/rfc6120#section-13.7.2.1). {{% /boxes/note %}} -The TLS certificate provided by the target server must be signed by a -known Certificate Authority. Servers are ultimately responsible for -determining the trusted Certificate Authorities, however are strongly -encouraged to rely on the operating system's judgement. Servers can -offer administrators a means to override the trusted authorities list. -Servers can additionally skip the certificate validation for a given -whitelist of domains or netmasks for the purposes of testing or in -networks where verification is done elsewhere, such as with `.onion` -addresses. Servers should respect SNI when making requests where -possible: a SNI should be sent for the certificate which is expected, -unless that certificate is expected to be an IP address in which case -SNI is not supported and should not be sent. - -Servers are encouraged to make use of the [Certificate -Transparency](https://www.certificate-transparency.org/) project. - {{% http-api spec="server-server" api="wellknown" %}} ### Server implementation From 5ee403a51d794b367ffc53faa3a6bed96730ad0a Mon Sep 17 00:00:00 2001 From: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> Date: Wed, 27 Jul 2022 11:29:26 +0100 Subject: [PATCH 2/4] Update content/identity-service-api.md --- content/identity-service-api.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/identity-service-api.md b/content/identity-service-api.md index d9dfa9c39..58fc9019c 100644 --- a/content/identity-service-api.md +++ b/content/identity-service-api.md @@ -40,7 +40,7 @@ The mandatory baseline for identity server communication in Matrix is exchanging JSON objects over HTTP APIs. HTTPS is required for communication. -All `POST` and `PUT` endpoints, with the exception of [`POST +All `POST` and `PUT` endpoints, with the exception (for historical reasons) of [`POST /_matrix/identity/v2/account/logout`](#post_matrixidentityv2accountlogout), require the client to supply a request body containing a (potentially empty) JSON object. Clients are *not* required to supply a `Content-Type` header. From 15726ed62e91c2bafdd950ccd3f2657a929a1cbe Mon Sep 17 00:00:00 2001 From: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> Date: Thu, 28 Jul 2022 13:20:59 +0100 Subject: [PATCH 3/4] Update content/client-server-api/_index.md --- content/client-server-api/_index.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/content/client-server-api/_index.md b/content/client-server-api/_index.md index 191aeaacf..7a1f9e4dc 100644 --- a/content/client-server-api/_index.md +++ b/content/client-server-api/_index.md @@ -25,7 +25,8 @@ Authentication](#client-authentication) for details). All `POST` and `PUT` endpoints, with the exception of [`POST /_matrix/media/v3/upload`](#post_matrixmediav3upload), require the client to supply a request body containing a (potentially empty) JSON object. -Clients are *not* required to supply a `Content-Type` header. +Clients should supply a `Content-Type` header of `application/json` for all requests with JSON bodies, +but this is not required. Similarly, all endpoints require the server to return a JSON object, with the exception of 200 responses to From 2320b91a5f70eb6dd6a4e5dbb608130e0df67a22 Mon Sep 17 00:00:00 2001 From: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> Date: Thu, 28 Jul 2022 13:24:57 +0100 Subject: [PATCH 4/4] Apply suggestions from code review --- content/client-server-api/_index.md | 2 +- content/identity-service-api.md | 5 +++-- content/server-server-api.md | 5 +++-- 3 files changed, 7 insertions(+), 5 deletions(-) diff --git a/content/client-server-api/_index.md b/content/client-server-api/_index.md index 7a1f9e4dc..f391aeb1a 100644 --- a/content/client-server-api/_index.md +++ b/content/client-server-api/_index.md @@ -32,7 +32,7 @@ Similarly, all endpoints require the server to return a JSON object, with the exception of 200 responses to [`GET /_matrix/media/v3/download/{serverName}/{mediaId}`](#get_matrixmediav3downloadservernamemediaid) and [`GET /_matrix/media/v3/thumbnail/{serverName}/{mediaId}`](#get_matrixmediav3thumbnailservernamemediaid). -Servers should include a `Content-Type` header of `application/json` for all JSON responses. +Servers msut include a `Content-Type` header of `application/json` for all JSON responses. All JSON data, in requests or responses, must be encoded using UTF-8. diff --git a/content/identity-service-api.md b/content/identity-service-api.md index 58fc9019c..c4d4a31a7 100644 --- a/content/identity-service-api.md +++ b/content/identity-service-api.md @@ -43,10 +43,11 @@ communication. All `POST` and `PUT` endpoints, with the exception (for historical reasons) of [`POST /_matrix/identity/v2/account/logout`](#post_matrixidentityv2accountlogout), require the client to supply a request body containing a (potentially empty) -JSON object. Clients are *not* required to supply a `Content-Type` header. +JSON object. Clients should supply a `Content-Type` header of `application/json` +for all requests with JSON bodies, but this is not required. Similarly, all endpoints require the server to return a JSON object. Servers -should include a `Content-Type` header of `application/json` for all JSON +must include a `Content-Type` header of `application/json` for all JSON responses. All JSON data, in requests or responses, must be encoded using UTF-8. diff --git a/content/server-server-api.md b/content/server-server-api.md index fa485d173..2a034c355 100644 --- a/content/server-server-api.md +++ b/content/server-server-api.md @@ -53,10 +53,11 @@ specified in future as optional extensions. All `POST` and `PUT` endpoints require the requesting server to supply a request body containing a (potentially empty) JSON object. Requesting servers -are *not* required to supply a `Content-Type` header. +should supply a `Content-Type` header of `application/json` for all requests +with JSON bodies, but this is not required. Similarly, all endpoints in this specification require the destination server -to return a JSON object. Servers should include a `Content-Type` header of +to return a JSON object. Servers must include a `Content-Type` header of `application/json` for all JSON responses. All JSON data, in requests or responses, must be encoded using UTF-8.