Avoid adding charset to Content-Type header

[samrat]

Hello! 👋

I'm trying to use the OpenAI Realtime API and need to send Content-Type: application/sdp as one of the headers.

Here's what my Dart code looks like

      final response = await http.post(
        Uri.parse('$baseUrl?model=$model'),
        headers: {
          'Authorization': 'Bearer $ephemeralKey',
          'Content-Type': 'application/sdp',
        },
        body: sdp,
        encoding: null
      );

Even though I'm setting the header to application/sdp the request sent has the header set to application/sdp; charset=utf-8(I tested this with iOS, macOS and Web debug targets)

And looks like the OpenAI endpoint is very strict about the value it accepts for this header:

Response body: {"error":{"message":"Unsupported content type. This API method only accepts 'application/sdp' requests, but you specified the header 'Content-Type: application/sdp; charset=utf-8'. Please try again with a supported content type.","type":"invalid_request_error","param":null,"code":"unsupported_content_type"}}

Is there any way to prevent the http library from modifying headers before the request is made?

[seifibrahim32]

@brianquinlan I will be looking into this issue.

[brianquinlan]

The documentation for encoding says that setting it will add the charset parameter.

[seifibrahim32]

@brianquinlan Is it obligatory from http to add charset parameter into the headers?

[brianquinlan]

"charset" is not required, see https://datatracker.ietf.org/doc/html/rfc2046#section-4.1.2

@samrat I think that your problem is that you are setting the HTTP payload using body and package:http is assuming a default encoding when converting it into bytes. Instead, you can set bodyBytes and use whatever encoding that you want. See https://pub.dev/documentation/http/latest/http/Request/body.html

@seifibrahim32 Changing package:http's charset behavior would be breaking. We may just need better documentation for the top-level functions e.g. https://pub.dev/documentation/http/latest/http/post.html

[seifibrahim32]

@brianquinlan Yes, charset should be with default utf8 or utf16 atmost.

@samrat Have you tried to use encoding property to maneuver this request?

[chedwin41]

I am having the same issue here when using an opentelemetry collector. The rules are strict :

The client and the server MUST set “Content-Type: application/json” request and response headers when sending JSON Protobuf encoded payload. (https://opentelemetry.io/docs/specs/otlp/#json-protobuf-encoding)

Here is my code:

http.Response response = await http.post(
              Uri.parse(
                otelUrl,
              ),
              headers: <String, String>{
                "content-type": "application/json",
              },
              body: jsonBody,
            );

As I understand, there is no good solution to avoid "charset=utf-8" to be added in content-type node while using http ? We should directly use request instead ?

@seifibrahim32 If encoding is set to anything else different from utf8 it will encode in another format the body isn't it. What we would need is that the body is still encoded but not specified in the content-type node

[samrat]

As I understand, there is no good solution to avoid "charset=utf-8" to be added in content-type node while using http ? We should directly use request instead ?

@chedwin41 It seems the way to prevent this behaviour is to set the request body first before setting the headers. Here is what I did: https://samrat.me/til-avoid-modifying-headers-with-dart-http-package/ .

Would love to know if there is a better way.