UnicodeEncodeError when using WSDL client

[kylepeterson]

I am using xsdata with a wsdl definition to make a request.

I am using this client

from xsdata.formats.dataclass.client import Client

and instantiating it like this

Client.from_service(TalentPortGiveFeedback, location=ENDPOINT)

However I get the following error when sending unicode characters, such as mandarin, in the body.

[ERROR] UnicodeEncodeError: 'latin-1' codec can't encode characters in position 609-613: Body ('你好干得好') is not valid Latin-1. Use body.encode('utf-8') if you want to send it encoded in UTF-8.\nTraceback (most recent call last):\n File ... response = client.send(request, headers={\"Authorization\": f\"Bearer {bearer_token}\"})\n File \"/var/task/xsdata/formats/dataclass/client.py\", line 93, in send\n response = self.transport.post(self.config.location, data=data, headers=headers)\n File \"/var/task/xsdata/formats/dataclass/transports.py\", line 44, in post\n res = requests.post(url, data=data, headers=headers, timeout=self.timeout)\n File \"/var/task/requests/api.py\", line 115, in post\n return request(\"post\", url, data=data, json=json, **kwargs)\n File \"/var/task/requests/api.py\", line 59, in request\n return session.request(method=method, url=url, **kwargs)\n File \"/var/task/requests/sessions.py\", line 587, in request\n resp = self.send(prep, **send_kwargs)\n File \"/var/task/requests/sessions.py\", line 701, in send\n r = adapter.send(request, **kwargs)\n File \"/var/task/requests/adapters.py\", line 489, in send\n resp = conn.urlopen(\n File \"/var/task/urllib3/connectionpool.py\", line 703, in urlopen\n httplib_response = self._make_request(\n File \"/var/task/urllib3/connectionpool.py\", line 398, in _make_request\n conn.request(method, url, **httplib_request_kw)\n File \"/var/task/urllib3/connection.py\", line 239, in request\n super(HTTPConnection, self).request(method, url, body=body, headers=headers)\n File \"/var/lang/lib/python3.9/http/client.py\", line 1285, in request\n self._send_request(method, url, body, headers, encode_chunked)\n File \"/var/lang/lib/python3.9/http/client.py\", line 1330, in _send_request\n body = _encode(body, 'body')\n File \"/var/lang/lib/python3.9/http/client.py\", line 168, in _encode\n raise UnicodeEncodeError("

It seems the body is being encoded to 'latin-1' but i cant find any mention of this behavior or how to override it to use 'utf-8' in your docs.

I tried setting headers for charset like this: "Content-Type": "application/xml; charset=utf-8" but to no avail.

Do you have any advice on how to proceed?

[skinkie]

@kylepeterson I don't know how you are actually sending this. But what I noticed while building something similar last week was that Python's request library had some issues with the encoding as well. This is how I solved it. So it might be an upstream issue. https://github.com/openTdataCH/ojp-nova/blob/master/network_flow.py#L27

[tefra]

I have an idea, can you please try something like this? I think we are missing an option to pass a custom encoding

from xsdata.formats.dataclass.client import Client

class MyClient(Client):

    def prepare_payload(self, obj: Any) -> Any:
        return super().prepare_payload(obj).encode("utf-8")

[tefra]

The fix is on master

client = Client.from_service(HelloGetHelloAsString, encoding="utf-8")