Failure when downloading any remote media

[Kimiblock]

Description

Synapse fails to download any remote media

Steps to reproduce

• Fetch a remote media

Homeserver

moechat.kimiblock.top

Synapse Version

1.117

Installation Method

Other (please mention below)

Database

PostgreSQL single server

Workers

Single process

Platform

Arch Linux rolling, all packages has been updated

Configuration

experimental_features:
msc3575_enabled: true
msc3852_enabled: true
msc3026_enabled: true
msc3266_enabled: true
msc2815_enabled: true
msc2815_enabled: true
msc3773_enabled: true
msc3664_enabled: true
msc3881_enabled: true

Push

msc3890_enabled: true
msc3881_enabled: true
msc3391_enabled: true
msc4028_push_encrypted_events: true

enable_media_repo: true

Keep this off!

enable_authenticated_media: false

media_store_path: "media_repo"
max_pending_media_uploads: 4
max_upload_size: 3M
max_image_pixels: 32M
dynamic_thumbnails: true
media_retention:
local_media_lifetime: 14d
remote_media_lifetime: 4d

web_client_location: https://plane.kimiblock.top

presence:
enabled: true

Relevant log output

2024-11-13 08:44:51,926 - synapse.http.client -
 1118 - WARNING - sentinel - Exception writing 
to multipart parser: 'MultipartParser' object h
as no attribute 'write'
2024-11-13 08:44:51,927 - synapse.media.media_r
epository - 910 - ERROR - GET-5074 - Failed to 
fetch remote media 2o4.ca/d14KyhF8hOafktomhOlRt
wC2zK0SiZlM
Traceback (most recent call last):
  File "/usr/lib/python3.12/site-packages/synap
se/media/media_repository.py", line 866, in _fe
deration_download_remote_file
    res = await self.client.federation_download
_media(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^
  File "/usr/lib/python3.12/site-packages/synap
se/federation/federation_client.py", line 1888,
 in federation_download_media
    return await self.transport_layer.federatio
n_download_media(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/synap
se/federation/transport/client.py", line 885, i
n federation_download_media
    return await self.client.federation_get_fil
e(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^
  File "/usr/lib/python3.12/site-packages/synap
se/http/matrixfederationclient.py", line 1742, 
in federation_get_file
    multipart_response = await make_deferred_yi
eldable(deferred)
                         ^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/synap
se/http/client.py", line 1116, in dataReceived
    self.parser.write(incoming_data)
    ^^^^^^^^^^^^^^^^^
AttributeError: 'MultipartParser' object has no
 attribute 'write'
2024-11-13 08:44:52,763 - synapse.http.client -
 1118 - WARNING - sentinel - Exception writing 
to multipart parser: 'MultipartParser' object h
as no attribute 'write'
2024-11-13 08:44:52,763 - synapse.media.media_r
epository - 910 - ERROR - GET-5044 - Failed to 
fetch remote media uwu.zemos.net/6Oc7VA1lboOpV6
qltdMeoZ21W9BR79Rv
Traceback (most recent call last):
  File "/usr/lib/python3.12/site-packages/synap
se/media/media_repository.py", line 866, in _fe
deration_download_remote_file
    res = await self.client.federation_download
_media(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^
  File "/usr/lib/python3.12/site-packages/synap
se/federation/federation_client.py", line 1888,
 in federation_download_media
    return await self.transport_layer.federatio
n_download_media(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/synap
se/federation/transport/client.py", line 885, i
n federation_download_media
    return await self.client.federation_get_fil
e(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^
  File "/usr/lib/python3.12/site-packages/synap
se/http/matrixfederationclient.py", line 1742, 
in federation_get_file
    multipart_response = await make_deferred_yi
eldable(deferred)
                         ^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/synap
se/http/client.py", line 1116, in dataReceived
    self.parser.write(incoming_data)
    ^^^^^^^^^^^^^^^^^
AttributeError: 'MultipartParser' object has no
 attribute 'write'
2024-11-13 08:44:57,462 - synapse.http.client -
 1118 - WARNING - sentinel - Exception writing 
to multipart parser: 'MultipartParser' object h
as no attribute 'write'
2024-11-13 08:44:57,462 - synapse.media.media_r
epository - 910 - ERROR - GET-5153 - Failed to 
fetch remote media nichi.co/BJghvnTXFaPPPPuVomq
GOyCo
Traceback (most recent call last):
  File "/usr/lib/python3.12/site-packages/synap
se/media/media_repository.py", line 866, in _fe
deration_download_remote_file
    res = await self.client.federation_download
_media(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^
  File "/usr/lib/python3.12/site-packages/synap
se/federation/federation_client.py", line 1888,
 in federation_download_media
    return await self.transport_layer.federatio
n_download_media(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/synap
se/federation/transport/client.py", line 885, i
n federation_download_media
    return await self.client.federation_get_fil
e(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^
  File "/usr/lib/python3.12/site-packages/synap
se/http/matrixfederationclient.py", line 1742, 
in federation_get_file
    multipart_response = await make_deferred_yi
eldable(deferred)
                         ^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/synap
se/http/client.py", line 1116, in dataReceived
    self.parser.write(incoming_data)
    ^^^^^^^^^^^^^^^^^
AttributeError: 'MultipartParser' object has no
 attribute 'write'
2024-11-13 08:44:58,430 - synapse.http.client -
 1118 - WARNING - sentinel - Exception writing 
to multipart parser: 'MultipartParser' object h
as no attribute 'write'
2024-11-13 08:44:58,430 - synapse.media.media_r
epository - 910 - ERROR - GET-5160 - Failed to 
fetch remote media nichi.co/BJghvnTXFaPPPPuVomq
GOyCo
Traceback (most recent call last):
  File "/usr/lib/python3.12/site-packages/synap
se/media/media_repository.py", line 866, in _fe
deration_download_remote_file
    res = await self.client.federation_download
_media(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^
  File "/usr/lib/python3.12/site-packages/synap
se/federation/federation_client.py", line 1888,
 in federation_download_media
    return await self.transport_layer.federatio
n_download_media(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/synap
se/federation/transport/client.py", line 885, i
n federation_download_media
    return await self.client.federation_get_fil
e(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^
  File "/usr/lib/python3.12/site-packages/synap
se/http/matrixfederationclient.py", line 1742, 
in federation_get_file
    multipart_response = await make_deferred_yi
eldable(deferred)
                         ^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/synap
se/http/client.py", line 1116, in dataReceived
    self.parser.write(incoming_data)
    ^^^^^^^^^^^^^^^^^
AttributeError: 'MultipartParser' object has no
 attribute 'write'
➜  ~

Anything else that would be useful to know?

No response

[erikjohnston]

That is really odd, the error is:

AttributeError: 'MultipartParser' object has no attribute 'write'

Which is a class from python_multipart package. It's possible that another package is colliding with it, which doesn't have write? You should try re-installing python-multipart perhaps?

Using a system-wide installation can be slightly brittle, as multiple python packages can clash, i.e. two different packages can install modules under multipart.

[Kimiblock]

I have both multipart and python-multipart installed, that is the cause.

[alexanderustinov]

I've had the same with v1.119.0 on gentoo with distro packages. It seems, synapse uses both multipart and python-multipart (doesn't start with multipart removed)
Workarounded problem by changing

synapse/synapse/http/client.py

Line 39 in 850ff14

import multipart

to

import python_multipart as multipart

[Kimiblock]

#17932

[arkamar]

@alexanderustinov that sounds weird, because gentoo package patches the line as you suggest, see https://github.com/gentoo/gentoo/blob/67abb5404860cf0bdb60b2454e1b98a3b5fb881c/net-im/synapse/synapse-1.119.0.ebuild#L181-L183 and all tests pass for ~amd64. We do this change since v1.116.0-r2.

[alexanderustinov]

@alexanderustinov that sounds weird, because gentoo package patches the line as you suggest, see https://github.com/gentoo/gentoo/blob/67abb5404860cf0bdb60b2454e1b98a3b5fb881c/net-im/synapse/synapse-1.119.0.ebuild#L181-L183 and all tests pass for ~amd64. We do this change since v1.116.0-r2.

@arkamar i've had custom ebuild in an overlay for years. Haven't noticed synapse in ::gentoo, thanks.