This repository has been archived by the owner on Apr 26, 2024. It is now read-only.
Can't join nor leave a space after running buggy version of portdb #13426
Labels
A-Spaces
Hierarchical organization of rooms
X-Needs-Info
This issue is blocked awaiting information from the reporter
Comments
|
Hi, thanks for your report. This error condition ( Have you ever done anything unusual to the database (e.g. manually deleting rows, running external tools on it, ported it from SQLite to Postgres, etc)? Are you using any experimental features (faster joins, message retention policies, etc)? |
reivilibre
added
the
X-Needs-Info
|
Yes we did port the database from SQLite to Postgres just before this issue occurred (no other problems have been reported so far). We didn't manually delete rows, we didn't run external tools on it and we don't use any experimental features (faster joins, message retention policies, etc). |
|
When you ported from sqlite, which version of synapse was installed at the time? (You may have ported before the fix in #13226 was available.) |
|
It was version 1.59.1. We port the database from SQLite to Postgres and the we updated from v 1.59.1 to v 1.63. |
if I compare the date of the fix and the date of the v 1.59.1 release, I guess you are right, we ported before the fix then... |
|
Do you still have the old SQLite database? If so, please could you help to confirm whether this is why this happened:
SELECT rowid FROM events WHERE event_id = '$d07wypTuQbxA5SUD3OwBPBUNl1mCF6ydmDwsdUGhxrw';and SELECT rowid FROM event_json WHERE event_id = '$d07wypTuQbxA5SUD3OwBPBUNl1mCF6ydmDwsdUGhxrw';These queries will let us know whether the event exists in those tables or not, and if it does, what SQLite-internal rowid it has (which may be useful to confirm our hypothesis). It would also be good if you can confirm on your Postgres database: SELECT 1 FROM events WHERE event_id = '$d07wypTuQbxA5SUD3OwBPBUNl1mCF6ydmDwsdUGhxrw';and SELECT 1 FROM event_json WHERE event_id = '$d07wypTuQbxA5SUD3OwBPBUNl1mCF6ydmDwsdUGhxrw';These queries will just let us know whether the event exists in Postgres. |
|
Here are what I got : Does it help? |
|
@Thatoo I don't think you're connected to your Postgres database properly; maybe you need to try However the SQLite portion of your result starts to confirm the hypothesis. |
|
I think your best bet will be to use the Delete Rooms Admin API to remove the room entirely from your homeserver. |
I guess you were right ! :-)
I'm ok with deleting the room with the admin api but I'm not sure how to do it properly. |
|
Just looking at the docs that curl command seems sane, is it not working? |
|
What I want to be sure is if this command will indeed remove the room from our homeserver but will allow us to rejoin it after. |
|
Aha - that is the suggested command, no guarantee that it will work but that's the hope. |
|
it failed... |
|
Looks like it failed because it was unable to kick user |
|
It is indeed my account but I can't leave the space, this is the subject of this issue... |
|
inspired from the join API... it doesn't work... |
|
Sorry, I think there might be some confusion here-the issue states that you cannot leave or join a space, but are you talking about a room rather than a space? Do you have logs for the failed kick command? |
It is a space : #monnaie-libre:matrix.org
Only that :
What log do you need? |
|
Okay great. Can you try leaving (probably just via a client) the room you are trying to delete: |
|
I tried leaving from Fluffychat. then from Element. Here are both logs. Does that help? |
|
@Thatoo did you try the |
|
No, not yet, should I? |
yes
unfortunately that's probably the case, at least until the user tells his client to flush its cache. That may be unavoidable. |
|
Here is what I got : I empty the cache and reloaded my client and indeed the space #monnaie-libre:matrix.org isn't listed any more? Do you think I could now try to attempt joining back this space frome the same user? or is it better to do an other step before? |
|
best to restart synapse before rejoining, due to #11521 - but otherwise, yes. |
|
I restarted synapse, I empty the cache of Schildichat and try to join the space again, !nsxQkuXAmwktOKyUNc:matrix.org (nsxQkuXAmwktOKyUNc), it fails saying "internal server error" and logs are as follow : |
|
|
Are those queries before or after attempting to rejoin? Also can you |
|
Both before and after attempting to rejoin (I tried again both time). |
richvdh
removed
the
X-Needs-Info
|
|
squahtx
added
the
X-Needs-Info
|
Was this resolved or is this still an issue? |
|
I didn't have time to work on it yet. |
|
I just tried again and this time What should I do then? |
|
So So I didn't restart synapse and I didn't try to join the room again. |
|
did you do all the steps I suggested? |
it contains 1 row so I don't know what to do. |
have you done it again since you tried and failed to rejoin? |
|
I did it all again : Plus I empty the cache of my schildichat client app. I'll try to join again later. |
|
It's weird because now it says again "internal server error" |
|
So I expulsed this user from the space and invited him again. The user received the invitation but when clicking "Accept", I get "internal server error" message again. |
if you get an "internal server error", there will be an error message and stacktrace in the Synapse logs. We cannot do anything without those logs.
This is not a strange thing. It is a known issue, matrix-org/matrix-spec#752.
Again, share the logs. |
|
Here are some logs when I tried two time one after the other to join the space: |
|
Thanks for pulling out the logs. The error of interest is Can you try the following, including
These steps have to be done in order. If any step fails, please stop and post the errors before doing anything else. |
First attempt to rejoin the space I got this error : Second attempt gives me the internal server error. Here are the logs : |
|
I guess the problem is with and then I should add one more Let me know what are these yyyyy |
Yes, pretty much. The table in question is It's possible that an additional error will be discovered after trying the above. You can determine which table is involved by using |
|
I don't understand what this command will do |
It should give you an error, because
We want trying to remove rows from
The next best thing would be to join to the So instead, we want to delete anything in
It is safe, but you should take a backup before you run the command. (And you should take regular database backups anyway.) |
First attempt to rejoin the space... and... it works!!!! |
DMRobertson
changed the title
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Description
I was in a space and for some reason I wanted to leave and come back to check something. I couldn't leave so I asked the admin to expulse me.
Then I tried to join again that space. Another user invited me.
Now I get the invitation message offering me to accept or reject.
None of the two action are working.
Accept is taking for ever without any answer, Reject is telling me "internal server error".
And now, no other user of my server can't join this space.
Steps to reproduce
Homeserver
defis.info
Synapse Version
1.63
Installation Method
Debian packages from packages.matrix.org
Platform
proxmox container with debian behind a nginx reverse proxy
Relevant log output
Anything else that would be useful to know?
the space is #monnaie-libre:matrix.org and it's id is !nsxQkuXAmwktOKyUNc:matrix.org
The text was updated successfully, but these errors were encountered: