Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MSC2529: Proposal to use existing events as captions for images #2529

Closed
wants to merge 3 commits into from
Closed

MSC2529: Proposal to use existing events as captions for images #2529

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
f40d175
proposal to use existing events as captions for images
benparsons May 7, 2020
720b76f
line-wrapping
benparsons May 7, 2020
531afe8
there are different types of space
benparsons May 7, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions proposals/2529-text-messages-as-captions.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Use existing m.room.message/m.text events as captions for images
uhoreg marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, splitting image and caption into separate events will make bridging very complicated. When to send the image? How long until the caption is sent, then?

it'll still be practically impossible to bridge an image+caption away from matrix, the caption would basically have to be rendered as a separate message.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would that be very bad? For example, bridging to Discord would result in two messages next to each other if they cannot be combined by debouncing.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, as messages can be sent in between the image and the caption the entire caption context is lost.

Soru personally considers this a blocker, TBH.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Easier bridging (and handling captions in general) is one of the main benefits of #2530 btw

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 to embed the caption in the image event content


## Background

There is a demand to be able to apply a text caption to an image, as is possible in other chat platforms. In Matrix this is not possible, so people will generally send two events: one `m.image`, then a `m.text` event immediately afterward to simulate a caption.
benparsons marked this conversation as resolved.
Show resolved Hide resolved

Better would be to able to explicitly mark an event as a caption.

## Proposal

Allow an optional `m.relates_to` field in the `content` field of a text message event.

Example:

```
...
"content": {
"body": "Caption text",
"msgtype": "m.text",
"m.relates_to": {
"event_id": "$(some image event)",
"rel_type": "m.caption"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

iirc this is a further metadata leak in encrypted rooms.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure this is the right type under MSC1849: there was fairly long discussion at some point about how we shouldn't need to define new relationship types very often, if at all.

}
},
```

If a client recognises the `rel_type`, they can render the caption with the image rather than as a separate message in the timeline.

The benefit of this is that if a client doesn't support or recognise the `m.caption`, it can ignore the relation and just render the message inline.

This would not require aggregation from the server since there will always be a need to send the event separately anyway.

## Potential issues

* Not sure how this relates to the broader questions discussed in MSC1849
* This is catering to a narrow use-case requirement. There may be a more general solution available
* Would MSC1767 (extensible events) obsolete this?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extensible events would definitely include captions, but IMO basic features like captions shouldn't be blocked on MSCs that aren't going to happen any time soon