Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MM-52657] AV1 support #795

Merged
merged 6 commits into from
Jul 26, 2024
Merged

[MM-52657] AV1 support #795

merged 6 commits into from
Jul 26, 2024

Conversation

streamer45
Copy link
Collaborator

Summary

PR adds experimental support for encoding screen-sharing tracks using the AV1 codec.

Changes should be backward compatible with regard to older clients since we must always send a fallback VP8 track.

We are also bumping the minimum rtcd version since some breaking changes to the join event payload were required.

Related PRs

mattermost/rtcd#150

Ticket Link

https://mattermost.atlassian.net/browse/MM-52657

@streamer45 streamer45 added 2: Dev Review Requires review by a core committer 2: QA Review Requires review by a QA tester Do Not Merge/Awaiting PR Awaiting another pull request before merging (e.g. server changes) Docs Needed Requires documentation labels Jun 24, 2024
@streamer45 streamer45 requested a review from cpoile June 24, 2024 14:46
@streamer45 streamer45 self-assigned this Jun 24, 2024
Copy link
Member

@cpoile cpoile left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Non-blocking suggestions. :)

Comment on lines 574 to 575
if (this.config.simulcast) {
logWarn('both simulcast and av1 support are enabled');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about prioritizing AV1 over simulcast?
And either way, maybe we should throw a configuration error if the system admin selects both? Otherwise they will select them and then wonder why they're not seeing any improvement...

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cpoile I honestly didn't want to overcomplicate it for some experimental features that we don't yet know if we want to retain. Our config can still be changed by different means (e.g., JSON file), so then we'd have to decide what to do in case the mutually exclusive settings are turned on (e.g. fail on start?).

The idea of logging the warning was that it would be trivial to determine if they fall into such a case. This is also documented in the config itself. I feel that may be enough for the time being.

@cpoile cpoile removed the 2: Dev Review Requires review by a core committer label Jun 27, 2024
@streamer45
Copy link
Collaborator Author

streamer45 commented Jun 28, 2024

@DHaussermann This is ready for review. What I am looking for QA wise is making sure we are not introducing regressions with screen sharing on any of our supported clients, with a special focus on mobile.

So I'd like to test the following cases:

Presenter side:

  • Chrome
  • Desktop App
  • Firefox

Receiving side:

  • Chrome
  • Desktop App
  • Firefox
  • Android App
  • iOS App

For mobile, we should test both existing app and a build from the new PR branch -> mattermost/mattermost-mobile#8037

@streamer45 streamer45 requested a review from DHaussermann June 28, 2024 07:08
@streamer45 streamer45 added this to the v1.0.0 🚀 / MM 10.0 milestone Jul 12, 2024
Copy link

@DHaussermann DHaussermann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested and passed

  • Deployed the plugin locally and enabled AV1 codec
  • Tested that browsers can send and receive video for screen share
  • Tested Firefox, Chrome, Safari and MM Desktop app for send and recieve
  • Tested production mobile build can receive video (android and iOS)
  • Tested Mobile builds from linked PR can receive video (android and iOS)
  • With all tests ensure latency was about on par by scrolling vertically on a static page from the client sending and ensuring lag was nothing unexpected on the client receiving.

LGTM!

@DHaussermann DHaussermann removed the 2: QA Review Requires review by a QA tester label Jul 15, 2024
@DHaussermann
Copy link

@streamer45 a couple unrelated questions here:

  1. This build won't run on a cloud server for Mattermost v9.10.0 I don't think the config make use of standalone RTCD but I see this when I try enabling the plugin. Has something changed recently on the plugin side? Or maybe it was just misconfigured in a way I did not notice.
    image

  2. Probably a silly question but... Why is the share screen option sometimes in the modal and sometimes not? Is there some simple condition I have missed here? Sometimes the share option is only in the expanded view and not the modal.
    image

@streamer45
Copy link
Collaborator Author

@DHaussermann

  1. All Cloud servers use RTCD as it's enforced through environment overrides so it doesn't appear in the config. We haven't released v0.17.0 yet, which is required by these changes ([MM-52657] AV1 support rtcd#150), so this is best tested locally.
  2. It looks like you may have screen sharing disabled in config (AllowScreenSharing). There's also a condition based on having multiple teams. With the team sidebar in place, we can expand the widget and fit more buttons, including screen sharing. Otherwise, we should have the button in the widget menu. I don't see it in either location there, though.

@DHaussermann
Copy link

Thanks. Re: 2. I don't actually see any instances where the share is not in the modal once it is enabled.

@streamer45 streamer45 added 3: Reviews Complete All reviewers have approved the pull request and removed Do Not Merge/Awaiting PR Awaiting another pull request before merging (e.g. server changes) labels Jul 26, 2024
@streamer45 streamer45 merged commit a7dabb0 into main Jul 26, 2024
18 checks passed
@streamer45 streamer45 deleted the MM-52657 branch July 26, 2024 08:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3: Reviews Complete All reviewers have approved the pull request Docs Needed Requires documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants