Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CIS-2152] Add support for CDN v2 #2339

Merged
merged 31 commits into from
Oct 26, 2022
Merged

Conversation

nuno-vieira
Copy link
Member

@nuno-vieira nuno-vieira commented Oct 19, 2022

🔗 Issue Links

CIS-2152

🎯 Goal

Use Stream's CDN v2 to improve the memory footprint of the application. Now, images are downloaded with a lower resolution instead of the original one.

📝 Summary

  • Stream CDN v2 Implementation
  • Add the possibility to limit the resolution of image attachments
  • Refactors the ImageLoading API
  • Refactors he ImageCDN with some needed breaking changes
  • ComposerVC.addAttachmentToContent() has a new info property (minor breaking change)
  • Reorganization of Utils Folder
  • Deprecates CGSize.avatarThumbnailSize and moves it to Components.default

🛠 Implementation

  • The ImageCDN required breaking changes so that the API is more understandable but also more scalable. The breaking changes are also minimal and will be documented in the changelog.
  • In some cases, we know the size of the view where we are loading the image, but in other cases like cells, we don't know, especially in the message list where the message cell can be totally customizable and since the cell is reusable there is no way to get the size upfront unless the view has a static size. So by default, right now the image attachments in the message list have a maximum resolution, if the image attachment goes beyond this limit, it is capped to the maximum resolution which is 2MP (+/- what WhatsApp uses).

🎨 Showcase

Since we are reducing the resolution of the loaded images, the memory footprint reduces a lot. Doing a test with a message list with about 16 images that have high resolution, the memory footprint is reduced by ~6x times.

Before After
Memory_Use_Before Memory_Use_After

After exporting the memory graph and analysing with VVMap, we can see that especially the Image Raster data is much lower:

Before After
VVMAP_Before VVMAP_After

The reason for these improvements is that for memory use the image resolution is the most important, not the file size. The memory is used by the decoding and decompression of the image which is dependent on the resolution.

For example, if we are talking about an image of 3300x4500, that will be 57MB! For only one image. Since each pixel takes up 4 bytes (if sRGB), we can calculate this with the following formula 3300 * 4500 * 4 / 1024 / 1024 = 56.6MB.

🧪 Manual Testing Notes

  • A smoke test around images should be done, although snapshot tests should have found any regression if there was any.

☑️ Contributor Checklist

  • I have signed the Stream CLA (required)
  • This change follows zero ⚠️ policy (required)
  • This change should be manually QAed
  • Changelog is updated with client-facing changes
  • New code is covered by unit tests
  • Comparison screenshots added for visual changes
  • Affected documentation updated (docusaurus, tutorial, CMS)

🎁 Meme

@nuno-vieira nuno-vieira added 🎨 SDK: StreamChatUI Tasks related to the StreamChatUI SDK ✅ Feature An issue or PR related to a feature labels Oct 19, 2022
@nuno-vieira nuno-vieira self-assigned this Oct 19, 2022
@nuno-vieira nuno-vieira requested a review from a team as a code owner October 19, 2022 11:07
@Stream-SDK-Bot
Copy link
Collaborator

Stream-SDK-Bot commented Oct 19, 2022

1 Warning
⚠️ Big PR

Generated by 🚫 Danger

@nuno-vieira nuno-vieira force-pushed the add/cdn-v2-implementation branch from 2247a33 to 90b67e1 Compare October 19, 2022 11:13
@nuno-vieira nuno-vieira added the 🤞 Ready For QA A PR that is Ready for QA label Oct 19, 2022
@emerge-tools
Copy link

emerge-tools bot commented Oct 19, 2022

📏 Size Analysis

Total install size 10.5 MB | This change: ⬆️ +33.3 kB (+0.317%)

Image of diff

🗂 See size breakdown
Item Install size
➕ StreamChatUI.framework/StreamChatUI.StreamImageCDN.urlRequest(forImageUrl,resize) ⬆️ 3.5 kB
➕ StreamChat.framework/StreamChat.AnyAttachmentPayload.init(attachmentType,localFileURL,localMetadata,extraData) ⬆️ 3.2 kB
➖ StreamChat.framework/StreamChat.AnyAttachmentPayload.init(localFileURL,attachmentType,extraData) ⬇️ 3.1 kB
➖ StreamChatUI.framework/StreamChatUI.StreamImageCDN.thumbnailURL(originalURL,preferredSize) ⬇️ 2.7 kB
StreamChatUI.framework/DYLD.Exports ⬆️ 2.6 kB
➖ StreamChatUI.framework/StreamChatUI.NukeImageLoader.loadImage(into,url,imageCDN,placeholder,resize,preferredSize,completion) ⬇️ 2.3 kB
➕ StreamChatUI.framework/StreamChatUI.NukeImageLoader.loadImage(into,from,with,completion) ⬆️ 2.2 kB
➖ StreamChatUI.framework/StreamChatUI.NukeImageLoader.loadImages(from,placeholders,loadThumbnails,thumbnailSize,imageCDN,completion) ⬇️ 2.2 kB
StreamChatUI.framework/DYLD.String Table ⬆️ 1.7 kB
➕ StreamChatUI.framework/StreamChatUI.NukeImageLoader.downloadMultipleImages(with,completion) ⬆️ 1.7 kB
➕ StreamChatUI.framework/StreamChatUI.ComposerVC.addAttachmentToContent(from,type,info) ⬆️ 1.6 kB
➕ StreamChatUI.framework/StreamChatUI.NukeImageLoader.downloadImage(with,completion) ⬆️ 1.5 kB
➕ StreamChatUI.framework/StreamChatUI.StreamImageCDN.cachingKey(forImageUrl) ⬆️ 1.4 kB
➖ StreamChatUI.framework/StreamChatUI.StreamImageCDN.cachingKey(forImage) ⬇️ 1.3 kB
➕ StreamChatUI.framework/StreamChatUI.ImageDownloadRequest.value witness ⬆️ 1.3 kB
➖ StreamChatUI.framework/StreamChatUI.ImageProcessors.LateResize ⬇️ 1.3 kB
➕ StreamChatUI.framework/StreamChatUI.ImageLoading.loadImage(into,from,maxResolutionInPixels,completion) ⬆️ 1.2 kB
➕ StreamChatUI.framework/StreamChatUI.ImageResultsMapper.mapErrors(with) ⬆️ 1.2 kB
StreamChatUI.framework/StreamChatUI.ComposerVC.addAttachmentToContent(from,type) ⬇️ 1.1 kB
StreamChatUI.framework/Code Signature ⬆️ 1.0 kB

🔎 See the full size analysis (2459d11) merging into develop (f065659)

Provide a baseSha and pullRequestNumber with your upload to ensure diffs are correct. Read more in the docs

@nuno-vieira nuno-vieira removed the 🤞 Ready For QA A PR that is Ready for QA label Oct 20, 2022
Copy link
Contributor

@martinmitrevski martinmitrevski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work @nuno-vieira 👏

I've added some comments and I still need to do the manual QA. Also, please see why the checks are failing.

Additionally, let's revisit the breaking changes again.

CHANGELOG.md Outdated Show resolved Hide resolved
CHANGELOG.md Outdated Show resolved Hide resolved
CHANGELOG.md Outdated Show resolved Hide resolved
Sources/StreamChatUI/Utils/ImageLoading/ImageLoading.swift Outdated Show resolved Hide resolved
Sources/StreamChatUI/Utils/ImageLoading/ImageLoading.swift Outdated Show resolved Hide resolved
CHANGELOG.md Outdated Show resolved Hide resolved
Sources/StreamChatUI/Utils/ImageLoading/ImageCDN.swift Outdated Show resolved Hide resolved
@nuno-vieira nuno-vieira force-pushed the add/cdn-v2-implementation branch from c1f66d2 to 7183420 Compare October 20, 2022 17:10
Copy link
Contributor

@martinmitrevski martinmitrevski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM ✅ Also did some manual QA, everything works smoothly.
About the tuples thing, it's really up-to you, if you feel that's more intuitive, we can merge it like this.

Copy link
Contributor

@polqf polqf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lovely to see this finally happening.
I just have one concern around braking changes and possible behavior changes.

Is there any way to keep the current APIs but with a deprecation warning instead of completely removing them? Also, is there any chance that the behavior changes for existing customers after these changes?

@nuno-vieira nuno-vieira force-pushed the add/cdn-v2-implementation branch from 3d45409 to 1802b0b Compare October 25, 2022 12:13
@nuno-vieira nuno-vieira added the 🟢 QAed A PR that was QAed label Oct 25, 2022
Copy link
Contributor

@martinmitrevski martinmitrevski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great work 👏 ready to be merged IMO.

@sonarqubecloud
Copy link

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 1 Code Smell

95.0% 95.0% Coverage
0.0% 0.0% Duplication

@testableapple
Copy link
Contributor

Did a round of QA, LGTM 🚢

@nuno-vieira nuno-vieira merged commit 6a058c9 into develop Oct 26, 2022
@nuno-vieira nuno-vieira deleted the add/cdn-v2-implementation branch October 26, 2022 09:22
@nuno-vieira nuno-vieira mentioned this pull request Oct 27, 2022
}

/// The formula to calculate the new resolution is based on the max pixels and
/// the original aspect ratio. To get the formula, a system of questions is needed.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

System of equations 🙃 🤦‍♂️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
✅ Feature An issue or PR related to a feature 🟢 QAed A PR that was QAed 🎨 SDK: StreamChatUI Tasks related to the StreamChatUI SDK
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants