Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add maximum image file size to be processed by the thumbnailer #9069

Merged
merged 1 commit into from
May 8, 2024

Conversation

DeepDiver1975
Copy link
Member

@DeepDiver1975 DeepDiver1975 commented May 3, 2024

Description

Yet another hardening around the thumbnail service.

How Has This Been Tested?

Screenshots (if appropriate):

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Technical debt
  • Tests only (no source changes)

Checklist:

  • Code changes
  • Unit tests added
  • Acceptance tests added
  • Documentation ticket raised:

Copy link

update-docs bot commented May 3, 2024

Thanks for opening this pull request! The maintainers of this repository would appreciate it if you would create a changelog item based on your changes.

Comment on lines 47 to 49
MaxInputWidth int `yaml:"max_input_width" env:"THUMBNAILS_MAX_INPUT_WIDTH" desc:"The maximum width of an input image which is being processed." introductionVersion:"6.0"`
MaxInputHeight int `yaml:"max_input_height" env:"THUMBNAILS_MAX_INPUT_HEIGHT" desc:"The maximum height of an input image which is being processed." introductionVersion:"6.0"`
MaxInputImageFileSize uint64 `yaml:"max_input_image_file_size" env:"THUMBNAILS_MAX_INPUT_IMAGE_FILE_SIZE" desc:"The maximum file size of an input image which is being processed." introductionVersion:"6.0"`
Copy link
Contributor

@mmattel mmattel May 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to add the unit of the value in the description. int and uint64 is just the computational range.

Example:
The maximum file size of an input image which is being processed. -->
The maximum file size in bytes of an input image that will be processed.

We also should describe what will happen with unprocessed images. How will the thumbnail look like? (Imho it would be great to have a kind of dummy image available showing - "no thumbnail available" to avoid confusion)

@DeepDiver1975 DeepDiver1975 force-pushed the feat/max-image-file-size-thumbnailer branch from e1abd8c to 7d92c1c Compare May 7, 2024 07:26
Copy link
Contributor

@mmattel mmattel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM from a docs pov 👍

@DeepDiver1975 DeepDiver1975 force-pushed the feat/max-image-file-size-thumbnailer branch 2 times, most recently from 25e05db to e3f4115 Compare May 8, 2024 11:03
@DeepDiver1975 DeepDiver1975 requested a review from kobergj May 8, 2024 11:03
Copy link
Contributor

@rhafer rhafer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this need a Changelog. Looks good otherwise.

@DeepDiver1975 DeepDiver1975 force-pushed the feat/max-image-file-size-thumbnailer branch from e3f4115 to a0990ce Compare May 8, 2024 11:58
@DeepDiver1975 DeepDiver1975 requested a review from rhafer May 8, 2024 11:58
@DeepDiver1975
Copy link
Member Author

I think this need a Changelog. Looks good otherwise.

done - added to other changelog item which falls into the same topic

@DeepDiver1975 DeepDiver1975 force-pushed the feat/max-image-file-size-thumbnailer branch from a0990ce to 8117fe3 Compare May 8, 2024 12:16
DataEndpoint string `yaml:"data_endpoint" env:"THUMBNAILS_DATA_ENDPOINT" desc:"The HTTP endpoint where the actual thumbnail file can be downloaded." introductionVersion:"pre5.0"`
MaxInputWidth int `yaml:"max_input_width" env:"THUMBNAILS_MAX_INPUT_WIDTH" desc:"The maximum width of an input image which is being processed." introductionVersion:"6.0"`
MaxInputHeight int `yaml:"max_input_height" env:"THUMBNAILS_MAX_INPUT_HEIGHT" desc:"The maximum height of an input image which is being processed." introductionVersion:"6.0"`
MaxInputImageFileSize string `yaml:"max_input_image_file_size" env:"THUMBNAILS_MAX_INPUT_IMAGE_FILE_SIZE" desc:"The maximum file size of an input image which is being processed. Usable common abbreviations: [KB, KiB, MB, MiB, GB, GiB, TB, TiB, PB, PiB, EB, EiB], example: 2GB." introductionVersion:"6.0"`
Copy link
Contributor

@wkloucek wkloucek May 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have some experience how the file size maps to memory consumption when generating a thumbnail?

Just as a "unrelated" question: are there any plans to limit the concurrency for generating thumbnails, too? Because only if we know the approx memory to be used for a thumbnail generation and how many can be generated in parallel, we could effectively constrain the memory limits (eg. in Kubernetes).

cc @d7oc

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have some experience how the file size maps to memory consumption when generating a thumbnail?

not really - and this is rather complicated...

From my understanding the image dimensions are more critical to this. A rather small file can hold an image with gigantic dimensions which can blow up the thumbnailer pretty easy ....

Just as a "unrelated" question: are there any plans to limit the concurrency for generating thumbnails, too?

I did not look into this yet. Feel free to add this to https://github.com/owncloud/enterprise/issues/6388
THX

@DeepDiver1975 DeepDiver1975 force-pushed the feat/max-image-file-size-thumbnailer branch from 8117fe3 to 0be461b Compare May 8, 2024 13:38
Copy link

sonarqubecloud bot commented May 8, 2024

@DeepDiver1975 DeepDiver1975 merged commit 06bb8c9 into master May 8, 2024
4 checks passed
@delete-merged-branch delete-merged-branch bot deleted the feat/max-image-file-size-thumbnailer branch May 8, 2024 14:28
rhafer pushed a commit to rhafer/ocis that referenced this pull request Oct 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants