Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: configurable artifact max payload size #320

Merged
merged 2 commits into from
Nov 30, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions changes/pr320.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# An example changelog entry
#
# 1. Choose one (or more if a PR encompasses multiple changes) of the following headers:
# - feature
# - enhancement
# - fix
# - deprecation
# - breaking (for breaking changes)
# - migration (for database migrations)
#
# 2. Fill in one (or more) bullet points under the heading, describing the change.
# Markdown syntax may be used.
#
# 3. If you would like to be credited as helping with this release, add a
# contributor section with your name and github username.
#
# Here's an example of a PR that adds an enhancement

enhancement:
- "Configure the maximum artifact payload - [#320](https://github.com/PrefectHQ/server/pull/320)"

contributor:
- "[Joël Luijmes](https://github.com/joelluijmes)"
3 changes: 3 additions & 0 deletions src/prefect_server/config.toml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,9 @@ insert_many_batch_size = 200
# path for user configuration file
user_config_path = "~/.prefect_server/config.toml"

# maximum amount of bytes artifact accepts, defaults to 1 mb.
artifacts_max_payload_bytes = 1000000

[api]
url = 'http://localhost:4200'

Expand Down
7 changes: 5 additions & 2 deletions src/prefect_server/graphql/artifacts.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
from graphql import GraphQLResolveInfo

from prefect import api
from prefect_server import config
from prefect_server.utilities.graphql import mutation


Expand All @@ -17,8 +18,10 @@ async def resolve_create_task_run_artifact(
data = input.get("data")

data_size = sys.getsizeof(json.dumps(data))
if data_size > 1000000: # 1 mb max
raise ValueError("Artifact data payload exceedes 1Mb limit.")
if data_size > config.artifacts_max_payload_bytes:
raise ValueError(
f"Artifact data payload exceeds {config.artifacts_max_payload_bytes} bytes limit."
)

task_run_artifact_id = await api.artifacts.create_task_run_artifact(
task_run_id=input.get("task_run_id"),
Expand Down