Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test(e2e): add checks for properties in reference tests #8214

Draft
wants to merge 3 commits into
base: next
Choose a base branch
from

Conversation

RitaDias
Copy link
Contributor

@RitaDias RitaDias commented Jan 8, 2025

Description

Tests added after the latest round of Actions API feedback

❗ some of the tests will fail as we wait for the fixes to land in the backend

  • Add e2e to check that properties are being added when reference is created
  • Add e2e to check that the properties are being removed when the reference and document are published
  • Add e2e to check that the properties are not removed when the weak property is set in the schema

What to review

Do the tests make sense?

Notes for release

N/A internal test

Copy link

vercel bot commented Jan 8, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
page-building-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jan 8, 2025 2:27pm
performance-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jan 8, 2025 2:27pm
test-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jan 8, 2025 2:27pm
2 Skipped Deployments
Name Status Preview Comments Updated (UTC)
studio-workshop ⬜️ Ignored (Inspect) Visit Preview Jan 8, 2025 2:27pm
test-next-studio ⬜️ Ignored (Inspect) Jan 8, 2025 2:27pm

Copy link
Contributor

github-actions bot commented Jan 8, 2025

No changes to documentation

Copy link
Contributor

github-actions bot commented Jan 8, 2025

Component Testing Report Updated Jan 8, 2025 2:31 PM (UTC)

❌ Failed Tests (1) -- expand for details
File Status Duration Passed Skipped Failed
comments/CommentInput.spec.tsx ✅ Passed (Inspect) 1m 5s 15 0 0
formBuilder/ArrayInput.spec.tsx ✅ Passed (Inspect) 13s 3 0 0
formBuilder/inputs/PortableText/Annotations.spec.tsx ✅ Passed (Inspect) 39s 6 0 0
formBuilder/inputs/PortableText/copyPaste/CopyPaste.spec.tsx ✅ Passed (Inspect) 51s 11 7 0
formBuilder/inputs/PortableText/copyPaste/CopyPasteFields.spec.tsx ✅ Passed (Inspect) 0s 0 12 0
formBuilder/inputs/PortableText/Decorators.spec.tsx ✅ Passed (Inspect) 26s 6 0 0
formBuilder/inputs/PortableText/DisableFocusAndUnset.spec.tsx ✅ Passed (Inspect) 14s 3 0 0
formBuilder/inputs/PortableText/DragAndDrop.spec.tsx ❌ Failed (Inspect) 1m 6s 5 0 1
formBuilder/inputs/PortableText/FocusTracking.spec.tsx ✅ Passed (Inspect) 1m 7s 15 0 0
formBuilder/inputs/PortableText/Input.spec.tsx ✅ Passed (Inspect) 1m 28s 21 0 0
formBuilder/inputs/PortableText/ObjectBlock.spec.tsx ✅ Passed (Inspect) 1m 40s 18 0 0
formBuilder/inputs/PortableText/PresenceCursors.spec.tsx ✅ Passed (Inspect) 12s 3 9 0
formBuilder/inputs/PortableText/Styles.spec.tsx ✅ Passed (Inspect) 26s 6 0 0
formBuilder/inputs/PortableText/Toolbar.spec.tsx ✅ Passed (Inspect) 1m 42s 21 0 0
formBuilder/tree-editing/TreeEditing.spec.tsx ✅ Passed (Inspect) 0s 0 3 0
formBuilder/tree-editing/TreeEditingNestedObjects.spec.tsx ✅ Passed (Inspect) 0s 0 3 0

Copy link
Contributor

github-actions bot commented Jan 8, 2025

⚡️ Editor Performance Report

Updated Wed, 08 Jan 2025 14:32:49 GMT

Benchmark reference
latency of sanity@latest
experiment
latency of this branch
Δ (%)
latency difference
article (title) 25.0 efps (40ms) 24.4 efps (41ms) +1ms (+2.5%)
article (body) 47.1 efps (21ms) 45.9 efps (22ms) +1ms (+2.6%)
article (string inside object) 27.0 efps (37ms) 26.3 efps (38ms) +1ms (+2.7%)
article (string inside array) 23.3 efps (43ms) 22.7 efps (44ms) +1ms (+2.3%)
recipe (name) 51.3 efps (20ms) 51.3 efps (20ms) +0ms (-/-%)
recipe (description) 58.8 efps (17ms) 55.6 efps (18ms) +1ms (+5.9%)
recipe (instructions) 99.9+ efps (6ms) 99.9+ efps (6ms) +0ms (-/-%)
synthetic (title) 19.0 efps (53ms) 18.9 efps (53ms) +1ms (+1.0%)
synthetic (string inside object) 20.8 efps (48ms) 19.6 efps (51ms) +3ms (+6.3%)

efps — editor "frames per second". The number of updates assumed to be possible within a second.

Derived from input latency. efps = 1000 / input_latency

Detailed information

🏠 Reference result

The performance result of sanity@latest

Benchmark latency p75 p90 p99 blocking time test duration
article (title) 40ms 43ms 75ms 399ms 753ms 10.8s
article (body) 21ms 23ms 27ms 66ms 127ms 5.8s
article (string inside object) 37ms 39ms 46ms 220ms 268ms 6.7s
article (string inside array) 43ms 44ms 49ms 84ms 196ms 7.0s
recipe (name) 20ms 21ms 31ms 48ms 0ms 6.7s
recipe (description) 17ms 19ms 20ms 37ms 0ms 4.3s
recipe (instructions) 6ms 8ms 9ms 28ms 0ms 3.6s
synthetic (title) 53ms 55ms 61ms 265ms 640ms 11.9s
synthetic (string inside object) 48ms 50ms 55ms 67ms 196ms 7.6s

🧪 Experiment result

The performance result of this branch

Benchmark latency p75 p90 p99 blocking time test duration
article (title) 41ms 48ms 64ms 380ms 986ms 10.8s
article (body) 22ms 25ms 37ms 128ms 380ms 6.0s
article (string inside object) 38ms 40ms 44ms 85ms 164ms 6.7s
article (string inside array) 44ms 48ms 56ms 329ms 447ms 7.5s
recipe (name) 20ms 21ms 23ms 40ms 0ms 6.8s
recipe (description) 18ms 20ms 21ms 33ms 0ms 4.4s
recipe (instructions) 6ms 8ms 8ms 16ms 0ms 3.1s
synthetic (title) 53ms 56ms 67ms 123ms 506ms 13.2s
synthetic (string inside object) 51ms 53ms 58ms 260ms 740ms 8.2s

📚 Glossary

column definitions

  • benchmark — the name of the test, e.g. "article", followed by the label of the field being measured, e.g. "(title)".
  • latency — the time between when a key was pressed and when it was rendered. derived from a set of samples. the median (p50) is shown to show the most common latency.
  • p75 — the 75th percentile of the input latency in the test run. 75% of the sampled inputs in this benchmark were processed faster than this value. this provides insight into the upper range of typical performance.
  • p90 — the 90th percentile of the input latency in the test run. 90% of the sampled inputs were faster than this. this metric helps identify slower interactions that occurred less frequently during the benchmark.
  • p99 — the 99th percentile of the input latency in the test run. only 1% of sampled inputs were slower than this. this represents the worst-case scenarios encountered during the benchmark, useful for identifying potential performance outliers.
  • blocking time — the total time during which the main thread was blocked, preventing user input and UI updates. this metric helps identify performance bottlenecks that may cause the interface to feel unresponsive.
  • test duration — how long the test run took to complete.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant