Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add PHPUnit tests for Image Loading Optimization #898

Merged
merged 52 commits into from
Jan 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
5183678
Add tests for detection.php and fix prefix on filter
westonruter Dec 5, 2023
b1e6f9d
Rename load tests to hooks tests
westonruter Dec 5, 2023
365bd43
Remove needless reliance on ImagesTestCase
westonruter Dec 5, 2023
2190c28
Add tests for storage/lock.php
westonruter Dec 5, 2023
3b5a049
Add tests for post type; improve validation of parsed URL metric
westonruter Dec 5, 2023
c0ef937
Add stubs for data tests
westonruter Dec 5, 2023
e1a13d6
Add tests for REST API
westonruter Dec 6, 2023
5c83545
Discontinue excluding all of WordPress-Extra sniffs from tests and fi…
westonruter Dec 6, 2023
7d8afd8
Add half of tests for storage/data.php functions
westonruter Dec 6, 2023
00b9ccb
Make ilo_unshift_url_metrics() easier to test
westonruter Dec 7, 2023
c9d41c3
Add test for ilo_unshift_url_metrics() and improve logic
westonruter Dec 7, 2023
ea08eda
Add tests for functions to get sample size and breakpoint max widths
westonruter Dec 7, 2023
0f922f3
Add test for ilo_group_url_metrics_by_breakpoint
westonruter Dec 7, 2023
a6d6997
Add tests for ilo_get_lcp_elements_by_minimum_viewport_widths()
westonruter Dec 7, 2023
1d92756
Add remainint test stubs
westonruter Dec 8, 2023
63136ab
Add tests for 2 optimization functions and improve comments
westonruter Dec 8, 2023
746f463
Fix XPath for foreign elements and add tests for ILO_HTML_Tag_Processor
westonruter Dec 8, 2023
59fb175
Add remaining tests for ILO_HTML_Tag_Processor
westonruter Dec 12, 2023
3e77988
Add test for ilo_needs_url_metric_for_breakpoint and fix stub
westonruter Dec 12, 2023
46080ef
Add two test cases for ilo_construct_preload_links and remove integri…
westonruter Dec 12, 2023
e12b798
Add array shape typing for ilo_construct_preload_links() arg
westonruter Dec 13, 2023
0410678
Add remaining tests for ilo_construct_preload_links()
westonruter Dec 13, 2023
1b16857
Add tests for ilo_optimize_template_output_buffer()
westonruter Dec 13, 2023
9265b9d
Fix REST API endpoint to reject metrics for specific unneeded viewpor…
westonruter Dec 14, 2023
1b219f5
Remove debug code
westonruter Dec 14, 2023
e1ac28e
Remove todo
westonruter Dec 14, 2023
7324ce2
Use ILO prefix for test classes
westonruter Dec 14, 2023
966c5b3
Use ILO prefix for test classes
westonruter Dec 21, 2023
7455a2e
Merge branch 'feature/image-loading-optimization' into add/ilo-tests
westonruter Dec 21, 2023
28db827
Run composer update
westonruter Dec 21, 2023
3709688
Fix Generic.CodeAnalysis.UnusedFunctionParameter
westonruter Dec 21, 2023
1b3b859
Fix filter return type for PHPStan static analysis
westonruter Dec 21, 2023
fd43aaa
Fix sending header() in test_perflab_aea_clean_aea_audit_action
westonruter Dec 21, 2023
df0d09e
Add since tags to ILO_HTML_Tag_Processor methods
westonruter Jan 9, 2024
1fa877c
Use test method prefix consistenty instead of redundant tag
westonruter Jan 9, 2024
3da1363
Remove unnecessary ilo_needs_url_metric_for_breakpoint()
westonruter Jan 9, 2024
2181d88
Revert test changes moved to #924
westonruter Jan 9, 2024
930d4ed
Unrevert 2181d88 for server-timing-tests.php
westonruter Jan 9, 2024
8dd3283
Unrevert 2181d88 for audit-enqueued-assets-test.php
westonruter Jan 10, 2024
bf1bd17
Reset SERVER global after each test
westonruter Jan 10, 2024
54396ec
Unset REQUEST_URI in tests that call go_to
westonruter Jan 10, 2024
492fde9
Merge branch 'feature/image-loading-optimization' of https://github.c…
westonruter Jan 11, 2024
461ca6b
Fix PHPStan errors in tests
westonruter Jan 11, 2024
5c009ba
Run composer update
westonruter Jan 11, 2024
c5053f1
Re-run composer update with PHP 8.1
westonruter Jan 12, 2024
d865bac
Update ilo_verify_url_metrics_storage_nonce() to return bool
westonruter Jan 12, 2024
ed305a9
Move ilo_can_optimize_response() to optimization.php
westonruter Jan 12, 2024
e1c05c7
Merge pull request #932 from WordPress/fix/phpstan-errors
mukeshpanchal27 Jan 12, 2024
8874d07
Merge branch 'trunk' of https://github.com/WordPress/performance into…
westonruter Jan 12, 2024
06ac5fe
Add missing since tag
westonruter Jan 12, 2024
4bb2ca8
Add line break before dataProvider tag
westonruter Jan 12, 2024
d62d710
Add additional line breaks in phpdoc
westonruter Jan 12, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@
"require": {
"composer/installers": "~1.0",
"php": ">=7|^8",
"ext-dom": "*",
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added because DOMDocument is used in tests now.

"ext-json": "*"
},
"scripts": {
Expand Down
266 changes: 162 additions & 104 deletions composer.lock

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,8 @@ public function __construct( string $html ) {
* A generator is used so that when iterating at a specific tag, additional information about the tag at that point
* can be queried from the class. Similarly, mutations may be performed when iterating at an open tag.
*
* @since n.e.x.t
*
* @return Generator<string> Tag name of current open tag.
*/
public function open_tags(): Generator {
Expand Down Expand Up @@ -191,7 +193,11 @@ public function open_tags(): Generator {
yield $tag_name;

// Immediately pop off self-closing tags.
if ( in_array( $tag_name, self::VOID_TAGS, true ) ) {
if (
in_array( $tag_name, self::VOID_TAGS, true )
||
( $p->has_self_closing_flag() && $this->is_foreign_element() )
) {
array_pop( $this->open_stack_tags );
}
Comment on lines -194 to 202
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dmsnell: FYI, I'm pretty happy with how has_self_closing_flag and the foreign element aspect of the HTML spec simplified things! Now it can properly get breadcrumbs/XPaths for each SVG/MathML element as well.

} else {
Expand All @@ -200,32 +206,21 @@ public function open_tags(): Generator {
continue;
}

// Since SVG and MathML can have a lot more self-closing/empty tags, potentially pop off the stack until getting to the open tag.
$did_splice = false;
if ( 'SVG' === $tag_name || 'MATH' === $tag_name ) {
$i = array_search( $tag_name, $this->open_stack_tags, true );
if ( false !== $i ) {
array_splice( $this->open_stack_tags, $i );
$did_splice = true;
}
}

if ( ! $did_splice ) {
$popped_tag_name = array_pop( $this->open_stack_tags );
if ( $popped_tag_name !== $tag_name && function_exists( 'wp_trigger_error' ) ) {
wp_trigger_error(
__METHOD__,
esc_html(
sprintf(
/* translators: 1: Popped tag name, 2: Closing tag name */
__( 'Expected popped tag stack element %1$s to match the currently visited closing tag %2$s.', 'performance-lab' ),
$popped_tag_name,
$tag_name
)
$popped_tag_name = array_pop( $this->open_stack_tags );
if ( $popped_tag_name !== $tag_name && function_exists( 'wp_trigger_error' ) ) {
wp_trigger_error(
__METHOD__,
esc_html(
sprintf(
/* translators: 1: Popped tag name, 2: Closing tag name */
__( 'Expected popped tag stack element %1$s to match the currently visited closing tag %2$s.', 'performance-lab' ),
$popped_tag_name,
$tag_name
)
);
}
)
);
}

array_splice( $this->open_stack_indices, count( $this->open_stack_tags ) + 1 );
}
}
Expand All @@ -236,6 +231,8 @@ public function open_tags(): Generator {
*
* A breadcrumb consists of a tag name and its sibling index.
*
* @since n.e.x.t
*
* @return Generator<array{string, int}> Breadcrumb.
*/
private function get_breadcrumbs(): Generator {
Expand All @@ -244,12 +241,30 @@ private function get_breadcrumbs(): Generator {
}
}

/**
* Determines whether currently inside a foreign element (MATH or SVG).
*
westonruter marked this conversation as resolved.
Show resolved Hide resolved
westonruter marked this conversation as resolved.
Show resolved Hide resolved
* @since n.e.x.t
*
* @return bool In foreign element.
*/
private function is_foreign_element(): bool {
foreach ( $this->open_stack_tags as $open_stack_tag ) {
mukeshpanchal27 marked this conversation as resolved.
Show resolved Hide resolved
if ( 'MATH' === $open_stack_tag || 'SVG' === $open_stack_tag ) {
return true;
}
}
return false;
}

/**
* Gets XPath for the current open tag.
*
* It would be nicer if this were like `/html[1]/body[2]` but in XPath the position() here refers to the
* index of the preceding node set. So it has to rather be written `/*[1][self::html]/*[2][self::body]`.
*
* @since n.e.x.t
*
* @return string XPath.
*/
public function get_xpath(): string {
Expand All @@ -266,6 +281,7 @@ public function get_xpath(): string {
* This is a wrapper around the underlying HTML_Tag_Processor method of the same name since only a limited number of
* methods can be exposed to prevent moving the pointer in such a way as the breadcrumb calculation is invalidated.
*
* @since n.e.x.t
* @see WP_HTML_Tag_Processor::get_attribute()
*
* @param string $name Name of attribute whose value is requested.
Expand All @@ -281,6 +297,7 @@ public function get_attribute( string $name ) {
* This is a wrapper around the underlying HTML_Tag_Processor method of the same name since only a limited number of
* methods can be exposed to prevent moving the pointer in such a way as the breadcrumb calculation is invalidated.
*
* @since n.e.x.t
* @see WP_HTML_Tag_Processor::set_attribute()
*
* @param string $name The attribute name to target.
Expand All @@ -297,6 +314,7 @@ public function set_attribute( string $name, $value ): bool {
* This is a wrapper around the underlying HTML_Tag_Processor method of the same name since only a limited number of
* methods can be exposed to prevent moving the pointer in such a way as the breadcrumb calculation is invalidated.
*
* @since n.e.x.t
* @see WP_HTML_Tag_Processor::remove_attribute()
*
* @param string $name The attribute name to remove.
Expand All @@ -312,6 +330,7 @@ public function remove_attribute( string $name ): bool {
* This is a wrapper around the underlying HTML_Tag_Processor method of the same name since only a limited number of
* methods can be exposed to prevent moving the pointer in such a way as the breadcrumb calculation is invalidated.
*
* @since n.e.x.t
* @see WP_HTML_Tag_Processor::get_updated_html()
*
* @return string The processed HTML.
Expand Down
2 changes: 1 addition & 1 deletion modules/images/image-loading-optimization/detection.php
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ function ilo_get_detection_script( string $slug, array $needed_minimum_viewport_
*
* @param int $detection_time_window Detection time window in milliseconds.
*/
$detection_time_window = apply_filters( 'perflab_image_loading_detection_time_window', 5000 );
$detection_time_window = apply_filters( 'ilo_detection_time_window', 5000 );

$detect_args = array(
'serveTime' => microtime( true ) * 1000, // In milliseconds for comparison with `Date.now()` in JavaScript.
Expand Down
67 changes: 54 additions & 13 deletions modules/images/image-loading-optimization/optimization.php
Original file line number Diff line number Diff line change
Expand Up @@ -24,32 +24,65 @@ function ilo_maybe_add_template_output_buffer_filter() {
}
add_action( 'wp', 'ilo_maybe_add_template_output_buffer_filter' );

/**
* Determines whether the current response can be optimized.
*
* Only search results are not eligible by default for optimization. This is because there is no predictability in
* whether posts in the loop will have featured images assigned or not. If a theme template for search results doesn't
* even show featured images, then this isn't an issue.
*
* @since n.e.x.t
* @access private
*
* @return bool Whether response can be optimized.
*/
function ilo_can_optimize_response(): bool {
$able = ! (
// Since the URL space is infinite.
is_search() ||
// Since injection of inline-editing controls interfere with breadcrumbs, while also just not necessary in this context.
is_customize_preview() ||
// The images detected in the response body of a POST request cannot, by definition, be cached.
'GET' !== $_SERVER['REQUEST_METHOD']
);

/**
* Filters whether the current response can be optimized.
*
* @since n.e.x.t
*
* @param bool $able Whether response can be optimized.
*/
return (bool) apply_filters( 'ilo_can_optimize_response', $able );
}

/**
* Constructs preload links.
*
* @since n.e.x.t
* @access private
*
* @param array $lcp_images_by_minimum_viewport_widths LCP images keyed by minimum viewport width, amended with attributes key for the IMG attributes.
* @param array<int, array{attributes: array{src?: string, srcset?: string, sizes?: string, crossorigin?: string}}|false> $lcp_elements_by_minimum_viewport_widths LCP images keyed by minimum viewport width, amended with attributes key for the IMG attributes.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe a personal preference, but I find this notation overkill. This isn't used anywhere in WordPress. If we want to describe more complex shapes like this, we should probably introduce a class for them. This would be safer in any case than using an associative array which by definition takes any arbitrary data.

I think a simple ILO_LCP_Element class or similar would be helpful here. Then this would become ILO_LCP_Element[].

Separate minor question: Why was this renamed to "LCP elements" instead of "LCP images"? Would it ever not be an LCP image? If so, the choice of fields would be a bit odd, since almost all of them are specific to images.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe a personal preference, but I find this notation overkill. This isn't used anywhere in WordPress. If we want to describe more complex shapes like this, we should probably introduce a class for them. This would be safer in any case than using an associative array which by definition takes any arbitrary data.

Actually, it is used in WordPress, specifically in the sodium_compat code. Granted, this is not really organic WordPress code. In any case, with precise phpdoc typing here, PHPStan will detect any anomalies with static analysis. So it's trying to gain the same benefits as TypeScript gives.

I think a simple ILO_LCP_Element class or similar would be helpful here. Then this would become ILO_LCP_Element[].

Yeah, I like that idea. However, believe it or not, I was wanting to limit the refactor of the existing code in this PR once the test coverage was completed. With the coverage in place, refactoring will be safer as I'll be able to detect regressions better.

Separate minor question: Why was this renamed to "LCP elements" instead of "LCP images"? Would it ever not be an LCP image? If so, the choice of fields would be a bit odd, since almost all of them are specific to images.

Yes, for example in #914 this is extended to support elements with CSS background images.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I like that idea. However, believe it or not, I was wanting to limit the refactor of the existing code in this PR once the test coverage was completed. With the coverage in place, refactoring will be safer as I'll be able to detect regressions better.

+1, that makes total sense to me. Maybe open an issue already?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've just now added it as a todo item on the overview issue.

* @return string Markup for zero or more preload link tags.
*/
function ilo_construct_preload_links( array $lcp_images_by_minimum_viewport_widths ): string {
function ilo_construct_preload_links( array $lcp_elements_by_minimum_viewport_widths ): string {
$preload_links = array();

// This uses a for loop to be able to access the following element within the iteration, using a numeric index.
$minimum_viewport_widths = array_keys( $lcp_images_by_minimum_viewport_widths );
$minimum_viewport_widths = array_keys( $lcp_elements_by_minimum_viewport_widths );
for ( $i = 0, $len = count( $minimum_viewport_widths ); $i < $len; $i++ ) {
$lcp_element = $lcp_images_by_minimum_viewport_widths[ $minimum_viewport_widths[ $i ] ];
if ( false === $lcp_element || empty( $lcp_element['attributes'] ) ) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The empty( $lcp_element['attributes'] ) check is removed here because it is always set.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another benefit of using a class to represent LCP elements would be that that would be truly enforced :)

// No LCP element at this breakpoint, so nothing to preload.
$lcp_element = $lcp_elements_by_minimum_viewport_widths[ $minimum_viewport_widths[ $i ] ];
if ( false === $lcp_element ) {
// No supported LCP element at this breakpoint, so nothing to preload.
continue;
}

$img_attributes = $lcp_element['attributes'];
// TODO: Add support for background images.
$attributes = $lcp_element['attributes'];

// Prevent preloading src for browsers that don't support imagesrcset on the link element.
if ( isset( $img_attributes['src'], $img_attributes['srcset'] ) ) {
unset( $img_attributes['src'] );
if ( isset( $attributes['src'], $attributes['srcset'] ) ) {
unset( $attributes['src'] );
}

// Add media query if it's going to be something other than just `min-width: 0px`.
Expand All @@ -60,12 +93,12 @@ function ilo_construct_preload_links( array $lcp_images_by_minimum_viewport_widt
if ( null !== $maximum_viewport_width ) {
$media_query .= sprintf( ' and ( max-width: %dpx )', $maximum_viewport_width );
}
$img_attributes['media'] = $media_query;
$attributes['media'] = $media_query;
}

// Construct preload link.
$link_tag = '<link data-ilo-added-tag rel="preload" fetchpriority="high" as="image"';
foreach ( array_filter( $img_attributes ) as $name => $value ) {
foreach ( array_filter( $attributes ) as $name => $value ) {
// Map img attribute name to link attribute name.
if ( 'srcset' === $name || 'sizes' === $name ) {
$name = 'image' . $name;
Expand Down Expand Up @@ -107,7 +140,12 @@ function ilo_optimize_template_output_buffer( string $buffer ): string {
);

// Whether we need to add the data-ilo-xpath attribute to elements and whether the detection script should be injected.
$needs_detection = ilo_needs_url_metric_for_breakpoint( $needed_minimum_viewport_widths );
$needs_detection = in_array(
true,
// Each array item is array{int, bool}, with the second item being whether the viewport width is needed.
array_column( $needed_minimum_viewport_widths, 1 ),
true
);
Comment on lines +143 to +148
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some cool things with PHPStan's static analysis here. If 0 is supplied as the second arg to array_column() it detects an error:

phpstan: Call to function in_array() with arguments true, list and true will always evaluate to false.

This is because from the array shape returned by ilo_get_needed_minimum_viewport_widths() it knows that the first column will all be integers (the minimum viewport widths) and thus the in_array() check will always be false.

Conversely, if I supply a higher index like 2 in array_column(), I get:

phpstan: Call to function in_array() with arguments true, array{} and true will always evaluate to false.

Here PHPStan knows that since there are only two items in each tuple, the return of array_column() will be an empty array, and thus in_array() will also always return false.


$breakpoint_max_widths = ilo_get_breakpoint_max_widths();
$url_metrics_grouped_by_breakpoint = ilo_group_url_metrics_by_breakpoint( $url_metrics, $breakpoint_max_widths );
Expand Down Expand Up @@ -169,10 +207,13 @@ function ilo_optimize_template_output_buffer( string $buffer ): string {
$processor->remove_attribute( 'fetchpriority' );
}

// TODO: If the image is visible (intersectionRatio!=0) in any of the URL metrics, remove loading=lazy.
// TODO: Conversely, if an image is the LCP element for one breakpoint but not another, add loading=lazy. This won't hurt performance since the image is being preloaded.

// Capture the attributes from the LCP elements to use in preload links.
if ( isset( $lcp_element_minimum_viewport_width_by_xpath[ $xpath ] ) ) {
$attributes = array();
foreach ( array( 'src', 'srcset', 'sizes', 'crossorigin', 'integrity' ) as $attr_name ) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The integrity attribute doesn't exist on IMG so it shouldn't be copied.

foreach ( array( 'src', 'srcset', 'sizes', 'crossorigin' ) as $attr_name ) {
$attributes[ $attr_name ] = $processor->get_attribute( $attr_name );
}
foreach ( $lcp_element_minimum_viewport_width_by_xpath[ $xpath ] as $minimum_viewport_width ) {
Expand Down
Loading