Do we really want to clip the sourceRect of createImageBitmap to the input's dimensions?

[Kaiido]

#1546 introduced a normative change that made the source rectangle clipped to the image's dimensions (#1546 (comment)).

When this change was introduced it was described as being "consistent with drawImage", however as a web-author I would like to dispute this claim.

If I try to make a parallel between the use of drawImage and the clipping of createImageBitmap I would use the sx, sy, sw, sh parameters of createImageBitmap to produce the clipped source like the same parameters are being used in the 9-length version of drawImage, then use the 5-length version to resize this cropped ImageBitmap.

// using the 9-length version directly
imgCanvas.getContext( "2d" ).drawImage( imgEl, sx, sy, sw, sh, dx, dy, dw, dh );
// using createImageBitmap's cropping + 5-length drawImage
const bmp = await createImageBitmap( imgEl, sx, sy, sw, sh );
bmpCanvas.getContext( "2d" ).drawImage( bmp, dx, dy, dw, dh );
// [Blink only] use resizeWidth and resizeHeight
const bmpR = await createImageBitmap( imgEl, sx, sy, sw, sh, { resizeWidth: dw, resizeHeight: dh } );
bmpResizeCanvas.getContext( "2d" ).drawImage( bmpR, dx, dy );

And doing so, we end up with very different results, as demonstrated in this fiddle:

• If ran in Blink, which does implement the current specified behavior, both results using createImageBitmap are stretched on the x-axis because both the call to drawImage(bmp, dx, dy, dw, dh) and the resizeWidth and resizeHeight options assumed the input would keep the square aspect-ratio defined by sw and sh.
• In Firefox, where this change hasn't made it yet, (I opened https://bugzilla.mozilla.org/show_bug.cgi?id=1687216), both working results are the same, which is more like what I'd call a "consistent" result. (The third result is different because this browser doesn't support the resizeWidth and resizeHeight options).

So I'd like to gather thoughts on this behavior, which in my own opinion is particularly surprising.

Also, I'll take the chance to note that this "Clip x to y's dimensions" doesn't seem to be linked to any definition an implementer should follow. I am myself in the process of writing a js monkey-patch for this method and I have to assume that for instance when the sourceRectangle is completely outside the input's bounds we have to return a transparent ImageBitmap with the dimensions from sourceRectangle, only because I saw it's what Blink does when I was writing a (now aborted) patch there.

Ps: Related WPT tests PR.

cc @jakearchibald

[jakearchibald]

I think the confusing detail is from https://html.spec.whatwg.org/multipage/canvas.html#the-canvas-element

When the source rectangle is outside the source image, the source rectangle must be clipped to the source image and the destination rectangle must be clipped in the same proportion.

It seems like createImageBitmap doesn't apply the second part of this, where the destination rectangle is clipped in the same proportion.

Feels like the drawImage line would be less confusing if it was just "pixels outside the source image bounds are transparent black", but I can't figure out if that might produce an observable difference.

[Kaiido]

Regarding drawImage's clipping, (may be worth its own ticket?)

The only place I could find where changing the drawImage clipping behavior would make a difference is when using the "clear" composite mode, currently only available in Blink: https://jsfiddle.net/m9zL17gq/

There, even transparent pixels are used for the compositing (at least in current implementation, I'm not a specialist of compositing and I did not try to understand the specs there).

I am not sure what would be web-authors' expectations here though.

And while maybe less important, one benefit of clipping here is that implementations can early exit in case of empty sourceRect (e.g in Blink), but I'm not sure the specs should enforce the clipping just for that, maybe simply allowing it would be enough, but forcing transparent pixels may also cause some useless overhead.

---

Back to createImageBitmap

So we agree the current behavior is unexpected? I believe removing this step is quite straightforward, only Blink has implemented it and I already have a patch ready for them if needed.

[annevk]

cc @whatwg/canvas

[jakearchibald]

So we agree the current behavior is unexpected?

I think so, yeah.

I believe removing this step is quite straightforward, only Blink has implemented it and I already have a patch ready for them if needed.

Seems fair. The test should be changed too.

[junov]

Feels like the drawImage line would be less confusing if it was just "pixels outside the source image bounds are transparent black", but I can't figure out if that might produce an observable difference.

The reason that the rectangles are clipped rather than filling with transparent black is so that drawImage will render a rectangle with sharp edges. Filling with transparent black would result, in some cases, in an edge that is blurry (when imageSmoothingEnabled is set to true).

I know this because I've had to fix that bug a long time ago. If I recall correctly, a web developer who was rendering content tile by tile reported a bug that there were seams in the rendered output. So this little detail actually matters.

[kdashg]

For e.g. createImageBitmap([3x3 image], 2, 2, 2, 1) I expect it to give me a 2x1 ImageBitmap with the left pixel being the src's bottom-right pixel, and right pixel transparent black.
I think it's important for devs to explicitly opt-in to clipping/content-dependent-resizing, and that it's a good invariant that you get a result of the same size that you requested.

There's some divergence here between partial-pixel-targetable drawImage and working with whole-integer-pixels like for ImageBitmap, which is that drawImage expects to be able to filter non-integer-scale resizes smoothly, and there's no equivalent solution for ImageBitmap except rounding, which I think is fraught to do behind the dev's back.

Ideally we spec the most "obvious" default behavior, and enable users to do more sophisticated things themselves.

[fserb]

Was there some discussion regarding if this breaks current content?

[kdashg]

When content is currently broken in at least one browser, we have pretty strong latitude to standardize it however we think is best. Put another way, content will be broken whatever direction we choose.

[fserb]

ahn, okey. I didn't notice that. My bad. Yeah, if that's the case, I agree with you.

[junov]

Good point. According to MDN, createImageBitmap is not yet shipping in Safari. So yeah... any page that uses it is currently broken.

…

On Tue, Feb 2, 2021 at 1:45 PM Jeff Gilbert ***@***.***> wrote: When content is currently broken in at least one browser, we have pretty strong latitude to standardize it however we think is best. Put another way, content will be broken whatever direction we choose. — You are receiving this because you are on a team that was mentioned. Reply to this email directly, view it on GitHub <#6306 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAT4V6DPNNWT7I6ZKGPDOBTS5BB23ANCNFSM4WGVDAMQ> .