Enhancement: support multiple overlayWith in single pipeline (including blend modes) via vips_composite

[ghost]

I'm looking for option to combine multiple images, side by side or one under another.
I looked the overlayWith function but seems it can't do what is required. I also looked in
libvips options and maybe .reduce is what we need. Please share with me any idea of realization if there is nothing implemented yet.

[lovell]

Hello, if you've not already seen them, a few similar questions are #405 #597 and #699

Using the current API, I'd probably suggest looking at #470 (comment) for how to start with a blank canvas. This can then be processed multiple times, each using overlayWith with the relevant values for top and left.

[Etiene]

I'm trying to do the same thing. So far the example code on #405 looks the cleanest to me.

However, it creates a new sharp object of the whole thing on each small image that is overlaid over a bigger one. In terms of performance this is impractical for me. Are there any other solutions?

Thanks

[lovell]

Hi @Etiene, your performance concerns may be covered by #573.

[Etiene]

Hello @lovell, I installed the branch version, performance is still undoable. Are there any modifications to the code example I need to make?

Is there no way to do something like

sharp('base.jpg')
  .overlayWith('mask1.png',{top:10, left:10})
  .overlayWith('mask2.png',{top:80, left:100})
  .overlayWith(.....)
  .....
  .toBuffer(....)

?

[lovell]

@Etiene The branch referred to in #573 is a WIP and does not yet improve things. The API you propose, although unsupported at present, would make a great future possible enhancement, thank you.

[wmertens]

How about, in the meantime, printing a warning when using a second overlayWith in the same pipeline? I was pretty confused when it wasn't working…

[Lasto13]

This would be great

[lovell]

There's a discussion about adding an image composition operation to the underlying libvips library that will help with this feature request - see https://github.com/jcupitt/libvips/issues/657

[CWSpear]

The upstream issue was closed a few weeks ago. Is this unblocked now?

[lovell]

@CWSpear This will require libvips v8.6.0 (sharp v0.19.x). If you're interested in submitting a PR then sharp has a work-in-progress suit branch you can experiment with now.

[rui-cruz]

Hi @lovell / @CWSpear any news on adopting this great feature? I would really appreciate having this feature. I can help on https://github.com/DefinitelyTyped/DefinitelyTyped/tree/master/types/sharp.

[rui-cruz]

Hi @lovell, thank you for working on this. Did you made any progress?

Kind regards

[lovell]

@rui-cruz This task is now unblocked, thanks for the nudge. PRs welcome as always.

[rui-cruz]

I wish I could help, but I have no knowledge for such thing :( I can help with the types for typescript though..

[adityapatadia]

+1

[uriesk]

+1
i am creating images out of a tiled map and it is painfully slow to create a new sharp object

[lovell]

Commit 7cafd43 on the uptake branch adds a new composite operation that accepts an array of images and blend modes to composite with. As a result the existing overlayWith function will be deprecated.

The proposed API is documented at https://github.com/lovell/sharp/blob/uptake/docs/api-composite.md

It can be tested now by using npm install --build-from-source lovell/sharp#uptake

This will be in sharp v0.22.0.

[jamesvillarrubia]

It can be tested now by using npm install lovell/sharp#uptake

Hi @lovell This library is great! I've got 0.21.3 running and was looking to fiddle with the composite api. I've set up a light weight node script with a duplicate of the red/blue/green boxes that you use in the unit tests as well as the sample code, but toBuffer seems to return only the buffer for the first image in the sharp constructor. The composite images never seem to be added, though the code is run (and I've stepped through it).

Is there a final pipeline trigger required to get the composites to combine?

[lovell]

@jamesvillarrubia This feature is not in v0.21.3 - it is only available in the uptake branch - it will be part of the forthcoming v0.22.0.

[jamesvillarrubia]

Sorry, I was unclear. I’m on the uptake branch and I’m trying to help test it, but I can’t get the composite function to trigger.

[lovell]

@jamesvillarrubia Please can you provide a code sample, perhaps as a separate repo/gist, that exhibits this behaviour.

[mtbottens]

@lovell

I'm seeing a similar thing. If i pull down the source of this repo, it seems to work, but when using it as a dependency it doesn't seem to work.

Check this out: https://github.com/mtbottens/sharp-composite-example/blob/master/index.js

If you clone down that repo and run node index.js it will generate a test.png file. I'm seeing actual.png as a result, where expected.png is the file generated by the unit tests in this repo here: https://github.com/lovell/sharp/blob/uptake/test/unit/composite.js#L69-L77

[jamesvillarrubia]

Of course! Should have done that to begin with:
https://github.com/jamesvillarrubia/sharp-buffer-rest-api-example/blob/master/app.js

When I hit localhost:3000/build-image, I only see the image of the bookshelf, but no overlay of the checkmark. When I step through, the sharp object just after composite and before toBuffer() is:

{
  "_readableState": {
    "objectMode": false,
    "highWaterMark": 16384,
    "buffer": {
      "head": null,
      "tail": null,
      "length": 0
    },
    "length": 0,
    "pipes": null,
    "pipesCount": 0,
    "flowing": null,
    "ended": false,
    "endEmitted": false,
    "reading": false,
    "sync": true,
    "needReadable": false,
    "emittedReadable": false,
    "readableListening": false,
    "resumeScheduled": false,
    "destroyed": false,
    "defaultEncoding": "utf8",
    "awaitDrain": 0,
    "readingMore": false,
    "decoder": null,
    "encoding": null
  },
  "readable": true,
  "domain": null,
  "_events": {},
  "_eventsCount": 1,
  "_writableState": {
    "objectMode": false,
    "highWaterMark": 16384,
    "finalCalled": false,
    "needDrain": false,
    "ending": false,
    "ended": false,
    "finished": false,
    "destroyed": false,
    "decodeStrings": true,
    "defaultEncoding": "utf8",
    "length": 0,
    "writing": false,
    "corked": 0,
    "sync": true,
    "bufferProcessing": false,
    "writecb": null,
    "writelen": 0,
    "bufferedRequest": null,
    "lastBufferedRequest": null,
    "pendingcb": 0,
    "prefinished": false,
    "errorEmitted": false,
    "bufferedRequestCount": 0,
    "corkedRequestsFree": {
      "next": null,
      "entry": null
    }
  },
  "writable": true,
  "allowHalfOpen": true,
  "options": {
    "sequentialRead": false,
    "limitInputPixels": 268402689,
    "iccProfilePath": "/Users/neptune/Sites/mtc_v5/sharp-buffer-rest-api-example/node_modules/sharp/lib/icc/",
    "topOffsetPre": -1,
    "leftOffsetPre": -1,
    "widthPre": -1,
    "heightPre": -1,
    "topOffsetPost": -1,
    "leftOffsetPost": -1,
    "widthPost": -1,
    "heightPost": -1,
    "width": 800,
    "height": 400,
    "canvas": "crop",
    "position": 0,
    "resizeBackground": [
      0,
      0,
      0,
      255
    ],
    "useExifOrientation": false,
    "angle": 0,
    "rotationAngle": 0,
    "rotationBackground": [
      0,
      0,
      0,
      255
    ],
    "rotateBeforePreExtract": false,
    "flip": false,
    "flop": false,
    "extendTop": 0,
    "extendBottom": 0,
    "extendLeft": 0,
    "extendRight": 0,
    "extendBackground": [
      0,
      0,
      0,
      255
    ],
    "withoutEnlargement": false,
    "kernel": "lanczos3",
    "fastShrinkOnLoad": true,
    "tintA": 128,
    "tintB": 128,
    "flatten": false,
    "flattenBackground": [
      0,
      0,
      0
    ],
    "negate": false,
    "medianSize": 0,
    "blurSigma": 0,
    "sharpenSigma": 0,
    "sharpenFlat": 1,
    "sharpenJagged": 2,
    "threshold": 0,
    "thresholdGrayscale": true,
    "trimThreshold": 0,
    "gamma": 0,
    "gammaOut": 0,
    "greyscale": false,
    "normalise": 0,
    "booleanBufferIn": null,
    "booleanFileIn": "",
    "joinChannelIn": [],
    "extractChannel": -1,
    "removeAlpha": false,
    "ensureAlpha": false,
    "colourspace": "srgb",
    "composite": [
      {
        "input": {
          "failOnError": true,
          "file": "overlay.png"
        },
        "blend": "over",
        "tile": false,
        "left": -1,
        "top": -1,
        "gravity": 6
      }
    ],
    "fileOut": "",
    "formatOut": "png",
    "streamOut": false,
    "withMetadata": false,
    "withMetadataOrientation": -1,
    "resolveWithObject": false,
    "jpegQuality": 80,
    "jpegProgressive": false,
    "jpegChromaSubsampling": "4:2:0",
    "jpegTrellisQuantisation": false,
    "jpegOvershootDeringing": false,
    "jpegOptimiseScans": false,
    "jpegOptimiseCoding": true,
    "jpegQuantisationTable": 0,
    "pngProgressive": false,
    "pngCompressionLevel": 9,
    "pngAdaptiveFiltering": false,
    "pngPalette": false,
    "pngQuality": 100,
    "pngColours": 256,
    "pngDither": 1,
    "webpQuality": 80,
    "webpAlphaQuality": 100,
    "webpLossless": false,
    "webpNearLossless": false,
    "tiffQuality": 80,
    "tiffCompression": "jpeg",
    "tiffPredictor": "horizontal",
    "tiffPyramid": false,
    "tiffSquash": false,
    "tiffTile": false,
    "tiffTileHeight": 256,
    "tiffTileWidth": 256,
    "tiffXres": 1,
    "tiffYres": 1,
    "tileSize": 256,
    "tileOverlap": 0,
    "linearA": 1,
    "linearB": 0,
    "input": {
      "failOnError": true,
      "file": "bookshelf.jpeg"
    }
  }
}

[lovell]

Ah, sorry, you'll need to pass the --build-from-source flag to npm (I've updated my comment to include this).

[jamesvillarrubia]

Perfect! Figured it was something simple. For those that read this later, I also had to point pyenv python2 to any appropriate version for the node-gyp to work. In the repo, I ran:

pyenv local 3.6.6 2.7.8
npm install --build-from-source

and that solved it!

[ovaris]

Hi, testing out the new composite function but for some reason the end result contains only the first image from inputsteam array, here's my code:

const composeImage = async ([background, ...layers]: Array<ReadStream>): Promise<Sharp>=> {
  const baseSharpInstance = sharp();
  background.pipe(baseSharpInstance);

  const promises = layers.map(async stream => {
    const s = sharp();
    stream.pipe(s);
    const buffer = await s.toBuffer();
    return { input: buffer };
  })
  const inputs = await Promise.all(promises);
  return baseSharpInstance.composite(inputs);
};

So for some reason output image only contains only first image (background).

Any Ideas?

[jamesvillarrubia]

You might check out the comment above. I had the same issue - I wasn't using npm install --build-from-source

[lovell]

v0.22.0 now available with this feature, thanks all for the testing and feedback.

[tcz]

This is awesome! Thank you so much for this!

[Dramex]

i replaced overlayWith with composite but cutout parameters doesn't work.
composite([{ input, cutout: true }])

[lovell]

@Dramex Use blend: 'dest-in' - there's an example in the readme - https://github.com/lovell/sharp/blob/master/README.md#stream