Cache stampede problem in cacheable package

[mlois-efimob]

Describe the bug
Concurrent calls to wrap for the same key, results in a cache stampede and all concurrent calls falls in the expensive wrapped function call. This behaviour is not present on cache-manager.

How To Reproduce (best to provide workable code or tests!)
To reproduce the problem with cacheable:

import { Cacheable } from 'cacheable';

// Memory store by default
const memoryCache = new Cacheable();

function doWork(id: number) {
  return new Promise((resolve) => {
    setTimeout(() => {
      console.log(`return data doWork: ${id}`);
      resolve('data');
    }, 3000);
  });
}

const getCachedUser = memoryCache.wrap(doWork, {
  keyPrefix: 'user'
});

(async () => {
  const start = process.hrtime.bigint();
  console.log('start');
  console.log(
    await Promise.all([
      getCachedUser(1),
      getCachedUser(2),
      getCachedUser(3),
      getCachedUser(1),
      getCachedUser(2),
      getCachedUser(3),
      getCachedUser(1),
      getCachedUser(2),
      getCachedUser(3)
    ])
  );
  const duration = Number(process.hrtime.bigint() - start) * 1e-9;
  console.log(`end: ${duration} s`);
})().catch(console.error);

This produces the output, with the stampede problem:

start
return data doWork: 1
return data doWork: 2
return data doWork: 3
return data doWork: 1
return data doWork: 2
return data doWork: 3
return data doWork: 1
return data doWork: 2
return data doWork: 3
[
  'data', 'data',
  'data', 'data',
  'data', 'data',
  'data', 'data',
  'data'
]
end: 3.0066516020000003 s

But cache-manager manages it correctly even with layered caches:

import { createCache } from 'cache-manager';
import Keyv from 'keyv';

// Memory store by default
const memoryCache = createCache({
  stores: [new Keyv()]
});

function doWork(id: number) {
  return new Promise((resolve) => {
    setTimeout(() => {
      console.log(`return data doWork ${id}`);
      resolve('data');
    }, 3000);
  });
}

async function getCachedUser(userId: number) {
  return memoryCache.wrap(`user_${userId}`, () => doWork(userId));
}

(async () => {
  const start = process.hrtime.bigint();

  console.log('start');
  console.log(
    await Promise.all([
      getCachedUser(1),
      getCachedUser(2),
      getCachedUser(3),
      getCachedUser(1),
      getCachedUser(2),
      getCachedUser(3),
      getCachedUser(1),
      getCachedUser(2),
      getCachedUser(3)
    ])
  );

  const duration = Number(process.hrtime.bigint() - start) * 1e-9;
  console.log(`end: ${duration} s`);
})().catch(console.error);

This produces the output, with only one call per Id:

start
return data doWork 1
return data doWork 2
return data doWork 3
[
  'data', 'data',
  'data', 'data',
  'data', 'data',
  'data', 'data',
  'data'
]
end: 3.0093118490000004 s

[jaredwray]

@mlois-efimob - thanks so much for finding this issue and we will be looking into this.

[jaredwray]

#891

[jaredwray]

@mlois-efimob - thanks so much for your example and also trying out Cacheable 🎉 We have added in coalesce-async to the wrap function now and verified it is now handling stampede issues. This will be released with v1.8.4 today. 🍻

[mlois-efimob]

Hi, no problem at all if we can make the lib better, I saw the PR and seems that is the missing piece between the packages cacheable and cache-manager.