Key based array merging

[NightProgramming]

I think providing the possibility to merge based on key instead of order would be nice:

const x = [{id: 1, value: "a"}, {id: 2, value: "b"}];
const y = [{id: 2, value: "b updated"}, {id: 1, value: "a updated"}];

const mergeSettings = {key: "id"};
const merged = deepmergeCustom(mergeSettings)(x, y);

console.log(merged);
// [{id: 1, value: "a updated"}, {id: 2, value: "b updated"}]

Maybe also provide the option to provide paths:

const x = {arr: [{id: 1, value: "a"}, {id: 2, value: "b"}], otherProp: {otherArray: [{myKey: "myKey1"}]}};
const mergeSettings = {keys: ["arr.id", "otherProp.otherArray.myKey"]};

[RebeccaStevens]

This can already be done with deepmergeCustom as it is written to be very customizable.

Here's an implementation of merging arrays based on a element's key:

////////////////////
// utils.ts

import { deepmerge, deepmergeCustom, type DeepMergeOptions } from "deepmerge-ts";

export function hasProp<T, K extends PropertyKey>(
  value: T,
  prop: K
): value is T & Record<K, unknown> {
  return typeof value === "object" && value !== null && prop in value;
}

export function myArrayDeepmerge<K extends PropertyKey>(id: K) {
  const mergeSettings: DeepMergeOptions = {
    mergeArrays: (values, utils) => {
      const valuesById = values.reduce<Map<unknown, unknown[]>>(
        (carry, current) => {
          const currentElementsById = new Map<unknown, unknown>();
          current.forEach((element) => {
            if (!hasProp(element, id)) {
              throw Error("Invalid element type"); // You can change this to simply ignore the value or keep it as is, rather than throwing.
            }
            if (currentElementsById.has(element[id])) {
              throw Error("multiple elements with the same id"); // You can change this to simply ignore duplicates. In which case `currentElementsById` is not longer needed. If you can guarantee this else where then that also justifies removing this check.
            }
            currentElementsById.set(element[id], element);

            const currentList = carry.get(element[id]) ?? [];
            carry.set(element[id], [...currentList, element]);
          });
          return carry;
        },
        new Map<unknown, unknown[]>()
      );

      return [...valuesById.values()].reduce(
        (carry, values) => [...carry, deepmerge(...values)],
        []
      );
    }
  };

  return deepmergeCustom(mergeSettings);
}

////////////////////
// some-other-file.ts

import { myArrayDeepmerge} from "./path/to/utils.ts";

const x = [
  { id: 1, value: "a" },
  { id: 2, value: "b" }
];
const y = [
  { id: 2, value: "b updated" },
  { id: 1, value: "a updated" }
];
const merged = myArrayDeepmerge("id")(x, y);

console.log(merged);
// [{id: 1, value: "a updated"}, {id: 2, value: "b updated"}]

As this merge function is designed to merge arrays of the same type; you shouldn't need to provide any fancy HKT definitions. The default ones should work just fine.

Using key paths is also possible but you would need to provide me with more details on the how you want it to work for me to implement it.

With regard to adding this sort of merge function directly to the library. I am open to it but I'll need more feedback from everyone. We'd also need to nail down what should happen in the error cases.

[NightProgramming]

Thank you for your detailed answer! Basically what I mean is that the key paths provide a way to reach a key, starting from the root (the objects we pass to deepmerge) and ending at the key property. Then that key should be used to merge the array that is one layer above the object that contains the key.

An example object and key path:

const x = {outerArr: [{arr: [{id: 1}]}, {arr: [{id: 2}]}]};
const keyPaths = ["outerArr.arr.id"]

So id will be used to merge the objects of the two inner arrays. Lets say we have y:

const y = {outerArr: [{arr: [{id: 3, newProp: "newProp"}]}]};

For the outer array we don't have any key path, so default merging should apply. But it wouldn't be useful it the default array merging logic here would be concatenating the arrays. It would need to be a combine approach that is based on same-index-logic meaning there is an attempt to merge {arr: [{id: 1}]} (coming from x.outerArr[0]) with {arr: [{id: 3, newProp: "newProp"}]} (coming from y.outerArr[0]). To proceed we have to merge the properties of those two objects. We see they both have arr and it's not a primitive so no we know we have to merge those two arrays. Now we are at a point where the key path applies so we don't do the combine-based approach anymore but a key-based approach. We take the last part from the key path (which is "id" ) to identify the keys of the objects that are inside the two arrays we are about to merge. Id 1 and 3 don't match so we cannot merge{id: 1} with{id: 3, newProp: "newProp"}. Instead we have to keep both. So the overall result would be

{outerArr: [{arr: [{id: 1}, {id: 3, newProp: "newProp"}]}, {arr: [{id: 2}]}] }

Instead if the ids would match (imagine y.outerArr[0].arr[0] gives {id: 1, newProp: "newProp"}) we would get

{outerArr: [{arr: [{id: 1, newProp: "newProp"}]}, {arr: [{id: 2}]}] }

It could also be that the key path isn't applicable to one (or even both) object(s) passed to deepmerge. Like with the following y:

const y = {outerArr: []};

In that case normal merging applies (which would return an object that is identical to x in this example). I wouldn't throw an error.

Also I wouldn't make the decision to apply combine-based-logic instead of concat-based-logic on key paths, instead I would maybe offer it as option for the whole merging process.

[RebeccaStevens]

I'm going to close this issue for now and track the progress in #33.
Once that feature is added, your original use case should fairly easily to implement.

[RebeccaStevens]

@NightProgramming I've just released a beta for version 3 that has the meta data feature in it.

Here's a test I added that does key-based array merging:

deepmerge-ts/tests/deepmerge-custom.test.ts

Lines 409 to 530 in 42e0207

	test("key path based array merging", (t) => {
	const x = {
	foo: [
	{ id: 1, value: ["a"] },
	{ id: 2, value: ["b"] },
	],
	bar: [1, 2, 3],
	baz: {
	qux: [
	{ id: 1, value: ["c"] },
	{ id: 2, value: ["d"] },
	],
	},
	qux: [
	{ id: 1, value: ["e"] },
	{ id: 2, value: ["f"] },
	],
	};
	const y = {
	foo: [
	{ id: 2, value: ["g"] },
	{ id: 1, value: ["h"] },
	],
	bar: [4, 5, 6],
	baz: {
	qux: [
	{ id: 2, value: ["i"] },
	{ id: 1, value: ["j"] },
	],
	},
	qux: [
	{ id: 2, value: ["k"] },
	{ id: 1, value: ["l"] },
	],
	};
	const expected = {
	foo: [
	{ id: 1, value: ["a", "h"] },
	{ id: 2, value: ["b", "g"] },
	],
	bar: [1, 2, 3, 4, 5, 6],
	baz: {
	qux: [
	{ id: 1, value: ["c", "j"] },
	{ id: 2, value: ["d", "i"] },
	],
	},
	qux: [
	{ id: 1, value: ["e"] },
	{ id: 2, value: ["f"] },
	{ id: 2, value: ["k"] },
	{ id: 1, value: ["l"] },
	],
	};
	const customizedDeepmergeEntry = <K extends PropertyKey>(
	...idsPaths: ReadonlyArray<ReadonlyArray<K>>
	) => {
	const mergeSettings: DeepMergeOptions<
	ReadonlyArray<unknown>,
	Readonly<Partial<{ id: unknown; key: PropertyKey }>>
	> = {
	metaDataUpdater: (previousMeta, metaMeta) => {
	return [...previousMeta, metaMeta.key ?? metaMeta.id];
	},
	mergeArrays: (values, utils, meta) => {
	const idPath = idsPaths.find((idPath) => {
	const parentPath = idPath.slice(0, -1);
	return (
	parentPath.length === meta.length &&
	parentPath.every((part, i) => part === meta[i])
	);
	});
	if (idPath === undefined) {
	return utils.defaultMergeFunctions.mergeArrays(values);
	}
	const id = idPath[idPath.length - 1];
	const valuesById = values.reduce<Map<unknown, unknown[]>>(
	(carry, current) => {
	const currentElementsById = new Map<unknown, unknown>();
	for (const element of current) {
	if (!hasProp(element, id)) {
	throw new Error("Invalid element type");
	}
	if (currentElementsById.has(element[id])) {
	throw new Error("multiple elements with the same id");
	}
	currentElementsById.set(element[id], element);
	const currentList = carry.get(element[id]) ?? [];
	carry.set(element[id], [...currentList, element]);
	}
	return carry;
	},
	new Map<unknown, unknown[]>()
	);
	return [...valuesById.entries()].reduce<unknown[]>(
	(carry, [id, values]) => {
	const childMeta = utils.metaDataUpdater(meta, { id });
	return [
	...carry,
	deepmergeCustom(mergeSettings, childMeta)(...values),
	];
	},
	[]
	);
	},
	};
	return deepmergeCustom(mergeSettings, []);
	};
	const merged = customizedDeepmergeEntry(["foo", "id"], ["baz", "qux", "id"])(
	x,
	y
	);
	t.deepEqual(merged, expected);
	});

[NightProgramming]

@RebeccaStevens Would it be possible to make the custom merge code a bit shorter by not having to have to code the default merging?

So that writing only return; would be equal to return utils.defaultMergeFunctions.mergeArrays(values);. Meaning that the lib would apply default merging if the custom merger returns undefined. It would make the custom code shorter when not explicitly having to apply the default behavior.

Of course we would then also need a way to be able to tell the library explicitly that we want to use undefined as the result of the custom merge function. I guess we could do it with a sambol e.g. return utils.useUndefined; where useUndefined is a symbol coming from the deepmerge lib, so that the lib code could check if the return value of the call of the custom merger is this symbol and then use undefined as result of the custom merging.

Or the custom merger could receive a callback (used like utils.useUndefined()) in order to tell the lib that undefined should be used.

Maybe we could even have the option to skip merging (via utils.skip()). Meaning the result won't contain the current property key at all.

[NightProgramming]

@RebeccaStevens Also, would it be possible to get some more meta data? Assuming a property value should be set, based on values from other properties. It would be nice to not only get the property values and the property name but also be able to get the property owners.

Example:

const x = {a: 1, isBadObject: true};
const y = {a: 2, isBadObject: false};

...

mergeOthers: (values, utils, meta) => {
    const { key } = meta;
    if (key === "isBadObject") {
        return false;  // The overall merge result object will be a good object :)
    }        
    if (key === "a") {
        const owners = meta.getOwners();
        let resultSum = 0;
        for (let i = 0; i < values.length; i++) {
            if (!owners[i].isBadObject) {
                resultSum += values[i];
            }
            else {
                // Skip bad object.
            }
        }
        return resultSum ;
    }
    return; // Trigger default merging (see my above post)
}

I used a function call to get the owners, because otherwise it might not be good for performance if meta would always provide the owners array out-of-the-box, instead of only providing it when commanded to do so.

Via metaDataUpdater a client could also provide the full ancestor chain to the custom merger, if for some reason the client needs that info. Note: In that ancestor chain also arrays could occur (not just plain objects). So when merging something like {arr: [{a: 1}]}. The ancestors of the inner object would be the array and then the outer object.