Discussion regarding diff patch updation instead of entire bundle.

[oddlyspaced]

I went through the code to understand the workflow as I want to implement support for patch updating instead of pushing entire bundles to improve the updater performance. I have the following plan in mind which I want to discuss.

Let's take Android for example, currently if an update is found, the file is downloaded by the native module and it unzips the bundle and stores it locally and then changes the bundle path.

Architecturally if we would be including support for diff patches, then there needs to be a way for the native module to be able to check if the downloaded file is a patch or is it an entire bundle. I was thinking this could just be an additional data file that can be sent with the bundle, so that locally the differentiation can be done. If the file is a patch file, we apply bsdiff algorithm to merge the patch with the existing bundle to create a new bundle and then store this locally and set the js bundle path accordingly.

I have the following questions with regards to this :

• how would the differentiation between the bundle / patch thing be done ?
• if we implement patch support, do we provide support for stacking patches or just 1 patch at a time ?
• The bsdiff patch package would also need to be shipped with hot updater. Could that be a concern regarding size etc ?

I want to work on implementing this feature and have a decent idea on how to do it. If these questions can be answered, I can start working on a POC.

Thanks a lot in advance and thanks a lot for building this !

[gronxb]

Thank you for the great suggestion.

https://github.com/gronxb/hot-updater/blob/main/packages/core/src/types.ts

1. This type represents the base bundle entity. Modifying it would affect all database tables, but I believe an is_patch field needs to be added.

2. The bundles table should store one row per patch file. If the is_patch field is true, instead of downloading a full zip file, the patch logic should be applied.

When a build is generated, a UUID v7-based bundleId is created in the bundle using a Babel plugin, and this ID is recorded in the database. The update logic checks the database using this ID.

However, if is_patch is introduced, the existing logic, which checks a single row, would no longer be sufficient. Instead, patch files would need to be applied in a migration-like manner. I’ll need to carefully consider how to handle this on the server side.

3. I’m not sure how large the bsdiff package is. Do you have an estimate of how much difference it would make in terms of size ? My philosophy when building libraries is to aim for zero-dependency as much as possible. I’m not sure if this approach would be viable in this case.

Additionally, one concern is the future support plan for repack’s module federation. While I haven’t thought about it in detail yet, my rough idea is that instead of uploading full bundles, we could upload micro-bundles and update only the necessary parts. In this case, I’m worried that the bsdiff functionality might become a bottleneck.

I’m also unsure whether these three update methods can coexist:

1. Full file updates (Current)
2. Diff-based file updates (bsdiff)
3. Single-file updates (re.pack)

What are your thoughts?

[oddlyspaced]

Thanks a lot for responding, I'll check your points regarding the patch handling and come back here later.

With regards to the 3 update methods, my idea for introducing bsdiff was to provide an option with regards to the update process. At my org we use the base react native framework without any other framework and module federation is something that would require any team to restructure their application and possibly re architecture their project which sometimes might not be feasible. My goal with bsdiff incremental updates is to provide a base level solution that anyone can use. Functionally repack is superior however considering hot updater is becoming more widely adopted especially with codepush going away and a lot of people migrating to hot updater, patch updates is something that I have always missed with codepush and would love to have it here.

[oddlyspaced]

Regarding existence of the 3 methods, repack can exist independently and separately as modules can be updated with it. Patch support would only be available for people who have been doing normal bundle updates.

[gronxb]

Good. Let's discuss the database scheme

How about structuring the patchedParentId field like this, forming a linked list?

[
  {
    "id": "123e4567-e89b-12d3-a456-426614174000",
    "platform": "ios",
    "targetAppVersion": "1.0.0",
    "shouldForceUpdate": false,
    "enabled": true,
    "fileUrl": "https://example.com/bundle/1.0.0-ios.js",
    "fileHash": "abc123hash",
    "gitCommitHash": "abcd1234commit",
    "message": "Initial release",
    "patchedParentId": null
  },
  {
    "id": "223e4567-e89b-12d3-a456-426614174001",
    "platform": "ios",
    "targetAppVersion": "1.0.0",
    "shouldForceUpdate": false,
    "enabled": true,
    "fileUrl": "https://example.com/bundle/1.0.1-ios.js",
    "fileHash": "def456hash",
    "gitCommitHash": "efgh5678commit",
    "message": "Bug fix patch 1",
    "patchedParentId": "123e4567-e89b-12d3-a456-426614174000"
  },
  {
    "id": "323e4567-e89b-12d3-a456-426614174002",
    "platform": "ios",
    "targetAppVersion": "1.0.0",
    "shouldForceUpdate": false,
    "enabled": true,
    "fileUrl": "https://example.com/bundle/1.0.2-ios.js",
    "fileHash": "ghi789hash",
    "gitCommitHash": "ijkl9012commit",
    "message": "Bug fix patch 2",
    "patchedParentId": "223e4567-e89b-12d3-a456-426614174001"
  },
  {
    "id": "423e4567-e89b-12d3-a456-426614174003",
    "platform": "ios",
    "targetAppVersion": "1.0.0",
    "shouldForceUpdate": false,
    "enabled": true,
    "fileUrl": "https://example.com/bundle/1.0.3-ios.js",
    "fileHash": "jkl012hash",
    "gitCommitHash": "mnop3456commit",
    "message": "Bug fix patch 3",
    "patchedParentId": "323e4567-e89b-12d3-a456-426614174002"
  }
]

By structuring it as a linked list, you can easily track which bundle should be patched next. If patchedParentId is null, it means a full update is required.

[gronxb]

Also, PATCHED should be added to the status of getUpdateInfo.

https://github.com/gronxb/hot-updater/blob/main/packages/core/src/test-utils/setupGetUpdateInfoTestSuite.ts

[oddlyspaced]

The linked list concept makes sense to me; however, allowing stacked patch updates could introduce unnecessary complexities.

For example, if a base bundle is X and we have A, B and C updates then having them update like :
X <- A <- B <- C
can lead to a bad UX since in scenarios where the user has a fresh install of the app with bundle X, and then all three patches would need to be applied, which could be compute expensive and also might introduce anomalies.

Instead, we can limit the patch support to 1 parent id only, and at a point in time, let's only have 1 patch to that parent available to begin with.

Thus we can have the Linked List structure you mentioned above, and can easily figure out what patch needs to be applied on which base bundle.

Also, I suppose this parsing would happen in the server right? The app would send the id and details and the server would return the patch / bundle.

Also on a side note :
https://github.com/JimmyDaddy/react-native-bs-diff-patch/

This is the library I have been checking out for bs diff handling. At its core it contains a cpp binary which is then linked and run by the different platforms. Was thinking that if we use this, we can run the same binary pre and post update, that is to generate the bundle in our system server and then using the same binary to patch the binary on the device itself.

[oddlyspaced]

@gronxb can you please point me to the code which is used to figure out if an update is available? I will go through the logic and understand it for better context 👍🏾

[gronxb]

The following is a shared test case designed to ensure that every database-related plugin passes the same test suite:

Common Test Case:

• Test Suite

Supabase (Postgres)

• Core: get_update_info.sql
• Test: get_update_info.spec.ts
• Edge Function: update-server

Cloudflare

• Core: getUpdateInfo.ts
• Test: getUpdateInfo.spec.ts
• Worker: index.ts

[gronxb]

You can modify the package and test it in the examples directory.

Since Supabase is easier to test, I’ll explain using Supabase as an example.

1. Modify the Type Definitions

• Update the types in:
  types.ts
• Likely, the field patchedParentId: string | null will be added.

2. Add a Migration File

• Navigate to:
  migrations folder
• Create a new migration file to add the patchedBundleId field to the existing bundles table.

3. Modify the Edge Function Logic

• Update the logic in:
  update-server/index.ts

4. Build and Deploy

• Move to examples/v0.77.0 and run:

pnpm i
pnpm -w build

to build all workspaces.

• Deploy the updated DB migration and server code using:

pnpm hot-updater init

Now, you can test the changes.
The same process applies to Cloudflare as well.

[gronxb]

Instead, we can limit the patch support to 1 parent id only, and at a point in time, let's only have 1 patch to that parent available to begin with

So, does this mean that a patch can only be applied once?

How should the following scenario work?

X (base) → A (patch) → B (patch) → C (patch)

[oddlyspaced]

Thanks for the steps 🙌🏾
I will check them out and do my POC

So, does this mean that a patch can only be applied once?

How should the following scenario work?

X (base) → A (patch) → B (patch) → C (patch)

My initial idea is to have only 1 patch available for a base bundle.
So for example lets say X is the base bundle that is already shipped via play store / app store.

Now it could be that there is some minor fix that is done, it can be patch A

So it would be :
X (base) <- A (fix patch)

Now considering we already have a patch A live, but we get another minor fix we would need to upload another patch update.

In this case the new patch would include the fix from A and the new fix for B

So now it would be :
X (base) <- B (fix patch A + new fix patch)

At a time only 1 patch would be live. It would be up to the developer what all fixes they are integrating in the patch.

This way, the local updater wouldn't have to deal with complex queue handling and can simply update the patch.

For example in the same scenario, if in the X base bundle, we first apply the patch A, the updater will make a new bundle by merging X and A and use this new bundle. Now lets say we get patch B, the local updater can simply merge X and B and use that new bundle.

With the patch, it would be up to the dev to ensure the patch is functional 👍🏾

[gronxb]

It seems challenging to perform a phased rollback when patches have been applied 3 times. While the bundles table follows the initially proposed linked list approach for patches, the server logic could provide a single diff file by merging the patches sequentially.

We should also consider the rollback process.

[oddlyspaced]

For rollback we can do what we currently and just set the preference to null so as to use the app bundle.

Phased rollback can make things complicated for the feature development 😓. Good points though, I'll also take care of this in my poc.