Getting 'Unexpected end of JSON input' exception in encryption.js

[rmisio]

Case

bug

Issue

Getting the following exception:

encryption.js:25 Uncaught (in promise) SyntaxError: Unexpected end of JSON input
    at Object.parse (<anonymous>)
    at Crypter._decryptValue (encryption.js:25)
    at crypter.js:62
    at Array.map (<anonymous>)
    at Crypter.decrypt (crypter.js:59)
    at RxCollection._handleFromPouch (rx-collection.js:192)
    at rx-collection.js:314
    at Array.map (<anonymous>)
    at rx-collection.js:313

Not exactly sure which part of my code triggers it, since the stack trace references none of my files.

Info

• Environment: browser
• Adapter: IndexedDB
• Stack: React

Code

Here's my schema:

const schema = {
  title: 'User profile schema',
  description: 'Database schema for the profile of a user',
  version: 0,
  type: 'object',
  properties: {
    peerID: {
      type: 'string',
      primary: true,
    },
    handle: {
      type: 'string',
      encrypted: true,
    },    
    name: {
      type: 'string',
      encrypted: true,
    },
    location: {
      type: 'string',
      encrypted: true,
    },
    about: {
      type: 'string',
      encrypted: true,
    },
    shortDescription: {
      type: 'string',
      encrypted: true,
    },
    nsfw: {
      type: 'boolean',
      encrypted: true,
    },
    vendor: {
      type: 'boolean',
      encrypted: true,
    },
    moderator: {
      type: 'boolean',
      encrypted: true,
    },
    // Will work this in later
    moderatorInfo: {
      type: ['object', 'null'],
      encrypted: true,
    },
    contactInfo: {
      type: ['object', 'null'],
      encrypted: true,
      properties: {
        website: {
          type: 'string',
        },
        email: {
          type: 'string',
        },
        phoneNumber: {
          type: 'string',
        },
        social: {
          type: 'array',
          uniqueItems: true,
          items: {
            type: 'object',
            properties: {
              type: {
                type: 'string'
              },
              username: {
                type: 'string'
              },
              proof: {
                type: 'string'
              },
            }
          }
        }
      }
    },
    colors: {
      type: ['object', 'null'],
      encrypted: true,
      properties: {
        primary: {
          type: 'string',
        },
        secondary: {
          type: 'string',
        },
        text: {
          type: 'string',
        },
        highlight: {
          type: 'string',
        },
        highlightText: {
          type: 'string',
        },
      }
    },
    avatarHashes: {
      type: ['object', 'null'],
      encrypted: true,
      properties: {
        tiny: {
          type: 'string'
        },
        small: {
          type: 'string'
        },
        medium: {
          type: 'string'
        },
        large: {
          type: 'string'
        },
        original: {
          type: 'string'
        },
      }
    },    
  },
  required: ['peerID', 'name']
};

Here's a dump of my db:

{
  "name": "a0e62fc96f1ca251998d512db1353c21f008d8c071b5c46a6212f9ab8e854c1c9",
  "instanceToken": "oodzoxksdi",
  "encrypted": true,
  "passwordHash": "a76bb4891fe30596c51e410f63e42bd9",
  "collections": [
    {
      "name": "profile",
      "schemaHash": "132ae4ac54c7552c18d7e47881deca12",
      "encrypted": true,
      "passwordHash": "a76bb4891fe30596c51e410f63e42bd9",
      "docs": [
        {
          "moderatorInfo": "U2FsdGVkX1+/wnlXNw6+M43WciG7cJLSWGAMFZswKiM=",
          "contactInfo": "U2FsdGVkX18vJeCWXulo+yzOH8LZvo/116TZz2G44xg=",
          "colors": "U2FsdGVkX19NDXFNWacnl6lhnIrvWZWVj1tCbxKyuik=",
          "avatarHashes": "U2FsdGVkX19H+VaarclSvJrfKQAENQ0UNM3N/zsmBIg=",
          "name": "U2FsdGVkX1/FV1R+rnIAokULhcZJLl+gkvcstrtgzgU=",
          "shortDescription": "U2FsdGVkX19oniIoqdi2wfrzfTtooaKoVhRGIWvTDRI=",
          "handle": "U2FsdGVkX1+ZlspmtKqfPLQiQ69poS6Yus5GE6MUHt4=",
          "location": "U2FsdGVkX19RpHIltTXvMcKJIX5mAJxiHgoMLbGA+sE=",
          "about": "U2FsdGVkX184Sc8xWQSI00mE/TwlPeiZSZi9/XxGw0o=",
          "nsfw": "U2FsdGVkX1+lfcIIxHwN6vvMHUiEPCMH33ux2frXpnI=",
          "vendor": "U2FsdGVkX1+nqr63Th0KW8qdgjzrZCTC+k+ytIdye4w=",
          "moderator": "U2FsdGVkX19qmzgLZJ2lO6b51u9C85Rt8VlXWlRB9oc=",
          "peerID": "QmRHvW41Ga6wQuayQNbrhwrTghasabxwWwqmT5hrWXQLoj"
        }
      ]
    }
  ]
}

[pubkey]

Hi @rmisio thanks for reporting.
This looks like a serious bug. Can you create some code to reproduce it?
It seems that it has something to do with the encryption/decryption.

[rmisio]

Ok, here's basically what I'm doing... I tweaked the code a little to simplify it:

// actual schema included in the issue description
import profileSchema from 'schema/profile';

const collections = [
  {
    name: 'profile',
    schema: profileSchema,
    sync: true,
  },
];

// logging into the database
  const db = await RxDB.create({
    name,
    adapter: 'idb',
    password
  });

// create collections
await Promise.all(collections.map(data => db.collection(data)));

// sync
collections
  .filter(col => col.sync)
  .map(col => col.name)
  .map(colName => db[colName].sync({
    remote: syncUrl + colName + '/'
  }));

It looks like the sync part (the last code block) is what's causing the issue because if I remove that part, I don't get the error.

[rmisio]

I don't see anything of note in the sync db server logs. And, the issue happens whether the sync server is running or not.

[info] GET /profile/ 200 - 127.0.0.1
[info] GET / 200 - 127.0.0.1
[info] GET / 200 - 127.0.0.1
[info] GET /profile/_local/UDhjNoSHlMAKG2i4MVh4Lg%3D%3D? 404 - 127.0.0.1
[info] GET /profile/_changes?style=all_docs&since=0&limit=100 200 - 127.0.0.1
[info] GET /profile/_local/UDhjNoSHlMAKG2i4MVh4Lg%3D%3D? 404 - 127.0.0.1
[info] GET /profile/_changes?style=all_docs&since=1&limit=100 200 - 127.0.0.1
[info] POST /profile/_all_docs?conflicts=true&include_docs=true 200 - 127.0.0.1
[info] PUT /profile/_local/UDhjNoSHlMAKG2i4MVh4Lg%3D%3D 201 - 127.0.0.1
[info] GET /profile/_local/0zRedgtcE9JzV6BTSxjzEg%3D%3D? 404 - 127.0.0.1
[info] POST /profile/_revs_diff 200 - 127.0.0.1
[info] PUT /profile/_local/0zRedgtcE9JzV6BTSxjzEg%3D%3D 201 - 127.0.0.1
[info] GET /profile/_local/UDhjNoSHlMAKG2i4MVh4Lg%3D%3D? 200 - 127.0.0.1
[info] PUT /profile/_local/UDhjNoSHlMAKG2i4MVh4Lg%3D%3D 201 - 127.0.0.1

[pubkey]

It look like you replicate unencrypted data into the rxdb collection

[rmisio]

It look like you replicate unencrypted data into the rxdb collection

I'm not sure what you mean?

Actually in it's simplest form I could replicate this without any data. I just follow the code that I posted and prior to adding in any data to the db, the error happens.

Where are you seeing that I am "replicate unencrypted data into the rxdb collection"?

[pubkey]

I think you sync with a couchdb-instance that has stored the fields without encryption. Then you have non-encrypted data locally and rxdb tries to decrypt it, which is not possible and causes the crash

[rmisio]

I think you sync with a couchdb-instance that has stored the fields without encryption.

hmm... I'm not sure how I did that. I've used rxdb before and set-up encrypted data which I've synced remotely. I've used the same set-up I'm using now, outside of perhaps now using newer versions of rxdb and pouchdb-server.

So, what exactly do I need to do to be able to sync this encrypted data?

This is the db server I'm using:
https://github.com/pouchdb/pouchdb-server

[pubkey]

Can you query the documents directly from the colelctions pouchdb myRxDBCollection.pouch.find.. and post them.

[rmisio]

@pubkey Sorry for the delayed response. i got side-tracked onto another project.

Anyhow, here's the sole document in the collection (this is via the Fauxton UI):

{
  "_id": "QmRHvW41Ga6wQuayQNbrhwrTghasabxwWwqmT5hrWXQLoj",
  "moderatorInfo": "U2FsdGVkX1816aBGy5mKmjOvK8oEHmsbf1DKeLaixjQ=",
  "contactInfo": "U2FsdGVkX18tLTTvLqTuIuF5M+YOInD47l2rtzF1Dqs=",
  "colors": "U2FsdGVkX19MlotFN/N0MPSdWGKZ/Xruy7aHDWIZk2k=",
  "avatarHashes": "U2FsdGVkX1++YWxwSvyu6Qoh0MGzYlfy5blQMR/VdiY=",
  "name": "U2FsdGVkX19AGZ36WxwyhlmIUzWQ/mpGmF0rkjUXPtA=",
  "shortDescription": "U2FsdGVkX1/N2LvmR1KAUfRyWl5/q1CUck4zqdHDprWMuo6Wf9g5BfO6IVUXj4WtMPnpupKFCTAW/T4x102mEw==",
  "handle": "U2FsdGVkX19RmTUzOYe7y2AHYKlQBWbCp4sGMJ3af5M=",
  "location": "U2FsdGVkX18v2dTpw+3/heOVULVvfwZZEe6cmJt3PsY=",
  "about": "U2FsdGVkX1/B+NkuSzbyxuNI1W5z5R7CaRPdthtY/S4=",
  "nsfw": "U2FsdGVkX184CA22Y9VXGNmRIY5+47OjnAc/v2g7LmQ=",
  "vendor": "U2FsdGVkX19n1OuvMofGzOC7L7xDgM9n71U9e9GzXVc=",
  "moderator": "U2FsdGVkX1+g1l3hNDlZH4zWcbhKXKdMDFfTpyvOX6g=",
  "_rev": "1-8a6d0b1e2298dce21042a25013c6f5c4"
}

[rmisio]

Not sure if this is relevant, but I'm also getting this in the console:

GET http://localhost:5984/profile/_changes?style=all_docs&feed=longpoll&heartbeat=10000&since=1&limit=100 net::ERR_INCOMPLETE_CHUNKED_ENCODING 200 (OK)

[rmisio]

Also that same exception in encryption.js happens if I change the data from pouchdb web UI. So, something in the data exchange between the remote and local db's is getting borked.

[pubkey]

How was the initial document created? The base64-string are not encrypted json from rxdb or?

[rmisio]

I pull the name and shortDescription from a form:

{
  "name": "John Doe",
  "shortDescription": "Hi! How are you?"
}

and then I pass them here:

export const save = (data = {}) => {
  const profileData = {
    moderatorInfo: null,
    contactInfo: null,
    colors: null,
    avatarHashes: null,
    ...data,
  }

  return new Promise((resolve, reject) => {
    getDB()
      .then(
        db => {
          return db.profile.upsert(profileData);
        }
      )
      .then(
        profile => resolve(profile),
      )
      .catch(e => {
        reject(e);
        throw e;
      });
  });
}

There's no encryption happening on my end. I think it's happening in RxDB or one of it's dependencies.

[rmisio]

@pubkey Any update on this?

[pubkey]

@rmisio no sorry, there is no update from my side.
Can you modify this file to reproduce the error?
Or create a minimal repo that reproduces this.

[rmisio]

@pubkey Here's a test that reproduces this one.

https://github.com/rmisio/rxdb/blob/master/test/unit/encrypt-bug-917.test.js

I've found the issue is that it throws this exception on the read of an encrypted field of type boolean or string (maybe others, haven't tried) which both doesn't have a default declared in the schema and was not passed in a value when saving.

So, it does let you save without issue, just chokes on a subsequent read.

[pubkey]

@rmisio thanks for the test.
I could pin this down and fixed it with the attached commit
The problem was that when a optional encrypted field is not set, rxdb did encrypt the empty string and later could not decrypt that because an empty string is no valid json.