Storing user data to chrome.storage

[zombieFox]

Investigate storing data to user account: https://developer.chrome.com/extensions/storage

Problems solving:

• Syncing data between devices on same chrome account

To investigate:

• Does Firefox also support user data storage?

[adrianmihalko]

Isn't possible to completely drop any local storage / chrome storage? It would be great to store settings in a simple file, or a database. I have many devices, phone/desktop and I am using multiple browsers Safari, FF, Chrome. Maybe somebody is good at server side languages, for simplicity like PHP and can help us. It can't be so hard to write/read settings to/from a file. 🤔

Yesterday I checked many startpages at /r/startpages and overall this was the best. 👍

[zombieFox]

Yesterday I checked many startpages at /r/startpages and overall this was the best. 👍

Thanks!

It would be great to store settings in a simple file, or a database.

Just so we are on the same page, are you using the export/import feature on nightTab? This does save the settings and data in a JSON file:

[adrianmihalko]

This is almost what I am looking for.

What do you think, it will be possible to read json from file? I mean automatically read (for example) settings.json from root directory? In this case I can export data, write manually to file (settings.json), but I don't need to import and store locally in local storage in every browser, since it will be automatically imported from settings.json.

[zombieFox]

I think I understand what you want, but that kind of configuration is not user friendly for a browser extension. nightTab is primarily an extension and browsers users shouldn't need to edit files to use it.

However if you want to host the files locally, as I think you are describing, you can directly edit the bookmarks.js file. This is read from the project on first load -- so not from local storage.

[zombieFox]

Also, @hartk1213 has made an awesome tool which generates state.js and bookmark.js files from a JSON for scenarios like this. It could be what you're looking for:
NightTabHelper
(Kudos to hartk1213!)

[adrianmihalko]

Very cool, this is what I am looking for, because I want to use as a standalone web app, instead of extension.

[orwells-ghost]

Any updates on this? I spent two hours trying to edit the files directly with no luck, and the NightTabHelper is dated. It would be awesome if you could add an option for persistent storage outside of local storage for those of us who are privacy focused and automatically clear our caches.

[JetLaggedJackal]

I am also joining the bandwagon for this possible feature as I am sure some of us will definitely want to host it locally.
As few people previously mentioned, this is one of the most polished start/dash pages I have seen even thou it was intended to be used as browser extension only.
I am willing to contribute for that cup of coffee.

[zombieFox]

Any updates on this? I spent two hours trying to edit the files directly with no luck, and the NightTabHelper is dated. It would be awesome if you could add an option for persistent storage outside of local storage for those of us who are privacy focused and automatically clear our caches.

@orwells-ghost It should be possible to edit the bookmarks.js and state.js files and host the project with your preferred bookmarks. What specifically are you having trouble with?

[zombieFox]

I am also joining the bandwagon for this possible feature as I am sure some of us will definitely want to host it locally.

As it stands, I'm still investigating what I need to do to make this happen. nightTab is a side project and I don't devote all my free time to it. So I'm slowly reading up and learning about chrome.storage.

[zombieFox]

Docs:
https://developer.chrome.com/extensions/storage
https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/storage/sync

Notes:

• Firefox and Chrome have different API methods:
  • chrome.storage.sync.set and storage.StorageArea.set
• Firefox has a limit of 100KB of sync storage
  • This would mean local image backgrounds are out of the questions

[JetLaggedJackal]

I am also joining the bandwagon for this possible feature as I am sure some of us will definitely want to host it locally.

As it stands, I'm still investigating what I need to do to make this happen. nightTab is a side project and I don't devote all my free time to it. So I'm slowing reading up and learning about chrome.storage.

Thanks for quick reply nonetheless. I ended up editing two files mentioned earlier and hosting it as web page, but yeah it would be nice to be able to save/read a local config file if it's hosted as a web page.
Then I could just dockerize it and you get a out-of-the-box dashboard.

[AlfaJackal]

This startpage is by far the best I have ever seen. Unfortunately I have the same issue: Hosting it on a server with my edited bookmarks. I used gulp and built the project and I can use everything except saving across devices.

Editing and compiling all these .js-files on every change is too much trouble. It really would be nice if there would be a single .json for bookmarks and a config file for the rest.

[zombieFox]

Editing and compiling all these .js-files on every change is too much trouble. It really would be nice if there would be a single .json for bookmarks and a config file for the rest.

Just so I am clear, are you editing bookmarks.js and moving that to a different computer?

Does the Export feature not work for you @AlfaJackal ?

[zombieFox]

I have been developing more test projects usingchrome.storage. Here are my findings, tl;dr: Not looking good and will probably not be possible:

• Firefox and Chrome use different methods to sync, which makes implementation difficult.
• Sync size limits for Firefox and Chrome are very small (8,192 bytes), much smaller than the size of a nightTab exported backup object, (which is what would need to be synced for this to work).

Not good at all.

[AlecDusheck]

String compression is 100% worth a look here. For example, there are multiple implementations of lz-string in pure Javascript.

Using Dave Brown's compression algorithm got my config all the way down to 6421 bytes (compared to 10769 originally!). I'll just dump his code below for everyone to look at:

function en(c){var x='charCodeAt',b,e={},f=c.split(""),d=[],a=f[0],g=256;for(b=1;b<f.length;b++)c=f[b],null!=e[a+c]?a+=c:(d.push(1<a.length?e[a]:a[x](0)),e[a+c]=g,g++,a=c);d.push(1<a.length?e[a]:a[x](0));for(b=0;b<d.length;b++)d[b]=String.fromCharCode(d[b]);return d.join("")}

function de(b){var a,e={},d=b.split(""),c=f=d[0],g=[c],h=o=256;for(b=1;b<d.length;b++)a=d[b].charCodeAt(0),a=h>a?d[b]:e[a]?e[a]:f+c,g.push(a),c=a.charAt(0),e[o]=f+c,o++,f=a;return g.join("")}

The en function will compress the string and the de function will decompress. Some people claim there may be some issues but I couldn’t find any

Source: Stack Overflow

TLDR: Check out string compression with lz-string. Should reduce most configs below limit.

[AlecDusheck]

But without string compression (mentioned above) or by creating some API (which costs $$); there isn't a way to do this afik

[JalinWang]

Does the "unpin" operation mean this feature gonna release soon?

[zombieFox]

Does the "unpin" operation mean this feature gonna release soon?

I'm afraid not, it still seems like this feature request is too difficult to support (for reasons stated above). This issue was unpinned so that another ticket could gain focus.

[sansmoraxz]

[> Docs: https://developer.chrome.com/extensions/storage https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/storage/sync

Notes:

• Firefox and Chrome have different API methods:

  • chrome.storage.sync.set and storage.StorageArea.set

• Firefox has a limit of 100KB of sync storage

  • This would mean local image backgrounds are out of the questions

](#93 (comment))

The apis are now fully in sync. unlimitedStorage permission maybe used to bypass the limit imposed by storage.local. But for storage.sync it's different for each browser.

But storage is object based rather than string based. So maybe limit to storing only truthy objects and prevent sync storage memory hogging. viz. If there's no image or video url present don't store it. And storage.sync.getBytesInUse() can be used to generate warnings when synced data is near limit.

Or implement UI for what items they want to sync.

Still the default limit of 5 MB is pretty huge. And there's some native compressions already in play for synced data.

[metruzanca]

Still the default limit of 5 MB is pretty huge. And there's some native compressions already in play for synced data.

Unfortunately we're constrained by feature parity so, like ZombieFox mentioned, its 100kb. (Chrome may have 5MB but if Firefox only has 100KB, we have to go with firefox's maximum as a universal max.

---

An issue of the compression/decompression is that would likely result in a load time delay.

[CrazyWolf13]

Hi @zombieFox

Have there yet been any successful attempts at syncing via Browser api sync?

Possibly some other nice feature, if implementation via browser api is not possible, why not add a field where one can enter a link to a raw json file serving the config?, Nighttab then caches that locally and updates in the background.

This would make it really easy, to just dump the config to any pastebin, github raw link, wastebin server etc.

This way the url itself could be synced over via browser api.