Support Mac App Store apps

[vitorgalvao]

Mac App Store apps keep files inside ~/Library/Containers/. These cannot be replaced with soft links (ln -s), but do work with hard links (ln) and OS X aliases. Since hard links cannot be used on directories, this would work for many apps, but not all.

My suggestion would be to add CONTAINERS = 'Library/Containers/' to # Commonly used paths #, and then when mackup encounters an app that references something that starts with CONTAINERS, use the applescript way; else, make a symbolic link as always.

[lra]

That's annoying.

I tried with Vox though and it seems that replacing the ~/Library/Containers/com.coppertino.Vox folder with a link elsewhere did not cause any problem.

What broken behavior did you experienced ?

[vitorgalvao]

The same one you’ve experienced with sparrow. Maybe it happens only when we try to sync entire directories, and not for singular files. I’ll have to perform some tests on this.

[vitorgalvao]

Yep, from my limited testing right now, (sip, where only one file needs to be linked, and sketch, where we need to link one file and a directory), it seems like linking directories inside containers is the only case where it causes problems (in sketch’s case you lose important settings, like the customisation of your toolbar).

[lra]

Do you mean linking files directly works ?
(e.g. instead of linking Containers/Sip you link Containers/Sip/File1 and Containers/Sip/File2)

[vitorgalvao]

Seems to be the case, yes. Linking ~/Library/Cotainers/com.ruiaureliano.Sip/Data/Library/Preferences/com.ruiaureliano.Sip.plist works for sip, but linking ~/Library/Cotainers/com.bohemiancoding.sketch/Data/Library/Preferences/com.bohemiancoding.sketch.plist and ~/Library/Cotainers/com.bohemiancoding.sketch/Data/Library/Application Support/sketch does not work for sketch.

Linking individual files would not be a long-term solution, however, as in some cases whole directories do have to be copied over. In sketch, for instance, inside that linked directory is where user plugins reside, and those will be different depending on the user.

[vitorgalvao]

New development. It seems that this can break or not on a case-by-case basis, so we should probably just get a solution that’ll act on files inside Containers, and settle on that, for those cases. Readkit, while only needing two files to maintain it’s settings, will break if the one inside Application Support is a symlink. Also, other apps (like Sip, mentioned previously), seem to no longer work with symlinks, after running new tests. I’m not sure what’s different, but we shouldn’t rely on something that might-or-not work, and find a more permanent solution.

[lra]

Yeah, links are ruled out. The app dev need to ask for permissions when submitting to the app store and tell which file outside of the Container app folder it needs to access, with a good reason.
Hard links or file copy are the only options here.
But hard links are not supported by dropbox (it wont detect changes)
And file copy means mackup need to watch over those files for changes from time to time.

Mackup could add itself to the user's crontab and diff/sync those files every X min, keeping latest modified all the time (local or dropbox), but there might be problems when multiple computers are on.

[vitorgalvao]

I agree that using cron (or launchd, even), is not an optimal solution, and should really be avoided.

The issue here seems to be how to convey to the user that some apps will have limited backup capabilities, so it is clear that those will need a bit more care to be kept up to date.

I don’t have a solution to this, but maybe the start of a suggestion. Since applications will essentially be divided into two groups (“always in sync” and “sync on demand”), maybe actually splitting them could help make that distinction (I’m thinking something to the effect of how homebrew uses taps). This way there would be an “opt-in” to these apps, making users aware of the why of this limitation.

Another option would be to only sync apps specified in the .mackup.cfg file, effectively forcing users to set it up, and be conscious about the limitations of certain cases (maybe apps would have to be organised under different sections of the file).

Both this options have the particularity of not making users worry about “is this app automatically synced, or do I have to do it myself regularly?”, since they’d be made aware of that when setting up, but they do come with the burden of requiring a bit extra work when initially setting up. I believe that initial hurdle to be more advantageous than having users puzzled about why some backups “just work”, while others do not.

These solutions would change mackup’s design philosophy a bit, true, but right now it is a choice between being more convenient and being more complete.

[lra]

We don't need 2 "taps" to handle apps differently, a simple config option in the app cfg can work.
e.g.

[application]
name = Super Duper
sync_method = copy

For this app mackup will not link the conf files, but just copy them when it's being run.

Then as you mentioned, the real problem here is that we let users in the dark, not knowing what's in sync.

That's where the mackup status feature in the TODO can help.
It could display the list of apps both installed locally or not installed but with a conf in Dropbox.
For each of them it would say in sync or show when the last sync have been done, or even if it has never been done.
Obviously, for apps with linked config files, it would always display in sync, but at least the user could know what config is outdated (from the apps with a copy sync_method) and showing that it has not been synced for 20 days can help the user decide.

Making mackup optin instead of optout is another question.

[vitorgalvao]

I actually very much like that solution. Having mackup status work in such a way would allow mackup to function as usual, and would give a user the tools to understand the limitations of some cases right from the app. I’m all for it.

[lra]

Cool I guess we have a plan then, thanks for helping clarifying this out.
IMHO the plan should be:

1. split out mackup from one big main.py to a bunch of modules (that was the point of 0.6 work)
2. implement a status option
3. implement a copy sync method
4. add app store apps in

[vitorgalvao]

Sounds good, but shouldn’t option 2 be the last one? If there’s some change that needs to be done to the copy sync method (something we’re not foreseeing now that’ll only be apparent during implementation), status can be rethought to accommodate that. That would be more efficient than having to change status after it has been implemented, in case some problem does occur.

[lra]

status being just a display option, it can work and be usuful now, it will just display "in sync" for the applications that are currently synced.
We don't need 3. or 4. to build 2., but we can't have 3. or 4. without 2. so it's better to build 2. before no?

[vitorgalvao]

I was imagining them as being more independent. 3 and 4 can exist without 2 (the limitations just won’t be as clear), and vice-versa. I was thinking of 3 and 4 first since some complications might occur that will change the design of the feature, while 2, being an option that will complement that, could adapt itself to how it turns out, instead of having to change later in case the feature does end up having to be rethought.

However, you know the tool better, and the sequence of implementation shouldn’t make that big of difference, so I trust the way you find to be more efficient.

[lra]

That's true, but I don't see support for copy/sync methods without letting the user know what's happening, given by the status.
At worse, if we hit a wall, the display feature will be there and useful anyway.

[yangm97]

Have you tried mount -o bind? Or put the link in dropbox instead of in the app's dir? Maybe mounting the app's dir in another directory then symlink it?
Copying files every minute looks very bad for me. Also, it can take tons of precious space on SSDs, and maybe increase battery usage.
EDIT: Something like this: symlink /something to dropbox then mount app's dir at /something.

[lra]

I don't see how mount -o bind can help here, can you get into more details if you have a solution ? Keep in mind that Dropbox need real files in his folder to detect changes and sync them, or it will do nothing.

Putting symlinks in Dropbox does not work as the point is to sync the files content.

[yangm97]

I was thinking about making a symlink to a folder inside Dropbox and then mount the contents of the app's did in the symlink.
Something like this: http://cl.ly/image/0P0o1C161A3X
Also, for apps that aren't always creating files, we can hardlink all of its contents by using cp -al

[lra]

@yangm97:

• You can't mound -o bind a folder on a folder, at least on stock OS X, and it would not work for files anyway
• Putting a link in the middle won't fool the OS.
• There is no -l option on OS X cp and hardlinks are not an option as dropbox is not able to detect changes.

I might have totally missed your point, but I don't get it.

[yangm97]

Ok, now I'm really out of ideas now, looks like Apple has moded GNU tools, I was testing these things on Ubuntu.

[vitorgalvao]

looks like Apple has moded GNU tools

That wouldn’t be very productive. I think they simply use the BSD versions (since OS X is based on it).

[torarnv]

What's the status of this? Is anyone working on a solution? Has a consensus been formed on how to solve it?

[scruffydan]

Also interested in the status of this.

Thanks.

[lra]

Well, unless we have a working solution, there is no progress on this.
Anyone has a "working" plan?

[lra]

Once #613 is in, it should not be a problem