-
-
Notifications
You must be signed in to change notification settings - Fork 137
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Prevent editing of unsynchronized notes #161
Comments
Thanks for reporting this issue, this is really a problem which should be solved. The problem also occurs in another scenario:
However, in this scenario, your approach of preventing the editing is not sufficient (the mobile user will still overwrite changes done by the other device). Furthermore, it slows down the interaction with the app -- although concurrent writes of notes are typically rather rarely. In some cases this will be just some milliseconds, but in other cases, this will be several seconds up to a minute (worst case: many large notes, slow device, slow and erroneous connection). Therefore, another solution for this issue should be found. Unfortunately, this problem is not that easy. Therefore, I'm writing some thoughts down, but please feel free to add own ideas or concerns. One enhancement would put your note-update to the background and then try to merge the remote changes with the (hopefully few) local changes. This could be done using a diff/merge-algorithm. A challenge is, that this has to be done in realtime (the user can do more changes in the meanwhile) and merging can produce messy conflicts. Furthermore, note-update is only possible if the network connection is active and stable. Therefore, a solution for any scenario must realize an optimistic conflict-resolution which checks for concurrent writes when the note is synchronized back to the server. However, this requires changes in the server app (we could do that) which then has to reject the change (this should not happen) or likewise implement the diff/merge-algorithm. This has not to be done in realtime but it still can produce messy conflicts. Finally, some background info: the problem we have here, is that we would like to facilitate consistency and availability during existing network partitions. Unfortunately, this is not possible (cp. CAP theorem). Therefore, other applications which use distributed data often use the approach server wins (e.g. DAVdroid) or client wins (e.g. this app currently), which means that in case of conflicts, all changes from the other side are overwritten by the client resp. server. |
Thanks for your response and explanation! Actually I also thought about the possibility of merging the local and remote changes, but imo that would be very hard to implement because the notes app (if I understand correctly) changes the notes directly. Imho, in order to implement a reliable merge operation, you'd need all clients (and that includes the web interface) to make local / intermediate changes first and then try to merge them to the server when closing the editor (similar to a commit in a VCS). I assume that this would need a lot of modifications in both apps. However, the problem I described is a lot easier, because there are no concurrent changes involved and it could therefore be solved by just making sure that a note is synced before it can be modified. Even if the synchronization should take up to a minute it has to be done in order to prevent data loss! Although I can't imagine that a single note takes that long to synchronize, you could give the user the possibility to cancel synchronization if it takes too long (e. g. by displaying a corresponding button along with the sync. animation). Finally, to your last note: I think it's a good idea to let the user choose between "server wins", "client wins" and "ask user". Other apps are doing this too and I find it very helpful. |
Does "Client wins" mean that if there are two clients which make changes at the same time, there will be an endless fighting loop? – Edit. Ah, I see, client A will overwrite the data first, then mark it synced, then client B will overwrite with its data and mark it synced, so there will be no loop. |
"Client wins" means "The last one who writes, wins". Waiting for whole sync is not acceptable for the reasons @korelstar mentioned. Assuming network connection is available:
Assuming no network connection is available:
Though this is still not an optimal solution since, as mentioned, it requires another server roundtrip which will slow down the "live edit" feature (maybe we have to throttle it down to each 3 seconds?) |
That's one way to look at it. For me, it's perfectly acceptable to wait a few seconds but it's not acceptable to lose my data. And just to be clear: I'm talking only about the note that is opened for editing. Only that note needs to by synchronized asap, the others can be synchronized in the background (you can't edit multiple notes on the same device at the same time anyway). @korelstar only mentioned that it can take long to synchronize but didn't explain why. How is synchronization currently implemented? From what you've written above it sounds like a whole note is transferred in order to check if it's up-to-date. That would indeed be very inefficient and could be optimized. Some ideas:
If you manage to make synchronization fast, you can solve the problem I described above pretty easy.
I think it's important to distinguish between the scenario I described above ("single user, multiple clients") and the one you talk about ("multiple users, concurrent changes"). The first scenario is about making sure that a user never works with an outdated version of a note and the second one is about resolving merge conflicts. That are totally different problems. To me it seems that you're trying to solve the first one by solving the second one. However the second one is much harder to solve and probably much less common. Therefore it makes sense to solve scenario one first (and help the majority of users) and scenario two later. It doesn't matter if you solve scenario two by using a merge algorithm or locking or whatever, it would still be necessary to synchronize a note before editing it, so the user always has the latest version to work with. As a first step, you could make a menu option that prevents editing of unsynchronized notes if enabled. If that would be possible, I didn't have to manually synchronize each time before editing a note and that would already make me happy ;-) |
I agree that we have to optimize (speed up) the synchronization. I've already created an issue for that (#154). Details on realizing the optimization should be discussed there. But apart from optimizing the synchronization, I think we should still have a synchronization of all notes when the notes app is opened resp. the main activity is called. Hence, typically, the notes should be up-to-date when you are starting to edit one. Only if you are faster than the synchronization and the note was changed remotely, you'll have a problem. Furthermore, you will only lose data, if you then change the note. Therefore, I still don't like to slow down the overall app interaction (please note, that in the next release, you jump directly into editing when opening a note, see #141; so you can't even read an unchanged note before the synchronization has finished with your approach!). We have to find a better solution for this. And yes, you are right, my approaches are rather too complex for a soon implementation. Here are some other (easier) ideas:
What do you think about these ideas? We don't prevent lost updates, but the user can identify situations, when it's likely to have one. Then she/he can wait in order to prevent a lost update or go on if she/he knows that there is no remote change on this note. To be completely safe (my scenario), we have to do a conflict detection on synchronization like @stefan-niedermann and I proposed. |
Thanks for sharing your ideas!
I agree. I never meant to change that. But it should also be possible to synchronize a single note, i. e. there could be a function "synchronize(note X)" that can be called if a user opens a note that has not been synchronized yet. It should than be processes with max. priority.
You're right, but nonetheless I managed to trigger this problem within the first week after purchasing the app, so it's not that hard to reproduce ;-)
Are you sure about that? If I remember correctly, I closed the note directly after opening the edit mode, because I realized that the content is outdated, but it still was overwritten. However, I'm only 90% sure about that...
I understand what you're saying, but it's a very subjective issue. Currently I have to manually trigger a synchronization each time I start the app and wait for it to finish, before I can open a note. If there would be a mechanism that makes sure that the note is up-to-date when I edit it (i.e. do that automatically what I now do manually), that would improve my user experience and reduce the waiting time before I can finally edit the note. If you don't like to enable this "protective mechanism" for all users, you could make it optional. If the note is directly opened for editing or not, does not make any difference for me. I neither want to read nor edit an outdated note (except if the device is offline or the server is unreachable, that case has to be handled somehow).
I like that idea 👍
That's easy for the "single user, multiple clients" scenario: it's (potentially) outdated if it has not been synchronized since the user lastly opened the app (remember that there are no other users and I think we can safely assume that a single user is not editing the same note on different clients at the same time ;-)).
That would also be an improvement. But updating a note that is currently edited is not that simple. You'd still need a merging algorithm to make it work...
Like I said, they would improve the situation. But the user still would have to manually check all the "warning signs" each time (and wait) if he wants to make sure that he gets the latest version of a note. Imho that's inconvenient and worses user experience. For users that prefer data integrity over speed (like me, and I'm sure that there are others out there) there should be an option like I described above.
There's another option: locking. Many people may consider locking as "not elegant" but it solves a lot of problems :D. If a note is always locked before somebody is editing it, there would be no need for a merge mechanism. You'd only have to make sure that the note is up-to-date before it is modified. Editing a note would then mean:
--> Multi-user safety without the risk of merge conflicts ;-) For collaborative editing of complex documents, locking is not an option. But for a simple app like "notes", it would be feasible imho. But of course, you'd need to distinguish between viewing and editing a note... |
Instead of going into all details, we should concentrate on the real goal: avoid concurrent writes on the same note and provide the most current state of a note for reading. You are requesting this workflow for opening a note:
Please correct me, if I understood something wrong. I don't want the user to force to wait, therefore my desired workflow is:
Please note, that
Advantages of this solution:
Disadvantages of this solution:
I've made a mockup in order to get a feeling how this could look like: Open a note while being online: Open a note while begin offline: @stefan-niedermann @jancborchardt What do you think about this solution? |
It's correct! Of course, the progress bar would only have to be shown if the note has not yet been updated (which gets more unlikely, as soon as performance is improved). I like the workflow / mockup you proposed, because it provides valuable feedback to the user and reduces the risk of making conflicting changes. It does not actually prevent them, but I could live with that ;-)
Comparing notes to e-mails is actually not so good, because e-mails can't be modified, so they are just visible or not, but never outdated. But I think I know what you mean ;-)
So yo don't plan to merge the changes from the server? That's OK for me. It saves a lot of work while still providing an improvement to the user. Overall, I like your ideas 👍 |
@korelstar thanks for this proposal. i think its really good compromise. Only one thing that grinds my gears: The progress bar (Mockup 1) with the notice will show up every single time i open a note, right? I think this is way to often since in 99% there will be no conflicts. So we should work out a way to show this progress bar and the message as less as possible. By the way: Let me say a clear no to any automatic merge algorithm since it is way too fault-prone. (Ever had fun with subversion merging? 😆 ) |
I thought it would only be shown if the note has not been synchronized since the user started the app... |
I'm still thinking, that this is an important thing. But since it's much harder to implement, we should concentrate on the first step -- for now. But if someone wants to implement it: pull requests are very welcome (I think -- I'm not the owner of this 😉 ).
Yes, could be annoying. I think we have to try out, how it behaves in practice. I hope that we'll never see this progress bar, since the synchronization should typically have finished when opening a note. But your idea to delay the presentation could be a good solution to improve the situtation. |
I didn’t read the whole thread because wow is it long. But the simple fact it suggests »prevent editing« is wrong. ;) People should never be prevented from editing. Offline should not be an error state because it isn’t. Offline is just something you sometimes are, like on the train, or in a plane, or with bad connection, etc. It needs to properly be worked into the app, rather than treated like an unusual state. :)) |
Since this requires some work on the server side, I wrote (one year ago) down some ideas on how to solve the concurrent issue in nextcloud/notes#56. |
Wouldn't it be easiest to simply write out an additional file when conflicted copies are detected? That's how the *Cloud desktop clients and really any sync client handles it. It wouldn't require any additional server-side code and allow the user to manually merge contents. |
@haarp the issue is that this is a very basic way to handle this, and people are left to clean it up. It’s not a nice experience at all, especially when for example in a long note only one line changed. Whenever I see something like "File conflict" I still shudder, and I’m not even so bad with computers. ;) |
True, conflicts are terrible UX. But loosing data is certainly worse, no? What about a simple optimistic locking strategy: Store the ETag for each note and then send an if-condition with that ETag with every write request. That would mean writing to a changed note fails and only in that case would conflict copies have to be created. |
Nevermind, has been already mentioned in nextcloud/notes#56. Sorry for the noise. |
No problem, we are happy about every kind of contribution @tribut :) I thought trying to merge automatically if possible (like git does, when there are no conflicting changes in the same line), maybe there is some diff lib and maybe this should be part of the server app which should respond a 400 on failed automatically merge..... |
@stefan-niedermann Please see nextcloud/notes#56 😃 |
OK, 9 mentions of data loss in this thread. Nobody likes data loss and most people here seem to be okay with slower updates or at least a warning message and being able to decide. This issue is quite old and it still makes my shared shopping list a constant cause for confusion, frustration, fights, tears and BBQs without charcoal. (Yes, I'm exaggerating a bit.) 😁 So, whom do I have to pay how much to implement this at least in a wary that nothing is being silently lost? Update: |
5 years later and I still hit the very same problem while testing my local setup. |
Five years later I still have to share my quite limited free time over several projects I contribute to. This is a complex topic and not just a "single line commit". My experience showed that starting several big branches is not the best idea - recent fundamental architecture changes like switching to
Good starting points are
|
Wouldn't it be already a big improvement if there is a clear visual indication that the notes are currently not synchronized and that loss of data might occur if you start editing the document? |
I'd love to have a feature according to the title. Many thanks @minimeee for creating this issue: I agree. I suffered thinking doing right but actually did wrong on several occasions now. I had a pleasant UX at first: Everything opened up directly and I read and edited the note. But what I did was terribly misguided because of the outdated data that was presented. I would have loved to wait for 10 minutes if I could then work on recent data. Why merging and conflicts can not be out of scope of this issue? If more people prefer a smooth appearance of the app more than preventing it from showing outdated data, this feature could be optional. |
I'm unlikely to add anything new to this seven year old issue, but (representing dupe #1846) I'm in the camp of those who believe newer changes on the server shouldn't knowingly be silently overwritten during sync. A minimal change would be to notify the user that the destructive action has been taken (and to which note). This allows them to retrieve the lost changes if they choose. An extension on that would be to provide the ability to open a browser showing the versions of the note/file in Files (.. without knowing if Files has a URL for that). This could just be an Android notification with an action button. I don't believe any of building full conflict resolution on mobile, blocking editing, or what is effectively a large "you are offline" warning, make sense. I prefer duplicating the note but know that too has its problems. I think the above is (greatly) better than nothing 🤷 |
I like the app, but I recently found a problem that can cause loss of content (0.9.0 on Galaxy S5, Android 6.0.1). How to reproduce:
Result: the changes made to the note using the web interface are lost and overwritten with the local changes (or the old version, if no local changes have been made).
I propose the following solution: the app should deny editing a note before synchronization has been finished! I. e. if I tap the "edit" button and the note has not yet been synchronized, instead of opening the editor with the outdated content, synchronization should be started and the editor should only be opened after the note has been synchronized. In order to indicate activity to the user, the same animation can be shown that is already used when triggering a manual synchronization (by swiping down). In case synchronization is not possible (e. g. no connection), a warning should be shown that makes it clear that all unsynchronized changes made through the web interface will be lost when the note is edited locally. What do you think?
The text was updated successfully, but these errors were encountered: