Show and jump to WhatsApp quoted messages in reply messages in WhatsApp preview

[thalespr]

Actually, WhatsApp parser, at least for WhatsApp installed in Android devices, doesn't show in the preview TAB the original message when you have a reply message. For example, let's say user_A replies to a message that user_B sent, the user_B's message isn't shown 'linked' (in a cascade format) to user_A' message, so you don't see, at least graphically, that user_A is replying a message that user_B sent.

[lfcnassif]

AFAIK, this isn't supported for iOS too, this would be an useful enhancement.

[thalespr]

I'll try to take a look at it and dev some codes for this.

[lfcnassif]

Thank you very much @thalespr, just be careful about regressions...

[thalespr]

After a while, returning to work on this feature. Sorry for being away from this project. I'll be careful about regressions.

[thalespr]

Started developing this feature for Android... Some notes to keep track on:

1. Added quoted_row_id attr to Message class (maybe the attr name can change in the future to make sense for iOS too);
2. I think it will be necessary to create a new class to store quoted messages;
3. I think the Select message query in ExtractorAndroid (at extractMessages method) should be reevaluated since the messages table is changing every now and then...
4. What should be the best approach to show quoted message in the preview tab (report) when it is a media (like audio, video, etc.)? Is it enough to put the media type label and the remote_jid?

[lfcnassif]

4. What should be the best approach to show quoted message in the preview tab (report) when it is a media (like audio, video, etc.)? Is it enough to put the media type label and the remote_jid?

Not sure about the needed effort, but if it is possible, I think displaying image/video thumbs or the audio generic icon would be better.

[thalespr]

Some notes:

Included a new class to store quoted messages;
This feature is, initially, being implemented without considering deleted messages that might be recovered by IPED;
I guess we need to think better (and maybe refactor it) how to deal with the SELECT Messages for sqlite query in ExtractorAndroid... It's very critical in my opinion, but no ideas for now...

[lfcnassif]

This feature is, initially, being implemented without considering deleted messages that might be recovered by IPED;

This may be OK for a first implementation...

I guess we need to think better (and maybe refactor it) how to deal with the SELECT Messages for sqlite query in ExtractorAndroid... It's very critical in my opinion

What is your current concern?

[thalespr]

This feature is, initially, being implemented without considering deleted messages that might be recovered by IPED;

This may be OK for a first implementation...

I guess we need to think better (and maybe refactor it) how to deal with the SELECT Messages for sqlite query in ExtractorAndroid... It's very critical in my opinion

What is your current concern?

My concern is about "versioning" the messages table. It may be difficult to keep control on it as different versions of this table are released... If I want to get a New field from this table, I need to create a new message query to grab it...
Maybe can we change to select all the fields from this tablet and then make the sanity checks on the code? Don't know If It would be a performance issue...

[thalespr]

Need samples of DB's that are dealed by ExtractorAndroidNew... Can someone provide them please? Thanks in advance.

[wladimirleite]

Need samples of DB's that are dealed by ExtractorAndroidNew... Can someone provide them please? Thanks in advance.

Hi @thalespr! Just sent you (through Teams) a 7z file with ~20 WhatsApp Android databases.

[thalespr]

Need samples of DB's that are dealed by ExtractorAndroidNew... Can someone provide them please? Thanks in advance.

Hi @thalespr! Just sent you (through Teams) a 7z file with ~20 WhatsApp Android databases.

I got it. Thanks @tc-wleite.

[lfcnassif]

@thalespr, @gfd2020 didn't know about your work and started an independent implementation and opened #1916.

@thalespr and @gfd2020, could you both submit a PR in draft mode? So we all can evaluate which one is more advanced and continue the work from it? Of course, we can copy interesting things from the less advanced one to the implementation to be chosen by both of you.

[thalespr]

Hello @lfcnassif! Nice to have someone else working on it too. Let me catch up with you about this feature:

• I have developed message quotes for mime type = text in the old ExtractorAndroid;
• I was working to put a thumbnail when the quoted message was an audio, image or video (didn't finish yet);
• I'll begin to implement this feature for the new ExtractorAndroid, since they are implemented separetely because of different databases schemas.
• I did NOT started developing this feature for iOS devices. Maybe @gfd2020 can do this job? So we can work in paralell ?

Since @gfd2020 is willing to help in this feature, there are some concerns I think he should be aware of:

• You should be careful changing the SELECT messages queries in ExtractorAndroid. It can break old and new functionalities...
• Keep in mind the report view can be sliced in different fragments. So the quoted message can be in a different fragment.
• There's a feature that can recover deleted messages, but, at this first moment, I'm not considering deleted messages that might be recovered by IPED.

I'll try to take a look this week at the branch I'm working on and submit it as PR draft as suggested.

Thanks.

[thalespr]

P.S: looks like @gfd2020's work is more advanced than mine. So feel free to go ahead and assign this ticket to him. Not a problem for me.

[gfd2020]

• I did NOT started developing this feature for iOS devices. Maybe @gfd2020 can do this job? So we can work in paralell ?

I was working on ExtractorAndroidNew only until now. We can work in parallel.

• You should be careful changing the SELECT messages queries in ExtractorAndroid. It can break old and new functionalities...

I used the same select that search for messages. The ID of the original message is captured, then a pointer to the original message is created, any data you need, just look for it there.
It has to be well tested. So far in my tests it's ok.

• Keep in mind the report view can be sliced in different fragments. So the quoted message can be in a different fragment.
  ell remembered. In the case of my PR, the message will be retrieved but the click to go to the anchor will not work.

Well remembered. In the case of my PR, the message will be retrieved but the click to go to the anchor will not work.

• There's a feature that can recover deleted messages, but, at this first moment, I'm not considering deleted messages that might be recovered by IPED.

If you delete a message that was quoted, WhatsApp simply will not accept the anchor click. However, I believe that the data from this quote is still in the message_quoted table.
Edit: Confirmed. The information about the text, which user sent it and the type of media is in the message_quoted table. Then you can search for this information.

My Draft PR is #1920

[gfd2020]

P.S: looks like @gfd2020's work is more advanced than mine. So feel free to go ahead and assign this ticket to him. Not a problem for me.

@thalespr , I believe you could also help with this PR, especially since you have already made several implementations and pertinent observations.

[gfd2020]

@lfcnassif , @thalespr .

I managed to make it work on IOS, but I don't know if the columns I added are used in older databases. Do you have databases for regression testing ( Android too)?

PS: Even without finding the quote messages (if deleted), it is possible to retrieve the reply data, but I was unable to understand the entire logic of the ZMETADATA field ...

[lfcnassif]

Yes, I have many databases from hundreds of cases for regression tests. I can run the test when you finish a first version. Or I can try to share them with you if you prefer (dozens of GBs).

[gfd2020]

Yes, I have many databases from hundreds of cases for regression tests. I can run the test when you finish a first version. Or I can try to share them with you if you prefer (dozens of GBs).

If you could test it for me I would appreciate it. In new databases is working.

[lfcnassif]

Sure!

[gfd2020]

Sure!

Just make work in Old Whatsapp Databases ..

[gfd2020]

@lfcnassif , I think I handled exceptions from older databases that could cause SQL errors. The implementation is closer to the end. Could you do a test, please?

[lfcnassif]

Sure! Thank you @gfd2020 for helping with this!

[lfcnassif]

Closed by #1920