Messages can appear in the wrong order due to low time precision

[fulalas]

If I send 2 messages in a row really quick or forward (on mobile app) 2 messages at once, sometimes nchat show them in the wrong order. I presume this happens because nchat is using low time precision (maybe just seconds, not milliseconds?).

This is not a critical bug and this is not a regression.

nchat-5.1.29

[d99kris]

Hi @fulalas - thanks for reporting a bug. I can see the issue here too, if I forward two messages using WhatsApp from the phone on a different account, and receive them in nchat. I assume you also see this only for WhatsApp, right?

The WhatsApp glue layer implementation has some (likely unecessary) time stamp rounding to seconds (like you guessed), I will look into switching to millisecond timestamps (which is what nchat generally uses).

[fulalas]

I just confirmed it also happens using Telegram protocol.

[d99kris]

Thanks for sharing. I checked the WhatsApp (whatsmeow) and Telegram (tdlib) API's, and the message timestamp nchat receives is only at second-resolution. Internally nchat converts the timestamp to ms-resolution and uses a message id hash for the "ms-part", this is to have a consistent sorting order (even if it's wrong at least it's not random).

Anyway, the official Telegram and WhatsApp clients manage to list messages in correct order despite the timestamp resolution, so they likely sort based on some other data. I believe Telegram might sort based on message id, and for WhatsApp perhaps the messages are just sorted based on order received by the client (just a guess).

I will need to do a bit more research to see how this can be fixed. Just wanted to share a brief update on the troubleshooting done so far.

[fulalas]

Thanks a lot, man! Very kind of you :)

[graves501]

As of nchat 5.6.7 the message order in telegram can still be wrong, I also assume that the messages are internally sorted by message ID 🤔

[d99kris]

Yeah, there haven't been any fix implemented for this yet.

I've done some research and whatsapp / whatsmeow messages are received in temporal order (older before newer), so it would be possible to locally generate ordered higher precision timestamps based on that, especially since nchat never fetches whatsapp messages older than initial sync, and only fetches each message once.

For telegram / tdlib the message ids are strictly increasing in each chat, so the information is available, it's just that it's difficult to create a timestamp to reflect this. Unless going to 128-bit timestamps, which would have some performance impact (esp. on cache side). Or always fetching all messages, which would have quite a bit of performance impact.

Another option would be to sort messages on timestamp first and on message ID second, it's just that some cache db queries become a bit complicated.

Once I find some time, I'lll try see what other multi-protocol chat clients do to handle this..

[Bellavene]

Sometimes when people edit their messages, they appear as newer messages in nchat too.

[fulalas]

Sometimes when people edit their messages, they appear as newer messages in nchat too.

Yes!

Also, it would be nice to have the 'edited' keyword next to the edited messages.

[d99kris]

Sometimes when people edit their messages, they appear as newer messages in nchat too.

Thanks for reporting! I'll check on this, it might be a quicker/simpler fix than the low resolution issue.

Also, it would be nice to have the 'edited' keyword next to the edited messages.

Will try look into this..

[d99kris]

Sometimes when people edit their messages, they appear as newer messages in nchat too.

A fix to retain original timestamp of edited whatsapp messages has been committed in 7fdd37e / v5.11.17.

[fulalas]

Yep, the issue with edited messages jumping to the most recent position in the history is fixed now. Great! :)

But the issue I mentioned in the first post can still be reproduced.