A hypothesis to prevent message omission

hkk · February 24, 2024, 9:39am

In fact, I have encountered many situations where messages are missing.

My solution is as follows: in a group of multiple people, each device should not only send a timestamp, but also tell other devices the sequence number of the message they sent themselves (each device has its own independent sequence number, arranged in a certain order).

In this case, normal message records should be arranged in a certain order.

When the message numbers of a certain device are not consecutive, we know that the device has failed to send certain (determined) messages.

So can this effectively solve the synchronization problem?

We can set a separate serial number for each device, such as 130 million serial numbers, and then reuse them when the device runs out of serial numbers.

For example, A sent the first message, A sent the second message,

A sent the x-th message

Below is a special situation that I am thinking about.

Is there a situation where A, B, and C are in the same group.

When they are all online

A: 1

B: 2

C: 3

(All three messages were successfully sent, and then B and C went offline)

A: 4

(4 failed to send)

B and C are online simultaneously

B: 5

(5 successfully sent between B and C, then C went offline)

If A and B are both online, then 4 has also been successfully sent.

At this point, D, F, and G sent a lot of messages.

Will there be a device synchronizing the fourth message before C goes online?

This is my question. I encountered a situation yesterday where the message was not synchronized

sblin · February 25, 2024, 1:28am

Synchronization is explained in the doc:

Swarm — Jami documentation or Synchronizing conversation history with Swarm

And to summarize very very quickly the history shown is the representation of the graph stored in the git repository (by topological order + chronological).

So technically if a message send 2 messages (1,2), if 2 is sent, 1 is here (cause the git pull can’t succeed for missing commit) and the only explanation for the UI if 2 is shown but not 1 would be a UI bug at this point. Should not be the case since some versions now.

Also in your example say A sends 4 and fail to send, B can’t even know he needs to send 5, as they don’t even know 4 exist.

hkk · February 25, 2024, 11:36am

What I mean is, assuming member A is sending messages in the group.

A will number the messages he sends,

For example, a001, a002, a003, and so on.

When the serial number is missing, it indicates that there is a message in the middle that has not been sent.

And not just A, each device in the group will have a message number assigned to it.

hkk · February 25, 2024, 11:41am

Each device will inform other devices of the timestamp for sending messages, as well as the number of messages it sent itself. This will be used by other devices to verify the chat information they receive.

And can we manage chat records according to this number?

We can delete chat records before a certain number, which will help us manage the data generated by the chat and also solve the synchronization problem in peer-to-peer networks.

hkk · February 25, 2024, 11:52am

If A sends the fourth message to B and others are not online, then in the future, if multiple messages are successfully sent. At this point, the fourth message was not received by most people, while messages after the fifth message were received by others. Will b synchronize the fourth message with other users? And 4 mistakenly thought that their message had been successfully sent, but in reality, this message was only sent to B. From a resource saving perspective, A may no longer send the fourth message repeatedly.

In fact, waste and conservation are opposite. If we use numbering, will it help reduce waste of resources.

We only need to broadcast the last numbered message to know who has missed the message.

hkk · February 25, 2024, 1:00pm

If we use timestamps, we cannot guarantee whether there are messages that we have not yet received before a certain point in time, which increases the difficulty of managing messages.

But if we use message numbering, we can guarantee that no message will be missed.

hkk · February 25, 2024, 4:26pm

I raised this question because I lost some messages. When I searched locally, I found the number of a new message sent by the same account

For example, git show 83c6d9ad3eb8af83b8ff19ac15ca3c11ffd58ab8

But I couldn’t find the number of the lost old message.

During that period, I restarted my device multiple times, but the message was still not received,

I didn’t receive that message until a contact person used the reply function

hkk · February 25, 2024, 4:35pm

It should be noted that after using the reply function, I clicked on reply but did not redirect to the lost message. It was not until I restarted that I successfully received the message. At that time, I opened the log, but I was too anxious and forgot to save it

hkk · February 25, 2024, 4:58pm

According to a certain numbering rule, the device can detect missing messages on its own. At this point, the fastest thing for the device is that if the sender is online, the device should request the sender to resend the missing message

hkk · February 26, 2024, 5:41am

This is the error record saved by the group members at that time,

February 23 15:28:52 Fedora Jami [12365]: [device 75e1864d70f83c69533dc55c4fa8a7e28012e5ca49a19c0ebcc69a59036561c2] Peer already connected Add a new channel

February 23 15:28:52 Fedora Jami [12365]: Upload pack command detected

February 23 15:28:52 Fedora Jami [12365]: [device 75e1864d70f83c69533dc55c4fa8a7e28012e5ca49a19c0ebcc69a59036561c2] Peer already connected Add a new channel

February 23 15:28:53 Fedora Jami [12365]: [device 75e1864d70f83c69533dc55c4fa8a7e28012e5ca49a19c0ebcc69a59036561c2] Peer already connected Add a new channel

February 23 15:28:53 Fedora Jami [12365]: Upload pack command detected

February 23 15:28:53 Fedora Jami [12365]: [device 75e1864d70f83c69533dc55c4fa8a7e28012e5ca49a19c0ebcc69a59036561c2] Peer already connected Add a new channel

February 23 15:28:53 Fedora Jami [12365]: Invalid add file detected: invited/{Jami ID} 2

February 23 15:28:53 Fedora Jami [12365]: Malformed text/plain commit 2c3b7ebd29b93a650504ba3b72af5a25098835ec Please check you use the latest version of Jami, or that your contact is not doing unwanted stuff

I failed at the time, but later succeeded, indicating that the message itself was not a problem, and I only sent normal text messages and emojis.

My desktop is the latest stable version (as of 2.26), Android is 405, and the versions in my contacts are relatively new