backlog often appear duplicated on the IRC side.

[terceiro]

When I open my IRC client,I will often see the full backlokg appear twice, like this:

msg1
msg2
msg3
msg1
msg2
msg3

for example I am connected to 3 workspaces, and this morning I got this in all of them.

[ltworf]

Uhm I haven't encountered this.

Basically it calls an API to get history from a timestamp. Are the messages in threads maybe?

[terceiro]

no, normal messages.

[ltworf]

Could you check what the history api call is returning? I can't say I've seen the issue. In my experience they even remove all edits and stuff and just send the final message including all the reactions.

[terceiro]

This doesn't seem related to the history api call. What is happening is that it gets called twice for each channel. Looking back at my logs from when this happened (it's not always), this is what I see:

abr 29 13:46:02 lemur localslackirc-myworkspace[72094]: Downloading logs from channel channel1
abr 29 13:46:02 lemur localslackirc-myworkspace[72094]: Calling cursor
abr 29 13:46:02 lemur localslackirc-myworkspace[72094]: Downloading logs from channel channel2
abr 29 13:46:02 lemur localslackirc-myworkspace[72094]: Calling cursor
abr 29 13:46:03 lemur localslackirc-myworkspace[72094]: Downloading logs from channel channel3
abr 29 13:46:03 lemur localslackirc-myworkspace[72094]: Calling cursor
abr 29 13:46:03 lemur localslackirc-myworkspace[72094]: Downloading logs from channel channel4
abr 29 13:46:03 lemur localslackirc-myworkspace[72094]: Calling cursor
abr 29 13:46:03 lemur localslackirc-myworkspace[72094]: Downloading logs from channel channel5
abr 29 13:46:03 lemur localslackirc-myworkspace[72094]: Calling cursor
abr 29 13:46:03 lemur localslackirc-myworkspace[72094]: Downloading logs from channel channel6
abr 29 13:46:03 lemur localslackirc-myworkspace[72094]: Calling cursor
abr 29 13:46:03 lemur localslackirc-myworkspace[72094]: Downloading logs from channel channel1
abr 29 13:46:03 lemur localslackirc-myworkspace[72094]: Calling cursor
abr 29 13:46:04 lemur localslackirc-myworkspace[72094]: Downloading logs from channel channel2
abr 29 13:46:04 lemur localslackirc-myworkspace[72094]: Calling cursor
abr 29 13:46:04 lemur localslackirc-myworkspace[72094]: Downloading logs from channel channel3
abr 29 13:46:04 lemur localslackirc-myworkspace[72094]: Calling cursor
abr 29 13:46:04 lemur localslackirc-myworkspace[72094]: Downloading logs from channel channel4
abr 29 13:46:04 lemur localslackirc-myworkspace[72094]: Calling cursor
abr 29 13:46:04 lemur localslackirc-myworkspace[72094]: Downloading logs from channel channel5
abr 29 13:46:04 lemur localslackirc-myworkspace[72094]: Calling cursor
abr 29 13:46:05 lemur localslackirc-myworkspace[72094]: Downloading logs from channel channel6
abr 29 13:46:05 lemur localslackirc-myworkspace[72094]: Calling cursor

so my guess is that these lines in the body of Slack._history are producing duplicated entries in chats somehow:

        chats: Sequence[Union[IM, Channel]] = []
        chats += await self.channels() + await self.get_ims()  # type: ignore
        for channel in chats:
            # ...

[ltworf]

Is this still an issue?

[terceiro]

very much, happens to me all the time

[terceiro]

to be fair, I hadn't updated localslackirc in a while (was running 148dab8). I just pulled the latest master (9582618) and will let you know

[terceiro]

very much, happens to me all the time

I mean, it happens very frequently, but not all the time or on all channels.

[ltworf]

Hm ok. I did have the issue as well but I haven't encountered it for a while. But there are still issues if the history is too long, because it blocks for too long to fetch it, in those cases I remove the state file and give up on the history.

[ltworf]

This should be fixed now, unless you kill the process without letting it store the state.