td icon indicating copy to clipboard operation
td copied to clipboard

min_id and max_id in messages.getHistory()

Open MeanSquaredError opened this issue 3 years ago • 2 comments

I want to use the min_id and max_id parameters in calls to messages.getHistory(). However after looking into the code it seems that the high-level API in telegram/messages.GetHistory() does not support min_id and max_id.

It seems that the low-level code in tg.MessagesGetHistoryRequest supports these parameters so it should be possible to add support for them in the high-level API.

Is my understanding of the code correct? Are there plans to add support for these two parameters?

MeanSquaredError avatar Jan 07 '22 17:01 MeanSquaredError

I did not add min_id and max_id parameters because it can cause some side effects (as I know, Telegram ignores min_id and max_id, when offset_id is used)

I planning to update and refactor query package soon:

  • Use simple TL-based codegen instead of slow AST-based.
  • Split iterator and its internal buffer, to use same code for methods without pagination.
  • Rewrite iterator and buffer with generics.
  • Add some wrappers/helpers for dialogs and messages.

Generics are upcoming in Go 1.18. According to Go Release Cycle, it will at the beginning of February.

I can add these parameters, but I think it better to use a custom query implementation. You can implement messages.Query and use it with messages.NewIterator.

tdakkota avatar Jan 08 '22 02:01 tdakkota

I have been experimenting with the low-level tg.* functions downloading history messages from channels and so far my experiments show that min_id and max_id are not really ignored but rather work as additional filter applied on the server after results have been selected by offset_id, add_offset and limit

So let's say that there is some channel with messages with IDs ranging from 1000 to 1 and that they don't have any gaps in their numbering.

If you pass offset_id=920, add_offset=1, limit=10 then the server will return ten messages with ID's 919, 918, ..., 910

However if you send a query with offset_id=920, add_offset=1, limit=10, max_id=917, min_id=911, then you will get only five messages with IDs 916, 915, 914, 913, 912

Or if you send a query with offset_id=920, add_offset=1, limit=10, max_id=812, min_id=750 then you will get zero messages because the initial offset_id=920, add_offset=1, limit=10 will select messages from 919 to 910 and afterwards max_id=812, min_id=750 will remove all the selected messages from the list as not matching the range 750 < message_id < 812.

Overall max_id is not very useful but min_id can be used when downloading large ranges of messages, e.g. when filling gaps in downloaded messages, e.g. when one needs to remove all messages with ID less than a certain number because they have been already downloaded.

However all of the above information is from my experiments so it may be incorrect and/or incomplete.

I decided to use the low-level functions in tg.* anyway because the high-level iterators in telegram.Messages().Foreach() make it difficult to limit server-requests and react to flood_wait errors.

Other than that I can say that overall gotd is a rather useful library so thank you for developing and maintaining it.

MeanSquaredError avatar Jan 08 '22 11:01 MeanSquaredError

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale[bot] avatar Jan 15 '23 15:01 stale[bot]