sarama Exact fetch size to retry partial record set delivery

The handling of partial delivery of a record set can be improved. instead of trying to double the fetch size and try again: https://github.com/Shopify/sarama/blob/v1.26.1/consumer.go#L602-L609 we can make use of the size returned from the partial delivery: https://github.com/Shopify/sarama/blob/v1.26.1/fetch_response.go#L86 i.e. in addition to the Partial boolean flag https://github.com/Shopify/sarama/blob/v1.26.1/fetch_response.go#L40 we can add a RawSize field and use it in the next fetch request.

Each fetch should request at least 12 bytes to ensure RawSize is included in the fetch response.

Apr 01 '20 12:04 zmstone

Thank you for taking the time to raise this issue. However, it has not had any activity on it in the past 90 days and will be closed in 30 days if no updates occur. Please check if the master branch has already resolved the issue since it was raised. If you believe the issue is still valid and you would like input from the maintainers then please comment to ask for it to be reviewed.

Mar 17 '21 00:03 ghost

No bot. Please don’t close

Mar 17 '21 05:03 zmstone

Thank you for taking the time to raise this issue. However, it has not had any activity on it in the past 90 days and will be closed in 30 days if no updates occur. Please check if the main branch has already resolved the issue since it was raised. If you believe the issue is still valid and you would like input from the maintainers then please comment to ask for it to be reviewed.

Aug 24 '23 16:08 github-actions[bot]

Thank you for taking the time to raise this issue. However, it has not had any activity on it in the past 90 days and will be closed in 30 days if no updates occur. Please check if the main branch has already resolved the issue since it was raised. If you believe the issue is still valid and you would like input from the maintainers then please comment to ask for it to be reviewed.

Dec 12 '23 12:12 github-actions[bot]