[Bug]: pubsub attributes too big

[dermasmid]

Related Template(s)

pubsub_to_elasticsearch

What happened?

sometimes when using the temple and writing to es fails the temple will fail when trying to send the message back to pubusb with this log a log saying the errorMessage attribute key is bigger than the 1024 bytes limit.

i think this happens when the batch size is high and the number of failed messages is high as well

Beam Version

Newer than 2.46.0

Relevant log output

Error message from worker: java.lang.IllegalArgumentException: javax.naming.SizeLimitExceededException: Pubsub message attribute value for key 'errorMessage' starting with 'java.io.IOException: Error writing to Elasticsearch, some elements could not be inserted:
Document id VGJMq4cB19pO6TW9jVUO: failed to parse field [jsonPayload.level] of type [long] in document with id 'VGJMq4cB19pO6TW9jVUO'. Preview of field's value: 'info' exceeds the maximum of 1024 bytes. See https://cloud.google.com/pubsub/quotas#resource_limits