Posting new transactions take a long time (over 10s)

[l3utterfly]

What version are you using?

Horizon v2.15.1 Go v1.17

What did you do?

Post new stellar transaction to horizon server. The response sometimes takes over 10 seconds.

What did you expect to see?

Fast response times. Server memory and CPU usage are all below 20%. Postgres database cpu usage is also minimal.

What did you see instead?

Slow response times.

Here are the logs for one of the requests:

time="2022-03-08T14:21:57.300Z" level=info msg="Processing transaction" hash=f7487ef79da3b160276b0a8d31d494a0f6e10e37368b01a1d1dd28bd3027b0fd pid=39985 req=ip-12-0-3-101/F7KIBzTCvu-097330 service=txsub.System tx=AAAAAgAAAAAVZDKVZ+daUrpJxKUxrq7e/tCEdcAyu02FlLzEo4RIxQAAAGQARyF4AAAAvwAAAAAAAAAAAAAAAQAAAAEAAAAAFWQylWfnWlK6ScSlMa6u3v7QhHXAMrtNhZS8xKOESMUAAAABAAAAAG82tYNZVHSJYbxX+/jgOyiuwvcXfc2pDr/L4n4gtML2AAAAAVNHQ1QAAAAAoTG30zMLdzwci0i0bQ8IyiTPywP0souVSjM/C15LxZ4AAAAAAAAACgAAAAAAAAABo4RIxQAAAEDlzK4X0Nlj++p+Xf5kaHubftIT1o4XO7mlqs6v8AHd2UdDAa0RwlFWzRAAO4c/6yYpsZSybG9Q/2P+MeQkzpMF tx_type=EnvelopeTypeEnvelopeTypeTx

time="2022-03-08T14:22:02.514Z" level=info msg="Submitter result" duration=0.000806085 err="<nil>" pid=39985 req=ip-12-0-3-101/F7KIBzTCvu-097330 service=txsub.submitter

time="2022-03-08T14:22:07.511Z" level=info msg="Finished request" app_name=undefined app_version=undefined bytes=3578 client_name=stellar-dotnet-sdk client_version=7.1.4.0 duration=10.211339663 host=horizon.mmocircles.com ip=12.0.3.149 ip_port=12.0.3.149 method=POST path=/transactions pid=39985 referer=undefined req=ip-12-0-3-101/F7KIBzTCvu-097330 route=/transactions status=200 streaming=false x_forwarder_for=12.0.3.149