[Docs] Define hedging and add to documentation for Tempo

[knylander-grafana]

The terms "hedge" and "hedging" are used in the Tempo documentation as well as in some of the configuration files.

We should write a definition for hedging and include it in the Tempo documentation for clarity.

In the configuration page, the options that refer to hedging include query_frontend, querier, and storage.

Pages that refer to hedging:

• Configuration
• Backend search

Where should the definition be placed? Maybe on the configuration/polling page?

[knylander-grafana]

Possible definition from @zalegrala:

A hedged request is one that is sent if the first request doesn't come back right away. So if we delay for 400 milliseconds before the hedged request is sent, then whichever one comes back first is what is returned to the caller.

Best practices (from Marty): https://cloud.google.com/storage/docs/best-practices

If your application is latency sensitive, use hedged requests. Hedged requests allow you to retry faster and cut down on tail latency. They do this while not reducing your request deadline, which could cause requests to time out prematurely. For more information, see The Tail at Scale.

[knylander-grafana]

Draft definition:

Hedging helps reduce latency by issuing a new request if the first one doesn't return within the expected time frame. If we delay for 400 milliseconds before the hedged request is sent, then whichever one comes back first is what is returned to the caller. Hedging is considered a best practice for cloud storage. For more information, refer to https://cloud.google.com/storage/docs/best-practices

[osg-grafana]

Relates to https://github.com/grafana/technical-documentation/issues/154.

[osg-grafana]

Draft definition:

Hedging helps reduce latency by issuing a new request if the first one doesn't return within the expected time frame. If we delay for 400 milliseconds before the hedged request is sent, then whichever one comes back first is what is returned to the caller. Hedging is considered a best practice for cloud storage. For more information, refer to https://cloud.google.com/storage/docs/best-practices

Hedging reduces latency by issuing a new request, if the first one does not return within the expected time frame. For example, if you add a delay of 400 milliseconds before the hedged request is sent, then whichever one returns first is the one that is returned to the caller. Hedging considered a best practice for cloud storage. For more information, refer to https://cloud.google.com/storage/docs/best-practices.

• Is it clearer to use "comes back" instead of "returns", or vice versa (as in my change); does "comes back" mean something (at least slightly different than) "returns"?
• I think for definitions, spelling out contractions is the most clear; accuracy is at its highest in this context.
• My gut was that "to delay" ought to be a transitive verb (that requires an object), so I change it to a noun.
• "We" is now oriented as "you" to the reader directly.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had any activity in the past 60 days. The next time this stale check runs, the stale label will be removed if there is new activity. The issue will be closed after 15 days if there is no new activity. Please apply keepalive label to exempt this Issue.

[osg-grafana]

Reopening and closing related issue that contains less information.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had any activity in the past 60 days. The next time this stale check runs, the stale label will be removed if there is new activity. The issue will be closed after 15 days if there is no new activity. Please apply keepalive label to exempt this Issue.