AddOneHotObservation code example is incorrect and missing information

[omarathon]

Describe the bug The code example

enum ItemType { Sword, Shield, Bow, LastItem }
const int NUM_ITEM_TYPES = (int)ItemType.LastItem;

public override void CollectObservations(VectorSensor sensor)
{
    // The first argument is the selection index; the second is the
    // number of possibilities
    sensor.AddOneHotObservation((int)currentItem, NUM_ITEM_TYPES);
}

in the docs found here is incorrect. NUM_ITEM_TYPES should be (int)ItemType.LastItem + 1, because casting an enum item to an int obtains the 0-indexed ordinal value of the item, and thus the 0-indexed ordinal of the last item is actually the number of items in the enum minus 1. The implementation of AddOneHotObservation excludes the second parameter of AddOneHotObservation, yet its doc string is "The max index for any observation.", which is especially confusing as the doc string seems to falsely imply that the index is inclusive (which it is not).

Not a major issue but the inconsistency is a bit annoying, as following the way the code example says ends up in errors when training your model. Adding to this, the docs don't state that when adding a one-hot observation, the number of vector observations corresponding to that one-hot observation should be equal to the number of possible values for the one-hot observation. When first using one-hot observations, I assumed the number of vector observations would be automatically scaled, but that's not the case. This could be automated (for example, by having number of one-hot observations in the inspector and map them to vector observations appropriately), or the docs could clearly state that the user should increase the number of their vector observations.

Some additional clarification on when one-hot observations should be used could also be helpful. The docs say they should be used for "Categorical variables", but it's not very clear what exactly that means. Perhaps the docs could specify this means variables with no implicit natural ordering.

To Reproduce Steps to reproduce the behavior:

1. Go to https://github.com/Unity-Technologies/ml-agents/blob/main/docs/Learning-Environment-Design-Agents.md#one-hot-encoding-categorical-information
2. Scroll down to code example for AddOneHotObservation
3. See error on line 2

Also:

1. Go to https://docs.unity3d.com/Packages/[email protected]/api/Unity.MLAgents.Sensors.VectorSensor.html
2. Scroll down to docs for AddOneHotObservation(Int32, Int32)
3. See error in description of the range parameter

Console logs / stack traces N/A

Screenshots N/A

Environment (please complete the following information):

• Unity Version: Unity 2020.3.14f1
• OS + version: Windows 10
• ML-Agents version: main branch
• Torch version: N/A
• Environment: N/A

[hvpeteet]

Thanks for the report, we will update the documentation.

[hvpeteet]

For internal-team tracking: https://jira.unity3d.com/browse/MLA-2403

[hvpeteet]

The documentation update will be done with the next release. Changing this to a feature request since the remaining work is around auto-sizing observations when using one-hot obs.