Add Architecture Diagram and Clarify NeMo Gym + NeMo RL Integration

[cwing-nvidia]

Add Architecture Diagram and Clarify NeMo Gym + NeMo RL Integration

Background

Users were unclear on the data flow between NeMo Gym, NeMo RL, and the policy model. They expected the model to directly use NeMo Gym but didn't understand that the two systems are completely decoupled and communicate via HTTP.

Problem

Our current documentation should clarify:

• NeMo RL exposes the policy model as an OpenAI HTTP server
• NeMo Gym calls this HTTP server
• The two systems only communicate via HTTP
• This is architecturally different from most frameworks that pass vllm.LLM handles directly to their environments

Acceptance Criteria

• [ ] Create an architecture diagram showing how NeMo Gym, NeMo RL, and the policy model interact during RL training
• [ ] Add the diagram to the main documentation (likely in Concepts or Setup sections)
• [ ] Update documentation to explicitly explain the HTTP-based decoupling
• [ ] Clarify that this design is different from typical frameworks that pass model handles directly
• [ ] Ensure this is surfaced early in the tutorial flow (before users start building)

Priority

High