docs(weave): MVP Weave RAG docs QA with chat in UI and server

[J2-D2-3PO]

Description

Implements an MVP RAG-based QA system for the Weave documentation site. This MVP establishes a solid foundation to explore more advanced RAG techniques and to improve docs quality through live usage.

• Adds weave_rag.py to load, embed, and retrieve chunks from real markdown docs. Means you can also ask questions in the cli
• Adds lightweight source bias via llms.yaml to improve retrieval relevance
• Caches document embeddings to weave_rag_index.pkl for faster iteration
• Exposes FastAPI endpoint via weave_rag_server.py for local serving
• Adds basic CORS support to allow local Docusaurus UI calls
• New React component WeaveRAGChat.tsx for in-docs interactive QA
• UI now includes source doc and top 5 retrieved chunks with similarity scores
• Adds demo page at docs/weave-rag-demo.md for easy usage and fast testing

Test the chat app

0. Check out this branch:

git checkout weave-rag

gh pr checkout 4006

1. Set up a Python virtual environment and start it:

python3 -m venv weaverag-venv
source weaverag-venv/bin/activate

2. Install dependencies:

pip install numpy weave openai pyyaml langchain fastapi uvicorn pydantic

3. Set your OpenAI API key

export OPENAI_API_KEY=sk-...

4. From \scripts, run the FastAPI server

cd scripts
uvicorn weave_rag_server:app --reload

This starts a local server at http://localhost:8000 that powers the RAG endpoint.

5. Open a second terminal and start the Docusaurus site

cd ../  # back to root if needed
yarn install
yarn start

6. Navigate to your local docs chat at http://localhost:3000/weave-rag-demo
7. Type a real docs question. Wait a moment, and watch the RAG assistant answer — with traceable source + top-5 context chunks.

[circle-job-mirror[bot]]

Preview this PR with FeatureBee: https://beta.wandb.ai/?betaVersion=75f5d4b0f3ff892d4bb7685595fd0095c29aaa04