AI Engineer World Fair - AI Agent Breakout session fixes

[cedricvidal]

Description

Upgrade AI Agent Evaluation Notebooks to 1RP.

Those changes were made to present the notebooks at AI Engineer World Fair for this talk

[!NOTE]
Including this image because I'm going to share this PR with participants so they can try the code until it's merged hopefully

Details of all the changes:

• Using separate env vars for AI Foundry report upload REPORT_* to accomodate for the fact that 1RP Projects currently cannot be used to upload reports so you need a 1RP Project for evaluation and a Hub project to upload the report.
• Added pip installation commands
• Added environment variable loading using dotenv across notebooks.
• Updated Azure OpenAI model settings and environment variable names.
• Improved evaluation examples and output formatting for intent resolution, task adherence, tool call accuracy, and response completeness.
• Updated .gitignore to exclude all .env* files.
• Improved README with environment variable setup instructions.

Environment Variables

[!IMPORTANT]
Environment variables are loaded from .env file. Here are the expected variables

To run the evaluations:

• AZURE_OPENAI_ENDPOINT: The endpoint URL for Azure OpenAI.
• AZURE_OPENAI_DEPLOYMENT: The deployment name for the Azure OpenAI model (e.g., gpt-4o).
• MODEL_DEPLOYMENT_NAME: The deployment name for the model used in evaluations (e.g., gpt-4o).
• AGENT_MODEL_DEPLOYMENT_NAME: The deployment name for the agent model (e.g., gpt-4o).
• AZURE_OPENAI_API_VERSION: The API version for Azure OpenAI.
• AZURE_SUBSCRIPTION_ID: The Azure subscription ID.
• PROJECT_NAME: The name of the Azure project.
• RESOURCE_GROUP_NAME: The name of the Azure resource group.
• AZURE_AI_PROJECT: The Azure AI project identifier.
• AZURE_OPENAI_API_KEY: The API key for Azure OpenAI.

To upload reports to Azure AI Foundry:

• REPORT_AZURE_SUBSCRIPTION_ID: The Azure subscription ID for report uploads.
• REPORT_PROJECT_NAME: The project name for report uploads.
• REPORT_RESOURCE_GROUP_NAME: The resource group name for report uploads.

Ensure all these variables are properly configured in your .env file before running the notebooks.

Checklist

• [x] I have read the contribution guidelines
• [x] I have coordinated with the docs team ([email protected]) if this PR deletes files or changes any file names or file extensions.
• [x] This notebook or file is added to the CODEOWNERS file, pointing to the author or the author's team.

Thanks @singankit for the support helping me get ready for the conference!!

[hayescode]

@cedricvidal how can we use this with the new Foundry projects (no hub)? This code gives an error.

    azure_ai_project={
        "subscription_id": os.environ["REPORT_AZURE_SUBSCRIPTION_ID"],
        "project_name": os.environ["REPORT_PROJECT_NAME"],
        "resource_group_name": os.environ["REPORT_RESOURCE_GROUP_NAME"],
    },

EvaluationException: (InternalError) The get 'my_project' workspace request failed with HTTP 404 - (ResourceNotFound) The Resource 'Microsoft.MachineLearningServices/workspaces/my_project' under resource group 'my_rg' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix