BE: Organization filters

[EugeneLightsOn]

• Filtering all entities with a foreign key to Organization by organization_id if the Organization-Id request header is set.

AI Description

This PR introduces changes to the backend, enhancing the system's ability to handle organizations and user groups.

Summary of Changes:

• Added a new function, validate_organization_header, to validate requests containing the Organization-Id header.
• Modified the RouterName classes to include Depends(validate_organization_header) in their dependencies.
• Introduced a new class, FilterFields, with an attribute ORGANIZATION_ID, and a custom query class, CustomFilterQuery, to enable filtering by the organization_id field.
• Updated the get_organization_users function to retrieve a list of users within a specified organization.
• Adjustments to the ContextMiddleware class to handle the Organization-Id header and set the global context.
• Various test updates and additions to validate the functionality of organizations, user groups, and agents within organizations.

Detailed Code Changes:

New Code:

• validate_organization_header function in request_validators.py to validate the presence of the Organization-Id header.
• FilterFields class and CustomFilterQuery class in base.py for custom filtering capabilities.
• get_organization_users function in organization.py to retrieve users within an organization.
• Added organization_id and related methods to the Context class in context.py.
• Added handling for the Organization-Id header and global context management in the ContextMiddleware class in `context.py.
• Introduced organization-related factories and tests in test_list_organization_agents, test_list_organization_agents_query_param, and test_list_organization_agents_nonexistent_organization in test_agent.py.
• Added organization-related tests in test_get_organization_conversation_list in test_conversation.py.
• Implemented tests for listing organization users in test_list_organization_users in test_organization.py.

Modified Code:

• Modified RouterName classes in routers.py to include Depends(validate_organization_header) in their dependencies.
• Updated the get_session function in database.py to use CustomFilterQuery.
• Adjusted the list_agents function in agent.py to handle the organization_id for filtering agents.
• Modified the get_organization and delete_organization functions in organization.py to raise an exception if the "Organization not found" instead of "Model not found".
• Changed the return type of get_organization_users function in organization.py to return a list of User instead of ManagedTool.
• Updated the Context class in context.py to include organization_id, organization, and global filtering methods.
• Modified the session and session_chat functions in conftest.py to use `CustomFilterQuery.
• Adjusted tests in test_chat.py to include tools when creating an agent.
• Updated tests in test_conversation.py to include organization_id when creating conversations.
• Modified factories in agent.py and conversation.py to include organization_id.

Removed Code:

• None

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 85.23490% with 22 lines in your changes missing coverage. Please review.

Please upload report for BASE (main@dbc5b7b). Learn more about missing BASE report.

Files	Patch %	Lines
src/backend/schemas/context.py	64.00%	9 Missing :warning:
...ckend/alembic/versions/2024_08_19_c301506b3676_.py	57.14%	6 Missing :warning:
src/backend/routers/organization.py	66.66%	3 Missing :warning:
src/backend/tests/unit/routers/test_chat.py	0.00%	2 Missing :warning:
src/backend/database_models/base.py	94.73%	1 Missing :warning:
src/backend/tools/google_drive/tool.py	0.00%	1 Missing :warning:

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #706   +/-   ##
=======================================
  Coverage        ?   75.00%           
=======================================
  Files           ?      262           
  Lines           ?    11656           
  Branches        ?        0           
=======================================
  Hits            ?     8743           
  Misses          ?     2913           
  Partials        ?        0           

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[raphael-cohere]

Can you give a bit more context about this change? Is security relevant or will all users be able to access every organisation? Because if not I can easily change the organisation id in the header? Wouldn't it be better to just fetch it from the authenticated user?

[EugeneLightsOn]

Can you give a bit more context about this change? Is security relevant or will all users be able to access every organisation? Because if not I can easily change the organisation id in the header? Wouldn't it be better to just fetch it from the authenticated user?

For now, we do not have any restrictions related to organizations. However, I agree that we need to implement them in the future.