anything-llm icon indicating copy to clipboard operation
anything-llm copied to clipboard

[FEAT]: Default user file upload.

Open A3shTnT opened this issue 9 months ago • 11 comments

What would you like to see?

I want to give default users only chat and upload file analysis permissions, and only use their own workspace. However, currently only the manager role can do this, but the manager can also modify other workspaces. Generally speaking, most apps support ordinary users uploading documents. Do I need additional settings to enable it, or is this feature not currently available in the project?

A3shTnT avatar Feb 26 '25 02:02 A3shTnT

If a user uploaded a document into the chat, it would then have to be embedded in the workspace, which if shared would then be accessible to any other user on the same workspace.

timothycarambat avatar Feb 26 '25 20:02 timothycarambat

If a user uploaded a document into the chat, it would then have to be embedded in the workspace, which if shared would then be accessible to any other user on the same workspace.

So I give each ordinary user a workspace that doesn't interfere with each other, so they can't see the files uploaded by others. Is this reasonable?

A3shTnT avatar Feb 27 '25 01:02 A3shTnT

@A3shTnT That would be the recommended way to do this if we allowed default users to upload documents to a workspace.

timothycarambat avatar Feb 27 '25 18:02 timothycarambat

I also think that default users should be given permissions to create their own workspace, upload files there, and select an LLM model only within that workspace

Yaronbaroz18 avatar Mar 02 '25 07:03 Yaronbaroz18

Any updates on this? We'd like to start using anythingllm for multiple users, but current permissions model is not letting us do this

dimm0 avatar Mar 18 '25 22:03 dimm0

Creative thinking: Wouldn't it be sufficient for most use cases to allow user to only upload documents for the current (!) thread, i.e. not for RAG but just to textualize and embed the full content? Given the huge token window sizes nowadays and most users knowing exactly which specific document they want to "chat with" maybe that is way easier and would fulfill most needs ("summarize", "ask questions", "translate", ...). Comments appreciated @timothycarambat @dimm0 @Yaronbaroz18 @A3shTnT !

brainboutique avatar Mar 20 '25 08:03 brainboutique

Wouldn't it be sufficient for most use cases to allow user to only upload documents for the current (!) thread

It can be done in any other chat system (librechat, openwebui, etc). The benefit of using anythingllm is letting users build the knowledge base and chat with it.

dimm0 avatar Mar 20 '25 13:03 dimm0

I have tested AnythingLLM in a course setup, and having the option for people to upload their own document and chat with it separately from documents embedded in a workspace is something that would be very valuable. Now people all have to have admin rights to be able to do this or manager. Having a Document_User role where people could use their own document in thread would be very helpful.

felixfaassen avatar Apr 13 '25 09:04 felixfaassen

Analysis of Document Upload Solutions for Normal Users

I've used Claude Code to analyze the codebase and suggest a few options on how such a feature could be implemented. Would love to hear what you think, as I'm a very very very basic (old) coder not pretending to understand the AnythingLLM codebase and all the technicalities and of course, user experience philosophies behind the project. Just trying to help to get this feature implemented ;-)

After analyzing various approaches for enabling document uploads by normal users, Solution 4: Role-Based Document Permissions emerges as the best option for several key reasons:

  1. Minimal impact on the codebase - leverages existing authorization mechanisms
  2. Builds on the existing role system rather than modifying core vector database functionality
  3. Maintains consistent architecture by keeping authorization separate from data handling
  4. Future-proof approach that doesn't tie documents to threads or create complex namespacing
  5. Easier maintenance by following established patterns in the codebase
  6. Scalable solution allowing admins to grant permissions to specific users or roles

Implementation primarily involves adding permission flags, modifying middleware, updating the UI, and implementing optional quota controls.

All Potential Solutions Considered:

Solution 1: Thread-Only Document Embedding

This approach allows normal users to upload documents, but these documents are only embedded for the specific thread they're chatting in.

Implementation:

  1. Create a new database table to track thread-specific documents
  2. Modify document upload endpoints to accept a threadId parameter
  3. Add a toggle in admin settings to enable this feature
  4. Implement document lifecycle management with auto-cleanup after thread inactivity

Config options:

  • Maximum document size per upload
  • Maximum total size per user
  • Document retention period

Solution 2: User-Specific Workspace Namespaces

This approach creates sub-namespaces within the workspace's vector database.

Implementation:

  1. Extend vector database providers to support user-specific namespaces
  2. Modify document upload endpoints to embed documents in user namespaces
  3. Update similarity search to check both global and user namespaces
  4. Add admin controls for enabling/disabling user namespaces

Config options:

  • User document quota
  • Automatic document expiration
  • Visibility of user documents to others

Solution 3: Document Approval Workflow

This approach implements an approval workflow for user-uploaded documents.

Implementation:

  1. Create a pending documents queue
  2. Allow users to upload to this queue
  3. Admins/managers approve or reject documents
  4. Approved documents get embedded in the workspace vector database

Config options:

  • Auto-approval for certain file types/sizes
  • Notification settings for pending approvals
  • Default approval expiration

Solution 4: Role-Based Document Permissions

This approach extends the existing role system with granular document permissions.

Implementation:

  1. Add a new "can_upload_documents" permission
  2. Extend the role system to support this permission
  3. Modify middleware to check for this specific permission
  4. Add admin UI to toggle this permission per role or per user

Config options:

  • Per-role document quotas
  • Document type restrictions
  • Workspace-specific upload permissions

felixfaassen avatar Apr 13 '25 09:04 felixfaassen

⏺ Implementation Plan for Solution 4: Role-Based Document Permissions

  1. Update Role System

File: /server/utils/middleware/multiUserProtected.js

  • Add a new permission constant: CAN_UPLOAD_DOCUMENTS
  • Add this permission to the ROLES.manager by default
  • Create a helper function to check this specific permission
  1. Modify Authentication Middleware

File: /server/utils/middleware/validatedRequest.js

  • Update the middleware to check for the new permission
  • Create a new middleware function canUploadDocuments that checks if the user has the permission
  1. Update Document Upload Endpoints

File: /server/endpoints/document.js

  • Replace flexUserRoleValid([ROLES.admin, ROLES.manager]) with new middleware that checks for upload permission
  • Modify routes that handle document creation to use this middleware

File: /server/endpoints/api/document/index.js

  • Update API routes to check for the document upload permission
  • Ensure proper error messages for unauthorized users
  1. Add User Management UI Updates

File: /frontend/src/pages/Admin/Users/index.jsx

  • Add a checkbox for the "Can upload documents" permission
  • Update user editing modal to include this permission
  • Add permission state handling in the component
  1. Add Database Schema Updates

File: /server/prisma/schema.prisma

  • Update the User model to include the new permission flag
  • This might be done by extending existing permissions or adding a specific flag

File: /server/models/user.js

  • Update user methods to handle the new permission
  • Add functions to check, grant, and revoke this permission
  1. Add Admin Configuration UI

File: /frontend/src/pages/Admin/Users/index.jsx

  • Add UI controls for administrators to manage this permission
  • Include batch operations for enabling/disabling this permission for multiple users
  1. Add Quota Management (Optional)

File: /server/models/documents.js

  • Add methods to track document usage per user
  • Implement quota checking before allowing uploads

File: /server/endpoints/document.js

  • Add quota validation before processing uploads
  • Return appropriate error messages when quotas are exceeded
  1. Update Frontend Components

File: /frontend/src/components/WorkspaceChat/index.jsx

  • Update the UI to conditionally show upload controls based on user permissions
  • Modify document upload components to check permissions client-side
  1. Documentation Updates
  • Update API documentation to reflect new permission
  • Add user guide information about the new feature

felixfaassen avatar Apr 13 '25 09:04 felixfaassen

@timothycarambat I created a draft PR for this feature. I'm feeling very uncomfortable with this, as I'm no expert coder, and this feature is above my knowledge/experience and knowledge about the codebase.

You can see the draft PR here: https://github.com/Mintplex-Labs/anything-llm/pull/3638 It follows the implementation solution using Role-Based Document Permissions as mentioned earlier in this issue.

I attempted to create a basic implementation to demonstrate its functionality. I tested the feature locally in a Docker instance, and it functions.

There currently is no implementation for automatically deleting documents uploaded by users. This is something that requires more thought and work.

I hope this is helpful. Please check very thoroughly, as I used Claude Code to assist me with these changes.

felixfaassen avatar Apr 13 '25 13:04 felixfaassen

I've run into this issue today - a client was very confused about why the file they uploaded wasn't fully referenced by the AI modal being used.

I was surprised to see file uploads actually get embedded in the workspace, rather than just being "pinned" to the thread. It's an odd UX choice, as now a private file from a private thread has been embedded into the workspace and is available to all users of that workspace.

From your suggestions above @felixfaassen I think solution 1 is the most sensible - allow people to upload files to their threads, don't embed them, but instead provide the entire context to the AI to respond.

scottybo avatar May 07 '25 18:05 scottybo

Related issues:

  • https://github.com/Mintplex-Labs/anything-llm/issues/2301
  • https://github.com/Mintplex-Labs/anything-llm/issues/1543
  • https://github.com/Mintplex-Labs/anything-llm/issues/3287
  • https://github.com/Mintplex-Labs/anything-llm/issues/3137

scottybo avatar May 07 '25 18:05 scottybo

@scottybo I think you are right; when I implemented solution 4, a lot more details needed to be considered as well, and it is kinda ackward for users to see them upload documents in a chat and then have them available for the entire workspace.

felixfaassen avatar May 07 '25 19:05 felixfaassen