Implement `.q-ignore` functionality for Amazon Q uploads

[selenehyun]

Problem

The current implementation of the Amazon Q feature in the AWS Toolkit for VS Code does not provide a way to exclude certain files or directories when uploading a project. This becomes problematic for users working with monorepo structures or projects that exceed the 200MB upload limit.

Specifically:

1. The 200MB upload limit can be quickly reached, especially in monorepo setups.
2. Files unrelated to the actual code (e.g., build artifacts, large data files) are unnecessarily included in the upload.
3. There's no built-in mechanism to selectively ignore files or directories during the upload process.

This situation leads to inefficient use of the upload capacity and potentially prevents users from utilizing Amazon Q for larger projects.

Expected behavior

We propose implementing a .q-ignore file functionality, similar to how .gitignore works for Git. This feature would allow users to:

1. Create a .q-ignore file in their project root.
2. Specify files, directories, or patterns to be excluded from the Amazon Q upload.
3. Reduce the upload size by excluding unnecessary files, thus staying within the 200MB limit.
4. Customize the upload content to focus on relevant code and documentation.

Example of a potential .q-ignore file:

# Ignore node modules
node_modules/

# Ignore build artifacts
build/
dist/

# Ignore large data files
*.csv
*.json

# Ignore specific directories
legacy-code/
test-data/

This feature would greatly enhance the usability of Amazon Q for larger projects and monorepos, allowing developers to efficiently utilize the service within the given constraints.

[justinmk3]

We propose implementing a .q-ignore file functionality, similar to how .gitignore works for Git.

Any reason Q should not just use .gitignore if it is found?

[selenehyun]

We propose implementing a .q-ignore file functionality, similar to how .gitignore works for Git.

Any reason Q should not just use .gitignore if it is found?

Thank you for your response.

While .gitignore lists files that should not be tracked by version control, it doesn’t necessarily align with the files that Amazon Q needs to analyze or upload.

My project is a monorepo and quite large in size. Even when I explicitly specify files or paths in the prompt, the upload often fails due to the size limit, primarily because of files related to Storybook and other environment configurations.

I proposed the .q-ignore file to exclude code or development environment-related files that are not necessary for analysis, hoping to avoid reaching the upload size limit.

Do you have any suggestions for a better solution?

[justinmk3]

I don't have a better solution. Ideally there would be a quasi-standard pattern to use here, but I didn't find anything. Github copilot has a yaml format but it appears to be part of their UI, not a file that is committed to a project.

[selenehyun]

I don't have a better solution. Ideally there would be a quasi-standard pattern to use here, but I didn't find anything. Github copilot has a yaml format but it appears to be part of their UI, not a file that is committed to a project.

I also couldn't think of a better solution than using a dedicated ignore file similar to .eslintignore or .npmignore. This approach would allow us to exclude specific types of files from being uploaded to Amazon Q, regardless of the language or development environment.

If you believe this idea has merit, I'd be interested to know if it could be included in the development roadmap or if there's any timeline associated with it. Alternatively, I'd be happy to contribute my time to help implement this feature if that's acceptable.

Thank you again for your attention!

[ctodd]

Add to this, the "Problems" tab reports 1000 errors for analyzed but unopened files which are artifacts generated in Amplify projects. No way I can see to clear this without restarting VS Code. +1 for .q-ignore.

[moltar]

It's pretty annoying that Q will ingest all of the output folders (e.g. cdk.out). It just wastes context with no benefit.

[justinmk3]

@moltar Q avoids anything in .gitignore, does that help?

[moltar]

Doesn't seem to work properly then. Because I saw it look at cdk.out which is most definitely in gitignore.

[benoriol]

Was this resolved? I am interested in making Amazon Q ignore my experiment "output" folder which will have 100k+ files and folders and I believe it might be breaking the indexing and Amazon Q's @ functionality.

[AlaaRAD]

Same problem here... Amazon Q gets the whole monolith context, which explodes memory usage. This makes it useless for reviews for instance!

[XiaoxuanLu]

Q currently respects .gitignore in the project, could you create a .gitignore and check if that works in your project.

[jbohren-amzn]

Another use case: if you have any Jupyter notebooks in your workspace, even if they're just referencing other source files, the output content (e.g. large base64-encoded images) will immediately saturate context. It would be great to exclude them (or maybe just make q know how to read only the relevant parts of them).

[brlin-tw]

@XiaoxuanLu

Q currently respects .gitignore in the project, could you create a .gitignore and check if that works in your project.

This specific gitignore rule doesn't seem to work in my project, unfortunately:

# Don't track unpacked product files
/project-template-*/