[Bug]: usage of `folders` attribute in context7.json

[raclettejs]

Context7 MCP Version

web

Bug Description

After adding a context7.json and triggering a refresh I'd expect to only see contents from the folders, specified in the folders config array in the context7.json, to be parsed and provided by context7. It seems however that the folder specified is not respected. Linked issue https://github.com/upstash/context7/issues/942#issuecomment-3539316761

Steps to Reproduce

1. add context7.json to your repo
2. add .md files in a specific folder in your repo
3. add foldername to folders configig in context7.json
4. check context7 content for you repo

Expected Behavior

Only see documents from the specified folder to be interpreted

Actual Behavior

still the whole repository seems to be scanned

Transport Method

stdio (default)

Configuration

context7.json:

{
  "$schema": "https://context7.com/schema/context7.json",
  "projectTitle": "racletteJS",
  "description": "A typescript framework/engine for modular platform and portal development",
  "folders": ["compiled-docs"],
  "excludeFolders": ["docs", "reference"],
  "excludeFiles": ["LICENSE.md", "CHANGELOG.md"]
}

Additional Context

Or did we just misconfigure our context7.json?

[fahreddinozcan]

Hey @raclettejs, thanks for the heads up. We'll investigate this, and send a fix ASAP

[raclettejs]

@fahreddinozcan any news?

[enesgules]

Hey @raclettejs this should be fixed, could you please retry? Also you can always claim your library and use the admin view to update these fields.

[enesgules]

Is this the library? https://context7.com/gitlab_raclettejs/docs

[raclettejs]

Is this the library? https://context7.com/gitlab_raclettejs/docs

Hi @enesgules! Thanks for the response. Our framework itself resides here. But as users of the framework use the pluginAPI we have compiled a set of code snipped based, markdown docs to describe its usage. This is the docs repository. For LLM training data we created a folder within our docs repo which will contain the compiled markdown files which we want context7 to consume. The context7 relevant docs content resides here

[AFCMS]

I have the same problem with https://github.com/oktomusic/oktomusic

Configured a documentation repository (both from UI and with the json file) and it still included stuff outside the folder.

https://github.com/oktomusic/oktomusic/blob/master/context7.json

[enesgules]

Hi @AFCMS did you refresh after updating? If not please do!

[enesgules]

Thanks for explaining @raclettejs Could you refresh and see if the selected folders are used? You can also claim ownership of this library using this page: https://context7.com/gitlab_raclettejs/docs/admin

[AFCMS]

Hi @AFCMS did you refresh after updating? If not please do!

Yes, I have tried to refresh multiple times.

I have added my documentation folder like this:

It seems to load the path asked but also files in root directory (in fact, it reads everything according to what is extracted) ? Do I need to explicitly exclude any other files?

Jan 10, 23:36:20
Repository cloned https://github.com/oktomusic/oktomusic in 618ms
Jan 10, 23:36:20
Finding markdown files in root directory
Jan 10, 23:36:20
No RST projects found (no conf.py files)
Jan 10, 23:36:20
Finding markdown files in folder: apps/website/docs
Jan 10, 23:36:20
Found root README file: README.md

[raclettejs]

Thanks for explaining @raclettejs Could you refresh and see if the selected folders are used? You can also claim ownership of this library using this page: https://context7.com/gitlab_raclettejs/docs/admin

I've added the key and url to the context7.json to claim it but context7 keeps telling me that there is no context7.json

[enesgules]

We are investigating

[fahreddinozcan]

It seems to load the path asked but also files in root directory (in fact, it reads everything according to what is extracted) ? Do I need to explicitly exclude any other files?

@AFCMS, this is intentional behavior. When you specify folders, we still include root README files by default since they typically contain important project documentation that users need. If you want to exclude the README.md, you can exclude it from "Files to Exclude" section

[fahreddinozcan]

I've added the key and url to the context7.json to claim it but context7 keeps telling me that there is no context7.json

@raclettejs There seems to be a bug with GitLab authentication that prevented us from fetching the context7.json from GitLab. The fix is on the way.

[fahreddinozcan]

It should be fixed now.

[raclettejs]

@fahreddinozcan I'm afraid but the content is still wrong. I've retriggered context7 but the detected content still seems to be mostly our vitepress plugin which resides in an excluded folder. We only want context7 to parse every .md file within the folder ./compiled-docs of our linked repository, therefore we setup the following context7.json

  "$schema": "https://context7.com/schema/context7.json",
  "projectTitle": "racletteJS",
  "description": "A typescript framework/engine for modular platform and portal development",
  "folders": [
    "compiled-docs"
  ],
  "excludeFolders": [
    "docs",
    "reference"
  ],
  "excludeFiles": [
    "LICENSE.md",
    "CHANGELOG.md"
  ],
  "url": "https://context7.com/gitlab_raclettejs/docs",
  "public_key": "pk_0hOcCed7PKQrJMrfXQhCB"
}

file described by context7 however resides in ./.vitepress/plugins