Error: Could not read directory: .autodoc/docs/markdown/. Did you run `sh download.sh`?

[yoaquim]

trafficstars

Followed the instruction as they're lined out. Everything should be setup correctly.

I keep getting Failed to get summary for file while indexing files, and at the end I get:

Error: Could not read directory: .autodoc/docs/markdown/. Did you run `sh download.sh`?
    at processDirectory (file:///opt/homebrew/lib/node_modules/@context-labs/autodoc/dist/cli/commands/index/createVectorStore.js:33:15)
    at RepoLoader.load (file:///opt/homebrew/lib/node_modules/@context-labs/autodoc/dist/cli/commands/index/createVectorStore.js:57:22)
    at createVectorStore (file:///opt/homebrew/lib/node_modules/@context-labs/autodoc/dist/cli/commands/index/createVectorStore.js:62:34)
    at index (file:///opt/homebrew/lib/node_modules/@context-labs/autodoc/dist/cli/commands/index/index.js:39:11)
✖ Create vector files...

[samheutmaker]

"Failed to get summary" can be caused by a handful of things. Right now logging isn't the best, so apologies for that.

Is your code public? I can take a look to see if I can figure what may be causing this error.

[yoaquim]

It is not public, sorry.

It is frontend code: Typescript and React.

I can try it on an open-source project I host and post here accordingly.

Edit: I also ran it on backend Typescript code (an API) and I get the same error.

[samheutmaker]

Please post more info about your environment.

[yoaquim]

Node: v19.5.0 Macbook Pro: Apple M2 Max

Worth noting that I just ran it on the open source project and it worked.

[samheutmaker]

I'll add better logging to surface the underlying error causing "could not get summary"

[onmax]

I tried with this repo and the same error was thrown.

[dahifi]

This is issue was fixed by checking node version is 19. I had installed in an older version and chdir to a different repo that changed it.

[polto]

I have the same error on one of the computers where I am using autodoc. it worked first and stopped after first indexing. Now I have this error on every repo, including the first that succeed once. Even if I wipe it , git clone again and try to re-run autodoc - I have this same error on that computer. I tried both nodejs 18 and 19.

[dahifi]

Make sure you install dependencies in 19. /shrug

[ChanJoon]

I have the same error, which conducted on the following environment. node -v : v18.12.1 OS: Windows 10

This is what appeared in my terminal. I modifed my private contents with <> tags.

Cost estimate to process this repository: $0.01
This is just an estimate. Actual cost may vary.
It recommended that you set a limit in your OpenAI account to prevent unexpected charges.
? Do you want to continue with indexing? Yes
Starting crawl...
⠙ Processing repository...(node:26184) ExperimentalWarning: The Fetch API is an experimental feature. This feature could change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
⠋ Processing 5 files...Error: Request failed with status code 429
    at createError (file:///C:/Users/<USERNAME>/AppData/Roaming/npm/node_modules/@context-labs/autodoc/node_modules/langchain/dist/util/axios-fetch-adapter.js:302:19)
    at settle (file:///C:/Users/<USERNAME>/AppData/Roaming/npm/node_modules/@context-labs/autodoc/node_modules/langchain/dist/util/axios-fetch-adapter.js:24:16)
    at file:///C:/Users/<USERNAME>/AppData/Roaming/npm/node_modules/@context-labs/autodoc/node_modules/langchain/dist/util/axios-fetch-adapter.js:185:19
    at new Promise (<anonymous>)
    at fetchAdapter (file:///C:/Users/<USERNAME>/AppData/Roaming/npm/node_modules/@context-labs/autodoc/node_modules/langchain/dist/util/axios-fetch-adapter.js:177:12)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
  config: {
    transitional: {
      silentJSONParsing: true,
      forcedJSONParsing: true,
      clarifyTimeoutError: false
    },
    adapter: [AsyncFunction: fetchAdapter],
    transformRequest: [ [Function: transformRequest] ],
    transformResponse: [ [Function: transformResponse] ],
    timeout: 0,
    xsrfCookieName: 'XSRF-TOKEN',
    xsrfHeaderName: 'X-XSRF-TOKEN',
    maxContentLength: -1,
    maxBodyLength: -1,
    validateStatus: [Function: validateStatus],
    headers: {
      Accept: 'application/json, text/plain, */*',
      'Content-Type': 'application/json',
      'User-Agent': 'OpenAI/NodeJS/3.2.1',
      Authorization: 'Bearer <MY_OPENAI_API_KEY>'
    },
    method: 'post',
    data: '{"model":"gpt-3.5-turbo","temperature":0.1,"top_p":1,"frequency_penalty":0,"presence_penalty":0,"n":1,"stream":false,"messages":[{"role":"user","content":"\\n    <PROMPT_WITH_MY_FILE_CONTENTS>"}]}',
    url: 'https://api.openai.com/v1/chat/completions'
  },
  request: Request {
    [Symbol(realm)]: { settingsObject: [Object] },
    [Symbol(state)]: {
      method: 'POST',
      localURLsOnly: false,
      unsafeRequest: false,
      body: [Object],
      client: [Object],
      reservedClient: null,
      replacesClientId: '',
      window: 'client',
      keepalive: false,
      serviceWorkers: 'all',
      initiator: '',
      destination: '',
      priority: null,
      origin: 'client',
      policyContainer: 'client',
      referrer: 'client',
      referrerPolicy: '',
      mode: 'cors',
      useCORSPreflightFlag: false,
      credentials: 'same-origin',
      useCredentials: false,
      cache: 'default',
      redirect: 'follow',
      integrity: '',
      cryptoGraphicsNonceMetadata: '',
      parserMetadata: '',
      reloadNavigation: false,
      historyNavigation: false,
      userActivation: false,
      taintedOrigin: false,
      redirectCount: 0,
      responseTainting: 'basic',
      preventNoCacheCacheControlHeaderModification: false,
      done: false,
      timingAllowFailed: false,
      headersList: [HeadersList],
      urlList: [Array],
      url: [URL]
    },
    [Symbol(signal)]: AbortSignal { aborted: false },
    [Symbol(headers)]: HeadersList {
      [Symbol(headers map)]: [Map],
      [Symbol(headers map sorted)]: null
    }
  },
  response: {
    ok: false,
    status: 429,
    statusText: 'Too Many Requests',
    headers: HeadersList {
      [Symbol(headers map)]: [Map],
      [Symbol(headers map sorted)]: null
    },
    config: {
      transitional: [Object],
      adapter: [AsyncFunction: fetchAdapter],
      transformRequest: [Array],
      transformResponse: [Array],
      timeout: 0,
      xsrfCookieName: 'XSRF-TOKEN',
      xsrfHeaderName: 'X-XSRF-TOKEN',
      maxContentLength: -1,
      maxBodyLength: -1,
      validateStatus: [Function: validateStatus],
      headers: [Object],
      method: 'post',
      data: '{"model":"gpt-3.5-turbo","temperature":0.1,"top_p":1,"frequency_penalty":0,"presence_penalty":0,"n":1,"stream":false,"messages":[{"role":"user","content":"\\n    <PROMPT_WITH_MY_FILE_CONTENTS>"}]}',
      url: 'https://api.openai.com/v1/chat/completions'
    },
    request: Request {
      [Symbol(realm)]: [Object],
      [Symbol(state)]: [Object],
      [Symbol(signal)]: [AbortSignal],
      [Symbol(headers)]: [HeadersList]
    },
    data: { error: [Object] }
  },
  isAxiosError: true,
  toJSON: [Function: toJSON]
}
Failed to get summary for file <MY_FILE_NAME>
// ...and similar errors continued.
// After that,
Error: Could not read directory: .autodoc\docs\markdown\. Did you run `sh download.sh`?
    at processDirectory (file:///C:/Users/82104/AppData/Roaming/npm/node_modules/@context-labs/autodoc/dist/cli/commands/index/createVectorStore.js:33:15)
    at RepoLoader.load (file:///C:/Users/82104/AppData/Roaming/npm/node_modules/@context-labs/autodoc/dist/cli/commands/index/createVectorStore.js:57:22)
    at createVectorStore (file:///C:/Users/82104/AppData/Roaming/npm/node_modules/@context-labs/autodoc/dist/cli/commands/index/createVectorStore.js:62:34)
    at index (file:///C:/Users/82104/AppData/Roaming/npm/node_modules/@context-labs/autodoc/dist/cli/commands/index/index.js:51:11)
✖ Create vector files...

In my opinion, the following reponses are the reason of this issue.

  response: {
    ok: false,
    status: 429,
    statusText: 'Too Many Requests',
    headers: HeadersList {
      [Symbol(headers map)]: [Map],
      [Symbol(headers map sorted)]: null
    },

Link to OPENAI Article

But as you can see in the first line, it only costs $0.01. How can we fix this issue?

[dana-orca]

Would love to get a solution for that as well :-)