causify-ai
HelpersTask207_create_index_for_readme
@gpsaggese According to my understand README.md is the index file. and the goal https://github.com/causify-ai/helpers/issues/207 is to generate the index file. The approach is based on this assumption and create and execute which:
- searches for the README.md in the root of the repo, and creates one if it is not present.
- Creates an index in based on the directory where the readme is located. This is done by traversing the repo and listing the markdown files in each dir.
- Generates 2 line summaries based for each md file if needed (controlled through set of args)
This is not a polished code, WDYT of the approach?
Cool stuff! See some comments below. But most importantly, have you tried it out? Does it work?:) Would be best, of course, if we could have some tests for the key functions.
We could create a set of tests by mocking the openai client. But the actual working depends a lot on the kind of the subscription we have to the OpenAI API (for now since it is personal api keys). For smaller docs it might not be a problem, but for bigger docs, we will hit the token limit.
Cool stuff! See some comments below. But most importantly, have you tried it out? Does it work?:) Would be best, of course, if we could have some tests for the key functions.
We could create a set of tests by mocking the openai client. But the actual working depends a lot on the kind of the subscription we have to the OpenAI API (for now since it is personal api keys). For smaller docs it might not be a problem, but for bigger docs, we will hit the token limit.
I agree, this can be a problem. Can we add a parameter to bypass summary generation and just add something in its place (e.g. "Insert summary for <file_name> here" or "Placeholder summary for <file_name>")? And then we can run the tests with that parameter enabled.
import openai raised an error 'No Module Found', which means that the docker container do not have openai installed. I also do not have the authority to install inside docker. So a temp fix is to encapsulate import hopenai in the function where it is called generate_summary_for_file().
Logic changes are in https://github.com/causify-ai/helpers/issues/207#issuecomment-2795737604.
