Connor Finn

feat: Pinecone Vector Database

1

Added Pinecone vector database support. To setup (also in readme) set `VECTOR_DB_TYPE=pinecone` and `PINECONE_API_KEY=` Tested with Pinecone, openai `text-embedding-3-small` 1) Embedding multiple text files 2) Getting all ids for multiple...

feat: Qdrant Vector Database

4

Qdrant vector database supported. Added section in readme in setting up Qdrant environmental variables as well as specifically running docker image. Implemented Async methods for qdrant. Tested using bedrock with...

chore: updating lite requirements to include ollama and others

1

Updated lite requirements list to better reflect requirements list: langchain-ollama==0.2.0 langchain-openai==0.2.0 langchain-huggingface==0.1.0

refactor: Async batch processing, limits, and configuration

1

Added async api calls to embed (process) and add chunks (documents) to vector storage. Added env variables: `MAX_CHUNKS`, `EMBEDDING_TIMEOUT`, `BATCH_SIZE`, and `CONCURRENT_LIMIT` to configure/limit batching api class. Description in Readme....

Unit Tests

1

Added unit tests with [Pytest](https://docs.pytest.org/en/stable/) can run `pytest` while in directory to run tests Sets up by embedding two text files in testFiles folder (superbowl.txt and short.txt) Tests getting IDs,...

Added `RAG_AZURE_OPENAI_BASEURL` and `RAG_AZURE_OPENAI_PROXY` environment variables to be passed into AzureOpenAIEmbeddings. Updated readme.