super-rag

super-rag copied to clipboard

# Describe the issue A user should also be able to filter the queries on the metadata.

elisalimli

# Describe the issue We should return an error if the content of the file is malformed, such as CSV files have missing columns.

elisalimli

Add Kafka

4

# What does this PR do? Fixes #39 #53 ## Steps to test 1. Run `docker-compose up -d` to spin up kafka, kafka-ui and redis 2. Start python server 3....

elisalimli

Scalable data ingestion architecture with microservices

11

Building the Docker image I noticed it was sized at 7GB in size and that it took 5 minutes for the build to complete. |REPOSITORY |TAG |IMAGE ID |CREATED |SIZE...

teocns

elisalimli

# Describe the issue In the [current implementation](https://github.com/superagent-ai/super-rag/pull/91) the Kafka consumer we process messages synchronously, so this causes the "Head-of-Line Blocking" problem. One possible fix to leverage would be leveraging...

elisalimli

# Describe the issue Somestimes interpreter mode doesn't work. SAML: ```yaml workflows: - superagent: llm: gpt-4-turbo-preview name: Browser assistant intro: 👋 Hi there! How can I help search for answers...

elisalimli

Can't summarise if the doc contains more than 16k tokens

1

Since we use gpt-3.5-turbo-16k model for summarising. If we try to summarise a doc is relatively long. It will not be able to do it. https://github.com/superagent-ai/super-rag/blob/main/utils/summarise.py#L33 Example file: https://github.com/datasciencedojo/datasets/blob/master/titanic.csv

elisalimli

Make it possible to use as a library.

2

# Describe the issue Make it possible to use Super-Rag as a Python library. Preferable syntax would be: ```python from super_rag import SuperRag client = SuperRag(config) # Create index client.ingest(payload)...

homanp

Expose configurable parameters: Ingest: - [x] - chunk_size - [x] - chunk_strategy - [x] - encoder (already done) Query: - [ ] - query transformation options (query fusion, step-back, query...

simjak