CASALIOY
CASALIOY copied to clipboard
su77ungr
♾️ toolkit for air-gapped LLMs on consumer-grade hardware
### Issue with current documentation: Some of the commands in the instructions does not support Conda. ### Idea or request for content: Anyone that have figured out how to set...
### .env # Generic MODEL_N_CTX=1024 TEXT_EMBEDDINGS_MODEL=sentence-transformers/all-MiniLM-L6-v2 TEXT_EMBEDDINGS_MODEL_TYPE=HF # LlamaCpp or HF USE_MLOCK=true # Ingestion PERSIST_DIRECTORY=db DOCUMENTS_DIRECTORY=source_documents INGEST_CHUNK_SIZE=500 INGEST_CHUNK_OVERLAP=50 # Generation MODEL_TYPE=LlamaCpp # GPT4All or LlamaCpp MODEL_PATH=eachadea/ggml-vicuna-7b-1.1/ggml-vic7b-q5_1.bin MODEL_TEMP=0.8 MODEL_STOP=[STOP] CHAIN_TYPE=stuff N_RETRIEVE_DOCUMENTS=100...
### .env # Generic TEXT_EMBEDDINGS_MODEL=sentence-transformers/all-MiniLM-L6-v2 TEXT_EMBEDDINGS_MODEL_TYPE=HF # LlamaCpp or HF USE_MLOCK=false # Ingestion PERSIST_DIRECTORY=db DOCUMENTS_DIRECTORY=source_documents INGEST_CHUNK_SIZE=500 INGEST_CHUNK_OVERLAP=50 INGEST_N_THREADS=4 # Generation MODEL_TYPE=GPT4All # GPT4All or LlamaCpp MODEL_PATH=thebloke/ggml-gpt4all-j-v1.3-groovy.bin MODEL_TEMP=0.8 MODEL_N_CTX=1024 # Max...
### .env ``` # Generic TEXT_EMBEDDINGS_MODEL=sentence-transformers/all-MiniLM-L6-v2 TEXT_EMBEDDINGS_MODEL_TYPE=HF # LlamaCpp or HF USE_MLOCK=false # Ingestion PERSIST_DIRECTORY=db DOCUMENTS_DIRECTORY=source_documents INGEST_CHUNK_SIZE=500 INGEST_CHUNK_OVERLAP=50 INGEST_N_THREADS=4 # Generation MODEL_TYPE=LlamaCpp # GPT4All or LlamaCpp MODEL_PATH=eachadea/ggml-vicuna-7b-1.1/ggml-vic7b-q5_1.bin MODEL_TEMP=0.8 MODEL_N_CTX=1024 #...
### .env # Generic TEXT_EMBEDDINGS_MODEL=sentence-transformers/all-MiniLM-L6-v2 TEXT_EMBEDDINGS_MODEL_TYPE=HF # LlamaCpp or HF USE_MLOCK=false # Ingestion PERSIST_DIRECTORY=db DOCUMENTS_DIRECTORY=source_documents INGEST_CHUNK_SIZE=500 INGEST_CHUNK_OVERLAP=50 INGEST_N_THREADS=5 # Generation # MODEL_TYPE=LlamaCpp # GPT4All or LlamaCpp MODEL_TYPE=GPT4All # MODEL_PATH=eachadea/ggml-vicuna-7b-1.1/ggml-vic7b-q5_1.bin MODEL_PATH=TheBloke/GPT4All-13B-snoozy-GGML/GPT4All-13B-snoozy.ggmlv3.q4_0.bin...
### Feature request Increase effitiency of system by translating input and output data to user language. What we need? - Translate input fragments from UserLang to English - Translate output...
### .env # Generic TEXT_EMBEDDINGS_MODEL=sentence-transformers/all-MiniLM-L6-v2 TEXT_EMBEDDINGS_MODEL_TYPE=HF # LlamaCpp or HF USE_MLOCK=false # Ingestion PERSIST_DIRECTORY=db DOCUMENTS_DIRECTORY=source_documents INGEST_CHUNK_SIZE=500 INGEST_CHUNK_OVERLAP=50 INGEST_N_THREADS=3 # Generation MODEL_TYPE=LlamaCpp # GPT4All or LlamaCpp MODEL_PATH=eachadea/ggml-vicuna-7b-1.1/ggml-vic7b-q5_1.bin MODEL_TEMP=0.8 MODEL_N_CTX=1024 # Max...
### .env # Generic TEXT_EMBEDDINGS_MODEL=sentence-transformers/all-MiniLM-L6-v2 TEXT_EMBEDDINGS_MODEL_TYPE=HF # LlamaCpp or HF USE_MLOCK=false # Ingestion PERSIST_DIRECTORY=db DOCUMENTS_DIRECTORY=source_documents INGEST_CHUNK_SIZE=500 INGEST_CHUNK_OVERLAP=50 INGEST_N_THREADS=1 # Generation MODEL_TYPE=LlamaCpp # GPT4All or LlamaCpp MODEL_PATH=eachadea/ggml-vicuna-7b-1.1/ggml-vic7b-q5_1.bin MODEL_TEMP=0.8 MODEL_N_CTX=2048 # Max...
originated #47 > Hi, is it possible to add a jsonarray loader (for huge json file)? And what about output stream functionality of ChatGpt? Is it possible to have a...
### Feature request Hi, some questions: 1) is it possible to integrate your solution on AWS SageMaker? How? 2) did you try your solution on Windows? After following your steps,...
