fyin

Open source alternative to Perplexity AI with ability to run locally.

Motivation

This project aims to build a tool that can be run locally, is open-source, and delivers faster answers, serving as an alternative to Perplexity AI.

Features

• [x] Run locally using ollama or use openai API
• [x] local VectorDB for fast search
• [x] very quick searching, scraping & answering due to parallelism
• [x] Configurable number of search results to parse
• [x] local scraping of websites

Demo

(You can watch the demo on YouTube too here)

Installation

1. Clone the repository - git clone https://github.com/shadowfax92/fyin.app
2. Get Bing API key
3. Get OpenAI API key or Ollama
4. Fill/setup the environment variables (see sample.env file, copy it to .fyin.env and fill the values))
5. cargo run --query "<Question>" -n <number of search results>

Environment Variables

# Open AI config; Ollama config in comments

# OPENAI_API_KEY="ollama"
OPENAI_API_KEY="your-openai-api-key"

# OPENAI_BASE_URL=http://localhost:11434/v1
# Leave blank for default
OPENAI_BASE_URL=

BING_SUBSCRIPTION_KEY="your-bing-subscription-key"
# Leave blank for default
BING_ENDPOINT=

# EMBEDDING_MODEL_NAME="llama3"
EMBEDDING_MODEL_NAME="text-embedding-ada-002"

# CHAT_MODEL_NAME="llama3"
CHAT_MODEL_NAME="gpt-4o"

Docker

Here is how you can run the app using docker:

1. Build the docker image - docker build -t fyin .
2. Create environment file - cp sample.env .env and populate the values
3. Run the docker container

docker run --rm --env-file .env fyin --query "<your question>" --search <optional: number of search results to parse>

Notes

• The app use Bing API for searching. You can get from Active Bing API.
• You can get OpenAI API key form OpenAI.
• Ollama setup instructions here.

TODO

• [ ] Simlar to perplexity.ai, use GPT to figure out 3-5 search queries based on prompt
  • This should give better results as we are translating human query into search query.
• [ ] Build a simple website
• [ ] Hosted version of the app