Farfalle
Open-source AI-powered search engine. (Perplexity Clone)
Run local LLMs (llama3, gemma, mistral, phi3), custom LLMs through LiteLLM, or use cloud models (Groq/Llama3, OpenAI/gpt4-o)
https://github.com/rashadphz/farfalle/assets/20783686/9527a8c9-a13b-4e53-9cda-a3ab28d671b2
Please feel free to contact me on Twitter or create an issue if you have any questions.
💻 Live Demo
farfalle.dev (Cloud models only)
📖 Overview
- 🛠️ Tech Stack
- 🏃🏿♂️ Getting Started
- 🚀 Deploy
🛣️ Roadmap
- Add support for local LLMs through Ollama
- Docker deployment setup
- Add support for searxng. Eliminates the need for external dependencies.
- Create a pre-built Docker Image
- Add support for custom LLMs through LiteLLM
- Chat History
- Expert Search
- Chat with local files
🛠️ Tech Stack
- Frontend: Next.js
- Backend: FastAPI
- Search API: SearXNG, Tavily, Serper, Bing
- Logging: Logfire
- Rate Limiting: Redis
- Components: shadcn/ui
Features
- Search with multiple search providers (Tavily, Searxng, Serper, Bing)
- Answer questions with cloud models (OpenAI/gpt4-o, OpenAI/gpt3.5-turbo, Groq/Llama3)
- Answer questions with local models (llama3, mistral, gemma, phi3)
- Answer questions with any custom LLMs through LiteLLM
- Search with an agent that plans and executes the search for better results
🏃🏿♂️ Getting Started Locally
Prerequisites
- Docker
- Ollama (If running local models)
- Download any of the supported models: llama3, mistral, gemma, phi3
- Start ollama server
ollama serve
Get API Keys
Quick Start:
git clone https://github.com/rashadphz/farfalle.git
cd farfalle && cp .env-template .env
Modify .env with your API keys (Optional, not required if using Ollama)
Start the app:
docker-compose -f docker-compose.dev.yaml up -d
Wait for the app to start then visit http://localhost:3000.
For custom setup instructions, see custom-setup-instructions.md
🚀 Deploy
Backend
After the backend is deployed, copy the web service URL to your clipboard. It should look something like: https://some-service-name.onrender.com.
Frontend
Use the copied backend URL in the NEXT_PUBLIC_API_URL environment variable when deploying with Vercel.
And you're done! 🥳
Use Farfalle as a Search Engine
To use Farfalle as your default search engine, follow these steps:
- Visit the settings of your browser
- Go to 'Search Engines'
- Create a new search engine entry using this URL: http://localhost:3000/?q=%s.
- Add the search engine.