Create and search a vector database from a wide variety of file types and get more reliable responses from an LLM. This is commonly referred to as "retrieval augmented generation."
Requirements
| Tool | Purpose |
|---|---|
| 🪟 Microsoft Windows | Only for Windows but open to pull requests |
| 🐍 Python 3.11–3.13 | Run the application |
| 🌿 Git | Clone / manage the repository |
| 🧲 Git LFS | Handle large model files |
| 📄 Pandoc | Document parsing support |
| 🛠️ Visual C++ Build Tools | Required for compiling dependencies |
Or you can try running these commands in Powershell on Windows:
Install:
winget install Microsoft.VisualStudio.2022.BuildTools --silent --accept-source-agreements --accept-package-agreements --override "--wait --quiet --add Microsoft.VisualStudio.Component.VC.Tools.x86.x64 --add Microsoft.VisualStudio.Component.Windows11SDK.22621"Verify installation:
Test-Path "C:\Program Files\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC"Installation
Download the latest "release," extract, navigate to the src folder, and run the following commands:
python -m venv ..\Scripts\activatepython setup_windows.pypython gui.pyInputs → Processing → Vector Database
| 📂 Ingest | 📄 .pdf, .docx, .txt, .html, .htm, .md, .csv, .xls, .xlsx, .xlsm, .rtf, .eml, .msg 🖼️ .png, .jpg, .jpeg, .bmp, .gif, .tif, .tiff 🎵 .mp3, .wav, .m4a, .ogg, .wma, .flac |
| ⚙️ Process | 📝 Extract text from documents 🖼️ Generate descriptions from images 🎧 Transcribe speech from audio |
| 🧠 Store | All processed content is embedded and saved into the vector database for searching. |
Query → LLM → Output
| ❓ Ask | ⌨️ Type or 🎙️ record a question |
| 🧠 Retrieve | Relevant chunks are pulled from the vector database |
| 🤖 Generate | Sent to an LLM (Local Model, Kobold, LM Studio, or ChatGPT) |
| 💬 Respond | LLM returns an answer based on the context you provided |
| 🔊 Optional | Text-to-speech can read the response aloud |
Usage
[!NOTE] Instructions on how to use the program are being consolidated into the
Ask Jeevesfunctionality, which can be accessed from the "Ask Jeeves" menu option. Please create an issue if Jeeves is not working.
Request a Feature or Report a Bug
Feel free to report bugs or request enhancements by creating an issue on github and I will respond promptly.
Contact
I welcome all suggestions - both positive and negative. You can e-mail me directly at "[email protected]" or I can frequently be seen on the KoboldAI Discord server (moniker is vic49). I am always happy to answer any quesitons or discuss anything vector database related! (no formal affiliation with KoboldAI).