Skip to content

Latest commit

 

History

History
77 lines (47 loc) · 2.1 KB

README.md

File metadata and controls

77 lines (47 loc) · 2.1 KB
Raw

Another local GPT UI

This is a user interface for your browser written in Python with the idea to provide a minimal UI for chat with local LLMs using llama.cpp featuring Retrieval Augmented Generation (RAG) and audio input using whisper.cpp.

The UI offers simple user identification without password authentication. This mainly serves to have a history with each user.

Installation

  • Checkout or download zip.
  • python3 -m venv env
  • . env/bin/activate
  • pip install -r requirements.txt
  • in the directory:
    • PYTHONPATH=<path>/server python3 server/app.py
  • Independently start the llama.cpp server
    • ./server -m ~/Downloads/models/dolphin-2.6-mixtral-8x7b.Q6_K.gguf --threads 8 -ngl 100 -c 32768 --cont-batching --parallel 1 -b 128
  • Independently start the GROBID docker image and expose it
    • docker run --rm --init --ulimit core=0 -p 8070:8070 grobid/grobid:0.8.0

Build Javascript from typescript

npm install
npm run build
npm run build_mini

Todos

  • Get rid of langchain