LLM Chatbot App ⚡

Frontend Original Link

Github

Features

• Chat history is maintained for each session (if you refresh, chat history clears)
• Intergration with Ollama
• Use Open Models on Local Environment

TODO

• Add endpoints for Non open Models (OpenAI, Gemini, etc)
• Hungging Face model integration
• Vectorize data other than PDF
• MCP Addons
• Agent2Agent

Installation

• Clone the repository

git clone https://github.com/mnc1423/llm_app.git

• Install Ollama (Docker or local Installation) Ollama Github

• Get Ollama Models Ollama Models

ollama pull deepseek-r1

• Setup .env

OLLAMA_ENDPOINT=
STREAMLIT_SERVER_RUN_ON_SAVE=
CHROMA_HOST=

• Run App

docker compose up -d 
OR streamlit run --server.port 8501 --server.address 0.0.0.0 Default_Chat.py

Usage

• General Chat (with Prompts)
• RAG based Chat (Currently Under Development)
• Customized Local Vector DB Exmaple Vector DB API
• Image Processing

Using HF Models (safetensors -> hhuf-> ollama)

• Clone HF repo (with safetensors Inside)
• Using llama.cpp Github

python llama.cpp/convert_hf_to_gguf.py <repo_path> --outfile <model>.gguf --outtype <type>

• Using Ollama

echo "FROM <created Model>" > Modelfile
ollama create <ollama Model> -f Modelfile

Using Gemini

• Get API Key from Google AI Studio link
• Add Key to .env