ChatLLM Web
|Updated:
8 Aug 2024
|633 Stars
Chat with LLM like Vicuna totally in your browser with WebGPU, safely, privately, and with no server. Powered by web llm.
Categories
Overview:
The ChatLLM Web is a browser-based chat application powered by WebGPU, offering a secure and private chatting experience with no server involvement. Users can access LLM like Vicuna within their browsers, thanks to the accelerated WebGPU support.
Features:
- Browser-Based: Everything runs within the browser with no server support.
- Web Worker Acceleration: Model runs in a web worker for seamless user experience.
- Easy Deployment: One-click deployment on Vercel in under 1 minute for ChatLLM Web.
- Model Caching: Supports caching to download the model just once.
- Multi-Conversation Chat: Allows multiple conversations with local data storage for privacy.
- Markdown Support: Supports math, code highlighting, and streaming response.
- Responsive UI: Well-designed UI with dark mode support.
- PWA Compatibility: Can be downloaded and run offline as a Progressive Web App.
Installation:
To use the ChatLLM Web app, follow these steps:
- Ensure you have a browser that supports WebGPU, like Chrome 113 or Chrome Canary.
- Make sure your GPU has around 6.4GB of memory for optimal performance.
- Upon the first use, download the 4GB Vicuna-7b model. Subsequent usage will load the model from the browser cache.
Summary:
The ChatLLM Web application offers a unique chatting experience with its browser-based operation, WebGPU acceleration, and privacy-focused features. Users can enjoy multi-conversation chats, markdown support, and a well-designed UI with the ability to deploy the app easily on Vercel. With model caching and PWA support, ChatLLM Web ensures a seamless and responsive chat environment.
