AIFIRST WebLLM: high-performance in-browser LLM inference engine
The brief
WebLLM enables high-performance large language model inference directly in the browser using WebGPU, with no server required.
Key points
- Models run fully client-side, making inference private and zero-cost after initial download.
- The project supports a range of open-weight models and is available as an open-source library.
Sources
- HNgithub.com