The progress of open source is too fast!
Just one file, easily deploy LLama on personal computers!
Source code
llamafile is an open source project, its main feature is to allow developers and end users to distribute and run large language models (LLM) using a single file. Here is a detailed introduction to the llamafile project:
Project goal: The goal of the llamafile project is to simplify access to and use of large language models. With this project, users can easily run LLM without complex installation and configuration processes.
Technical implementation: To achieve this goal, llamafile combines llama.cpp with Cosmopolitan Libc into a framework. This combination compresses all the complexity of LLM into a single executable file that can run natively on multiple operating systems, including macOS, Windows, Linux, FreeBSD, OpenBSD, and NetBSD.
Usability: Users only need to download the corresponding llamafile and perform simple operations depending on the operating system (such as adding the .exe extension and double-clicking to run on Windows) to start LLM. In addition, llamafile provides a WebUI interface that allows users to interact with LLM more conveniently.
Supported models: Currently, llamafile supports various large language models, including LLaVA, Mistral, Mixtral, and WizardCoder, etc. These models are all quantized models, so they can run smoothly even in pure CPU environments.
Community support: The llamafile project is hosted on GitHub and has received considerable attention. In just two months, the project has received over ten thousand stars, indicating recognition and interest from developers and users.
In summary, llamafile is an open source project aimed at simplifying the distribution and running of large language models. By compressing a complex LLM into a single executable file, it greatly reduces the user's threshold for use, allowing more people to easily experience and leverage the powerful capabilities of large language models.
The easiest way to try it yourself is to download the example llamafile of the LLaVA model (License: LLaMA 2, OpenAI). LLaVA is a new LLM that can do more than just chat; you can also upload images and ask questions about them. With llamafile, all of this happens locally; no data leaves your computer.
Download llava-v1.5-7b-q4.llamafile (3.97 GB).
Download lava-v1.5-7b-q4.llama file (3.97 GB).
Open your computer's terminal.
Open the terminal on your computer.
If you're using macOS, Linux, or BSD, you'll need to grant permission for your computer to execute this new file. (You only need to do this once.)
If you're using macOS, Linux, or BSD, you'll need to grant permission for your computer to execute this new file. (You only need to do this once.)
chmod +x llava-v1.5-7b-q4.llamafile
chmod +x llava-v1.5-7b-q4.llamafile
If you're on Windows, rename the file by adding ".exe" on the end.
If you're on Windows, rename the file by adding ".exe" at the end.
Run the llamafile. e.g.:
Run the llamafile. e.g.:
./llava-v1.5-7b-q4.llamafile
./llava-v1.5-7b-q4.llamafile
Your browser should open automatically and display a chat interface. (If it doesn't, just open your browser and point it at http://localhost:8080)
Your browser should open automatically and display a chat interface. (If it doesn't, just open your browser and point it at http://localhost:8080)
When you're done chatting, return to your terminal and hit Control-C to shut down llamafile.
When you're done chatting, return to your terminal and hit Control-C to shut down llamafile.