Accessible Chat Client for OpenAI-Compatible LLM Servers
VOLlama talks to any server that speaks the OpenAI API: Ollama, llama.cpp, LM Studio, vLLM, OpenAI, Gemini, OpenRouter, and so on. You describe each server with a preset that holds its base URL, API key, model, system prompt, and generation parameters.
If you want to run models locally, download and install Ollama, then pull a model. Replace gemma4:e4b-it-qat if you prefer a different model.
ollama pull gemma4:e4b-it-qat
Optionally, if you want to utilize the retrieval-augmented generation feature, you need an embedding model.
ollama pull embeddinggemma
Finally, download the latest and run VOLlama.
For Mac, VOLlama is not notarized by Apple, so you need to allow to run in system settings > privacy and security.
VOLlama may take a while to load especially on Mac, so be patient. You’ll eventually hear “VOLlama is starting.”
If you want responses to be read aloud automatically, you can enable the “Speak Response with System Voice” option from the chat menu.
On first launch VOLlama asks you to create a preset. After that, press control+p to open the preset menu. It lists your presets, so switching to one is a single keystroke, and it has one other item: Preset Manager. The toolbar button shows the active preset, which is remembered between sessions.
The Preset Manager is where presets are created, edited, copied and deleted. At the top is the same preset button the toolbar has: its menu lists your presets, and below them New, Duplicate and Delete. Under the button are three tabs, editing whichever preset the button names:
Nothing is saved until you press OK, so Cancel discards every change including new presets, deletions, parameters and prompts. The preset the button is showing when you press OK becomes the active one.
API keys are encrypted before they are written to disk. Leave the key empty for local servers that do not need one.
The Tools checkbox in the Chat menu lets the model read, write and edit files and run commands on your computer. It is off by default and stays where you left it between sessions.
Commands run and files change without asking you first. There is no confirmation prompt and no undo. Only turn it on for a model you trust; small local models often call tools badly.
Workspace in the chat menu is the folder the model works in.The default is VOLlama in your home folder unless you pick another. It is created when you turn tools on, remembered between sessions, and falls back to the default if it is gone the next time you start.
The workspace is a starting point rather than a sandbox. The model can run a command elsewhere and write outside it.
Escape stops the reply, and stops a command still running in the foreground.
Shortcuts for all the features can be found in the menu bar. Here are exceptions:
In order to ask a multimodal model questions about an image:
These parameters are sent to the server as OpenAI API options. All of them start empty, and an empty field is not sent at all, so the model uses its own default.
| Parameter | Description | Value Type |
|---|---|---|
| max_tokens | Maximum number of tokens to generate in the response. | int |
| temperature | Adjusts the model’s creativity. Higher values lead to more creative responses. Range: 0.0-2.0. | float |
| top_p | Nucleus sampling. Higher values lead to more diverse text, lower values to more focused text. Range: 0.0-1.0. | float |
| presence_penalty | Penalizes new tokens based on their presence so far. Range: -2.0-2.0. | float |
| frequency_penalty | Penalizes new tokens based on their frequency so far. Range: -2.0-2.0. | float |
| stop | Triggers the model to stop generating text when this pattern is encountered. List strings separated by “, “. | string Array |
| seed | Sets the random number seed for generation. Specific numbers ensure reproducibility. | int |
| reasoning_effort | Sets the reasoning effort: none, low, medium, or high. Ignored by models without reasoning. | string |
The context window is not a generation parameter. It sits on the Connection tab of the preset editor, next to the model, because it describes what that model on that server can hold. It is never sent: VOLlama uses it to decide when to compact the conversation, and RAG uses it to work out how many retrieved chunks fit in one prompt.
A long chat eventually stops fitting in the model’s context window. When that happens the server either refuses the message or quietly drops the oldest part of the chat, and the model forgets how the conversation started without telling you.
So when a reply uses more than 80 percent of the context window, VOLlama asks the model to summarize the conversation for a copy of itself that will never see the original, and the summary takes the place of everything before it in what gets sent from then on. The chat window says Compacted: on the line where it happened. You can also do it at any time from the Edit menu, or with control+shift+K.
If the server refuses a message because the chat is too long, VOLlama compacts and sends it again. Nothing is deleted. The whole chat stays in the window, saves in full, and you can use alt+up to walk back through all the previous messages. Only what is sent to the model changes. Saving keeps the summary, so a chat you reopen carries on from where it was compacted.
Starting a new chat, or clearing the last message, throws the summary away.
To retrieve a document and ask questions about it, follow these steps:
https://bbc.com/.
/q What are some positive news for today? without the quotes. Prefacing your message with /q triggers processing your prompt with RAG./q remains for models that ignore tools or call them badly.Note: RAG retrieves only snippets of text relevant to your question, not full summaries. If you want to see the retrieved context, turn on Show Context on the Rag menu.
This section describes the parameters related to the Retrieval-Augmented Generation (RAG) feature. They also belong to a preset, on the RAG page of the Preset Manager.
| Parameter | Description | Value Type |
|---|---|---|
| embedding_base_url | Base URL of the OpenAI-compatible server used for embeddings. | string |
| embedding_api_key | API key for the embedding server. Empty for local servers. | string |
| embedding_model | Model used to embed documents and questions. | string |
| chunk_size | Determines the size of text chunks for indexing. | int |
| chunk_overlap | Specifies the overlap between the start and end of each chunk. | int |
| similarity_top_k | Number of the most relevant chunks fed to the model. | int |
| similarity_cutoff | The threshold for filtering out less relevant chunks. Setting too high may exclude all chunks. | float |
If you prefer to run Ollama using Docker, follow the instructions below:
Install Ollama by executing the following command in the command line:
docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
Download a model to generate text. Replace gemma3:4b-it-qat with your desired model if you wish to use a different model:
docker exec ollama ollama pull gemma3:4b-it-qat
Optionally, If you wish to use the retrieval-augmented generation feature, download an embedding model:
docker exec ollama ollama pull embeddinggemma
To stop Ollama, use the following command:
docker stop ollama
To restart Ollama, use the command below:
docker start ollama
VOLlama uses uv, which is the only thing you need installed. It fetches the right Python and every dependency itself.
git clone https://github.com/chigkim/VOLlama
cd VOLlama
uv run vollama
The first run creates the environment and takes a minute; after that it starts faster.
Building runs PyInstaller inside the environment, so this is the one place the environment is created up front and activated:
uv sync
.venv\Scripts\activate
build
uv sync
source .venv/bin/activate
./build.sh
You will find the app inside dist folder.
The bootloader is the small executable PyInstaller puts at the front of a packaged app to start it. The publicshed pip package can sometimes trigger false positive as a malware.
To build, you need Visual Studio Build Tools on Windows.
.venv\Scripts\activate
build-pyinstaller
build
build-pyinstaller clones PyInstaller, compiles the bootloader with waf, and installs that build over the published one in .venv. Running uv sync replaces with the published version on pip, so re-run the script after a sync.