llama.cpp 0.8.0 adds the LFM3 family and lets the server listen and speak through /v1/realtime and /tts

llama.cpp 0.8.0 adds LiquidAI's LFM3 family of models — text, vision, audio and omni — and lets the server take speech in and give speech back through two new endpoints, /v1/realtime and /tts, with a new experimental streaming-audio API underneath. The release page dates it 28 October, 12:47. It also brings a batch of new architectures , router and tool-calling work in the server, and a web UI that finds and downloads models from Hugging Face by itself. One change can stop a command that worked yesterday: the command line now requires--model or -hf. If you start llama.cpp from a script, check it before you update.

What changed

LFM3 arrives in four models: LFM3-1.7B for text and LFM3-VL-2B for images , and LFM3-Audio-2B and LFM3-Omni-2B, which do speech recognition (ASR), text-to-speech (TTS) and voice chat from a single model, with audio streaming in and out. The changelog also enables video input for the LFM3 family. LiquidAI's d1-2B and d1-omni-2B decision models come with them.

mtmd is the part of llama.cpp that handles input other than text — images and sound. Its new mtmd_audio_streamAPI, marked experimental, takes PCM audio — the raw samples a microphone produces — as it arrives, and turns it into embedding rows ready for the model's decoder. The page says it is what supports voxtral-realtime. Audio generation in mtmd now keeps its own state:mtmd_gen_state replaces the raw state_databuffers of the audio generation API.

In the server, /v1/realtimeimplements OpenAI's realtime protocol for conversation and transcription , and the changelog adds tool calls on that endpoint. A newPOST /ttsendpoint serves text-to-speech models. The reasoning effort levels a model's chat template defines are now exposed and can be chosen through the API , and the chat API supports deferred tools andtool_referencecontent parts. Logprobs — the probability the model gave each word it wrote — now work together with speculative decoding.

The router, the server mode that runs several models and unloads them when memory runs short, gets a configurable device memory margin for that unloading. It now passes a client's disconnect on to the child task serving it, and stops adding the mmproj margin a second time when a model sleeps and resumes. The mmproj is the separate file that lets a model read images or audio. A new--connectflag allows peer-to-peer access to the server from outside your local network.

The new architectures are DeepSeek V4.1 Flash, SenseNova U1, Beam, Kolibri-1, Flint, Muse Spark and Granite Switch Decision, plus decision and embedding models from LiquidAI, Baidu and NVIDIA. Muse Spark Open handles text, vision and audio ; Nemotron-3(.5)-Embed comes with support for non-standard GQA and a BPE tokenizer ; H2O-Lightning-4B is a decision model ; MiniCPM-OCR 1.0 and MiniCPM-OCR-Jev read text in images ; and Soprano TTS, a text-to-speech model, is supported.

For mixture-of-experts (MoE) models — built from many blocks called experts, of which each word uses only a few — the experts kept in the computer's main memory now go through a 50/50 SLRU cache instead of a plain LRU cache. An LRU cache keeps what was used most recently and drops the rest; a segmented LRU splits its room in two, so experts used once cannot push out the ones in steady use, and 50/50 is the split the page gives. The shared download code now honors theHTTP_PROXY and HTTPS_PROXYsettings and allows concurrent downloads.

The built-in web UI can now discover and download models directly from HF Hub, Hugging Face's model library, with models that belong to no organization listed first. A new search dialog finds conversations and messages , and conversation compaction keeps long chats within the context — the amount of text the model can hold at once. The reasoning menu is built from the chat template's effort levels , OGG audio attachments are converted to WAV , and the models manager has its own page, with each model's details in a side drawer.

ggml, the library underneath that does the arithmetic on each kind of hardware, moves to v0.28.0. On NVIDIA cards (CUDA), MoE expert selection and the gate/up+GLU step are fused, with graph launches split into chunks, and the SYCL backend gains graph record and replay. OpenCL gains Q2_K and Q3_K weights, and Vulkan gains subgroup reductions ; the Hexagon HTP backend is restructured, and KleidiAI SVE kernels are added.

What it was before

The page lists all of this as the changes since v0.7.0. Until this release, the cache for experts kept in main memory was a plain LRU cache — the cache our item onllama.cpp 0.7.0explains. ggml was at v0.27.0 , and the command line did not yet require--model or -hf.

What it means for you

Start with your scripts. If a script, a service or a shortcut starts a llama.cpp tool without --model or -hf, that command needs one of them from 0.8.0 on. Add--model with the path to your GGUF file, or -hf with the model's name on Hugging Face, before you update a machine that runs on its own.

If you want to talk to a model on your own machine, LFM3-Audio-2B and LFM3-Omni-2B hear and speak from one model , and the server now answers the OpenAI realtime protocol at/v1/realtime. A voice app written for that protocol is what this endpoint is built for; try yours against your own server before you rely on it. For speech from text alone,/ttsis the endpoint. If you write code against mtmd, the streaming API is marked experimental , and code that used the oldstate_data buffers for audio generation needs to move to mtmd_gen_state.

If you run a large MoE model with part of it in main memory, update and measure words per second again: the cache's policy changed , and the page gives no figure for what that changes. How much memory a model needs, and why the speed of that memory matters, is in our guideWhich Machine Runs a Model Locally.

If you serve several models through the router on one card, the new memory margin is how you leave room on the card when it unloads models. The page does not give the option's name, so look it up in the server's documentation before you set it. Leave--connectoff unless you mean your server to be reachable from outside your network. If downloads failed behind a proxy, setHTTPS_PROXYand try again.

If you use the web UI, you can now pick and download a model without leaving the browser , and long chats can be compacted instead of running past what the model holds.

The source

The release page on GitHub: llama.cpp v0.8.0.