Vallama

Vallama is a high-level Vala library for running LLM inference with llama.cpp, with native tool calling. It wraps the model, context, sampler and tokenizer objects of the C API into a small set of Vala classes — Model, Context, Sampler, Generator, ChatSession, Embedder — and takes care of the parts of llama.cpp that bite: chat templates, grammar ordering, streaming callbacks, and the context parameters that are fixed at creation.

It is built the way the rest of the stack is built: the collections underneath are Invercargill, so transcripts and tool lists are Enumerable sequences rather than bare arrays, and the examples that ship in examples/ are the runnable versions of every idiom on this page.

Because the model file is a local GGUF and the inference is local compute, an application built on Vallama depends on no cloud service, sends nothing anywhere, and keeps working when the network does not — the same decentralised stance the rest of Astrologue takes.

• Native tool calling. OpenAI-style tool calling driven by grammar-constrained decoding: when tools are registered, the assistant turn is generated under a GBNF grammar compiled from the tool schemas, so the model can only emit a valid tool-call envelope.
• Chat sessions. ChatSession wraps Model + Context + Sampler, keeps the transcript, formats prompts, runs constrained turns, and offers run_with_tools to drive the whole loop via a callback.
• Streaming generation. generate_stream and respond_stream hand each decoded piece to a callback, so interfaces can print as the model thinks.
• KV-cache priming. prime() pre-evaluates a long system prompt once; save()/load() persist the resulting KV state to a file so later runs skip the expensive evaluation and start generating immediately.
• Embeddings. A high-level text-embedding API on llama.cpp's sequence pooling: CLS, MEAN and LAST_TOKEN pooling, L2-normalised vectors, and batched embed_all over independent sequences.
• Tokenizers. Exposed directly for piece-level work, with the grammar sampler attachable to any chain.
• Chat templates handled. The model's template when it expands, an exact ChatML fallback otherwise, and a hand-rolled ChatML renderer whenever tool turns are in the transcript.

With USM on your machine, one command acquires, builds and installs the library with its headers, Vala bindings and introspection:

usm install vallama

From source, the stack's one-build discipline:

git clone <the Vallama repository>
cd Vallama
meson setup build
ninja -C build
ninja -C build install

Then compile against it with Vala's usual package mechanism:

valac --pkg vallama-0.1 my-program.vala

Generation streams piece by piece, with model, context and sampler split the way llama.cpp splits them:

Vallama.init();

var model = new Model (model_path, ModelOptions.defaults ());
var ctx_opts = ContextOptions.defaults ();
ctx_opts.n_ctx = 2048;
ctx_opts.n_threads = 8;
var ctx = new Context (model, ctx_opts);
var sampler = new Sampler (SamplerOptions.defaults ());
var generator = new Generator (ctx, sampler);

generator.generate_stream (prompt, max_tokens, (piece, token, is_final) => {
    print ("%s", piece);
    return true;      // false stops generation
});

Tool calling, the headline feature

Tools are declared with a small JSON-schema-ish API, compiled into a grammar that only accepts a valid call envelope (or a respond envelope carrying plain language):

var city = new ToolParam ("city", ToolParamType.STRING, true, "City name");
var tool = new ToolSpec ("get_weather", "Get current weather", params);

ChatSession keeps the transcript and runs the constrained loop:

var session = new ChatSession (model, ctx, sampler);
session.add_tools (tools);
session.add_message (new ChatMessage (ChatRole.SYSTEM, system_prompt));

if (session.chat ("Weather in Paris?") == ChatTurnResult.TOOL_CALL) {
    var call = session.last_tool_call;
    string result = execute (call);           // your code
    session.append_tool_result (call, result);
    session.chat (null);                      // final prose turn
    print ("%s\n", session.last_message_text ());
}

The whole assistant turn stays inside one JSON envelope on purpose: GBNF cannot express "any text OR this JSON", so the respond pseudo-tool gives the model a clean escape hatch while every turn stays parseable. The parser accepts both the bare envelope and Qwen2.5-style <tool_call> markup.

Caching the system prompt

Pre-evaluate a long system prompt once and reuse the KV state — across turns in one run, or from a file in the next:

generator.prime (SYSTEM_PROMPT);        // evaluate once, keep KV state
generator.save ("/path/to/cache.bin");  // persist it

// a later run:
generator.load ("/path/to/cache.bin");  // skip prompt evaluation,
generator.respond ("Why is the sky blue?");

Embeddings

One text or many, L2-normalised, with the pooling mode the model wants:

var model = new Vallama.Model ("/path/Qwen3-Embedding-0.6B-Q8_0.gguf");
var embedder = new Vallama.Embedder (model, Vallama.Pooling.LAST_TOKEN, ctx_opts);

int dimensions = embedder.dimensions;      // model n_embd, e.g. 1024
float[] query = embedder.embed ("Query: a city at the bottom of New Zealand");

var texts = new Invercargill.DataStructures.Vector<string> ();
texts.add ("Invercargill is the southernmost city in New Zealand.");
float[] docs = embedder.embed_all (texts);          // flat row-major
float[] doc0 = embedder.vector_at (docs, 0);        // slice row 0

The test suite runs offline — meson test needs no model file: it pins the pooling mapping, normalisation maths, ChatML round-trips, envelope parsing (including malformed input) and template fallback selection, and validates the GBNF compiler's output through llama.cpp's own grammar parser. Live-model runs gate on VALLAMA_TEST_MODEL=/path/to/model.gguf; without the variable the examples print their API surface and exit cleanly.

Version

0.1.0 (from MANIFEST.usm)

Licence

LGPL-3.0-or-later — free to run, study, share and improve; the LGPL keeps it free to link from any program.

Package

usm install vallama

Provides

libvallama-0.1.so, vallama-0.1.h and vallama-llama-shim.h, vallama-0.1.vapi + .deps plus the llama.vapi binding in share/vala/vapi, Vallama-0.1.gir, vallama-0.1.pc

Needs at runtime

GLib, GObject, libinvercargill-1.so, libjson-glib-1.0.so, and llama.cpp's libllama.so. A GGUF model file of your choosing.

Source

The repository clanker/Vallama on the Astrologue forge (git over ssh, port 222). Free software: the licence guarantees your right to the source.