Astrologue-Website/developer/vallama/index.html

<!doctype html>
<html lang="en-nz">
<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <title>Vallama — Astrologue Developer</title>
  <meta name="description" content="Vallama is a high-level Vala library for running LLM inference with llama.cpp, with native tool calling driven by grammar-constrained decoding: chat sessions, embeddings, tokenizers and tools — on your own hardware. Free software, LGPL-3.0-or-later.">
  <meta name="color-scheme" content="light dark">
  <meta name="theme-color" content="#f2ebda" media="(prefers-color-scheme: light)">
  <meta name="theme-color" content="#0a0a12" media="(prefers-color-scheme: dark)">
  <link rel="icon" href="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 64 64'%3E%3Ccircle cx='32' cy='32' r='27' fill='none' stroke='%23a8801f' stroke-width='4'/%3E%3Cpath d='M32 8l6 18 18 6-18 6-6 18-6-18-18-6 18-6z' fill='%23a8801f'/%3E%3Ccircle cx='32' cy='32' r='4' fill='%23003078'/%3E%3C/svg%3E">
  <meta property="og:title" content="Vallama — Astrologue Developer">
  <meta property="og:description" content="High-level Vala bindings for llama.cpp inference: chat sessions, grammar-constrained tool calling, embeddings and tokenizers — running on your own hardware.">
  <meta property="og:type" content="website">
  <link rel="stylesheet" href="../../assets/css/style.css">
</head>
<body>
  <a class="skip-link" href="#main">Skip to content</a>

  <header class="masthead">
    <div class="container masthead__inner">
      <a class="brand" href="../../">
        <img class="brand__logo brand__logo--light" src="../../assets/img/astrologue-light.png" width="392" height="89" alt="Astrologue">
        <img class="brand__logo brand__logo--dark" src="../../assets/img/astrologue-dark.png" width="392" height="89" alt="Astrologue">
      </a>
      <nav class="site-nav" aria-label="Primary">
        <ul class="site-nav__list">
          <li><a href="../../">Home</a></li>
          <li><a href="../../#news">News</a></li>
          <li><a href="../../#download">Download</a></li>
          <li><a href="../" aria-current="page">Developer</a></li>
          <li><a class="nav-cta" href="../../#download">Get Astrologue</a></li>
        </ul>
      </nav>
    </div>
  </header>

  <main id="main">

    <nav class="breadcrumb" aria-label="Breadcrumb">
      <div class="container">
        <ol>
          <li><a href="../../">Astrologue</a></li>
          <li><a href="../">Developer</a></li>
          <li aria-current="page">Vallama</li>
        </ol>
      </div>
    </nav>

    <section class="docs-header">
      <div class="container">
        <p class="kicker"><span>Intelligence · Language models, locally</span></p>
        <h1>Vallama</h1>
        <p class="lede">High-level Vala bindings for llama.cpp LLM inference: chat sessions, grammar-constrained tool calling, embeddings and tokenizers — running on your own hardware, with no cloud in sight.</p>
        <dl class="docs-facts">
          <div><dt>Version</dt> <dd><code>0.1.0</code></dd></div>
          <div><dt>Licence</dt> <dd>LGPL-3.0-or-later</dd></div>
          <div><dt>Package</dt> <dd><code>usm&nbsp;install&nbsp;vallama</code></dd></div>
          <div><dt>Language</dt> <dd>Vala</dd></div>
        </dl>
      </div>
    </section>

    <section class="band band--plain" id="docs">
      <div class="container docs-layout">

        <nav class="docs-toc" aria-label="On this page">
          <h2>On this page</h2>
          <ul>
            <li><a href="#overview">Overview</a></li>
            <li><a href="#features">Features</a></li>
            <li><a href="#installation">Installation</a></li>
            <li><a href="#usage">Usage &amp; documentation</a></li>
            <li><a href="#package">Package &amp; source</a></li>
          </ul>
        </nav>

        <div class="docs-prose">

          <header class="section-head" id="overview">
            <p class="kicker"><span>Overview</span></p>
            <h2>Why it exists</h2>
          </header>
          <p>Vallama is a high-level Vala library for running LLM inference with llama.cpp, with native tool calling. It wraps the model, context, sampler and tokenizer objects of the C API into a small set of Vala classes — <code>Model</code>, <code>Context</code>, <code>Sampler</code>, <code>Generator</code>, <code>ChatSession</code>, <code>Embedder</code> — and takes care of the parts of llama.cpp that bite: chat templates, grammar ordering, streaming callbacks, and the context parameters that are fixed at creation.</p>
          <p>It is built the way the rest of the stack is built: the collections underneath are <a href="../invercargill/">Invercargill</a>, so transcripts and tool lists are <code>Enumerable</code> sequences rather than bare arrays, and the examples that ship in <code>examples/</code> are the runnable versions of every idiom on this page.</p>
          <p>Because the model file is a local GGUF and the inference is local compute, an application built on Vallama depends on no cloud service, sends nothing anywhere, and keeps working when the network does not — the same decentralised stance the rest of Astrologue takes.</p>

          <header class="section-head" id="features">
            <p class="kicker"><span>Features</span></p>
            <h2>What it hands you</h2>
          </header>
          <ul class="docs-features">
            <li><strong>Native tool calling.</strong> OpenAI-style tool calling driven by grammar-constrained decoding: when tools are registered, the assistant turn is generated under a GBNF grammar compiled from the tool schemas, so the model can only emit a valid tool-call envelope.</li>
            <li><strong>Chat sessions.</strong> <code>ChatSession</code> wraps Model + Context + Sampler, keeps the transcript, formats prompts, runs constrained turns, and offers <code>run_with_tools</code> to drive the whole loop via a callback.</li>
            <li><strong>Streaming generation.</strong> <code>generate_stream</code> and <code>respond_stream</code> hand each decoded piece to a callback, so interfaces can print as the model thinks.</li>
            <li><strong>KV-cache priming.</strong> <code>prime()</code> pre-evaluates a long system prompt once; <code>save()</code>/<code>load()</code> persist the resulting KV state to a file so later runs skip the expensive evaluation and start generating immediately.</li>
            <li><strong>Embeddings.</strong> A high-level text-embedding API on llama.cpp's sequence pooling: CLS, MEAN and LAST_TOKEN pooling, L2-normalised vectors, and batched <code>embed_all</code> over independent sequences.</li>
            <li><strong>Tokenizers.</strong> Exposed directly for piece-level work, with the grammar sampler attachable to any chain.</li>
            <li><strong>Chat templates handled.</strong> The model's template when it expands, an exact ChatML fallback otherwise, and a hand-rolled ChatML renderer whenever tool turns are in the transcript.</li>
          </ul>

          <header class="section-head" id="installation">
            <p class="kicker"><span>Installation</span></p>
            <h2>Getting the library</h2>
          </header>
          <p>With USM on your machine, one command acquires, builds and installs the library with its headers, Vala bindings and introspection:</p>
          <figure class="code-plate">
            <pre><code>usm install vallama</code></pre>
            <figcaption>USM resolves <code>invercargill-1</code> and llama.cpp itself.</figcaption>
          </figure>
          <p>From source, the stack's one-build discipline:</p>
          <figure class="code-plate">
            <pre><code>git clone &lt;the Vallama repository&gt;
cd Vallama
meson setup build
ninja -C build
ninja -C build install</code></pre>
            <figcaption>Needs llama.cpp (with <code>llama.pc</code>), GLib and <code>invercargill-1</code> — the version this page documents is verified against llama.cpp b6153.</figcaption>
          </figure>
          <p>Then compile against it with Vala's usual package mechanism:</p>
          <figure class="code-plate">
            <pre><code>valac --pkg vallama-0.1 my-program.vala</code></pre>
            <figcaption>The binding pulls in its <code>.deps</code> chain — GLib, GObject and Invercargill — automatically.</figcaption>
          </figure>

          <header class="section-head" id="usage">
            <p class="kicker"><span>Usage &amp; documentation</span></p>
            <h2>From the README and examples</h2>
          </header>
          <p>Generation streams piece by piece, with model, context and sampler split the way llama.cpp splits them:</p>
          <figure class="code-plate">
            <pre><code>Vallama.init();

var model = new Model (model_path, ModelOptions.defaults ());
var ctx_opts = ContextOptions.defaults ();
ctx_opts.n_ctx = 2048;
ctx_opts.n_threads = 8;
var ctx = new Context (model, ctx_opts);
var sampler = new Sampler (SamplerOptions.defaults ());
var generator = new Generator (ctx, sampler);

generator.generate_stream (prompt, max_tokens, (piece, token, is_final) =&gt; {
    print ("%s", piece);
    return true;      // false stops generation
});</code></pre>
            <figcaption>Condensed from <code>examples/simple.vala</code> — load a GGUF and generate.</figcaption>
          </figure>
          <h3>Tool calling, the headline feature</h3>
          <p>Tools are declared with a small JSON-schema-ish API, compiled into a grammar that only accepts a valid call envelope (or a <code>respond</code> envelope carrying plain language):</p>
          <figure class="code-plate">
            <pre><code>var city = new ToolParam ("city", ToolParamType.STRING, true, "City name");
var tool = new ToolSpec ("get_weather", "Get current weather", params);</code></pre>
            <figcaption>From the README. The envelope: <code>{"name": "get_weather", "arguments": {"city": "Paris", "unit": "celsius"}}</code>.</figcaption>
          </figure>
          <p><code>ChatSession</code> keeps the transcript and runs the constrained loop:</p>
          <figure class="code-plate">
            <pre><code>var session = new ChatSession (model, ctx, sampler);
session.add_tools (tools);
session.add_message (new ChatMessage (ChatRole.SYSTEM, system_prompt));

if (session.chat ("Weather in Paris?") == ChatTurnResult.TOOL_CALL) {
    var call = session.last_tool_call;
    string result = execute (call);           // your code
    session.append_tool_result (call, result);
    session.chat (null);                      // final prose turn
    print ("%s\n", session.last_message_text ());
}</code></pre>
            <figcaption>Verbatim from the README. <code>run_with_tools (message, executor)</code> drives the whole loop via a callback.</figcaption>
          </figure>
          <p>The whole assistant turn stays inside one JSON envelope on purpose: GBNF cannot express "any text OR this JSON", so the <code>respond</code> pseudo-tool gives the model a clean escape hatch while every turn stays parseable. The parser accepts both the bare envelope and Qwen2.5-style <code>&lt;tool_call&gt;</code> markup.</p>
          <h3>Caching the system prompt</h3>
          <p>Pre-evaluate a long system prompt once and reuse the KV state — across turns in one run, or from a file in the next:</p>
          <figure class="code-plate">
            <pre><code>generator.prime (SYSTEM_PROMPT);        // evaluate once, keep KV state
generator.save ("/path/to/cache.bin");  // persist it

// a later run:
generator.load ("/path/to/cache.bin");  // skip prompt evaluation,
generator.respond ("Why is the sky blue?");</code></pre>
            <figcaption><code>examples/cache.vala</code> primes, saves, restores and measures the speed-up end to end.</figcaption>
          </figure>
          <h3>Embeddings</h3>
          <p>One text or many, L2-normalised, with the pooling mode the model wants:</p>
          <figure class="code-plate">
            <pre><code>var model = new Vallama.Model ("/path/Qwen3-Embedding-0.6B-Q8_0.gguf");
var embedder = new Vallama.Embedder (model, Vallama.Pooling.LAST_TOKEN, ctx_opts);

int dimensions = embedder.dimensions;      // model n_embd, e.g. 1024
float[] query = embedder.embed ("Query: a city at the bottom of New Zealand");

var texts = new Invercargill.DataStructures.Vector&lt;string&gt; ();
texts.add ("Invercargill is the southernmost city in New Zealand.");
float[] docs = embedder.embed_all (texts);          // flat row-major
float[] doc0 = embedder.vector_at (docs, 0);        // slice row 0</code></pre>
            <figcaption>From the README. Cosine similarity of two vectors is their dot product — every vector comes back normalised.</figcaption>
          </figure>
          <p>The test suite runs offline — <code>meson test</code> needs no model file: it pins the pooling mapping, normalisation maths, ChatML round-trips, envelope parsing (including malformed input) and template fallback selection, and validates the GBNF compiler's output through llama.cpp's own grammar parser. Live-model runs gate on <code>VALLAMA_TEST_MODEL=/path/to/model.gguf</code>; without the variable the examples print their API surface and exit cleanly.</p>

          <header class="section-head" id="package">
            <p class="kicker"><span>Package &amp; source</span></p>
            <h2>The ledger</h2>
          </header>
          <dl class="docs-package">
            <div><dt>Version</dt><dd><code>0.1.0</code> (from <code>MANIFEST.usm</code>)</dd></div>
            <div><dt>Licence</dt><dd>LGPL-3.0-or-later — free to run, study, share and improve; the LGPL keeps it free to link from any program.</dd></div>
            <div><dt>Package</dt><dd><code>usm install vallama</code></dd></div>
            <div><dt>Provides</dt><dd><code>libvallama-0.1.so</code>, <code>vallama-0.1.h</code> and <code>vallama-llama-shim.h</code>, <code>vallama-0.1.vapi</code> + <code>.deps</code> plus the <code>llama.vapi</code> binding in <code>share/vala/vapi</code>, <code>Vallama-0.1.gir</code>, <code>vallama-0.1.pc</code></dd></div>
            <div><dt>Needs at runtime</dt><dd>GLib, GObject, <code>libinvercargill-1.so</code>, <code>libjson-glib-1.0.so</code>, and llama.cpp's <code>libllama.so</code>. A GGUF model file of your choosing.</dd></div>
            <div><dt>Source</dt><dd>The repository <code>clanker/Vallama</code> on the Astrologue forge (git over ssh, port 222). Free software: the licence guarantees your right to the source.</dd></div>
          </dl>

          <nav class="docs-siblings" aria-label="Related in the catalogue">
            <p>Its neighbours:</p>
            <ul>
              <li><a href="../invercargill/">Invercargill</a></li>
              <li><a href="../locksmith/">Locksmith</a></li>
              <li><a href="../usm/">USM</a></li>
            </ul>
          </nav>

          <nav class="docs-pager" aria-label="Project pages">
            <a class="docs-pager__link docs-pager__link--prev" href="../locksmith/">
              <span>Previous</span>
              <strong>Locksmith</strong>
            </a>
            <p class="docs-pager__up"><a href="../">All eleven</a></p>
            <a class="docs-pager__link docs-pager__link--next" href="../usm/">
              <span>Next in the catalogue</span>
              <strong>USM</strong>
            </a>
          </nav>

        </div>
      </div>
    </section>

  </main>

  <footer class="site-footer">
    <div class="container">
      <div class="footer-grid">
        <div class="foot-brand">
          <img class="brand__logo brand__logo--dark" src="../../assets/img/astrologue-dark.png" width="392" height="89" alt="Astrologue">
          <p>An operating system for change — kept for self sufficiency, resilience, and software freedom, so regular people can hold the power of general purpose computers in their own hands.</p>
        </div>
        <nav aria-label="The four freedoms">
          <p class="foot-title">The Four Freedoms</p>
          <ol class="freedoms">
            <li><strong>Run</strong> the software as you wish, for any purpose.</li>
            <li><strong>Study</strong> the source and change it to do your computing as you wish.</li>
            <li><strong>Share</strong> copies so you can help others.</li>
            <li><strong>Modify</strong> and publish your versions so everyone benefits.</li>
          </ol>
        </nav>
        <nav aria-label="Site">
          <p class="foot-title">Site</p>
          <ul class="foot-list">
            <li><a href="../../">Home</a></li>
            <li><a href="../../#news">News</a></li>
            <li><a href="../../#download">Download</a></li>
            <li><a href="../">Developer</a></li>
          </ul>
        </nav>
      </div>
      <p class="colophon">Set in Cinzel and EB Garamond (SIL Open Font License). Astrologue is free software, licensed GPL-3.0-or-later: take it, tend it, pass it on. The libraries are LGPL-3.0-or-later, so they remain free to link.</p>
    </div>
  </footer>
</body>
</html>