Implexus/Architecture/01-Overview.md

Overview

What is Implexus?

Implexus is a path-based document database library and engine for Vala. It provides a hierarchical data storage system where documents are identified by application-assigned paths, similar to filesystem paths with folders.

Design Goals

1. Path-Based Identification

Documents are identified by paths like /users/john/profile rather than opaque IDs. This makes the database intuitive to use and enables hierarchical organization.

2. Flexible Entity System

Four entity types support different access patterns:

• CONTAINER: Container for child entities (like folders)
• DOCUMENT: Typed objects with properties
• CATEGORY: Auto-generated categories based on expression evaluation
• INDEX: Text search with dynamic container results

3. Storage Abstraction

DBM-style key-value storage interface allows different backends while maintaining a consistent API.

4. Binary Serialization

Element types are serialized to/from binary format only - no GLib.Object specific logic. This keeps serialization simple and predictable.

5. Unified API

Applications use the same Engine interface whether running in embedded mode or connecting to a remote server.

6. Invercargill Integration

Built on Invercargill types:

• Element interface for type-safe values
• Properties interface for document properties
• Enumerable<T> and Lot<T> for collection returns
• Set<T> and ReadOnlySet<T> for set operations
• Expressions for category and index queries

Operating Modes

Embedded Mode

The database engine runs as a library within the application process. Direct method calls, no network overhead.

var engine = new EmbeddedEngine(storage);
var container = engine.get_root().create_container("users");

Client/Server Mode

A daemon process manages the database, clients connect via TCP. Same API through RemoteEngine.

var engine = new RemoteEngine.connect("localhost", 9090);
var container = engine.get_root().create_container("users");

Key Design Principles

No Libgee or GLib Collections

Uses Invercargill data structures exclusively:

• Invercargill.DataStructures.Vector<T> instead of GLib.List<T>
• Invercargill.DataStructures.Dictionary<K,V> instead of GLib.HashTable<K,V>
• Invercargill.DataStructures.HashSet<T> for sets
• Invercargill.DataStructures.Category<K,V> for categorized storage

Return Types

• Use Enumerable<T> for lazy evaluation of potentially large result sets
• Use Lot<T> for materialized collections
• Use ReadOnlySet<T> for set-based operations

Expression-Based Queries

Categorys and Indexes use Invercargill.Expressions for:

• Property access on documents
• Filtering and transformation
• Text matching for indexes

High-Level Architecture

┌────────────────────────────────────────────────────────────────┐
│                        Application Layer                        │
└────────────────────────────────────────────────────────────────┘
                                │
                                ▼
┌────────────────────────────────────────────────────────────────┐
│                        Engine Interface                         │
│                                                                 │
│   Provides unified API for embedded and remote operation        │
└────────────────────────────────────────────────────────────────┘
                                │
                ┌───────────────┴───────────────┐
                ▼                               ▼
┌───────────────────────────┐   ┌───────────────────────────────┐
│      EmbeddedEngine       │   │        RemoteEngine           │
│                           │   │                               │
│  Direct in-process calls  │   │  TCP communication with       │
│                           │   │  implexusd daemon             │
└───────────────────────────┘   └───────────────────────────────┘
                │                               │
                └───────────────┬───────────────┘
                                ▼
┌────────────────────────────────────────────────────────────────┐
│                        Entity System                            │
│                                                                 │
│   Entity interface with four implementations:                   │
│   Container, Document, Category, Index                          │
└────────────────────────────────────────────────────────────────┘
                                │
                                ▼
┌────────────────────────────────────────────────────────────────┐
│                        Storage Layer                            │
│                                                                 │
│   Storage interface for entity persistence                      │
│   DBM interface for key-value storage                           │
│   Binary serialization for Element types                        │
└────────────────────────────────────────────────────────────────┘

Relationship to Reference Projects

Implexus follows patterns from:

Project	Patterns Used
Spry/astralis	Project structure, meson build
Invercargill	Element, Properties, Enumerable, Lot, Set, Expressions
Invercargill-Json	Element-to-binary serialization pattern
Inversion	Dependency injection for engine configuration