Implexus/Architecture/04-Class-Hierarchy.md

Class Hierarchy

This document describes the class hierarchy and relationships between types in Implexus.

Async I/O Model

All I/O operations in Implexus are asynchronous. The class hierarchy reflects this:

• Engine implementations provide *_async methods for all I/O operations
• Entity implementations provide *_async methods for navigation, CRUD, and property access
• Identity properties (path, name, entity_type, engine) remain synchronous
• No wrapper classes - AsyncEngine and AsyncEntity have been removed; async is built into the base interfaces

Inheritance Diagram

graph TB
    subgraph Interfaces
        EntityI[Entity Interface]
        EngineI[Engine Interface]
        StorageI[Storage Interface]
        DbmI[Dbm Interface]
        TransactionI[Transaction Interface]
    end
    
    subgraph Abstract Classes
        AbstractEntity[AbstractEntity]
    end
    
    subgraph Entity Implementations
        Container[Container]
        Document[Document]
        Category[Category]
        Index[Index]
        Catalogue[Catalogue]
        IndexResult[IndexResult]
    end
    
    subgraph Engine Implementations
        EmbeddedEngine[EmbeddedEngine]
        RemoteEngine[RemoteEngine]
    end
    
    subgraph Storage Implementations
        DefaultStorage[DefaultStorage]
        AsyncDbmQueue[AsyncDbmQueue]
        FilesystemDbm[FilesystemDbm]
        GdbmDbm[GdbmDbm]
        LmdbDbm[LmdbDbm]
        EmbeddedTransaction[EmbeddedTransaction]
    end
    
    EntityI --> AbstractEntity
    AbstractEntity --> Container
    AbstractEntity --> Document
    AbstractEntity --> Category
    AbstractEntity --> Index
    AbstractEntity --> Catalogue
    Index --> IndexResult
    
    EngineI --> EmbeddedEngine
    EngineI --> RemoteEngine
    
    StorageI --> DefaultStorage
    DbmI --> FilesystemDbm
    DbmI --> GdbmDbm
    DbmI --> LmdbDbm
    AsyncDbmQueue --> DbmI
    TransactionI --> EmbeddedTransaction

AbstractEntity Class

Base implementation providing common functionality for all entity types. All I/O operations are async.

namespace Implexus.Entities {

public abstract class AbstractEntity : Object, Entity {
    
    // Protected fields
    protected weak Engine _engine;
    protected EntityPath _path;
    
    // === Identity (Synchronous - No I/O) ===
    public unowned Engine engine { get { return _engine; } }
    public EntityPath path { owned get { return _path; } }
    public string name { owned get { return _path.name; } }
    
    public abstract EntityType entity_type { get; }
    public abstract string type_label { owned get; }
    public abstract string configured_expression { owned get; }
    public abstract string configured_type_label { owned get; }
    
    // === Parent/Child Navigation (Async) ===
    
    public async Entity? get_parent_async() throws EntityError {
        if (_path.is_root) return null;
        return yield _engine.get_entity_or_null_async(_path.parent);
    }
    
    public virtual async Invercargill.ReadOnlySet<string> get_child_names_async() throws EntityError { 
        return engine.configuration.storage.get_child_names(_path);
    }
    
    public virtual async Entity? get_child_async(string name) throws EntityError {
        var child_path = _path.child(name);
        return yield _engine.get_entity_or_null_async(child_path);
    }
    
    public virtual async Entity[] get_children_async() throws EntityError {
        var names = yield get_child_names_async();
        var children = new Entity[0];
        foreach (var name in names) {
            var child = yield get_child_async(name);
            if (child != null) {
                children += (!) child;
            }
        }
        return children;
    }
    
    // === Creation methods - override in Container (Async) ===
    
    public virtual async Entity? create_container_async(string name) throws EntityError {
        throw new EntityError.INVALID_OPERATION(
            "Cannot create container on %s", entity_type.to_string()
        );
    }
    
    public virtual async Entity? create_document_async(string name, string type_label) throws EntityError {
        throw new EntityError.INVALID_OPERATION(
            "Cannot create document on %s", entity_type.to_string()
        );
    }
    
    public virtual async Entity? create_category_async(string name, string type_label, string expression) throws EntityError {
        throw new EntityError.INVALID_OPERATION(
            "Cannot create category on %s", entity_type.to_string()
        );
    }
    
    public virtual async Entity? create_index_async(string name, string type_label, string expression) throws EntityError {
        throw new EntityError.INVALID_OPERATION(
            "Cannot create index on %s", entity_type.to_string()
        );
    }
    
    public virtual async Entity? create_catalogue_async(string name, string type_label, string expression) throws EntityError {
        throw new EntityError.INVALID_OPERATION(
            "Cannot create catalogue on %s", entity_type.to_string()
        );
    }
    
    // === Document operations - override in Document (Async) ===
    
    public virtual async Invercargill.Properties get_properties_async() throws EntityError { 
        throw new EntityError.INVALID_OPERATION("Not a document"); 
    }
    
    public virtual async Invercargill.Element? get_entity_property_async(string name) throws EntityError {
        throw new EntityError.INVALID_OPERATION("Not a document");
    }
    
    public virtual async void set_entity_property_async(string name, Invercargill.Element value) throws EntityError {
        throw new EntityError.INVALID_OPERATION("Not a document");
    }
    
    public virtual async void remove_property_async(string name) throws EntityError {
        throw new EntityError.INVALID_OPERATION("Not a document");
    }
    
    // === Lifecycle (Async) ===
    
    public virtual async void delete_async() throws EntityError {
        engine.configuration.storage.delete_entity(_path);
        var parent = yield get_parent_async();
        if (parent != null) {
            engine.configuration.storage.unregister_child(parent.path, name);
        }
        engine.entity_deleted(_path);
    }
    
    public virtual bool exists {
        get { return engine.configuration.storage.has_entity(_path); }
    }
    
    // === Set Operations (Async) ===
    
    public async EntitySet as_set_async() {
        return new EntitySet(this);
    }
    
    // Note: Invercargill.Element methods are inherited and implemented here
}

} // namespace Implexus.Entities

Engine Implementations

EmbeddedEngine

Direct in-process engine implementation with async I/O.

namespace Implexus.Engine {

public class EmbeddedEngine : Object, Engine {
    
    private Storage _storage;
    private Entity? _root;
    private AsyncDbmQueue _queue;
    private Transaction? _current_transaction;
    
    public EmbeddedEngine(Storage storage) {
        _storage = storage;
        _queue = new AsyncDbmQueue(storage.dbm);
        _queue.start();
    }
    
    // === Root Access (Async) ===
    
    public async Entity get_root_async() throws EngineError {
        if (_root != null) return (!) _root;
        _root = new Container(this, new EntityPath.root());
        return _root;
    }
    
    // === Path-Based Access (Async) ===
    
    public async Entity? get_entity_async(EntityPath path) throws EngineError {
        var entity = yield get_entity_or_null_async(path);
        if (entity == null) {
            throw new EngineError.ENTITY_NOT_FOUND("Entity not found: %s", path.to_string());
        }
        return entity;
    }
    
    public async Entity? get_entity_or_null_async(EntityPath path) throws EngineError {
        if (!_storage.has_entity(path)) return null;
        return yield load_entity_async(path);
    }
    
    public async bool entity_exists_async(EntityPath path) throws EngineError {
        return _storage.has_entity(path);
    }
    
    // === Query Operations (Async) ===
    
    public async Entity[] query_by_type_async(string type_label) throws EngineError {
        var paths = _storage.get_paths_by_type(type_label);
        var entities = new Entity[0];
        foreach (var path in paths) {
            var entity = yield get_entity_or_null_async(path);
            if (entity != null) {
                entities += (!) entity;
            }
        }
        return entities;
    }
    
    public async Entity[] query_by_expression_async(string type_label, string expression) throws EngineError {
        var evaluator = new Invercargill.Expressions.ExpressionEvaluator();
        var all = yield query_by_type_async(type_label);
        var results = new Entity[0];
        foreach (var entity in all) {
            try {
                var props = yield entity.get_properties_async();
                var result = evaluator.evaluate(expression, props);
                if (result != null && !(result as Invercargill.NullElement).is_null()) {
                    results += entity;
                }
            } catch {
                // Skip entities that fail evaluation
            }
        }
        return results;
    }
    
    // === Transactions (Async) ===
    
    public async Transaction begin_transaction_async() throws EngineError {
        if (_current_transaction != null) {
            throw new EngineError.TRANSACTION_ERROR("Transaction already active");
        }
        _storage.begin_transaction();
        _current_transaction = new EmbeddedTransaction(() => {
            _storage.commit_transaction();
            _current_transaction = null;
        }, () => {
            _storage.rollback_transaction();
            _current_transaction = null;
        });
        return _current_transaction;
    }
    
    public async void commit_async() throws EngineError {
        if (_current_transaction == null) {
            throw new EngineError.TRANSACTION_ERROR("No transaction active");
        }
        yield _current_transaction.commit_async();
    }
    
    public async void rollback_async() {
        if (_current_transaction != null) {
            yield _current_transaction.rollback_async();
        }
    }
    
    public bool in_transaction { get { return _current_transaction != null; } }
    
    public StorageConfiguration configuration { 
        owned get { return new StorageConfiguration(_storage); }
    }
    
    private async Entity load_entity_async(EntityPath path) throws EngineError {
        var data = _storage.load_entity(path);
        if (data == null) return null;
        
        var deserializer = new EntityDeserializer();
        return deserializer.deserialize((!) data, this, path);
    }
}

} // namespace Implexus.Engine

RemoteEngine

Client for connecting to implexusd daemon. All operations are async over the network.

namespace Implexus.Engine {

public class RemoteEngine : Object, Engine {
    
    private SocketConnection _connection;
    private MessageReader _reader;
    private MessageWriter _writer;
    
    public RemoteEngine.connect(string host, uint16 port) throws EngineError {
        try {
            var client = new SocketClient();
            _connection = client.connect_to_host(host, port, null);
            var stream = _connection.get_input_stream();
            var output = _connection.get_output_stream();
            _reader = new MessageReader(stream);
            _writer = new MessageWriter(output);
        } catch (Error e) {
            throw new EngineError.CONNECTION_ERROR("Failed to connect: %s", e.message);
        }
    }
    
    public async Entity get_root_async() throws EngineError {
        var entity = yield get_entity_or_null_async(new EntityPath.root());
        if (entity == null) {
            return new RemoteContainer(this, new EntityPath.root());
        }
        return entity;
    }
    
    public async Entity? get_entity_async(EntityPath path) throws EngineError {
        var response = yield send_request_async(new GetEntityRequest(path));
        if (response is EntityNotFoundResponse) {
            throw new EngineError.ENTITY_NOT_FOUND("Entity not found: %s", path.to_string());
        }
        return ((EntityResponse) response).entity;
    }
    
    public async Entity? get_entity_or_null_async(EntityPath path) throws EngineError {
        try {
            return yield get_entity_async(path);
        } catch {
            return null;
        }
    }
    
    public async bool entity_exists_async(EntityPath path) throws EngineError {
        var response = yield send_request_async(new EntityExistsRequest(path));
        return ((BooleanResponse) response).value;
    }
    
    // ... other async methods send protocol messages
    
    private async Response send_request_async(Request request) throws EngineError {
        _writer.write(request);
        var response = yield _reader.read_response_async();
        if (response is ErrorResponse) {
            throw new EngineError.PROTOCOL_ERROR(((ErrorResponse) response).message);
        }
        return response;
    }
}

} // namespace Implexus.Engine

Storage Implementations

DefaultStorage

namespace Implexus.Storage {

public class DefaultStorage : Object, Storage {
    
    private Dbm _dbm;
    private AsyncDbmQueue _queue;
    private Invercargill.DataStructures.Dictionary<string, Invercargill.DataStructures.HashSet<string>> _children;
    private Invercargill.DataStructures.Category<string, Invercargill.DataStructures.Vector<string>> _type_index;
    
    public DefaultStorage(Dbm dbm) {
        _dbm = dbm;
        _queue = new AsyncDbmQueue(dbm);
        _children = new Invercargill.DataStructures.Dictionary<string, Invercargill.DataStructures.HashSet<string>>();
        _type_index = new Invercargill.DataStructures.Category<string, Invercargill.DataStructures.Vector<string>>();
    }
    
    public bool has_entity(EntityPath path) {
        return _dbm.has_key(entity_key(path));
    }
    
    public uint8[]? load_entity(EntityPath path) {
        var data = _dbm.get(entity_key(path));
        return data?.to_bytes();
    }
    
    public void save_entity(EntityPath path, uint8[] data) throws StorageError {
        _dbm.set(entity_key(path), new Invercargill.BinaryData.from_bytes(data));
    }
    
    public void delete_entity(EntityPath path) throws StorageError {
        _dbm.delete(entity_key(path));
    }
    
    public Invercargill.ReadOnlySet<string> get_child_names(EntityPath parent_path) {
        var key = children_key(parent_path);
        if (!_children.has(key)) {
            return new Invercargill.DataStructures.HashSet<string>().as_read_only();
        }
        return _children.get(key).as_read_only();
    }
    
    public void register_child(EntityPath parent, string child_name) {
        var key = children_key(parent);
        if (!_children.has(key)) {
            _children.set(key, new Invercargill.DataStructures.HashSet<string>());
        }
        _children.get(key).add(child_name);
    }
    
    public void unregister_child(EntityPath parent, string child_name) {
        var key = children_key(parent);
        if (_children.has(key)) {
            _children.get(key).remove(child_name);
        }
    }
    
    public Invercargill.Enumerable<EntityPath> get_paths_by_type(string type_label) {
        if (!_type_index.has(type_label)) {
            return new Invercargill.DataStructures.Vector<EntityPath>().as_enumerable();
        }
        return _type_index.get(type_label)
                          .select(path_str => new EntityPath(path_str));
    }
    
    public void register_type(EntityPath path, string type_label) {
        if (!_type_index.has(type_label)) {
            _type_index.set(type_label, new Invercargill.DataStructures.Vector<string>());
        }
        _type_index.get(type_label).add(path.to_string());
    }
    
    public void unregister_type(EntityPath path, string type_label) {
        if (_type_index.has(type_label)) {
            _type_index.get(type_label).remove_all(path.to_string());
        }
    }
    
    private string entity_key(EntityPath path) { return @"entity:$(path.to_string())"; }
    private string children_key(EntityPath path) { return @"children:$(path.to_string())"; }
    
    // Transaction methods delegate to Dbm
    public void begin_transaction() throws StorageError { _dbm.begin_transaction(); }
    public void commit_transaction() throws StorageError { _dbm.commit_transaction(); }
    public void rollback_transaction() { _dbm.rollback_transaction(); }
    public bool in_transaction { get { return _dbm.in_transaction; } }
    
    public void open() throws StorageError { 
        _dbm.open(); 
        _queue.start();
        load_indices();
    }
    
    public void close() { 
        _queue.shutdown();
        _dbm.close(); 
    }
    
    public void compact() throws StorageError { 
        _dbm.compact(); 
    }
    
    private void load_indices() {
        // Rebuild child and type indices from entity data
        foreach (var entry in _dbm.keys) {
            if (entry.has_prefix("entity:")) {
                // Parse entity and update indices
            }
        }
    }
}

} // namespace Implexus.Storage

FilesystemDbm

Simple file-based DBM implementation (single-threaded).

namespace Implexus.Storage {

public class FilesystemDbm : Object, Dbm {
    
    private string _path;
    private Invercargill.DataStructures.Dictionary<string, Invercargill.BinaryData> _data;
    private bool _open;
    private int _transaction_depth;
    
    /**
     * FilesystemDbm does not support concurrent reads.
     * All operations go through the AsyncDbmQueue worker thread.
     */
    public bool supports_concurrent_reads { get { return false; } }
    
    public FilesystemDbm(string path) {
        _path = path;
        _data = new Invercargill.DataStructures.Dictionary<string, Invercargill.BinaryData>();
        _open = false;
        _transaction_depth = 0;
    }
    
    public bool has_key(string key) {
        ensure_open();
        return _data.has(key);
    }
    
    public Invercargill.BinaryData? @get(string key) {
        ensure_open();
        if (!_data.has(key)) return null;
        return _data.get(key);
    }
    
    public void @set(string key, Invercargill.BinaryData value) throws StorageError {
        ensure_open();
        _data.set(key, value);
    }
    
    public void delete(string key) throws StorageError {
        ensure_open();
        _data.remove(key);
    }
    
    public Invercargill.Enumerable<string> keys {
        owned get {
            ensure_open();
            return _data.keys;
        }
    }
    
    public void begin_transaction() throws StorageError {
        ensure_open();
        _transaction_depth++;
    }
    
    public void commit_transaction() throws StorageError {
        ensure_open();
        if (_transaction_depth > 0) {
            _transaction_depth--;
            if (_transaction_depth == 0) {
                sync();
            }
        }
    }
    
    public void rollback_transaction() {
        if (_transaction_depth > 0) {
            _transaction_depth--;
            // Reload from disk
            load_from_disk();
        }
    }
    
    public bool in_transaction { get { return _transaction_depth > 0; } }
    
    public void open() throws StorageError {
        if (_open) return;
        
        var file = File.new_for_path(_path);
        if (file.query_exists()) {
            load_from_disk();
        } else {
            file.get_parent().make_directory_with_parents();
        }
        _open = true;
    }
    
    public void close() {
        if (!_open) return;
        sync();
        _open = false;
    }
    
    public void sync() throws StorageError {
        // Write all data to disk
        var file = File.new_for_path(_path);
        try {
            var stream = file.replace(null, false, FileCreateFlags.NONE);
            var writer = new DataOutputStream(stream);
            
            // Write header
            writer.write_string("IMPXDBM1\n");
            
            // Write entry count
            writer.put_int64(_data.count);
            
            // Write entries
            foreach (var entry in _data.entries) {
                writer.put_int64(entry.key.length);
                writer.write_string(entry.key);
                writer.put_int64(entry.value.length);
                writer.write(entry.data);
            }
        } catch (Error e) {
            throw new StorageError.WRITE_ERROR("Failed to sync: %s", e.message);
        }
    }
    
    public void compact() throws StorageError {
        sync();
    }
    
    private void ensure_open() {
        if (!_open) {
            critical("DBM not open");
        }
    }
    
    private void load_from_disk() throws StorageError {
        // Read all data from disk
        var file = File.new_for_path(_path);
        try {
            var stream = new DataInputStream(file.read());
            
            // Read header
            var header = stream.read_line();
            if (header != "IMPXDBM1") {
                throw new StorageError.CORRUPTION_ERROR("Invalid DBM file format");
            }
            
            // Read entries
            int64 count = stream.read_int64();
            for (int64 i = 0; i < count; i++) {
                int64 key_len = stream.read_int64();
                uint8[] key_bytes = new uint8[key_len];
                stream.read(key_bytes);
                string key = (string) key_bytes;
                
                int64 value_len = stream.read_int64();
                uint8[] value = new uint8[value_len];
                stream.read(value);
                
                _data.set(key, new Invercargill.BinaryData.from_bytes(value));
            }
        } catch (Error e) {
            throw new StorageError.READ_ERROR("Failed to load: %s", e.message);
        }
    }
}

} // namespace Implexus.Storage

GdbmDbm

GDBM-based implementation (single-threaded).

namespace Implexus.Storage {

public class GdbmDbm : Object, Dbm {
    
    private string _path;
    private void* _dbf;  // GDBM_FILE handle
    
    /**
     * GDBM does not support concurrent reads.
     * All operations go through the AsyncDbmQueue worker thread.
     */
    public bool supports_concurrent_reads { get { return false; } }
    
    // ... GDBM-specific implementation
}

} // namespace Implexus.Storage

LmdbDbm

LMDB-based implementation with concurrent read support.

namespace Implexus.Storage {

public class LmdbDbm : Object, Dbm {
    
    private string _path;
    private void* _env;  // MDB_env*
    private void* _txn;  // Current transaction MDB_txn*
    
    /**
     * LMDB supports concurrent reads via MVCC.
     * Read operations can spawn their own threads,
     * while writes go through the AsyncDbmQueue.
     */
    public bool supports_concurrent_reads { get { return true; } }
    
    // ... LMDB-specific implementation
}

} // namespace Implexus.Storage

Class Summary

Class	Extends	Implements	Purpose
AbstractEntity	Object	Entity	Base entity implementation with async methods
Container	AbstractEntity	-	Container for child entities
Document	AbstractEntity	-	Properties-based document
Category	AbstractEntity	-	Expression-based auto-categories
Index	AbstractEntity	-	Text search with dynamic results
Catalogue	AbstractEntity	-	Key-based document grouping
IndexResult	AbstractEntity	-	Container returned by index query
EmbeddedEngine	Object	Engine	In-process engine with async I/O
RemoteEngine	Object	Engine	Client for daemon with async network I/O
DefaultStorage	Object	Storage	Entity persistence
AsyncDbmQueue	Object	-	Queue for async DBM operations
FilesystemDbm	Object	Dbm	File-based key-value store (single-threaded)
GdbmDbm	Object	Dbm	GDBM-based store (single-threaded)
LmdbDbm	Object	Dbm	LMDB-based store (concurrent reads)
EmbeddedTransaction	Object	Transaction	Transaction implementation

Removed Classes

The following classes have been removed as part of the async refactor:

Class	Reason for Removal
AsyncEngine	Async is now built into the base Engine interface
AsyncEntity	Async is now built into the base Entity interface
with_write_transaction() helper	Vala doesn't support async delegates; use manual begin/commit/rollback

Async Method Summary

Engine Interface

Method	Description
get_root_async()	Get root entity
get_entity_async(path)	Get entity by path
get_entity_or_null_async(path)	Get entity or null
entity_exists_async(path)	Check entity existence
query_by_type_async(type_label)	Query by type
query_by_expression_async(type_label, expr)	Query with filter
begin_transaction_async()	Start transaction
commit_async()	Commit transaction
rollback_async()	Rollback transaction

Entity Interface

Method	Description
get_parent_async()	Get parent entity
get_child_names_async()	Get child names
get_child_async(name)	Get child by name
get_children_async()	Get all children
create_container_async(name)	Create container child
create_document_async(name, type)	Create document child
create_category_async(name, type, expr)	Create category child
create_index_async(name, type, expr)	Create index child
create_catalogue_async(name, type, expr)	Create catalogue child
get_properties_async()	Get document properties
get_entity_property_async(name)	Get property value
set_entity_property_async(name, value)	Set property value
remove_property_async(name)	Remove property
delete_async()	Delete entity
as_set_async()	Create EntitySet