#Serving & Operations

This document covers how to run kbpy — the DatomStore, registry reconstruction, CLI tools, web interface, scoped registries, and the domain graph. For domain modeling concepts, see Domain Modeling. For the .kb syntax, see DSL Reference. For foundational theory, see Core Concepts.

#Architecture: How Data Flows from Git to Browser

Git repo (.kb files + Python data loaders)
    → kbpy history → SQLite DatomStore
    → kbpy serve --db → Registry reconstruction
    → Scoped registries → Web UI

The pipeline has two phases:

Extraction (kbpy history): Walks through git commits in a domain repository, extracts attestation records and .kb definitions, and writes them as datoms into a SQLite database. This is a batch process that runs once (incrementally).
Serving (kbpy serve --db): Reads the SQLite database, parses .kb files, and reconstructs the full in-memory domain model. The web interface then uses scoped registries to evaluate expectations and display results.

This separation means the serving layer never imports Python modules or executes data loading code — it works entirely from the pre-built SQLite database and declarative .kb definitions.

#DatomStore and Versioning

kbpy stores all attestation data and domain definitions in a SQLite database using a datom model inspired by Datomic. A datom is a 5-tuple (e, a, v, tx, tx_order) representing an entity-attribute-value triple with version information.

#Why Datoms Instead of Tables

Traditional relational databases require schema migrations when entity types change. The datom model avoids this by storing everything as entity-attribute-value tuples:

No schema migrations when entity types gain or lose attributes
Naturally supports heterogeneous attributes across entity types
Version history is built in — every value is tagged with its transaction
Simple to query: filter by entity, attribute, and version

#Schema

CREATE TABLE datoms (
    e TEXT NOT NULL,         -- entity identifier
    a TEXT NOT NULL,         -- attribute name
    v TEXT,                  -- value (JSON serialized)
    tx TEXT NOT NULL,        -- transaction ID (git commit hash)
    tx_order INTEGER NOT NULL  -- sequential ordering
);

#Entity ID Format

Entity IDs are domain-qualified strings:

Type	Format	Example
Attestation entity	`domain/EntityType:id`	`example_3/Namespace:ns-prod-01`
Definition entity	`domain/Def:type:name`	`example_3/Def:entity_type:ITService`

#Version Model

Each git commit in a domain repository produces a transaction (tx) with a sequential tx_order. To query the state at a specific version, filter by tx_order <= N and take the latest value per entity-attribute pair.

All domains share the same tx_order space, enabling cross-domain queries at a consistent point in time.

#Domain Versioning and Caching

#Version Source

The current version is determined from:

HEROKU_SLUG_COMMIT environment variable (for Heroku deployments)
git rev-parse HEAD (for local development)

Version is automatically included in the cache key, so multiple versions can coexist in the cache.

#LRU Caching

The domain loading system uses LRU (Least Recently Used) caching:

Cache	Size	Key	Purpose
`_load_full_domain`	`maxsize=10`	`(domain_name, version)`	Full domain registries
`get_scoped_registries`	`maxsize=50`	`(domain, campaign)`	Filtered registries per scope
`get_domain_campaigns`	`maxsize=20`	`domain`	Campaign listings

When a domain version is evicted from cache:

All registries for that version are discarded together
The domain can be re-loaded on next access
No stale state remains on entity classes

Use invalidate_domain_cache() to clear the cache for hot reload during development.

#History Extraction

The kbpy history CLI command walks through git commits in a domain repository and extracts all attestation data and definitions into a SQLite database:

uv run kbpy history <repo-path> [-o data.sqlite] [-v] [-b branch]

The tool discovers all domains in src/, extracts attestation records and .kb definitions for every commit, and writes them as datoms. Extraction is incremental — already-processed commits are skipped.

#SQLite-Only Serving

kbpy serves entirely from a SQLite database. This is the primary serving mode.

#Workflow

# 1. Extract history into SQLite
uv run kbpy history <domain-repo-path> -o data.sqlite -v

# 2. Serve from SQLite
uv run kbpy serve --db data.sqlite --port 8000

#How It Works

When --db is provided:

On startup, domains are discovered from entity ID prefixes in the database
For each domain, .kb files are located in src/ (e.g., src/example_3/domain.kb)
.kb files are parsed and merged (multiple files per domain are supported)
reconstruct_registries() rebuilds the full in-memory domain model from SQLite data and DSL definitions
Registries are registered via register_preloaded_registries() so the web layer finds them through the normal path

#Registry Reconstruction

The reconstruct_registries() function in src/kbpy/reconstruct.py rebuilds domain registries in 11 phases:

Register entity types from DSL definitions
Build module context map from relationship of-chains
Register attributes and relationships on entity types
Create Entity subclasses with FluentFact descriptors
Create entity sets from SQLite attestation data
Create partitions with compiled DSL predicates
Apply steward relationships
Register expectations
Eagerly materialize all entity sets (while loading context is active)

Phase 9 is critical: partition branches must be materialized before the loading context is cleared, because FluentFact descriptors need the loading context to look up attestation data. If you see "called outside loading context" errors, it means an entity set is being accessed after the loading phase completed.

#Lazy Loading

kbpy uses lazy loading to ensure fast startup and efficient memory usage. Data is only loaded when actually needed.

#How It Works

At startup: .kb files are parsed and domain definitions are registered. No data is loaded from SQLite.
On first access: Data is loaded from SQLite only when you actually iterate over or query an entity set.

#What Triggers Loading

Data loads on the first:

Iteration: iterating over an entity set (e.g. in a partition predicate)
Length check: len(entity_set)
Membership test: entity in entity_set

#When Lazy Loading Helps and When It Hurts

Helps: Fast startup even with large datasets. Only the entity sets needed for the current campaign evaluation get loaded.

Hurts: The first access to any entity set incurs a loading delay. This is why reconstruct_registries() eagerly materializes all partition branches in Phase 9 — to ensure all loading happens during the reconstruction phase, not during request handling.

Gotcha: After the loading context is cleared, accessing entity data that hasn't been materialized will fail. The reconstruction pipeline handles this by eagerly walking all entity sets, but if you're extending the system, be aware that entity set access must happen within a loading context.

#Multiple Data Sources

When an entity set has multiple data sources (via from: "hr_data", "payroll_data"), they all load together on first access. Records are merged by match field (upsert semantics).

Benefit	Description
Fast startup	Parsing .kb files is instant, even with large datasets
Memory efficient	Only load data you actually use
Order independent	Define entities in any order; references resolve lazily
Incremental loading	Each entity set loads independently when accessed

#CLI Exploration

Use the kbpy CLI to explore domains:

# Extract history from a domain repo into SQLite
uv run kbpy history <repo-path> -o data.sqlite -v

# Serve from SQLite (primary mode)
uv run kbpy serve --db data.sqlite --port 8000

# With auto-reload for development
uv run kbpy serve --db data.sqlite --port 8000 --reload

# Interactive REPL (legacy, requires Python module)
uv run kbpy <module> -i

# Commands in REPL:
#   list              - Show all entity sets
#   domain <set>      - Show entity type domain model
#   stats             - Show summary statistics
#   find <set> age>30 - Filter entities
#   graph <set>       - Show relationships
#   reason <set>      - Show why an entity set exists
#   reason <set> <entity> - Show entity's reason chain
#   why <set>.<entity>.<field> - Explain Unknown/Void

#Web Interface

kbpy includes a web interface for exploring domains, expectations, and entity data.

#URL Structure

The web interface uses a hierarchical URL structure:

URL Pattern	Description
`/`	List of available domains
`/d/<domain>/`	Domain landing page with campaigns
`/d/<domain>/x/<campaign>/`	Campaign dashboard
`/d/<domain>/expectations/`	All expectations in the domain
`/d/<domain>/expectations/<name>/`	Single expectation detail
`/d/<domain>/entities/`	Browse all entity types
`/d/<domain>/entities/<type>/`	Browse entities of a type
`/d/<domain>/partitions/`	Browse all partitions
`/d/<domain>/x/<campaign>/diagnostics`	Performance metrics and cache stats

The sidebar is organised around personas — the different roles that interact with the system:

Section	Persona	What they see
Alignment	All users	Dashboard, expectations, questions, actions
Actors	Stakeholders	Expectors, expectees, attestors
Domain	Domain modelers	Entity types, entity sets, partitions, relationships, attestation sources, domain graph
System	Operators	Diagnostics (performance metrics, cache stats, memory usage)

This structure means each person finds their relevant pages without navigating through irrelevant data. Expectors see which expectations they hold; expectees see which entities they're responsible for; attestors see which questions are directed at them.

#Campaign Pages

The campaign page (/d/<domain>/x/<campaign>/) shows:

Alignment summary: Total expectations, met, not met, uncertain
Progress bar: Visual breakdown of alignment status
Questions count: Missing data needing investigation
Actions count: Steps to resolve misalignments
Expectations list: Each expectation with status, questions, and actions

#Diagnostics Page

The diagnostics page (/d/<domain>/x/<campaign>/diagnostics) provides performance visibility without requiring external tooling. It is accessible from the "System" section in the sidebar.

Every metric on this page is purely observational — viewing diagnostics does not trigger entity materialisation or predicate evaluation, so it is safe to use on large domains.

The page displays five categories of information:

Domain load time measures how long it took to parse .kb files and reconstruct the full in-memory domain model. This is recorded once when _load_full_domain runs and reflects a cache miss. Subsequent loads hit the LRU cache and return instantly. A high load time (>1s) indicates that the domain has many entity sets or large CSV attestation sources.

Process memory shows the resident set size of the server process. This is the total memory used by the Python interpreter, all loaded domains, and any cached registries. On domains with 10k+ entities, this is the primary indicator of whether pagination and streaming counters are needed.

LRU cache stats shows hit/miss rates and current occupancy for the four LRU-cached functions: _load_full_domain, get_scoped_registries, get_domain_campaigns, and get_domain_expectation_forest. A low hit rate on get_scoped_registries suggests that the maxsize may be too small for the number of campaigns being actively viewed.

Entity set sizes lists every entity set in the current scope with its entity count and whether it is partial or exhaustive. This helps identify which sets are large enough to benefit from pagination or sampling.

Expectation stats lists expectations with their evaluation status. Evaluation time tracking is available when instrumented (future work).

In addition to the diagnostics page, two HTTP response headers provide per-request timing:

X-Response-Time-Ms is added by the TimingMiddleware to every response. It measures the total wall-clock time for the request, including template rendering. Requests exceeding 500ms are logged as warnings.

X-Render-Time-Ms is added by the _render function to every HTML response. It measures only the Jinja2 template rendering step. A large gap between X-Response-Time-Ms and X-Render-Time-Ms indicates that most time is spent in view logic (data loading, predicate evaluation) rather than template rendering.

#Running the Web Server

# Serve from SQLite (primary mode)
uv run kbpy serve --db data.sqlite --port 8000

# With auto-reload for development
uv run kbpy serve --db data.sqlite --port 8000 --reload

The server requires authentication. Default credentials:

Username: IW
Password: spikeDemo

#Scoped Registries: Why They Exist

Full domain registries contain everything in a domain — all entity types, all entity sets, all partitions, all expectations. But when viewing a single campaign, most of that data is irrelevant. Scoped registries filter to exactly what's relevant for a given campaign.

Why this matters: Without scoping, the domain graph would show the entire domain regardless of which expectations you're looking at. Evaluation would be slower because every entity set would be materialized. And question counts would include questions from unrelated expectations.

#How Scoped Registries Are Built

_create_scoped_registries() builds scoped registries by:

Starting from the expectations in the scope
Collecting every entity set referenced by those expectations (partition sources, expectee traversals, expector lookups)
Following relationship chains to include related entity sets and entity types
Including only the expected-empty partition branches — the branches named in expect_empty that feed forward into the expectation. Happy-path branches are excluded because they have no forward path to any expectation.
For partition branches not in the full registry (unnamed branches like is_unknown or is_void), synthetic registry keys are created so the graph can show the edge from partition to branch to expectation.
Evaluating expectations and populating the question_registry — accessing exp.questions triggers lazy evaluation which generates questions.
Tracking which entity attributes were accessed during evaluation via _evaluation_context, storing the result in accessed_attributes.

The graph visualisation then shows exactly what is in the scoped registries — no more, no less. If something unexpected appears in the graph, it indicates a problem in registry construction, not a missing filter.

#Thread-Local Contexts: What They Do and When They Matter

kbpy uses three independent thread-local contexts for different runtime phases. Understanding these is critical for anyone extending the system with new code that touches entity data.

#The Three Contexts

Context	Purpose	Active During
`_loading_context`	Tracks which domain registries are being populated	Registry reconstruction (Phase 1-11)
`_evaluation_context`	Records which entity attributes are accessed during predicate evaluation	Scoped registry construction
`_lookup_question_context`	Records failed lookups against partial collections	Scoped registry construction

#`_loading_context`

Manages domain registries during reconstruction. Registration functions (entity_type(), attribute_from_data(), etc.) call _get_current_registries() to know where to store their registrations. Each domain load is isolated into its own registry instance, enabling multi-user concurrency.

Key gotcha: Accessing entity attribute data outside a loading context loses attestation metadata. If you see errors about "called outside loading context," an entity set is being accessed after the loading phase completed.

#`_evaluation_context`

Tracks which entity attributes are accessed during partition predicate evaluation. When FluentFact descriptors (_FluentFactDescriptor.__get__, BoolFluentFact.__get__, ConstantFluentFact.__get__) and relationship lookups (Unify.__get__) are invoked, they call _record_attribute_access() to log the access.

The collected accessed_attributes dict (entity_type_name → set[attr_name]) is stored on the scoped registries and used by the domain graph to show only attributes that evaluation actually touched. This is necessary because static analysis of predicates can't handle disjunctive logic — only runtime tracking reveals which attributes actually matter.

#`_lookup_question_context`

Tracks failed lookups against partial (non-exhaustive) collections during evaluation. When Unify.__get__ encounters a lookup miss against a partial collection, it calls _record_lookup_question().

These are keyed by entity identity. During _ensure_evaluated(), only entities that land in is_unknown (meaning the failed lookup actually affected the outcome) generate lookup questions. This handles disjunction naturally — if Unknown | True = True, the entity doesn't reach is_unknown and no question is generated.

#Domain Graph

The domain graph (/d/<domain>/x/<campaign>/model) visualises how entity sets, partitions, relationships, and expectations connect within a scope. It is rendered client-side as a Graphviz DOT diagram.

#What the Graph Shows

The graph displays exactly what is in the scoped registries:

Entity sets as nodes
Partitions as nodes with edges to their branch entity sets
Expectations as terminal nodes
Relationships as edges between entity sets
Attestation sources as source nodes

#Partition Discovery

Partitions appear in the graph only when at least one of their branch entity sets is present in the scoped registry. The discovery mechanism walks each entity set's parent chain: if an entity set has a parent (i.e. it is a branch of a partition), that partition is included provided its source entity set is also in scope. This naturally limits partitions to those structurally connected to the scope.

#Runtime Attribute Access Tracking

There are three levels of attribute/metadata handling in scoped registries:

Entity type definitions are vocabulary — they describe what could be known. They stay unscoped, showing all potential knowable attributes.
Entity attributes (instance data) are scoped by runtime tracking — only the attributes that were touched when predicates ran appear in the domain graph. Over-inclusion is not acceptable because disjunctive logic means static analysis can't determine which attributes are needed.
Entity set metadata is naturally scoped — entity sets in the scoped registry are only those needed by evaluation. Their metadata (_attestation_sources, is_partial, reason, parent) is already correct because only relevant sets are included.

#Questions as First-Class Registry Entries

During scoped registry construction, expectation evaluation generates questions — instances of EntityQuestion — which are stored in question_registry on the scoped registries. This avoids repeatedly iterating expectations to collect questions in web views.

Questions represent missing or misaligned data. EntityQuestion covers four question types:

Type	Meaning	Directed at
`"partition"`	Unknown attribute value in a predicate	Attestor of the entity's data source
`"traversal"`	Failed path traversal	Attestor of the source entity's data
`"lookup"`	Failed lookup against a partial collection	Attestor of the target collection
`"completeness"`	Partial entity set may have additional entities	Attestor of the entity set's data source

Lookup questions are directed at the attestor of the target collection, not the source entity — because the target collection is where the missing entity would need to appear.

#Forward-Only Connectivity Invariant

Every node in a scoped domain graph must have a directed forward path to at least one expectation node. This invariant is enforced by the test suite.

"Forward" means following the data flow direction: entity sets flow into partitions, partition branches flow into expectations. The test performs a backward BFS from expectation nodes, following edges in reverse, to find all nodes that can reach an expectation. Any node not reached is flagged as disconnected.

This invariant ensures:

No orphaned nodes clutter the graph
Every visible element contributes to at least one expectation
The scoped registry construction is correct — if a node appears without a forward path, it was incorrectly included in the scope

#Serving & Operations

#Architecture: How Data Flows from Git to Browser

#DatomStore and Versioning

#Why Datoms Instead of Tables

#Schema

#Entity ID Format

#Version Model

#Domain Versioning and Caching

#Version Source

#LRU Caching

#History Extraction

#SQLite-Only Serving

#Workflow

#How It Works

#Registry Reconstruction

#Lazy Loading

#How It Works

#What Triggers Loading

#When Lazy Loading Helps and When It Hurts

#Multiple Data Sources

#CLI Exploration

#Web Interface

#URL Structure

#Persona-Driven Navigation

#Campaign Pages

#Diagnostics Page

#Running the Web Server

#Scoped Registries: Why They Exist

#How Scoped Registries Are Built

#Thread-Local Contexts: What They Do and When They Matter

#The Three Contexts

#_loading_context

#_evaluation_context

#_lookup_question_context

#Domain Graph

#What the Graph Shows

#Partition Discovery

#Runtime Attribute Access Tracking

#Questions as First-Class Registry Entries

#Forward-Only Connectivity Invariant

#`_loading_context`

#`_evaluation_context`

#`_lookup_question_context`