Vantage Framework

Vantage is a data entity persistence and abstraction framework for Rust.

Vantage changes the way you think about your data. Instead of writing queries and loading rows, you work with sets of records — “unpaid invoices older than 30 days”, “customers who ordered today” — narrowing them, combining them, traversing from one to another, and acting on them where they live.

Vantage offers two ways to work with your data. In transactional mode there is no state to manage: load a set, act on it, done — each operation goes straight to the backend. This is the natural shape of a REST endpoint: handle the request, read or update the data, respond. (The name describes that request/response shape, not database transactions — BEGIN/COMMIT is the backend’s business, and Vantage doesn’t wrap your operations in one.) In live mode Vantage maintains a local representation of the data segment you’re working with: you operate on your local copy, and it reconciles with the source over time. That’s how a user interface behaves — any kind. Live mode is built on top of transactional mode, so everything you learn about the first carries into the second.

This documentation tracks the 2.0 line of vantage-table and vantage-vista, and the drivers and vantage-diorama releases built on it.

Ethos

A few principles run through every layer of the framework:

• Let the backend do the work. If a database or API can filter, join, aggregate, or paginate, Vantage pushes the work there — data is narrowed at the source, not in your process.
• Fill the gaps client-side, honestly. Where a backend can’t (a CSV file can’t sort; two databases can’t join), the layer above provides it — and every handle advertises exactly what it supports, so a missing capability is an explicit error, never a silent guess.
• Respect native types. Each backend keeps its own type system end to end — decimals stay precise, dates stay dates, nothing is funnelled through JSON.
• Business logic lives on the entity. Validation, audit, soft-delete, and domain methods attach to your model once and apply everywhere the entity is used.
• Fail loudly, retry safely. No panics, no silent zeros or match-alls. Every write is retry-safe and stays inside the set it goes through (Safe writes).
• Be aware of observers. Data knows who is watching: changes stream to subscribers, and edits reconcile instead of clobbering. (This is live mode — the second half of the guide builds it up.)

Three ways to work with data

Vantage lets you choose how strongly typed your data access is — per component, not per project:

1. Entity mode — records are plain Rust structs (Table<SqliteDB, Product>). Columns and conditions are compiler-checked, and business logic attaches to the entity: full type safety, the natural choice for hand-written Rust code.
2. Record mode — each record is an arbitrary structure of named values (Record<V>), with schema introspectable at runtime. No entity struct required — built for generic code: data grids, admin tools, import/export.
3. Rhai scripts — more generic still: schema declared in YAML, custom expressions and logic in Rhai. Defined, loaded, and sealed at runtime with no recompiling — made for configuration-driven and agent-driven tooling.

The three interoperate: a Rhai-declared table serves records, records deserialize into entities, and all of it sits behind the same capability-checked handles.

What makes Vantage different

Vantage is a framework, not a library. A library solves one concern and hands the rest back to you; Vantage takes over the data layer entirely — modelling, querying, types, caching, reactivity — and gives you well-defined places to plug in what’s yours. The implementation spans 10+ crates (query builders, backend drivers, the entity layer, the reactive stack), all built on the same cohesive, extensible principles: what you learn in one crate applies in the next, and every extension point looks the same wherever you meet it.

Vantage also doesn’t mimic frameworks from other languages. No reflection, no runtime magic, no inheritance hierarchies. It leans on what Rust is uniquely good at — traits for composition, ownership for safe sharing, generics that compile away, async throughout — so the framework feels native rather than translated.

The four layers

Everything above maps onto four layers. Each builds on the one below, and you climb only as far as your application needs:

Table<SqliteDB, Product>   entity mode — typed, compiler-checked, transactional
Vista                      record mode — schema known at runtime, capabilities honest
Dio                        live mode begins — local cache, write queue, change events
Scenery                    reactive views over a Dio — what a UI binds to and watches

A Table is where your model and business logic live. Wrap it into a Vista when generic code needs to consume it (record mode, above). Bind the Vista to a Dio when you want a live local representation — caching, write routing, reconciliation. Open a Scenery over the Dio for an ordered table, a single record, or an aggregate that updates as the data changes. Layers never leak upward: a Table doesn’t know it’s being cached, and a Scenery consumer can’t tell which backend is underneath.

Vantage and Vantage UI

Vantage UI is a native admin console built directly on these crates — point it at your databases, APIs, and tools, and an AI agent configures tables, forms, and dashboards over them. It is a closed-source product (free download), and this framework is its open foundation: Vantage is open-sourced so that you can build your own services, CLIs, and UIs on the same data layer — and extend it, adding persistences and capabilities that custom builds can carry further than the stock app does.

If you want the finished tool, start with Vantage UI. If you want the foundation, read on.

Getting Started

Vantage covers a lot of ground — multiple databases, type systems, entity frameworks, UI adapters — but none of that matters until you’ve seen it do something useful.

This guide introduces Vantage concepts one at a time, each building on the last. We start with something you already know — SQL — and build a shared foundation: a small product catalog that begins as a CLI, becomes an HTTP API, and switches databases without touching its handlers, then erases into a runtime-generic data handle.

From there the guide forks by the shape of your problem. The reactive stack that sits on top — caching, change events, watchable views — is the same whichever backend is underneath, so you follow it down whichever path is yours:

• A facade over an API you don’t control — a slow, read-only cloud API that can’t sort, search, or paginate. Diorama caches it and fills the gaps.
• A live view in front of your own database — a relational database you own and write to. Diorama gives it a cached, watchable facade, and the path moves the same app from SQLite to PostgreSQL with a single switch, then on to SurrealDB.

Same abstractions, opposite backends — that contrast is the point.

The foundation

1. SQLite and the Query Builder — connect to a database, build and execute typed queries, map rows to structs.
2. Tables and Typed Data Access — define entities and tables, narrow sets with conditions, traverse relationships, add computed fields; CRUD becomes one-liners.
3. A Standalone Axum Server — put the model behind HTTP with one generic CRUD handler for every entity, then migrate the whole server from SQLite to MongoDB by editing only the model.
4. Vista — the Universal Data Handle — erase the entity and backend into a schema-bearing runtime handle, with explicit capability contracts.

Choose your path

With a Vista in hand, pick the path that matches your data — a facade over an API you don’t control, or a live view over your own relational database. Both build the same caching, reactive, watch-streaming stack; they differ only in the backend beneath it.

You’ll need basic Rust experience (structs, traits, async/await, cargo). No prior Vantage knowledge required.

Start here: SQLite and the Query Builder

Beyond the guide

The rest of the book is reference material — read it when the guide points at it, or jump straight to what you need:

• Expressions & Queries and the per-backend chapters (SQL, SurrealDB) — the query-building layer in depth.
• Records: Traversal, Invariants & Hooks — the write pipeline: audit stamps, validation, soft-delete.
• Model-Driven Architecture — how Vantage expects you to structure business software.
• Config-Driven Vistas: YAML & Rhai — define and reshape data handles from configuration, sealed at runtime without recompiling.
• Adding a New Persistence — nine incremental steps to connect Vantage to a backend it doesn’t know yet; the main path for extending the framework.
• Augmentation and the Type System — enriching rows from a second source, and teaching backends your own types.
• Historical Timeline — how the framework got here, release by release.

SQLite and the Query Builder

Vantage is a big framework. It covers SQL databases, SurrealDB, MongoDB, CSV files, REST APIs — and ties them all together with a shared type system, expression engine, and data abstraction layer.

We’ll get to all of that. But right now, let’s start with something familiar: building SQL queries.

For this chapter we’ll use SQLite. It’s lightweight, needs no server, and works with a plain file on disk.

Set up

Create a new project:

cargo init learn-1 && cd learn-1
cargo add vantage-sql --features sqlite
cargo add vantage-expressions
cargo add tokio --features full

Three dependencies — vantage-sql gives us the SQLite query builder and connection pool, vantage-expressions is needed by the sqlite_expr! macro, and tokio provides the async runtime because all database operations are async.

Create and populate a database

We’ll make a small product catalog from scratch. Create seed.sql in your project root:

CREATE TABLE product (
    id INTEGER PRIMARY KEY,
    name TEXT NOT NULL,
    price INTEGER NOT NULL,
    category_id INTEGER,
    is_deleted BOOLEAN NOT NULL DEFAULT 0
);

INSERT INTO product VALUES (1, 'Cupcake',           120, 1, 0);
INSERT INTO product VALUES (2, 'Doughnut',          135, 1, 0);
INSERT INTO product VALUES (3, 'Tart',              220, 2, 0);
INSERT INTO product VALUES (4, 'Pie',               299, 2, 0);
INSERT INTO product VALUES (5, 'Cookies',           199, 1, 0);
INSERT INTO product VALUES (6, 'A Stale Cake',       80, 1, 1);
INSERT INTO product VALUES (7, 'Sourdough Loaf',    350, 3, 0);

CREATE TABLE category (
    id INTEGER PRIMARY KEY,
    name TEXT NOT NULL
);

INSERT INTO category VALUES (1, 'Sweet Treats');
INSERT INTO category VALUES (2, 'Pastries');
INSERT INTO category VALUES (3, 'Breads');

Run it:

sqlite3 products.db < seed.sql

You now have products.db — 7 products (6 active, 1 deleted) across 3 categories. Quick check:

sqlite3 products.db "SELECT name, price FROM product WHERE is_deleted = 0"

Start with an async main

All database operations in Vantage are async, so we need a Tokio runtime. Replace src/main.rs with:

use vantage_sql::prelude::*;

#[tokio::main]
async fn main() {
    if let Err(e) = run().await {
        e.report();
    }
}

async fn run() -> VantageResult<()> {
    println!("Ready!");
    Ok(())
}

A few things going on here:

• use vantage_sql::prelude::* brings in everything we need for this chapter — SqliteDB, SqliteSelect, the sqlite_expr! macro, error types, and the traits that make builder and execution methods work.
• VantageResult<()> is Vantage’s own Result type. It uses VantageError, which tracks context and error chains for readable diagnostics.
• e.report() prints the error in a structured format. We call it from main() because Rust’s default Result-returning main uses Debug formatting, which is ugly. This pattern gives us clean error output instead.

Run cargo run to make sure it compiles.

Connect to SQLite

Add this inside run():

use vantage_sql::prelude::*;
#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
let db = SqliteDB::connect("sqlite:products.db?mode=ro")
    .await
    .context("Failed to connect to products.db")?;
Ok(())
}

SqliteDB wraps an sqlx connection pool. The connection string is an sqlx URL — ?mode=ro opens read-only, which is all we need for now.

#![allow(unused)]
fn main() {
let db = SqliteDB::new(existing_pool);
}

Error: Failed to connect to products.db
│
╰─▶ error returned from database: (code: 14) unable to open database file

Build a SELECT

SqliteSelect is the query builder for SQLite. Other persistences have their own — PostgresSelect, MongoSelect — and they all implement the Selectable trait, so the interface is identical apart from vendor-specific extensions. None of them need a database connection — they’re just structs that accumulate query parts. You build them with a chain of .with_*() calls:

#![allow(unused)]
fn main() {
use vantage_sql::prelude::*;
let select = SqliteSelect::new()
    .with_source("product")
    .with_field("name")
    .with_field("price");

println!("{}", select.preview());
// SELECT "name", "price" FROM "product"
assert_eq!(select.preview(), r#"SELECT "name", "price" FROM "product""#);
}

.preview() renders the final SQL as a string — handy for debugging, but never used for execution.

#![allow(unused)]
fn main() {
// Builder style
let select = SqliteSelect::new()
    .with_source("product")
    .with_field("name");

// Mutable style
let mut select = SqliteSelect::new();
select.add_source("product", None);
select.add_field("name");
}

Execute it

use vantage_sql::prelude::*;
#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
let db = SqliteDB::connect("sqlite:products.db?mode=ro")
    .await
    .context("Failed to connect to products.db")?;
let select = SqliteSelect::new()
    .with_source("product")
    .with_field("name")
    .with_field("price");
let result = db.execute(&select.expr()).await?;
println!("{:?}", result);
Ok(())
}

Two steps here: .expr() turns the builder into an Expression — Vantage’s internal representation that keeps parameters separate from the SQL template. Then db.execute() sends it to the database.

The result is AnySqliteType — a type-tagged wrapper around whatever came back. The Debug output isn’t pretty, but you should see all 6 product rows in there.

Adding conditions

Our database has a soft-delete flag — “A Stale Cake” has is_deleted = 1. Let’s filter it out:

#![allow(unused)]
fn main() {
use vantage_sql::prelude::*;
let condition = sqlite_expr!("\"is_deleted\" = {}", false);

let select = SqliteSelect::new()
    .with_source("product")
    .with_field("name")
    .with_field("price")
    .with_condition(condition);

println!("{}", select.preview());
// SELECT "name", "price" FROM "product" WHERE "is_deleted" = 0
assert_eq!(
    select.preview(),
    r#"SELECT "name", "price" FROM "product" WHERE "is_deleted" = 0"#
);
}

sqlite_expr! creates an Expression — a SQL template with typed, bound parameters. That {} is not Rust’s format!: the value false is stored separately and bound through sqlx’s parameterized query interface at execution time — no injection risk, ever. The preview shows it inline for readability.

In SQL persistences, a condition is just an expression. When you pass it to .with_condition(), it’s nested inside the select’s own expression tree. At execution time the whole tree is flattened into a single template + parameter list that the database driver can bind safely.

Run it — you should see 5 rows, with “A Stale Cake” filtered out.

Typed columns and operators

Writing \"is_deleted\" in a raw expression works, but there’s a cleaner way. Column<T> creates a typed column reference, then chain an SqliteOperation like .eq() to build the condition. (Column lives in the vantage-table crate, but the vantage_sql prelude re-exports it — no new dependency needed.)

#![allow(unused)]
fn main() {
use vantage_sql::prelude::*;
let is_deleted = Column::<bool>::new("is_deleted");
let condition = is_deleted.eq(false);
}

The same condition, rendered as is_deleted = 0 (a typed column isn’t quoted), but the type parameter <bool> ensures you can only compare against matching types. Try is_deleted.eq(42) — it won’t compile. Other operators — .gt(), .lt(), .ne(), .in_() — enforce the same type safety.

The result is a SqliteCondition — the backend’s native condition type — ready to be passed directly to .with_condition().

Multiple conditions combine with AND:

#![allow(unused)]
fn main() {
use vantage_sql::prelude::*;
let is_deleted = Column::<bool>::new("is_deleted");
let price = Column::<i64>::new("price");

let select = SqliteSelect::new()
    .with_source("product")
    .with_field("name")
    .with_condition(is_deleted.eq(false))
    .with_condition(price.gt(150));
// ... WHERE is_deleted = 0 AND price > 150
assert_eq!(
    select.preview(),
    r#"SELECT "name" FROM "product" WHERE is_deleted = 0 AND price > 150"#
);
}

#![allow(unused)]
fn main() {
use vantage_sql::sqlite::sqlite_ident as ident;

let condition = ident("is_deleted").eq(false);
}

Primitives, query builders, Column, conditions — even native values like i64 and bool — all implement Expressive<T>, where T is your database’s any-type (AnySqliteType for SQLite, AnyMongoType for MongoDB). That shared trait is what makes any of them usable as a parameter inside any expression.

Working with Any-types

Record<V> is an ordered map (IndexMap<String, V>) — one record per row, with column names as keys. It lives in the vantage-types crate, so add that dependency and import its prelude:

cargo add vantage-types --features serde

use vantage_types::prelude::*;
fn main() {}

So far we’ve been printing raw AnySqliteType with Debug. That works for verifying queries, but it’s useless for real work. When db.execute() returns a multi-row result, the AnySqliteType holds an array of maps internally. Convert it to Vec<Record<AnySqliteType>> to work with individual rows:

use vantage_sql::prelude::*;
use vantage_types::prelude::*;
#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
let db = SqliteDB::connect("sqlite:products.db?mode=ro")
    .await
    .context("Failed to connect to products.db")?;
let select = SqliteSelect::new()
    .with_source("product")
    .with_field("name")
    .with_field("price");
let raw = db.execute(&select.expr()).await?;

let records = Vec::<Record<AnySqliteType>>::try_from(raw)
    .context("Failed to convert to records")?;

for rec in &records {
    let name: String = rec["name"].try_get::<String>().unwrap();
    let price: i64 = rec["price"].try_get::<i64>().unwrap();
    println!("{} — {} cents", name, price);
}
// Cupcake — 120 cents
// Doughnut — 135 cents
// ...
Ok(())
}

Access fields by column name with rec["name"]. Each value is still an AnySqliteType, so you call .try_get::<T>() to extract a typed Rust value. If the type doesn’t match (say you call .try_get::<i64>() on a text column), you get None — no panics, no garbage.

#![allow(unused)]
fn main() {
let json_rec: Record<serde_json::Value> = rec.into_record();
}

Under the hood, each persistence has its own type system for storing values. SQLite uses CBOR — a compact binary format that preserves types like Decimal, NaiveDate, and NaiveDateTime through tagged values. MongoDB uses BSON natively. The AnySqliteType / AnyMongoType wrappers hide these details — you interact with .try_get::<T>() regardless of which persistence you’re using. If you ever need to inspect the raw representation, .value() gives you the underlying CBOR value:

use vantage_sql::prelude::*;
use vantage_types::prelude::*;
#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
let db = SqliteDB::connect("sqlite:products.db?mode=ro")
    .await
    .context("Failed to connect to products.db")?;
let select = SqliteSelect::new()
    .with_source("product")
    .with_field("name")
    .with_field("price");
let records = Vec::<Record<AnySqliteType>>::try_from(db.execute(&select.expr()).await?)?;
let rec = &records[0];
let price_cbor = rec["price"].value();
println!("{:?}", price_cbor);
// Integer(Integer(120))
Ok(())
}

See Persistence-aligned Type System for the full picture.

Mapping rows to structs

Calling .try_get::<T>() on every field gets tedious. The #[entity] macro generates TryFromRecord<AnySqliteType> for your struct, so conversion happens in one call with no type information lost:

use vantage_sql::prelude::*;
use vantage_types::prelude::*;
#[entity(SqliteType)]
struct Product {
    name: String,
    price: i64,
}

#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
let db = SqliteDB::connect("sqlite:products.db?mode=ro")
    .await
    .context("Failed to connect to products.db")?;
let select = SqliteSelect::new()
    .with_source("product")
    .with_field("name")
    .with_field("price");
let raw = db.execute(&select.expr()).await?;
let records = Vec::<Record<AnySqliteType>>::try_from(raw)?;

for rec in records {
    let product = Product::from_record(rec)?;
    println!("{} — {} cents", product.name, product.price);
}
Ok(())
}

The macro needs vantage-core as a direct dependency (it’s already a transitive dep through vantage-sql, but the generated code references it in your crate):

cargo add vantage-core

The macro also supports multiple type systems in one attribute — #[entity(SqliteType, PostgresType, MongoType)] generates a separate TryFromRecord impl for each persistence. One struct, all backends.

Putting it together

Here’s the complete src/main.rs — connect, query, convert to entities, print:

use vantage_sql::prelude::*;
use vantage_types::prelude::*;

#[entity(SqliteType)]
struct Product {
    name: String,
    price: i64,
}

#[tokio::main]
async fn main() {
    if let Err(e) = run().await {
        e.report();
    }
}

async fn run() -> VantageResult<()> {
    let db = SqliteDB::connect("sqlite:products.db?mode=ro")
        .await
        .context("Failed to connect to products.db")?;

    let select = SqliteSelect::new()
        .with_source("product")
        .with_field("name")
        .with_field("price")
        .with_condition(Column::<bool>::new("is_deleted").eq(false));

    let raw = db.execute(&select.expr()).await?;
    let records = Vec::<Record<AnySqliteType>>::try_from(raw)?;

    for rec in records {
        let p = Product::from_record(rec)?;
        println!("{:<12} {:>3} cents", p.name, p.price);
    }

    Ok(())
}

cargo run
# Cupcake      120 cents
# Doughnut     135 cents
# Tart         220 cents
# Pie          299 cents
# Cookies      199 cents

What we covered

Tables and Typed Data Access

A table is a structure representing a collection of records in a database:

use vantage_sql::prelude::*;
use vantage_types::prelude::*;
#[entity(SqliteType)]
#[derive(Clone, Default)]
struct Product {
    name: String,
    price: i64,
}
impl Product {
    fn table(db: SqliteDB) -> Table<SqliteDB, Product> {
        Table::new("product", db)
            .with_id_column("id")
            .with_column_of::<String>("name")
            .with_column_of::<i64>("price")
    }
}
#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
let db = SqliteDB::connect("sqlite:products.db")
    .await
    .context("Failed to connect to products.db")?;
let products = Product::table(db);

for (id, product) in products.list().await? {
    println!("{id}: {} — {} cents", product.name, product.price);
}
Ok(())
}

Think of Table<SqliteDB, Product> as a Vec<Product> — except the records aren’t in memory. They live somewhere else (a database, a file, an API), and you don’t know how many there are or what they contain until you list them.

If you’ve used an ORM before, you might expect Vantage to work with individual records — load one, change it, save it back. Vantage works differently: you always operate on a set of records. A set might contain one record, no records, or millions — you don’t pull them into memory to find out.

Operations like counting, filtering, and updating happen on the database side — Vantage builds the right query and sends it over. Even traversal operations and sub-queries are done by building queries and executing remotely — if the database supports it, of course.

When you .clone() a table, you don’t clone the data — you clone the definition. From there you can narrow it down by adding conditions, turning it into a subset. Some examples of sets you might work with:

• all user records except those with a soft-delete flag
• orders placed today
• orders of paid customers
• customers who have an unpaid invoice older than 30 days
• products that sold at least 5 items today

Each of these is a set — defined by conditions, not a list of IDs — and one set can serve as the condition of another (“orders of paid customers”). Acting on a set is a single call: notify those customers, archive those orders, apply that discount. Put together, even complex operations are easy to express in code:

use vantage_sql::prelude::*;
use vantage_sql::primitives;
use vantage_types::prelude::*;
#[entity(SqliteType)]
#[derive(Clone, Default)]
struct Customer {
    name: String,
}
#[entity(SqliteType)]
#[derive(Clone, Default)]
struct Invoice {
    status: String,
}
#[entity(SqliteType)]
#[derive(Clone, Default)]
struct Order {
    total: i64,
}
impl Customer {
    fn table(db: SqliteDB) -> Table<SqliteDB, Customer> {
        Table::new("customer", db)
            .with_id_column("id")
            .with_column_of::<String>("name")
            .with_many("orders", "customer_id", Order::table)
    }
}
impl Invoice {
    fn table(db: SqliteDB) -> Table<SqliteDB, Invoice> {
        Table::new("invoice", db)
            .with_id_column("id")
            .with_column_of::<String>("status")
            .with_column_of::<String>("issued_at")
            .with_one("customer", "customer_id", Customer::table)
    }
}
impl Order {
    fn table(db: SqliteDB) -> Table<SqliteDB, Order> {
        Table::new("order", db)
            .with_id_column("id")
            .with_column_of::<i64>("total")
            .with_column_of::<String>("placed_at")
    }
}
fn days_ago(days: i64) -> Expression<AnySqliteType> {
    let offset = format!("-{days} days");
    sqlite_expr!("date('now', {})", offset)
}
trait InvoiceTable {
    fn ref_customer(&self) -> Table<SqliteDB, Customer>;
}
impl InvoiceTable for Table<SqliteDB, Invoice> {
    fn ref_customer(&self) -> Table<SqliteDB, Customer> {
        self.get_ref_as("customer").unwrap()
    }
}
trait CustomerTable {
    fn subquery_orders(&self) -> Table<SqliteDB, Order>;
    async fn send_reminder(&self) -> VantageResult<()>;
}
impl CustomerTable for Table<SqliteDB, Customer> {
    fn subquery_orders(&self) -> Table<SqliteDB, Order> {
        self.get_subquery_as("orders").unwrap()
    }
    async fn send_reminder(&self) -> VantageResult<()> {
        for (_, customer) in self.list().await? {
            println!("reminder sent to {}", customer.name);
        }
        Ok(())
    }
}
trait OrderTable {
    fn only_this_month(self) -> Self;
    fn field_sum(&self, column: &str) -> Expression<AnySqliteType>;
}
impl OrderTable for Table<SqliteDB, Order> {
    fn only_this_month(self) -> Self {
        let placed_at = self["placed_at"].clone();
        self.with_condition(placed_at.gt(sqlite_expr!("date('now', 'start of month')")))
    }
    fn field_sum(&self, column: &str) -> Expression<AnySqliteType> {
        self.get_sum_query(&self[column])
    }
}
#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
let db = SqliteDB::connect("sqlite:shop.db").await.context("Failed to connect")?;
// "notify customers who have an unpaid invoice older than 30 days"
let mut overdue = Invoice::table(db.clone());
overdue.add_condition(overdue["status"].eq("unpaid"));
overdue.add_condition(overdue["issued_at"].lt(days_ago(30)));

overdue.ref_customer().send_reminder().await?;

// "apply a discount to customers who spent more than $500 this month"
let customers = Customer::table(db)
    .with_expression("spent_this_month", |c| {
        c.subquery_orders().only_this_month().field_sum("total")
    })
    .with_expression("discount", |c| {
        let spent = c.get_column_expr("spent_this_month").unwrap();
        primitives::ternary(spent.gt(500), 10, 0).expr()
    });
Ok(())
}

Don’t worry about the exact syntax — we’ll get to all of it. ref_customer, send_reminder, subquery_orders, only_this_month and field_sum aren’t framework calls: they’re methods the model defines on its own tables, and Extension traits below shows how. The point is that sets compose naturally: define one, use it to narrow another, then act on the result.

Defining a Table

The syntax for defining a table looks similar to query building from chapter 1, but there are important differences:

A Table is typically defined in its own file alongside the entity. Create src/product.rs:

// src/product.rs
use vantage_sql::prelude::*;
use vantage_types::prelude::*;

#[entity(SqliteType)]
#[derive(Clone, Default)]
pub struct Product {
    pub name: String,
    pub price: i64,
}

impl Product {
    pub fn table(db: SqliteDB) -> Table<SqliteDB, Product> {
        let is_deleted = Column::<bool>::new("is_deleted");
        Table::new("product", db)
            .with_id_column("id")
            .with_column_of::<String>("name")
            .with_column_of::<i64>("price")
            .with_column_of::<bool>("is_deleted")
            .with_condition(is_deleted.eq(false))
    }
}
fn main() {}

#![allow(unused)]
fn main() {
impl Product {
    fn table() -> Table<SqliteDB, Product> {
        Table::new("product", get_sqlite_db())
            // ...columns...
    }
}
}

This defines the product table once. You can now generate queries from it:

use vantage_sql::prelude::*;
use vantage_types::prelude::*;
#[entity(SqliteType)]
#[derive(Clone, Default)]
struct Product {
    name: String,
    price: i64,
}
impl Product {
    fn table(db: SqliteDB) -> Table<SqliteDB, Product> {
        let is_deleted = Column::<bool>::new("is_deleted");
        Table::new("product", db)
            .with_id_column("id")
            .with_column_of::<String>("name")
            .with_column_of::<i64>("price")
            .with_column_of::<bool>("is_deleted")
            .with_condition(is_deleted.eq(false))
    }
}
#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
let db = SqliteDB::connect("sqlite::memory:").await.context("Failed to connect")?;
let table = Product::table(db);

// Full SELECT with all columns and conditions applied
let select = table.select();
println!("{}", select.preview());
// SELECT "id", "name", "price", "is_deleted" FROM "product" WHERE is_deleted = 0

// Count query (does not execute — just builds the expression)
let count_query = table.get_count_query();
// (SELECT COUNT(*) FROM "product" WHERE is_deleted = 0)

// Sum query for a specific column
let sum_query = table.get_sum_query(&table["price"]);
// (SELECT SUM(price) FROM "product" WHERE is_deleted = 0)
assert_eq!(
    select.preview(),
    r#"SELECT "id", "name", "price", "is_deleted" FROM "product" WHERE is_deleted = 0"#
);
assert_eq!(
    count_query.preview(),
    r#"(SELECT COUNT(*) FROM "product" WHERE is_deleted = 0)"#
);
assert_eq!(
    sum_query.preview(),
    r#"(SELECT SUM(price) FROM "product" WHERE is_deleted = 0)"#
);
Ok(())
}

These return the same SqliteSelect and Expression types from chapter 1 — the table just assembles them for you, with its conditions already applied. The count and sum queries come wrapped in parentheses so they can nest inside another query as a subquery. But most of the time you won’t need the raw query at all.

CRUD operations

With the table defined, all four operations — create, read, update, delete — are one-liners. Each comes from a trait in the prelude:

use vantage_sql::prelude::*;
use vantage_types::prelude::*;
#[entity(SqliteType)]
#[derive(Clone, Default)]
struct Product {
    name: String,
    price: i64,
}
impl Product {
    fn table(db: SqliteDB) -> Table<SqliteDB, Product> {
        let is_deleted = Column::<bool>::new("is_deleted");
        Table::new("product", db)
            .with_id_column("id")
            .with_column_of::<String>("name")
            .with_column_of::<i64>("price")
            .with_column_of::<bool>("is_deleted")
            .with_condition(is_deleted.eq(false))
    }
}
#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
let db = SqliteDB::connect("sqlite:products.db")
    .await
    .context("Failed to connect to products.db")?;
let table = Product::table(db);

// Read — list all, get one by ID
let all = table.list().await?;             // IndexMap<String, Product>
let pie = table.get("4").await?;           // Option<Product> — id 4 is Pie

// Create — insert with a known ID
let muffin = Product { name: "Muffin".into(), price: 175 };
table.insert("8", &muffin).await?;

// Update — replace the entire record
let updated = Product { name: "Blueberry Muffin".into(), price: 195 };
table.replace("8", &updated).await?;

// Delete
table.delete("8").await?;
Ok(())
}

That’s it. list() returns an IndexMap<Id, Product> — ordered and keyed by ID. get() returns Option<Product> — None when the ID doesn’t exist, so you can pattern-match or .ok_or(...) into your own not-found error. There’s also get_some() which returns an (Id, Product) pair (still Option-wrapped) for sampling, and insert_return_id() for when you want the database to generate the ID; see Safe writes for why a client-made id is the retry-safe choice.

Listing any table

Table<SqliteDB, Product> is great when you know the types at compile time. Sometimes you want a function that lists any table — products, orders, customers — without naming the entity type.

With generics you keep full type safety, limited to Rust callers:

use vantage_sql::prelude::*;
use vantage_types::prelude::*;
async fn list_table<D, E>(table: &D) -> VantageResult<()>
where
    D: ReadableDataSet<E>,
    D::Id: std::fmt::Display,
    E: Entity<D::Value> + std::fmt::Debug,
{
    for (id, entity) in table.list().await? {
        println!("  {}: {:?}", id, entity);
    }
    Ok(())
}
fn main() {}

The entity bound names the dataset’s own value type (Entity<D::Value>): an entity is only an entity of a particular type system, and that’s what lets list() decode rows losslessly.

To go fully type-erased — iterate columns by name, get runtime values, and expose tables outside Rust (a web admin UI, FFI, scripting) — wrap the table in a Vista, the universal schema-bearing data handle. That’s the subject of its own chapter; Vista replaced the older AnyTable carrier in 0.5.

Relationships

Our database has a category table that we haven’t used yet. Let’s define it and connect it to products.

Create src/category.rs:

// src/category.rs
use vantage_sql::prelude::*;
use vantage_types::prelude::*;

use crate::product::Product;

#[entity(SqliteType)]
#[derive(Debug, Clone, Default)]
pub struct Category {
    pub name: String,
}

impl Category {
    pub fn table(db: SqliteDB) -> Table<SqliteDB, Category> {
        Table::new("category", db)
            .with_id_column("id")
            .with_column_of::<String>("name")
            .with_many("products", "category_id", Product::table)
    }
}
mod product {
    use vantage_sql::prelude::*;
    use vantage_types::prelude::*;
    #[entity(SqliteType)]
    #[derive(Clone, Default)]
    pub struct Product {
        pub name: String,
        pub price: i64,
    }
    impl Product {
        pub fn table(db: SqliteDB) -> Table<SqliteDB, Product> {
            let is_deleted = Column::<bool>::new("is_deleted");
            Table::new("product", db)
                .with_id_column("id")
                .with_column_of::<String>("name")
                .with_column_of::<i64>("price")
                .with_column_of::<bool>("is_deleted")
                .with_condition(is_deleted.eq(false))
        }
    }
}
fn main() {}

The important line is .with_many("products", "category_id", Product::table). This declares:

• “products” — the name of the relationship (you’ll use this to traverse it)
• “category_id” — the foreign key column on the product table
• Product::table — a function that builds the target table (same one we defined earlier)

No changes to Product are needed — the relationship is declared on the side that “owns” it. Category has many products; the foreign key lives on product.

Given a set of categories, get_ref_as traverses the relationship and returns matching products:

use vantage_sql::prelude::*;
use vantage_types::prelude::*;
#[entity(SqliteType)]
#[derive(Clone, Default)]
struct Product {
    name: String,
    price: i64,
}
impl Product {
    fn table(db: SqliteDB) -> Table<SqliteDB, Product> {
        let is_deleted = Column::<bool>::new("is_deleted");
        Table::new("product", db)
            .with_id_column("id")
            .with_column_of::<String>("name")
            .with_column_of::<i64>("price")
            .with_column_of::<bool>("is_deleted")
            .with_condition(is_deleted.eq(false))
    }
}
#[entity(SqliteType)]
#[derive(Clone, Default)]
struct Category {
    name: String,
}
impl Category {
    fn table(db: SqliteDB) -> Table<SqliteDB, Category> {
        Table::new("category", db)
            .with_id_column("id")
            .with_column_of::<String>("name")
            .with_many("products", "category_id", Product::table)
    }
}
#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
let db = SqliteDB::connect("sqlite::memory:").await.context("Failed to connect")?;
let categories = Category::table(db.clone());
let products = categories.get_ref_as::<Product>("products")?;
Ok(())
}

The result is a Table<SqliteDB, Product> with an extra condition — only products whose category_id matches one of the category IDs. Narrow the categories first and the subquery narrows too. Let’s use this in a CLI that accepts an optional search filter. The .with_search() method adds a LIKE condition across all columns of the table — handy for quick filtering.

Update src/main.rs. Declare the two modules at the top with mod category; and mod product;, then:

mod product {
    use vantage_sql::prelude::*;
    use vantage_types::prelude::*;
    #[entity(SqliteType)]
    #[derive(Clone, Default)]
    pub struct Product {
        pub name: String,
        pub price: i64,
    }
    impl Product {
        pub fn table(db: SqliteDB) -> Table<SqliteDB, Product> {
            let is_deleted = Column::<bool>::new("is_deleted");
            Table::new("product", db)
                .with_id_column("id")
                .with_column_of::<String>("name")
                .with_column_of::<i64>("price")
                .with_column_of::<bool>("is_deleted")
                .with_condition(is_deleted.eq(false))
        }
    }
}
mod category {
    use vantage_sql::prelude::*;
    use vantage_types::prelude::*;
    use crate::product::Product;
    #[entity(SqliteType)]
    #[derive(Clone, Default)]
    pub struct Category {
        pub name: String,
    }
    impl Category {
        pub fn table(db: SqliteDB) -> Table<SqliteDB, Category> {
            Table::new("category", db)
                .with_id_column("id")
                .with_column_of::<String>("name")
                .with_many("products", "category_id", Product::table)
        }
    }
}
use category::Category;
use product::Product;
use vantage_sql::prelude::*;

async fn list_products(table: &Table<SqliteDB, Product>) -> VantageResult<()> {
    for (id, p) in table.list().await? {
        println!("  {:<4} {:<20} {:>3} cents", id, p.name, p.price);
    }
    Ok(())
}

#[tokio::main]
async fn main() {
    if let Err(e) = run().await {
        e.report();
    }
}

async fn run() -> VantageResult<()> {
    let db = SqliteDB::connect("sqlite:products.db")
        .await
        .context("Failed to connect to products.db")?;

    let filter = std::env::args().nth(1);

    let products = match &filter {
        Some(search) => Category::table(db.clone())
            .with_search(search)
            .get_ref_as::<Product>("products")?,
        None => Product::table(db),
    };

    list_products(&products).await?;

    Ok(())
}

Try it:

cargo run                # all active products
cargo run "Sweet"        # Cupcake, Doughnut, Cookies
cargo run "Pastries"     # Tart, Pie
cargo run "t"            # matches "Sweet Treats" AND "Pastries" — 5 products

The search "t" matches two categories (“Sweet Treats” and “Pastries”), so products from both are returned.

SELECT ... FROM "product"
WHERE "category_id" IN (SELECT "id" FROM "category" WHERE "name" LIKE '%t%')
AND "is_deleted" = 0

Computed fields with expressions

So far, list_products only shows name and price. It would be nice to show the category name too — but Product doesn’t have a category field, and the category name lives in a different table.

Vantage solves this with expressions — computed fields that are evaluated as part of the SELECT query. You define them on the table, and they appear alongside regular columns.

First, add a with_one relationship on Product (the reverse of with_many on Category):

use vantage_sql::prelude::*;
use vantage_types::prelude::*;
#[entity(SqliteType)]
#[derive(Clone, Default)]
struct Product {
    name: String,
    price: i64,
}
#[entity(SqliteType)]
#[derive(Clone, Default)]
struct Category {
    name: String,
}
impl Category {
    fn table(db: SqliteDB) -> Table<SqliteDB, Category> {
        Table::new("category", db)
            .with_id_column("id")
            .with_column_of::<String>("name")
    }
}
#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
let db = SqliteDB::connect("sqlite::memory:").await.context("Failed to connect")?;
let products: Table<SqliteDB, Product> = Table::new("product", db)
    .with_id_column("id")
    .with_column_of::<i64>("category_id")
.with_one("category", "category_id", Category::table)
;
Ok(())
}

This says: each product has one category, linked through category_id. Now you can use get_subquery_as inside with_expression to build a correlated subquery:

use vantage_sql::prelude::*;
use vantage_types::prelude::*;
#[entity(SqliteType)]
#[derive(Clone, Default)]
struct Product {
    name: String,
    price: i64,
}
#[entity(SqliteType)]
#[derive(Clone, Default)]
struct Category {
    name: String,
}
impl Category {
    fn table(db: SqliteDB) -> Table<SqliteDB, Category> {
        Table::new("category", db)
            .with_id_column("id")
            .with_column_of::<String>("name")
    }
}
#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
let db = SqliteDB::connect("sqlite::memory:").await.context("Failed to connect")?;
let products: Table<SqliteDB, Product> = Table::new("product", db)
    .with_id_column("id")
    .with_column_of::<i64>("category_id")
    .with_one("category", "category_id", Category::table)
.with_expression("category", |t| {
    t.get_subquery_as::<Category>("category")
        .unwrap()
        .select_column("name")
        .expect("Category has a 'name' column")
})
;
Ok(())
}

select_column("name") builds a subquery like SELECT "name" FROM "category" WHERE "id" = "product"."category_id" — a correlated subquery that fetches the category name for each product row. It returns an Option, None when the target table has no such column; the name here is hard-coded and known to exist, so .expect() is the honest unwrap.

Here’s the updated src/product.rs:

use vantage_sql::prelude::*;
use vantage_types::prelude::*;

use crate::category::Category;

#[entity(SqliteType)]
#[derive(Debug, Clone, Default)]
pub struct Product {
    pub name: String,
    pub price: i64,
    pub category: Option<String>,
}

impl Product {
    pub fn table(db: SqliteDB) -> Table<SqliteDB, Product> {
        let is_deleted = Column::<bool>::new("is_deleted");
        Table::new("product", db)
            .with_id_column("id")
            .with_column_of::<String>("name")
            .with_column_of::<i64>("price")
            .with_column_of::<bool>("is_deleted")
            .with_condition(is_deleted.eq(false))
            .with_one("category", "category_id", Category::table)
            .with_expression("category", |t| {
                t.get_subquery_as::<Category>("category")
                    .unwrap()
                    .select_column("name")
                    .expect("Category has a 'name' column")
            })
    }
}
mod category {
    use vantage_sql::prelude::*;
    use vantage_types::prelude::*;
    use crate::Product;
    #[entity(SqliteType)]
    #[derive(Debug, Clone, Default)]
    pub struct Category {
        pub name: String,
    }
    impl Category {
        pub fn table(db: SqliteDB) -> Table<SqliteDB, Category> {
            Table::new("category", db)
                .with_id_column("id")
                .with_column_of::<String>("name")
                .with_many("products", "category_id", Product::table)
        }
    }
}
fn main() {}

A few things to note:

• category: Option<String> — the field is Option because a product might not have a category (NULL in the database). The expression result is deserialized into this field automatically.
• No with_column_of for “category” — the expression replaces what would normally be a column. Vantage adds it to the SELECT as a computed field.
• with_one + with_expression — the relationship defines how to traverse; the expression defines what to fetch. They work together but serve different purposes.

Update list_products to show the category:

use vantage_sql::prelude::*;
use vantage_types::prelude::*;
#[entity(SqliteType)]
#[derive(Debug, Clone, Default)]
struct Product {
    name: String,
    price: i64,
    category: Option<String>,
}
async fn list_products(table: &Table<SqliteDB, Product>) -> VantageResult<()> {
    for (id, p) in table.list().await? {
        let cat = p.category.as_deref().unwrap_or("-");
        println!("  {:<4} {:<20} {:>3} cents  [{}]", id, p.name, p.price, cat);
    }
    Ok(())
}
fn main() {}

Run it:

cargo run

  1    Cupcake              120 cents  [Sweet Treats]
  2    Doughnut             135 cents  [Sweet Treats]
  3    Tart                 220 cents  [Pastries]
  4    Pie                  299 cents  [Pastries]
  5    Cookies              199 cents  [Sweet Treats]
  7    Sourdough Loaf       350 cents  [Breads]

The category name comes from a subquery — no JOIN, no extra round trip. And the category filter still works:

cargo run "t"

  1    Cupcake              120 cents  [Sweet Treats]
  2    Doughnut             135 cents  [Sweet Treats]
  3    Tart                 220 cents  [Pastries]
  4    Pie                  299 cents  [Pastries]
  5    Cookies              199 cents  [Sweet Treats]

#![allow(unused)]
fn main() {
// In Category::table()
.with_expression("product_count", |t| {
    t.get_subquery_as::<Product>("products")
        .unwrap()
        .get_count_query()
})
.with_expression("title", |t| {
    let name = t.get_column_expr("name").unwrap();
    let count = t.get_column_expr("product_count").unwrap();
    concat_!(name, " (", count, ")").expr()
})
}

#![allow(unused)]
fn main() {
.with_expression("category", |t| {
    t.get_subquery_as::<Category>("category")
        .unwrap()
        .select_column("title")
        .expect("Category has a 'title' column")
})
}

  1    Cupcake              120 cents  [Sweet Treats (3)]
  2    Doughnut             135 cents  [Sweet Treats (3)]
  3    Tart                 220 cents  [Pastries (2)]
  4    Pie                  299 cents  [Pastries (2)]
  5    Cookies              199 cents  [Sweet Treats (3)]
  7    Sourdough Loaf       350 cents  [Breads (1)]

Extension traits

Throughout this chapter you’ve seen calls like categories.get_ref_as::<Product>("products"). The turbofish ::<Product> and the string "products" are repetitive and error-prone — get the string wrong and you get a runtime error.

Rust’s extension traits solve this. Define a trait on Table<SqliteDB, Category> that wraps the traversal:

use vantage_sql::prelude::*;
use vantage_types::prelude::*;
#[entity(SqliteType)]
#[derive(Clone, Default)]
pub struct Product {
    name: String,
    price: i64,
}
#[entity(SqliteType)]
#[derive(Clone, Default)]
pub struct Category {
    name: String,
}
pub trait CategoryTable {
    fn ref_products(&self) -> Table<SqliteDB, Product>;
}

impl CategoryTable for Table<SqliteDB, Category> {
    fn ref_products(&self) -> Table<SqliteDB, Product> {
        self.get_ref_as("products").unwrap()
    }
}
fn main() {}

The trait declares the vocabulary; the impl block supplies the bodies. Nothing is auto-implemented — ref_products() appears on Table<SqliteDB, Category> the moment you write this impl, and on nothing else. The bodies must live impl-side, too: methods like get_ref_as are inherent to Table, so they’re only reachable where self is concretely that table — inside the impl, not inside a trait default.

The unwrap() is safe because we know “products” is defined on every CategoryTable. If the string were wrong, it would panic immediately during development, not silently fail at runtime.

Now callers write categories.ref_products() — no turbofish, no string, no ?, and the compiler catches typos.

The same pattern works for typed column access:

use vantage_sql::prelude::*;
use vantage_types::prelude::*;
#[entity(SqliteType)]
#[derive(Clone, Default)]
pub struct Product {
    name: String,
    price: i64,
}
pub trait ProductTable {
    fn price(&self) -> Column<i64>;
}

impl ProductTable for Table<SqliteDB, Product> {
    fn price(&self) -> Column<i64> {
        self.get_column("price").unwrap()
    }
}
fn main() {}

With both traits in place, code reads naturally:

use vantage_sql::prelude::*;
use vantage_types::prelude::*;
#[entity(SqliteType)]
#[derive(Clone, Default)]
pub struct Product {
    name: String,
    price: i64,
}
impl Product {
    pub fn table(db: SqliteDB) -> Table<SqliteDB, Product> {
        let is_deleted = Column::<bool>::new("is_deleted");
        Table::new("product", db)
            .with_id_column("id")
            .with_column_of::<String>("name")
            .with_column_of::<i64>("price")
            .with_column_of::<bool>("is_deleted")
            .with_condition(is_deleted.eq(false))
    }
}
#[entity(SqliteType)]
#[derive(Clone, Default)]
pub struct Category {
    name: String,
}
impl Category {
    pub fn table(db: SqliteDB) -> Table<SqliteDB, Category> {
        Table::new("category", db)
            .with_id_column("id")
            .with_column_of::<String>("name")
            .with_many("products", "category_id", Product::table)
    }
}
pub trait CategoryTable {
    fn ref_products(&self) -> Table<SqliteDB, Product>;
}
impl CategoryTable for Table<SqliteDB, Category> {
    fn ref_products(&self) -> Table<SqliteDB, Product> {
        self.get_ref_as("products").unwrap()
    }
}
pub trait ProductTable {
    fn price(&self) -> Column<i64>;
}
impl ProductTable for Table<SqliteDB, Product> {
    fn price(&self) -> Column<i64> {
        self.get_column("price").unwrap()
    }
}
#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
let db = SqliteDB::connect("sqlite::memory:").await.context("Failed to connect")?;
let categories = Category::table(db);
let products = categories.ref_products();
let price = products.price();
let expensive = products.with_condition(price.gt(200));
assert!(expensive.select().preview().contains("price > 200"));
Ok(())
}

price() is read before with_condition because with_condition consumes the table it’s called on; taking the column first keeps the borrow and the move apart.

#![allow(unused)]
fn main() {
pub trait ProductTable {
    fn price(&self) -> Column<i64>;
    async fn print(&self) -> VantageResult<()>;
}

impl ProductTable for Table<SqliteDB, Product> {
    fn price(&self) -> Column<i64> {
        self.get_column("price").unwrap()
    }

    async fn print(&self) -> VantageResult<()> {
        vantage_cli_util::print_table(self).await
    }
}
}

+----+----------------+-------+-------------------+
| id | name           | price | category          |
+----+----------------+-------+-------------------+
| 1  | Cupcake        | 120   | Sweet Treats (3)  |
| 2  | Doughnut       | 135   | Sweet Treats (3)  |
| 3  | Tart           | 220   | Pastries (2)      |
| ...                                              |
+----+----------------+-------+-------------------+

Persistence Abstraction

The files you have written so far — Cargo.toml, product.rs, category.rs, main.rs — are the makings of an extensible business software architecture: scalable, maintainable, testable, and portable across databases without rewriting a single line of business logic.

What you have is four distinct components, each with a clear job:

flowchart TD
    A["<b>Business code</b><br/><code>main.rs</code><br/>minimal, domain-focused"]
    B["<b>Model definitions</b><br/><code>product.rs</code>, <code>category.rs</code><br/>entities, tables, relationships, domain methods"]
    C["<b>Persistence crate</b><br/><code>vantage-sql</code> (or surrealdb, mongodb, csv…)<br/>query builder, CRUD, type system"]
    D["<b>Vantage framework</b><br/><code>vantage-table</code>, <code>vantage-expressions</code>, <code>vantage-types</code><br/>the extensible mechanism that ties it together"]
    A --> B
    B --> C
    C --> D

The majority of your business code can work with data without any knowledge of where it’s stored or how. It operates on typed entities, relationships, and domain methods — nothing else.

The model definitions focus on describing where and how data is stored, but they don’t micro-manage the process. If storage requirements change — a new backend, a schema migration, a cached layer in front — the model is where you make the tweaks. Business code remains untouched.

This separation gives you the ergonomics scripting-language ORMs are known for — concise, domain-focused code — with Rust’s type safety and performance underneath: the abstraction compiles down to direct calls. When storage requirements shift, the change stays contained in the model layer instead of rippling through business code, and the whole architecture travels as a single static binary — server, desktop, or embedded.

A Standalone Axum Server

Chapter 2 ended with a CLI that prints products to stdout. In this chapter we put the same model behind an HTTP API:

curl "http://localhost:3001/categories"

[{ "name": "Sweet Treats" }, { "name": "Pastries" }, { "name": "Breads" }]

# Products in a specific category
curl "http://localhost:3001/categories/1/products"

# Writes
curl -X POST "http://localhost:3001/categories" \
  -H 'content-type: application/json' -d '{"name":"Gluten-Free"}'

A few things about the shape of this API:

• /categories/{id}/products is a nested route — products narrowed by the relationship we defined in chapter 2. The handler for it is the same generic list as /categories, just applied to a scoped table.
• /products and /categories start out on SQLite, same as chapter 2. Toward the end of the chapter we migrate them to MongoDB. That migration is a change to the model file; the handlers and routes don’t know the difference.

The handler functions are each written once and mounted against any Table<Backend, Entity> the router has on hand — one list, one get, one post, one patch, one delete, parameterised over the backend and entity types. Adding another entity is a route registration, not a new handler.

The minimum Axum skeleton

Two pieces: a small adaptation to chapter 2’s entities so they round-trip through HTTP, then the server itself. One new dependency:

cargo add axum

Tokio and serde are already pulled in from earlier chapters. axum is the HTTP framework; it uses the existing serde to encode response bodies as JSON.

Entities. Chapter 2’s Product carried a computed category field, and the Expressions compose callout showed Category gaining a computed title. Both were assembled from subqueries — great for display, but they don’t round-trip cleanly through a POST body because the JSON would try to write columns that don’t exist. For a writable API we replace the computed fields with the plain FK column that sits under them. (See with_expression for the chapter-2 computed-field setup we’re walking away from here.)

src/category.rs:

use serde::{Deserialize, Serialize};
use vantage_sql::prelude::*;
use vantage_types::prelude::*;

use crate::product::Product;

#[entity(SqliteType)]
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct Category {
    pub name: String,
}

impl Category {
    pub fn table(db: SqliteDB) -> Table<SqliteDB, Category> {
        Table::new("category", db)
            .with_id_column("id")
            .with_column_of::<String>("name")
            .with_many("products", "category_id", Product::table)
    }
}
mod product {
    use serde::{Deserialize, Serialize};
    use vantage_sql::prelude::*;
    use vantage_types::prelude::*;
    use crate::Category;
    #[entity(SqliteType)]
    #[derive(Debug, Clone, Default, Serialize, Deserialize)]
    pub struct Product {
        pub name: String,
        pub price: i64,
        #[serde(skip_serializing_if = "Option::is_none", default)]
        pub category_id: Option<String>,
        #[serde(default)]
        pub is_deleted: bool,
    }
    impl Product {
        pub fn table(db: SqliteDB) -> Table<SqliteDB, Product> {
            let is_deleted = Column::<bool>::new("is_deleted");
            Table::new("product", db)
                .with_id_column("id")
                .with_column_of::<String>("name")
                .with_column_of::<i64>("price")
                .with_column_of::<String>("category_id")
                .with_column_of::<bool>("is_deleted")
                .with_condition(is_deleted.eq(false))
                .with_one("category", "category_id", Category::table)
        }
    }
}
fn main() {}

src/product.rs:

use serde::{Deserialize, Serialize};
use vantage_sql::prelude::*;
use vantage_types::prelude::*;

use crate::category::Category;

#[entity(SqliteType)]
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct Product {
    pub name: String,
    pub price: i64,
    #[serde(skip_serializing_if = "Option::is_none", default)]
    pub category_id: Option<String>,
    #[serde(default)]
    pub is_deleted: bool,
}

impl Product {
    pub fn table(db: SqliteDB) -> Table<SqliteDB, Product> {
        let is_deleted = Column::<bool>::new("is_deleted");
        Table::new("product", db)
            .with_id_column("id")
            .with_column_of::<String>("name")
            .with_column_of::<i64>("price")
            .with_column_of::<String>("category_id")
            .with_column_of::<bool>("is_deleted")
            .with_condition(is_deleted.eq(false))
            .with_one("category", "category_id", Category::table)
    }
}
mod category {
    use serde::{Deserialize, Serialize};
    use vantage_sql::prelude::*;
    use vantage_types::prelude::*;
    use crate::Product;
    #[entity(SqliteType)]
    #[derive(Debug, Clone, Default, Serialize, Deserialize)]
    pub struct Category {
        pub name: String,
    }
    impl Category {
        pub fn table(db: SqliteDB) -> Table<SqliteDB, Category> {
            Table::new("category", db)
                .with_id_column("id")
                .with_column_of::<String>("name")
                .with_many("products", "category_id", Product::table)
        }
    }
}
fn main() {}

Two things to notice:

• category_id is now a real column in both the struct and the table, not a computed expression. Same information as before, but clients can read it from GET and supply it on POST.
• is_deleted picks up #[serde(default)] so POST /products bodies don’t have to carry it; new products default to live.

The soft-delete condition from chapter 2 stays — GET /products will still hide rows with is_deleted = true.

Now the server. Replace src/main.rs with the code below, keeping chapter 2’s mod category; and mod product; declarations at the top:

mod product {
    use serde::{Deserialize, Serialize};
    use vantage_sql::prelude::*;
    use vantage_types::prelude::*;
    use crate::category::Category;
    #[entity(SqliteType)]
    #[derive(Debug, Clone, Default, Serialize, Deserialize)]
    pub struct Product {
        pub name: String,
        pub price: i64,
        #[serde(skip_serializing_if = "Option::is_none", default)]
        pub category_id: Option<String>,
        #[serde(default)]
        pub is_deleted: bool,
    }
    impl Product {
        pub fn table(db: SqliteDB) -> Table<SqliteDB, Product> {
            let is_deleted = Column::<bool>::new("is_deleted");
            Table::new("product", db)
                .with_id_column("id")
                .with_column_of::<String>("name")
                .with_column_of::<i64>("price")
                .with_column_of::<String>("category_id")
                .with_column_of::<bool>("is_deleted")
                .with_condition(is_deleted.eq(false))
                .with_one("category", "category_id", Category::table)
        }
    }
}
mod category {
    use serde::{Deserialize, Serialize};
    use vantage_sql::prelude::*;
    use vantage_types::prelude::*;
    use crate::product::Product;
    #[entity(SqliteType)]
    #[derive(Debug, Clone, Default, Serialize, Deserialize)]
    pub struct Category {
        pub name: String,
    }
    impl Category {
        pub fn table(db: SqliteDB) -> Table<SqliteDB, Category> {
            Table::new("category", db)
                .with_id_column("id")
                .with_column_of::<String>("name")
                .with_many("products", "category_id", Product::table)
        }
    }
}
use std::sync::OnceLock;

use axum::{routing::get, Json, Router};
use category::Category;
use vantage_sql::prelude::*;

static DB: OnceLock<SqliteDB> = OnceLock::new();

fn db() -> SqliteDB {
    DB.get().expect("database not initialised").clone()
}

async fn list_categories() -> Json<Vec<Category>> {
    let rows = Category::table(db()).list().await.unwrap();
    Json(rows.into_values().collect())
}

#[tokio::main]
async fn main() -> VantageResult<()> {
    let conn = SqliteDB::connect("sqlite:products.db")
        .await
        .context("Failed to connect to products.db")?;
    DB.set(conn).ok();

    let app = Router::new().route("/categories", get(list_categories));

    let listener = tokio::net::TcpListener::bind("0.0.0.0:3001").await.unwrap();
    axum::serve(listener, app).await.unwrap();

    Ok(())
}

A few notes on what this is doing:

• static DB: OnceLock<SqliteDB>. The handler needs a database handle but we don’t want to open a new connection per request. A program-wide OnceLock is the simplest possible holder — set once in main, read from anywhere.
• Each request calls Category::table(db()) — building the table (columns, relationships) and then lists it.
• The handler returns Json<Vec<Category>> directly. No DTO layer — the entity’s fields become the JSON fields.

Run it:

cargo run

In another terminal:

curl http://localhost:3001/categories

[
  { "name": "Sweet Treats" },
  { "name": "Pastries" },
  { "name": "Breads" }
]

Caching table definitions

We can cache each table definition so it’s built once and handed out by reference on every subsequent call. In src/category.rs, add use std::sync::OnceLock; at the top and wrap the body of Category::table:

use serde::{Deserialize, Serialize};
use vantage_sql::prelude::*;
use vantage_types::prelude::*;
use std::sync::OnceLock;

#[entity(SqliteType)]
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct Category {
    pub name: String,
}
impl Category {
    pub fn table(db: SqliteDB) -> &'static Table<SqliteDB, Category> {
        static CACHE: OnceLock<Table<SqliteDB, Category>> = OnceLock::new();
        CACHE.get_or_init(|| {
            Table::new("category", db)
                .with_id_column("id")
                .with_column_of::<String>("name")
                .with_many("products", "category_id", |db| Product::table(db).clone())
        })
    }
}
#[entity(SqliteType)]
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct Product {
    pub name: String,
    pub price: i64,
    #[serde(skip_serializing_if = "Option::is_none", default)]
    pub category_id: Option<String>,
    #[serde(default)]
    pub is_deleted: bool,
}
impl Product {
    pub fn table(db: SqliteDB) -> &'static Table<SqliteDB, Product> {
        static CACHE: OnceLock<Table<SqliteDB, Product>> = OnceLock::new();
        CACHE.get_or_init(|| {
            let is_deleted = Column::<bool>::new("is_deleted");
            Table::new("product", db)
                .with_id_column("id")
                .with_column_of::<String>("name")
                .with_column_of::<i64>("price")
                .with_column_of::<String>("category_id")
                .with_column_of::<bool>("is_deleted")
                .with_condition(is_deleted.eq(false))
                .with_one("category", "category_id", |db| Category::table(db).clone())
        })
    }
}
fn main() {}

Three changes from chapter 2’s version:

• Return type is &'static Table<...> instead of Table<...>. The cache owns the definition; callers get a shared reference. Most table operations (list, get, insert, replace, delete, get_ref_as) take &self, so they work on the reference without any clone.
• static CACHE: OnceLock<...> lives inside the function. A function-local static is still a program-wide single instance — Rust allows this and it keeps the cache private to the table that owns it.
• The with_many callback is a closure: |db| Product::table(db).clone(). The framework still hands us a db and expects an owned Table<SqliteDB, Product> back, so we call the cached accessor and clone the reference. On the first call this triggers Product’s cache to build; every subsequent call just clones a pre-built definition.

src/product.rs follows the same pattern — wrap the body in OnceLock::get_or_init, change the return type to &'static Table<SqliteDB, Product>, and rewrite with_one as |db| Category::table(db).clone().

The handler in main.rs doesn’t change at all — Category::table(db()) used to return an owned Table, and now it returns a &'static Table that auto-derefs through the same .list() call. Restart the server and hit /categories — the response is identical, but now the table definitions are built on the first request and reused forever after.

#![allow(unused)]
fn main() {
let narrowed = Category::table(db())
    .clone()                      // owned copy of the cached definition
    .with_condition(...)           // now we can narrow it
}

Products of a category

The with_many relationship we kept from chapter 2 lets us serve a nested route — /categories/{id}/products. Before writing the handler, give the relationship a proper name with an extension trait on Table<SqliteDB, Category>. Add this to src/category.rs:

use vantage_sql::prelude::*;
use vantage_types::prelude::*;
#[entity(SqliteType)]
#[derive(Debug, Clone, Default)]
pub struct Category {
    pub name: String,
}
#[entity(SqliteType)]
#[derive(Debug, Clone, Default)]
pub struct Product {
    pub name: String,
}
pub trait CategoryTable {
    fn ref_products(&self) -> Table<SqliteDB, Product>;
}

impl CategoryTable for Table<SqliteDB, Category> {
    fn ref_products(&self) -> Table<SqliteDB, Product> {
        self.get_ref_as("products").unwrap()
    }
}
fn main() {}

Chapter 2 introduced this pattern — a typed, discoverable name instead of the string-and-turbofish call. The .unwrap() is safe here because we’re the ones who registered "products"; a typo surfaces immediately at startup.

Now the handler. Take the cached category table, narrow it to a single id, and traverse the relationship:

mod product {
    use std::sync::OnceLock;
    use serde::{Deserialize, Serialize};
    use vantage_sql::prelude::*;
    use vantage_types::prelude::*;
    use crate::category::Category;
    #[entity(SqliteType)]
    #[derive(Debug, Clone, Default, Serialize, Deserialize)]
    pub struct Product {
        pub name: String,
        pub price: i64,
        #[serde(skip_serializing_if = "Option::is_none", default)]
        pub category_id: Option<String>,
        #[serde(default)]
        pub is_deleted: bool,
    }
    impl Product {
        pub fn table(db: SqliteDB) -> &'static Table<SqliteDB, Product> {
            static CACHE: OnceLock<Table<SqliteDB, Product>> = OnceLock::new();
            CACHE.get_or_init(|| {
                let is_deleted = Column::<bool>::new("is_deleted");
                Table::new("product", db)
                    .with_id_column("id")
                    .with_column_of::<String>("name")
                    .with_column_of::<i64>("price")
                    .with_column_of::<String>("category_id")
                    .with_column_of::<bool>("is_deleted")
                    .with_condition(is_deleted.eq(false))
                    .with_one("category", "category_id", |db| Category::table(db).clone())
            })
        }
    }
}
mod category {
    use std::sync::OnceLock;
    use serde::{Deserialize, Serialize};
    use vantage_sql::prelude::*;
    use vantage_types::prelude::*;
    use crate::product::Product;
    #[entity(SqliteType)]
    #[derive(Debug, Clone, Default, Serialize, Deserialize)]
    pub struct Category {
        pub name: String,
    }
    impl Category {
        pub fn table(db: SqliteDB) -> &'static Table<SqliteDB, Category> {
            static CACHE: OnceLock<Table<SqliteDB, Category>> = OnceLock::new();
            CACHE.get_or_init(|| {
                Table::new("category", db)
                    .with_id_column("id")
                    .with_column_of::<String>("name")
                    .with_many("products", "category_id", |db| Product::table(db).clone())
            })
        }
    }
    pub trait CategoryTable {
        fn ref_products(&self) -> Table<SqliteDB, Product>;
    }
    impl CategoryTable for Table<SqliteDB, Category> {
        fn ref_products(&self) -> Table<SqliteDB, Product> {
            self.get_ref_as("products").unwrap()
        }
    }
}
use std::sync::OnceLock;
use axum::Json;
use category::Category;
use vantage_sql::prelude::*;
static DB: OnceLock<SqliteDB> = OnceLock::new();
fn db() -> SqliteDB {
    DB.get().expect("database not initialised").clone()
}
use axum::extract::Path;
use category::CategoryTable;
use product::Product;

async fn list_category_products(Path(id): Path<i64>) -> Json<Vec<Product>> {
    let id_col = Column::<i64>::new("id");
    let products = Category::table(db())
        .clone()
        .with_condition(id_col.eq(id))
        .ref_products();
    let rows = products.list().await.unwrap();
    Json(rows.into_values().collect())
}
fn main() {}

Three things going on:

• Category::table(db()).clone() — we need an owned Table to chain with_condition onto, so we clone the cached definition — the shape, not any rows.
• with_condition(id_col.eq(id)) — narrows the category table to one row: the one we’re asking for. Nothing hits the database yet.
• ref_products() — traverses the with_many relationship we registered on Category in chapter 2, via get_ref_as. The returned table is Table<SqliteDB, Product>, already scoped to products whose category_id matches the narrowed category set. Chapter 2 walked through what the emitted SQL looks like; it’s the same here.

Register the route in main.rs:

mod product {
    use std::sync::OnceLock;
    use serde::{Deserialize, Serialize};
    use vantage_sql::prelude::*;
    use vantage_types::prelude::*;
    use crate::category::Category;
    #[entity(SqliteType)]
    #[derive(Debug, Clone, Default, Serialize, Deserialize)]
    pub struct Product {
        pub name: String,
        pub price: i64,
        #[serde(skip_serializing_if = "Option::is_none", default)]
        pub category_id: Option<String>,
        #[serde(default)]
        pub is_deleted: bool,
    }
    impl Product {
        pub fn table(db: SqliteDB) -> &'static Table<SqliteDB, Product> {
            static CACHE: OnceLock<Table<SqliteDB, Product>> = OnceLock::new();
            CACHE.get_or_init(|| {
                let is_deleted = Column::<bool>::new("is_deleted");
                Table::new("product", db)
                    .with_id_column("id")
                    .with_column_of::<String>("name")
                    .with_column_of::<i64>("price")
                    .with_column_of::<String>("category_id")
                    .with_column_of::<bool>("is_deleted")
                    .with_condition(is_deleted.eq(false))
                    .with_one("category", "category_id", |db| Category::table(db).clone())
            })
        }
    }
}
mod category {
    use std::sync::OnceLock;
    use serde::{Deserialize, Serialize};
    use vantage_sql::prelude::*;
    use vantage_types::prelude::*;
    use crate::product::Product;
    #[entity(SqliteType)]
    #[derive(Debug, Clone, Default, Serialize, Deserialize)]
    pub struct Category {
        pub name: String,
    }
    impl Category {
        pub fn table(db: SqliteDB) -> &'static Table<SqliteDB, Category> {
            static CACHE: OnceLock<Table<SqliteDB, Category>> = OnceLock::new();
            CACHE.get_or_init(|| {
                Table::new("category", db)
                    .with_id_column("id")
                    .with_column_of::<String>("name")
                    .with_many("products", "category_id", |db| Product::table(db).clone())
            })
        }
    }
    pub trait CategoryTable {
        fn ref_products(&self) -> Table<SqliteDB, Product>;
    }
    impl CategoryTable for Table<SqliteDB, Category> {
        fn ref_products(&self) -> Table<SqliteDB, Product> {
            self.get_ref_as("products").unwrap()
        }
    }
}
use std::sync::OnceLock;
use axum::extract::Path;
use axum::{Json, Router, routing::get};
use category::{Category, CategoryTable};
use product::Product;
use vantage_sql::prelude::*;
static DB: OnceLock<SqliteDB> = OnceLock::new();
fn db() -> SqliteDB {
    DB.get().expect("database not initialised").clone()
}
async fn list_categories() -> Json<Vec<Category>> {
    let rows = Category::table(db()).list().await.unwrap();
    Json(rows.into_values().collect())
}
async fn list_category_products(Path(id): Path<i64>) -> Json<Vec<Product>> {
    let id_col = Column::<i64>::new("id");
    let products = Category::table(db())
        .clone()
        .with_condition(id_col.eq(id))
        .ref_products();
    let rows = products.list().await.unwrap();
    Json(rows.into_values().collect())
}
fn main() {
let app = Router::new()
    .route("/categories", get(list_categories))
    .route("/categories/{id}/products", get(list_category_products));
let _: Router = app;
}

Hit it:

curl http://localhost:3001/categories/1/products

[
  { "name": "Cupcake", "price": 120, "category_id": "1", "is_deleted": false },
  { "name": "Doughnut", "price": 135, "category_id": "1", "is_deleted": false },
  { "name": "Cookies", "price": 199, "category_id": "1", "is_deleted": false }
]

curl http://localhost:3001/categories/2/products

[
  { "name": "Tart", "price": 220, "category_id": "2", "is_deleted": false },
  { "name": "Pie", "price": 299, "category_id": "2", "is_deleted": false }
]

Swap the id in the URL and the products narrow accordingly. The relationship was declared once, back in chapter 2, and we haven’t touched it since — every new nested route reuses the same declaration.

Notice "category_id": "1" in the responses — a string, even though the SQLite column is INTEGER. This is chapter 2’s id convention at work: ids travel as strings through Vantage, so the entity declares category_id: Option<String> and the value converts on its way out of the database.

A generic crud helper

Two handlers so far — list_categories and list_category_products — and they’re already doing the same thing: take a narrowed table, call .list(), return JSON. Adding POST /categories, PATCH /categories/{id}, and DELETE /categories/{id} would mean four more near-identical handlers per entity. That’s not how you scale a codebase.

The five HTTP methods of CRUD all map onto one of two operation shapes:

• List-level at /collection: GET (list all) and POST (create one).
• Item-level at /collection/{id}: GET (read one), PATCH (update), DELETE (remove).

If we can describe the set of rows this endpoint operates on with one closure, the same handler bodies serve every entity. That closure is Fn(SqliteDB, &Params) -> Table<SqliteDB, E> — given the database and whatever path params axum extracted, return the Table to act on. Put it in main.rs:

use std::sync::OnceLock;
use axum::extract::Path;
use axum::http::StatusCode;
use axum::{Json, Router, routing::get};
use serde::{Serialize, de::DeserializeOwned};
use vantage_sql::prelude::*;
use vantage_types::prelude::*;
static DB: OnceLock<SqliteDB> = OnceLock::new();
fn db() -> SqliteDB {
    DB.get().expect("database not initialised").clone()
}
use std::collections::HashMap;
use std::sync::Arc;

type Params = HashMap<String, String>;

fn crud<E, F>(make_table: F) -> Router
where
    F: Fn(SqliteDB, &Params) -> Table<SqliteDB, E> + Send + Sync + 'static,
    E: Entity<AnySqliteType> + Serialize + DeserializeOwned + Send + Sync + 'static,
    <E as vantage_types::TryIntoRecord<AnySqliteType>>::Error: std::fmt::Debug,
{
    let f = Arc::new(make_table);
    Router::new()
        .route(
            "/",
            get({
                let f = f.clone();
                move |p: Option<Path<Params>>| async move {
                    let params = p.map(|Path(p)| p).unwrap_or_default();
                    let rows = f(db(), &params).list().await.unwrap();
                    Json::<Vec<E>>(rows.into_values().collect())
                }
            })
            .post({
                let f = f.clone();
                move |p: Option<Path<Params>>, Json(entity): Json<E>| async move {
                    let params = p.map(|Path(p)| p).unwrap_or_default();
                    let id = f(db(), &params).insert_return_id(&entity).await.unwrap();
                    Json(serde_json::json!({ "id": id }))
                }
            }),
        )
        .route(
            "/{id}",
            get({
                let f = f.clone();
                move |Path(params): Path<Params>| async move {
                    let id = params["id"].clone();
                    let entity = f(db(), &params).get(&id).await.unwrap().unwrap();
                    Json(entity)
                }
            })
            .patch({
                let f = f.clone();
                move |Path(params): Path<Params>, Json(partial): Json<E>| async move {
                    let id = params["id"].clone();
                    let updated = f(db(), &params).patch(&id, &partial).await.unwrap();
                    Json(updated)
                }
            })
            .delete({
                let f = f;
                move |Path(params): Path<Params>| async move {
                    let id = params["id"].clone();
                    f(db(), &params).delete(&id).await.unwrap();
                    StatusCode::NO_CONTENT
                }
            }),
        )
}
fn main() {}

Five handler bodies, each tiny, all generic over the entity type. The last bound, TryIntoRecord<AnySqliteType>::Error: Debug, is what the table’s write path asks of an entity: a record that fails to serialize has to be reportable. Mount crud(...) under a prefix with .nest(...) and every route below it gets the full CRUD verb set for free.

/categories becomes a one-liner:

use std::collections::HashMap;
use std::sync::OnceLock;
use axum::Router;
use serde::{Deserialize, Serialize, de::DeserializeOwned};
use vantage_sql::prelude::*;
use vantage_types::prelude::*;
type Params = HashMap<String, String>;
fn crud<E, F>(make_table: F) -> Router
where
    F: Fn(SqliteDB, &Params) -> Table<SqliteDB, E> + Send + Sync + 'static,
    E: Entity<AnySqliteType> + Serialize + DeserializeOwned + Send + Sync + 'static,
    <E as vantage_types::TryIntoRecord<AnySqliteType>>::Error: std::fmt::Debug,
{
    let _ = make_table;
    Router::new()
}
#[entity(SqliteType)]
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct Category {
    pub name: String,
}
impl Category {
    pub fn table(db: SqliteDB) -> &'static Table<SqliteDB, Category> {
        static CACHE: OnceLock<Table<SqliteDB, Category>> = OnceLock::new();
        CACHE.get_or_init(|| {
            Table::new("category", db)
                .with_id_column("id")
                .with_column_of::<String>("name")
        })
    }
}
fn main() {
let _app: Router = Router::new()
.nest("/categories", crud(|db, _| Category::table(db).clone()))
;
}

That gives us:

The nested /categories/{cat_id}/products route uses the same helper. The closure reads cat_id out of the params map to narrow the set:

use std::collections::HashMap;
use std::sync::OnceLock;
use axum::Router;
use serde::{Deserialize, Serialize, de::DeserializeOwned};
use vantage_sql::prelude::*;
use vantage_types::prelude::*;
type Params = HashMap<String, String>;
fn crud<E, F>(make_table: F) -> Router
where
    F: Fn(SqliteDB, &Params) -> Table<SqliteDB, E> + Send + Sync + 'static,
    E: Entity<AnySqliteType> + Serialize + DeserializeOwned + Send + Sync + 'static,
    <E as vantage_types::TryIntoRecord<AnySqliteType>>::Error: std::fmt::Debug,
{
    let _ = make_table;
    Router::new()
}
#[entity(SqliteType)]
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct Product {
    pub name: String,
}
#[entity(SqliteType)]
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct Category {
    pub name: String,
}
impl Category {
    pub fn table(db: SqliteDB) -> &'static Table<SqliteDB, Category> {
        static CACHE: OnceLock<Table<SqliteDB, Category>> = OnceLock::new();
        CACHE.get_or_init(|| {
            Table::new("category", db)
                .with_id_column("id")
                .with_column_of::<String>("name")
        })
    }
}
pub trait CategoryTable {
    fn id(&self) -> Column<i64>;
    fn ref_products(&self) -> Table<SqliteDB, Product>;
}
impl CategoryTable for Table<SqliteDB, Category> {
    fn id(&self) -> Column<i64> {
        self.get_column("id").unwrap()
    }
    fn ref_products(&self) -> Table<SqliteDB, Product> {
        self.get_ref_as("products").unwrap()
    }
}
fn main() {
let _app: Router = Router::new()
.nest(
    "/categories/{cat_id}/products",
    crud(|db, p| {
        let cat_id: i64 = p["cat_id"].parse().unwrap();
        let mut c = Category::table(db).clone();
        c.add_condition(c.id().eq(cat_id));
        c.ref_products()
    }),
)
;
}

c.id() comes from extending the CategoryTable trait we already set up — add it next to ref_products:

use vantage_sql::prelude::*;
use vantage_types::prelude::*;
#[entity(SqliteType)]
#[derive(Debug, Clone, Default)]
pub struct Category {
    pub name: String,
}
#[entity(SqliteType)]
#[derive(Debug, Clone, Default)]
pub struct Product {
    pub name: String,
}
pub trait CategoryTable {
    fn id(&self) -> Column<i64>;
    fn ref_products(&self) -> Table<SqliteDB, Product>;
}

impl CategoryTable for Table<SqliteDB, Category> {
    fn id(&self) -> Column<i64> {
        self.get_column("id").unwrap()
    }
    fn ref_products(&self) -> Table<SqliteDB, Product> {
        self.get_ref_as("products").unwrap()
    }
}
fn main() {}

Nesting crud under /categories/{cat_id}/products gives a full CRUD surface for products belonging to that category:

Try it:

curl -X POST http://localhost:3001/categories \
  -H 'content-type: application/json' -d '{"name":"Gluten-Free"}'
# {"id":"4"}

curl http://localhost:3001/categories/1/products/1
# {"name":"Cupcake","price":120,"category_id":"1","is_deleted":false}

curl -X PATCH http://localhost:3001/categories/1 \
  -H 'content-type: application/json' -d '{"name":"Sweet Things"}'
# {"name":"Sweet Things"}

curl -X DELETE http://localhost:3001/categories/4
# HTTP/1.1 204 No Content

The inline list_categories and list_category_products functions can be deleted — crud covers them both.

Error handling

Every handler in crud still ends in .unwrap(). The happy path has been fine to demo, but an API that panics on the slightest database hiccup isn’t usable. The worst part isn’t the 500 — it’s what happens on the wire when axum’s request task panics: the connection is dropped and the client sees an empty reply, with no status code and no body to explain.

A proper REST API needs three things:

• Missing resources return 404, not 500.
• Bad JSON bodies return 400 with a useful message.
• Everything else returns 500 but with a structured JSON body, not silence.

Axum already gives us the middle one for free — it rejects malformed Json<E> bodies with 400. The other two come down to converting VantageError into an HTTP response. Add an ApiError type to main.rs:

use axum::Json;
use axum::http::StatusCode;
use vantage_sql::prelude::*;
use axum::response::{IntoResponse, Response};

struct ApiError {
    status: StatusCode,
    message: String,
}

impl IntoResponse for ApiError {
    fn into_response(self) -> Response {
        (
            self.status,
            Json(serde_json::json!({ "error": self.message })),
        )
            .into_response()
    }
}

impl From<VantageError> for ApiError {
    fn from(e: VantageError) -> Self {
        eprintln!("API error: {:?}", e);
        Self {
            status: StatusCode::INTERNAL_SERVER_ERROR,
            message: e.to_string(),
        }
    }
}

fn not_found(id: &str) -> ApiError {
    ApiError {
        status: StatusCode::NOT_FOUND,
        message: format!("not found: {}", id),
    }
}

type ApiResult<T> = Result<T, ApiError>;
fn main() {}

Three pieces matter here:

• IntoResponse makes ApiError returnable from a handler; axum calls into_response() to assemble status, headers, and body.
• From<VantageError> lets us use ? inside a handler — every .await? short-circuits to an ApiError mapped to a 500, which axum will render for us.
• not_found(id) is how we build a 404 explicitly. Vantage’s get returns Option<E>, so we .ok_or_else(|| not_found(&id))? the missing case straight into a 404 — no error-message string-matching, no brittleness.

With that in place, the unwraps inside crud turn into ?, and each handler returns ApiResult<T>:

use std::collections::HashMap;
use std::sync::{Arc, OnceLock};
use axum::extract::Path;
use axum::http::StatusCode;
use axum::response::{IntoResponse, Response};
use axum::{Json, Router, routing::get};
use serde::{Serialize, de::DeserializeOwned};
use vantage_sql::prelude::*;
use vantage_types::prelude::*;
static DB: OnceLock<SqliteDB> = OnceLock::new();
fn db() -> SqliteDB {
    DB.get().expect("database not initialised").clone()
}
type Params = HashMap<String, String>;
struct ApiError {
    status: StatusCode,
    message: String,
}
impl IntoResponse for ApiError {
    fn into_response(self) -> Response {
        (self.status, Json(serde_json::json!({ "error": self.message }))).into_response()
    }
}
impl From<VantageError> for ApiError {
    fn from(e: VantageError) -> Self {
        Self { status: StatusCode::INTERNAL_SERVER_ERROR, message: e.to_string() }
    }
}
type ApiResult<T> = Result<T, ApiError>;
fn crud<E, F>(make_table: F) -> Router
where
    F: Fn(SqliteDB, &Params) -> Table<SqliteDB, E> + Send + Sync + 'static,
    E: Entity<AnySqliteType> + Serialize + DeserializeOwned + Send + Sync + 'static,
    <E as vantage_types::TryIntoRecord<AnySqliteType>>::Error: std::fmt::Debug,
{
    let f = Arc::new(make_table);
    Router::new().route(
        "/",
get({
    let f = f.clone();
    move |p: Option<Path<Params>>| async move {
        let params = p.map(|Path(p)| p).unwrap_or_default();
        let rows = f(db(), &params).list().await?;
        ApiResult::Ok(Json::<Vec<E>>(rows.into_values().collect()))
    }
})
    )
}
fn main() {}

For the GET /{id} handler, convert the Option into a 404 at the handler:

use std::collections::HashMap;
use std::sync::{Arc, OnceLock};
use axum::extract::Path;
use axum::http::StatusCode;
use axum::response::{IntoResponse, Response};
use axum::{Json, Router, routing::get};
use serde::{Serialize, de::DeserializeOwned};
use vantage_sql::prelude::*;
use vantage_types::prelude::*;
static DB: OnceLock<SqliteDB> = OnceLock::new();
fn db() -> SqliteDB {
    DB.get().expect("database not initialised").clone()
}
type Params = HashMap<String, String>;
struct ApiError {
    status: StatusCode,
    message: String,
}
impl IntoResponse for ApiError {
    fn into_response(self) -> Response {
        (self.status, Json(serde_json::json!({ "error": self.message }))).into_response()
    }
}
impl From<VantageError> for ApiError {
    fn from(e: VantageError) -> Self {
        Self { status: StatusCode::INTERNAL_SERVER_ERROR, message: e.to_string() }
    }
}
fn not_found(id: &str) -> ApiError {
    ApiError { status: StatusCode::NOT_FOUND, message: format!("not found: {}", id) }
}
type ApiResult<T> = Result<T, ApiError>;
fn crud<E, F>(make_table: F) -> Router
where
    F: Fn(SqliteDB, &Params) -> Table<SqliteDB, E> + Send + Sync + 'static,
    E: Entity<AnySqliteType> + Serialize + DeserializeOwned + Send + Sync + 'static,
    <E as vantage_types::TryIntoRecord<AnySqliteType>>::Error: std::fmt::Debug,
{
    let f = Arc::new(make_table);
    Router::new().route(
        "/{id}",
get({
    let f = f.clone();
    move |Path(params): Path<Params>| async move {
        let id = params["id"].clone();
        let entity = f(db(), &params)
            .get(id.clone())
            .await?
            .ok_or_else(|| not_found(&id))?;
        ApiResult::Ok(Json(entity))
    }
})
    )
}
fn main() {}

Do the same shape for post, patch, delete — drop the .unwrap(), add ?, wrap the happy path in ApiResult::Ok(...). Handlers that don’t read by id just need the ?.

Try a few error cases:

curl -w "\nstatus=%{http_code}\n" http://localhost:3001/categories/999
# {"error":"not found: 999"}
# status=404

curl -w "\nstatus=%{http_code}\n" -X PATCH http://localhost:3001/categories/999 \
  -H 'content-type: application/json' -d '{"name":"Ghost"}'
# {"error":"Row not found (table: \"category\", id: \"999\")"}
# status=500

curl -w "\nstatus=%{http_code}\n" -X POST http://localhost:3001/categories \
  -H 'content-type: application/json' -d '{not-json}'
# Failed to parse the request body as JSON: ...
# status=400

curl -w "\nstatus=%{http_code}\n" -X DELETE http://localhost:3001/categories/999
# status=204

Missing ids produce 404s with a clean JSON body. The malformed body gets axum’s built-in 400 for free. DELETE on a missing id still returns 204 — vantage’s delete is idempotent, and “the resource is gone” is true whether or not it was ever there. PATCH on a missing id fails with an error whose is_not_found() is true; this handler maps every non-get error to 500, so add Err(e) if e.is_not_found() => not_found(&id) to answer 404 there too.

Pagination and search

Three categories is fine for dev; three thousand would crush any client that naively calls GET /categories and tries to render everything. Real APIs page through long lists and let callers filter by a search term — two query-string features that belong inside crud so every entity gets them.

Axum parses query strings for us via the Query<T> extractor. Add a small struct for the parameters, and a second extractor on the list handler:

use serde::Deserialize;
use axum::extract::Query;
use vantage_table::pagination::Pagination;

#[derive(Deserialize, Default)]
struct ListQuery {
    page: Option<i64>,
    per_page: Option<i64>,
    q: Option<String>,
}
fn main() {}

Inside crud, only the GET / handler changes. It takes both extractors, mutates the narrowed table with whatever the caller asked for, and lists:

use std::collections::HashMap;
use std::sync::{Arc, OnceLock};
use axum::extract::{Path, Query};
use axum::http::StatusCode;
use axum::response::{IntoResponse, Response};
use axum::{Json, Router, routing::get};
use serde::{Deserialize, Serialize, de::DeserializeOwned};
use vantage_sql::prelude::*;
use vantage_table::pagination::Pagination;
use vantage_types::prelude::*;
static DB: OnceLock<SqliteDB> = OnceLock::new();
fn db() -> SqliteDB {
    DB.get().expect("database not initialised").clone()
}
type Params = HashMap<String, String>;
struct ApiError {
    status: StatusCode,
    message: String,
}
impl IntoResponse for ApiError {
    fn into_response(self) -> Response {
        (self.status, Json(serde_json::json!({ "error": self.message }))).into_response()
    }
}
impl From<VantageError> for ApiError {
    fn from(e: VantageError) -> Self {
        Self { status: StatusCode::INTERNAL_SERVER_ERROR, message: e.to_string() }
    }
}
type ApiResult<T> = Result<T, ApiError>;
#[derive(Deserialize, Default)]
struct ListQuery {
    page: Option<i64>,
    per_page: Option<i64>,
    q: Option<String>,
}
fn crud<E, F>(make_table: F) -> Router
where
    F: Fn(SqliteDB, &Params) -> Table<SqliteDB, E> + Send + Sync + 'static,
    E: Entity<AnySqliteType> + Serialize + DeserializeOwned + Send + Sync + 'static,
    <E as vantage_types::TryIntoRecord<AnySqliteType>>::Error: std::fmt::Debug,
{
    let f = Arc::new(make_table);
    Router::new().route(
        "/",
get({
    let f = f.clone();
    move |p: Option<Path<Params>>, Query(q): Query<ListQuery>| async move {
        let params = p.map(|Path(p)| p).unwrap_or_default();
        let mut t = f(db(), &params);
        if q.page.is_some() || q.per_page.is_some() {
            t.set_pagination(Some(Pagination::new(
                q.page.unwrap_or(1),
                q.per_page.unwrap_or(50),
            )));
        }
        if let Some(term) = q.q.as_deref() {
            t = t.with_search(term);
        }
        let rows = t.list().await?;
        ApiResult::Ok(Json::<Vec<E>>(rows.into_values().collect()))
    }
})
    )
}
fn main() {}

Two things happen:

• set_pagination(Some(Pagination::new(page, per_page))) takes a page number and a page size. Vantage applies these as LIMIT … OFFSET … on the SELECT. Missing params fall back to the defaults (page 1, 50 per page) — and if neither is supplied we don’t touch pagination at all, so the unfiltered list still hits the whole set.
• with_search(term) is the same call we used in chapter 2 to add a LIKE filter across the table’s search columns (every column, unless some are flagged searchable). It consumes the table and hands back the narrowed one, hence t = t.with_search(term). Both end up as extra clauses on the query that vantage already compiles for us.

Try it:

curl "http://localhost:3001/categories?page=1&per_page=2"
# [{"name":"Sweet Treats"},{"name":"Pastries"}]

curl "http://localhost:3001/categories?page=2&per_page=2"
# [{"name":"Breads"}]

curl "http://localhost:3001/categories?q=Pastries"
# [{"name":"Pastries"}]

curl "http://localhost:3001/categories?q=e"
# [{"name":"Sweet Treats"},{"name":"Pastries"},{"name":"Breads"}]

The nested route gets these for free — crud is the same function. per_page works on /categories/{cat_id}/products out of the box:

curl "http://localhost:3001/categories/1/products?per_page=2"
# [
#   {"name":"Cupcake","price":120,"category_id":"1","is_deleted":false},
#   {"name":"Doughnut","price":135,"category_id":"1","is_deleted":false}
# ]

Because the closure for the nested mount narrows the table with with_condition before crud applies its own pagination/search, the filters compose cleanly: the category scope stays in effect, and pagination just counts rows within it.

Migrating to MongoDB

Chapter 2 closed on a claim: the model layer isolates business code from storage, so swapping databases is a change to the model, not to the routes, handlers, or business logic. Now we cash the check. /categories, /categories/{id}, and /categories/{cat_id}/products keep their exact URL shape, response bodies, and behaviour — but the data lives in MongoDB instead of SqliteDB.

Start Mongo — Docker is the easiest way:

docker run -d --name mongo-learn -p 27017:27017 mongo:7

Cargo.toml

Drop vantage-sql, add vantage-mongodb:

[dependencies]
axum = "0.8.9"
serde = { version = "1", features = ["derive"] }
serde_json = "1"
tokio = { version = "1", features = ["full"] }
vantage-core = "1.1"
vantage-dataset = "1.1"
vantage-expressions = "1.2"
vantage-mongodb = "2.0"
vantage-table = "2.0"
vantage-types = { version = "1.0", features = ["serde"] }

Entities

Swap the #[entity] type tag from SqliteType to MongoType and the id column name from id to the MongoDB-idiomatic _id. Imports collapse to a single prelude use-line.

src/category.rs:

use std::sync::OnceLock;

use vantage_mongodb::prelude::*;

use crate::product::Product;

#[entity(MongoType)]
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct Category {
    pub name: String,
}

impl Category {
    pub fn table(db: MongoDB) -> &'static Table<MongoDB, Category> {
        static CACHE: OnceLock<Table<MongoDB, Category>> = OnceLock::new();
        CACHE.get_or_init(|| {
            Table::new("category", db)
                .with_id_column("_id")
                .with_column_of::<String>("name")
                .with_many("products", "category_id", |db| Product::table(db).clone())
        })
    }
}
mod product {
    use std::sync::OnceLock;
    use vantage_mongodb::prelude::*;
    use crate::Category;
    #[entity(MongoType)]
    #[derive(Debug, Clone, Default, Serialize, Deserialize)]
    pub struct Product {
        pub name: String,
        pub price: i64,
        #[serde(skip_serializing_if = "Option::is_none", default)]
        pub category_id: Option<String>,
        #[serde(default)]
        pub is_deleted: bool,
    }
    impl Product {
        pub fn table(db: MongoDB) -> &'static Table<MongoDB, Product> {
            static CACHE: OnceLock<Table<MongoDB, Product>> = OnceLock::new();
            CACHE.get_or_init(|| {
                let is_deleted = Column::<bool>::new("is_deleted");
                Table::new("product", db)
                    .with_id_column("_id")
                    .with_column_of::<String>("name")
                    .with_column_of::<i64>("price")
                    .with_column_of::<String>("category_id")
                    .with_column_of::<bool>("is_deleted")
                    .with_condition(is_deleted.eq(false))
                    .with_one("category", "category_id", |db| Category::table(db).clone())
            })
        }
    }
}
fn main() {}

src/product.rs gets the symmetric changes:

use std::sync::OnceLock;

use vantage_mongodb::prelude::*;

use crate::category::Category;

#[entity(MongoType)]
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct Product {
    pub name: String,
    pub price: i64,
    #[serde(skip_serializing_if = "Option::is_none", default)]
    pub category_id: Option<String>,
    #[serde(default)]
    pub is_deleted: bool,
}

impl Product {
    pub fn table(db: MongoDB) -> &'static Table<MongoDB, Product> {
        static CACHE: OnceLock<Table<MongoDB, Product>> = OnceLock::new();
        CACHE.get_or_init(|| {
            let is_deleted = Column::<bool>::new("is_deleted");
            Table::new("product", db)
                .with_id_column("_id")
                .with_column_of::<String>("name")
                .with_column_of::<i64>("price")
                .with_column_of::<String>("category_id")
                .with_column_of::<bool>("is_deleted")
                .with_condition(is_deleted.eq(false))
                .with_one("category", "category_id", |db| Category::table(db).clone())
        })
    }
}
mod category {
    use std::sync::OnceLock;
    use vantage_mongodb::prelude::*;
    use crate::Product;
    #[entity(MongoType)]
    #[derive(Debug, Clone, Default, Serialize, Deserialize)]
    pub struct Category {
        pub name: String,
    }
    impl Category {
        pub fn table(db: MongoDB) -> &'static Table<MongoDB, Category> {
            static CACHE: OnceLock<Table<MongoDB, Category>> = OnceLock::new();
            CACHE.get_or_init(|| {
                Table::new("category", db)
                    .with_id_column("_id")
                    .with_column_of::<String>("name")
                    .with_many("products", "category_id", |db| Product::table(db).clone())
            })
        }
    }
}
fn main() {}

Nothing about the struct shape changed — category_id and is_deleted have been real columns since § 2. The with_* calls line up one-for-one with the SQLite version; only the id column name and the entity’s type tag shift.

The CategoryTable trait from § 3 also updates — MongoDB’s _id is a string, not an integer:

use vantage_mongodb::prelude::*;
#[entity(MongoType)]
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct Category {
    pub name: String,
}
#[entity(MongoType)]
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct Product {
    pub name: String,
}
pub trait CategoryTable {
    fn id(&self) -> Column<String>;
    fn ref_products(&self) -> Table<MongoDB, Product>;
}

impl CategoryTable for Table<MongoDB, Category> {
    fn id(&self) -> Column<String> {
        self.get_column("_id").unwrap()
    }
    fn ref_products(&self) -> Table<MongoDB, Product> {
        self.get_ref_as("products").unwrap()
    }
}
fn main() {}

The nested route’s closure gets a little simpler as a result — no parse step, just narrow by the URL’s cat_id string directly:

use std::collections::HashMap;
use std::sync::OnceLock;
use axum::Router;
use vantage_mongodb::prelude::*;
type Params = HashMap<String, String>;
fn crud<E, F>(make_table: F) -> Router
where
    F: Fn(MongoDB, &Params) -> Table<MongoDB, E> + Send + Sync + 'static,
    E: Entity<AnyMongoType> + Serialize + DeserializeOwned + Send + Sync + 'static,
    <E as vantage_types::TryIntoRecord<AnyMongoType>>::Error: std::fmt::Debug,
{
    let _ = make_table;
    Router::new()
}
#[entity(MongoType)]
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct Product {
    pub name: String,
}
#[entity(MongoType)]
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct Category {
    pub name: String,
}
impl Category {
    pub fn table(db: MongoDB) -> &'static Table<MongoDB, Category> {
        static CACHE: OnceLock<Table<MongoDB, Category>> = OnceLock::new();
        CACHE.get_or_init(|| {
            Table::new("category", db)
                .with_id_column("_id")
                .with_column_of::<String>("name")
        })
    }
}
pub trait CategoryTable {
    fn id(&self) -> Column<String>;
    fn ref_products(&self) -> Table<MongoDB, Product>;
}
impl CategoryTable for Table<MongoDB, Category> {
    fn id(&self) -> Column<String> {
        self.get_column("_id").unwrap()
    }
    fn ref_products(&self) -> Table<MongoDB, Product> {
        self.get_ref_as("products").unwrap()
    }
}
fn main() {
let _router: Router =
crud(|db, p| {
    let mut c = Category::table(db).clone();
    c.add_condition(c.id().eq(p["cat_id"].as_str()));
    c.ref_products()
})
;
}

.eq(&str) works here because vantage-mongodb’s From<&str> for AnyMongoType auto-promotes 24-character hex strings to ObjectId and leaves everything else as String. The comparison fires with the right BSON type whichever _id convention the collection uses.

main.rs

The crud function’s generic bounds shift from Entity<AnySqliteType> to Entity<AnyMongoType> and its Fn(SqliteDB, ...) becomes Fn(MongoDB, ...). Body unchanged.

use std::collections::HashMap;
use axum::Router;
use vantage_mongodb::prelude::*;
type Params = HashMap<String, String>;
fn crud<E, F>(make_table: F) -> Router
where
    F: Fn(MongoDB, &Params) -> Table<MongoDB, E> + Send + Sync + 'static,
    E: Entity<AnyMongoType> + Serialize + DeserializeOwned + Send + Sync + 'static,
    <E as vantage_types::TryIntoRecord<AnyMongoType>>::Error: std::fmt::Debug,
{
    let _ = make_table;
    Router::new()
}
fn main() {}

Connection and id handling are the only handler-level changes:

use std::collections::HashMap;
use vantage_mongodb::prelude::*;
#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
let params = HashMap::from([("id".to_string(), "69e2b9101a552c206f5f8468".to_string())]);
// db() returns MongoDB; connect with URL + database name.
let conn = MongoDB::connect("mongodb://localhost:27017", "learn3")
    .await
    .context("Failed to connect to MongoDB")?;

// item-level ids are MongoId. String → MongoId dispatches to ObjectId when the
// string is a 24-char hex, otherwise stays a plain String, via a `From<String>`
// smart-parse in vantage-mongodb.
let id: MongoId = params["id"].clone().into();
Ok(())
}

The 404 path needs no change at all. Mongo’s get returns Result<Option<E>> just like SQLite’s, so the same .ok_or_else(|| not_found(&id))? maps a missing document to 404 — the promise of the Why not match on the error message? box holds across backends.

Running it

cargo run

The collection is empty, so the first request returns an empty array:

curl http://localhost:3001/categories
# []

POST a few categories — responses carry the auto-generated MongoDB ObjectId as the new id:

SWEETS=$(curl -s -X POST http://localhost:3001/categories \
  -H 'content-type: application/json' -d '{"name":"Sweet Treats"}' \
  | jq -r .id)
echo "$SWEETS"
# 69e2b9101a552c206f5f8468

Create a product in that category by POSTing to the nested route, including the parent id as category_id in the body:

curl -X POST "http://localhost:3001/categories/$SWEETS/products" \
  -H 'content-type: application/json' \
  -d "{\"name\":\"Cupcake\",\"price\":120,\"category_id\":\"$SWEETS\"}"
# {"id":"69e2b9101a552c206f5f846a"}

curl "http://localhost:3001/categories/$SWEETS/products"
# [{"name":"Cupcake","price":120,"category_id":"69e2b9101a552c206f5f8468","is_deleted":false}]

Same URL shape as the SQLite version. Same JSON. Same error codes. Handlers, routing, pagination, filtering — nothing in the request path learned that storage moved from a local file to a document database.

Scaling up: CRUD as a one-liner

The crud function, ApiError, Params, and ListQuery aren’t really tied to this app — they’re generic over any Table<MongoDB, E>. Move them into their own module and main.rs collapses to exactly what it’s about: connecting the database and registering routes. Three files do the work:

src/db.rs (17 lines) — the static DB: OnceLock<MongoDB>, an init(url, db) helper that connects and stores the handle, and a pub fn db() -> MongoDB accessor that hands out cheap MongoDB clones:

mod db {
use std::sync::OnceLock;

use vantage_mongodb::prelude::*;

static DB: OnceLock<MongoDB> = OnceLock::new();

pub async fn init(url: &str, database: &str) -> VantageResult<()> {
    let conn = MongoDB::connect(url, database)
        .await
        .context("Failed to connect to MongoDB")?;
    DB.set(conn).ok();
    Ok(())
}

pub fn db() -> MongoDB {
    DB.get().expect("database not initialised").clone()
}
}
fn main() {}

src/vantage_axum.rs (one file, ~130 lines) — everything HTTP: ApiError, ListQuery, and the crud<E, F> helper. Accepts any entity E that implements Entity<AnyMongoType> plus the usual serde bounds. It’s the pieces from the sections above, collected:

mod db {
    use std::sync::OnceLock;
    use vantage_mongodb::prelude::*;
    static DB: OnceLock<MongoDB> = OnceLock::new();
    pub async fn init(url: &str, database: &str) -> VantageResult<()> {
        let conn = MongoDB::connect(url, database)
            .await
            .context("Failed to connect to MongoDB")?;
        DB.set(conn).ok();
        Ok(())
    }
    pub fn db() -> MongoDB {
        DB.get().expect("database not initialised").clone()
    }
}
mod vantage_axum {
use std::collections::HashMap;
use std::sync::Arc;

use axum::{
    Json, Router,
    extract::{Path, Query},
    http::StatusCode,
    response::{IntoResponse, Response},
    routing::get,
};
use vantage_mongodb::prelude::*;

use crate::db::db;

pub struct ApiError {
    pub status: StatusCode,
    pub message: String,
}

impl IntoResponse for ApiError {
    fn into_response(self) -> Response {
        (
            self.status,
            Json(serde_json::json!({ "error": self.message })),
        )
            .into_response()
    }
}

impl From<VantageError> for ApiError {
    fn from(e: VantageError) -> Self {
        eprintln!("API error: {:?}", e);
        Self {
            status: StatusCode::INTERNAL_SERVER_ERROR,
            message: e.to_string(),
        }
    }
}

fn not_found(id: &str) -> ApiError {
    ApiError {
        status: StatusCode::NOT_FOUND,
        message: format!("not found: {}", id),
    }
}

pub type Params = HashMap<String, String>;
pub type ApiResult<T> = Result<T, ApiError>;

#[derive(Deserialize, Default)]
pub struct ListQuery {
    pub page: Option<i64>,
    pub per_page: Option<i64>,
    pub q: Option<String>,
}

pub fn crud<E, F>(make_table: F) -> Router
where
    F: Fn(MongoDB, &Params) -> Table<MongoDB, E> + Send + Sync + 'static,
    E: Entity<AnyMongoType> + Serialize + DeserializeOwned + Send + Sync + 'static,
    <E as vantage_types::TryIntoRecord<AnyMongoType>>::Error: std::fmt::Debug,
{
    let f = Arc::new(make_table);
    Router::new()
        .route(
            "/",
            get({
                let f = f.clone();
                move |p: Option<Path<Params>>, Query(q): Query<ListQuery>| async move {
                    let params = p.map(|Path(p)| p).unwrap_or_default();
                    let mut t = f(db(), &params);
                    if q.page.is_some() || q.per_page.is_some() {
                        t.set_pagination(Some(Pagination::new(
                            q.page.unwrap_or(1),
                            q.per_page.unwrap_or(50),
                        )));
                    }
                    if let Some(term) = q.q.as_deref() {
                        t = t.with_search(term);
                    }
                    let rows = t.list().await?;
                    ApiResult::Ok(Json::<Vec<E>>(rows.into_values().collect()))
                }
            })
            .post({
                let f = f.clone();
                move |p: Option<Path<Params>>, Json(entity): Json<E>| async move {
                    let params = p.map(|Path(p)| p).unwrap_or_default();
                    let id = f(db(), &params).insert_return_id(&entity).await?;
                    ApiResult::Ok(Json(serde_json::json!({ "id": id })))
                }
            }),
        )
        .route(
            "/{id}",
            get({
                let f = f.clone();
                move |Path(params): Path<Params>| async move {
                    let id = params["id"].clone();
                    let entity = f(db(), &params)
                        .get(id.clone())
                        .await?
                        .ok_or_else(|| not_found(&id))?;
                    ApiResult::Ok(Json(entity))
                }
            })
            .patch({
                let f = f.clone();
                move |Path(params): Path<Params>, Json(partial): Json<E>| async move {
                    let id: MongoId = params["id"].clone().into();
                    let updated = f(db(), &params).patch(id.clone(), &partial).await?;
                    ApiResult::Ok(Json(updated))
                }
            })
            .delete({
                let f = f;
                move |Path(params): Path<Params>| async move {
                    let id: MongoId = params["id"].clone().into();
                    f(db(), &params).delete(id.clone()).await?;
                    ApiResult::Ok(StatusCode::NO_CONTENT)
                }
            }),
        )
}
}
fn main() {}

src/main.rs — now down to ~30 lines. It declares the four modules (mod category;, mod db;, mod product;, mod vantage_axum;), then:

mod db {
    use std::sync::OnceLock;
    use vantage_mongodb::prelude::*;
    static DB: OnceLock<MongoDB> = OnceLock::new();
    pub async fn init(url: &str, database: &str) -> VantageResult<()> {
        let conn = MongoDB::connect(url, database)
            .await
            .context("Failed to connect to MongoDB")?;
        DB.set(conn).ok();
        Ok(())
    }
    pub fn db() -> MongoDB {
        DB.get().expect("database not initialised").clone()
    }
}
mod product {
    use std::sync::OnceLock;
    use vantage_mongodb::prelude::*;
    use crate::category::Category;
    #[entity(MongoType)]
    #[derive(Debug, Clone, Default, Serialize, Deserialize)]
    pub struct Product {
        pub name: String,
        pub price: i64,
        #[serde(skip_serializing_if = "Option::is_none", default)]
        pub category_id: Option<String>,
        #[serde(default)]
        pub is_deleted: bool,
    }
    impl Product {
        pub fn table(db: MongoDB) -> &'static Table<MongoDB, Product> {
            static CACHE: OnceLock<Table<MongoDB, Product>> = OnceLock::new();
            CACHE.get_or_init(|| {
                let is_deleted = Column::<bool>::new("is_deleted");
                Table::new("product", db)
                    .with_id_column("_id")
                    .with_column_of::<String>("name")
                    .with_column_of::<i64>("price")
                    .with_column_of::<String>("category_id")
                    .with_column_of::<bool>("is_deleted")
                    .with_condition(is_deleted.eq(false))
                    .with_one("category", "category_id", |db| Category::table(db).clone())
            })
        }
    }
}
mod category {
    use std::sync::OnceLock;
    use vantage_mongodb::prelude::*;
    use crate::product::Product;
    #[entity(MongoType)]
    #[derive(Debug, Clone, Default, Serialize, Deserialize)]
    pub struct Category {
        pub name: String,
    }
    impl Category {
        pub fn table(db: MongoDB) -> &'static Table<MongoDB, Category> {
            static CACHE: OnceLock<Table<MongoDB, Category>> = OnceLock::new();
            CACHE.get_or_init(|| {
                Table::new("category", db)
                    .with_id_column("_id")
                    .with_column_of::<String>("name")
                    .with_many("products", "category_id", |db| Product::table(db).clone())
            })
        }
    }
    pub trait CategoryTable {
        fn id(&self) -> Column<String>;
        fn ref_products(&self) -> Table<MongoDB, Product>;
    }
    impl CategoryTable for Table<MongoDB, Category> {
        fn id(&self) -> Column<String> {
            self.get_column("_id").unwrap()
        }
        fn ref_products(&self) -> Table<MongoDB, Product> {
            self.get_ref_as("products").unwrap()
        }
    }
}
mod vantage_axum {
    use std::collections::HashMap;
    use std::sync::Arc;
    use axum::{
        Json, Router,
        extract::{Path, Query},
        http::StatusCode,
        response::{IntoResponse, Response},
        routing::get,
    };
    use vantage_mongodb::prelude::*;
    use crate::db::db;
    pub struct ApiError {
        pub status: StatusCode,
        pub message: String,
    }
    impl IntoResponse for ApiError {
        fn into_response(self) -> Response {
            (self.status, Json(serde_json::json!({ "error": self.message }))).into_response()
        }
    }
    impl From<VantageError> for ApiError {
        fn from(e: VantageError) -> Self {
            eprintln!("API error: {:?}", e);
            Self { status: StatusCode::INTERNAL_SERVER_ERROR, message: e.to_string() }
        }
    }
    fn not_found(id: &str) -> ApiError {
        ApiError { status: StatusCode::NOT_FOUND, message: format!("not found: {}", id) }
    }
    pub type Params = HashMap<String, String>;
    pub type ApiResult<T> = Result<T, ApiError>;
    #[derive(Deserialize, Default)]
    pub struct ListQuery {
        pub page: Option<i64>,
        pub per_page: Option<i64>,
        pub q: Option<String>,
    }
    pub fn crud<E, F>(make_table: F) -> Router
    where
        F: Fn(MongoDB, &Params) -> Table<MongoDB, E> + Send + Sync + 'static,
        E: Entity<AnyMongoType> + Serialize + DeserializeOwned + Send + Sync + 'static,
        <E as vantage_types::TryIntoRecord<AnyMongoType>>::Error: std::fmt::Debug,
    {
        let f = Arc::new(make_table);
        Router::new()
            .route(
                "/",
                get({
                    let f = f.clone();
                    move |p: Option<Path<Params>>, Query(q): Query<ListQuery>| async move {
                        let params = p.map(|Path(p)| p).unwrap_or_default();
                        let mut t = f(db(), &params);
                        if q.page.is_some() || q.per_page.is_some() {
                            t.set_pagination(Some(Pagination::new(
                                q.page.unwrap_or(1),
                                q.per_page.unwrap_or(50),
                            )));
                        }
                        if let Some(term) = q.q.as_deref() {
                            t = t.with_search(term);
                        }
                        let rows = t.list().await?;
                        ApiResult::Ok(Json::<Vec<E>>(rows.into_values().collect()))
                    }
                })
                .post({
                    let f = f.clone();
                    move |p: Option<Path<Params>>, Json(entity): Json<E>| async move {
                        let params = p.map(|Path(p)| p).unwrap_or_default();
                        let id = f(db(), &params).insert_return_id(&entity).await?;
                        ApiResult::Ok(Json(serde_json::json!({ "id": id })))
                    }
                }),
            )
            .route(
                "/{id}",
                get({
                    let f = f.clone();
                    move |Path(params): Path<Params>| async move {
                        let id = params["id"].clone();
                        let entity = f(db(), &params)
                            .get(id.clone())
                            .await?
                            .ok_or_else(|| not_found(&id))?;
                        ApiResult::Ok(Json(entity))
                    }
                })
                .patch({
                    let f = f.clone();
                    move |Path(params): Path<Params>, Json(partial): Json<E>| async move {
                        let id: MongoId = params["id"].clone().into();
                        let updated = f(db(), &params).patch(id.clone(), &partial).await?;
                        ApiResult::Ok(Json(updated))
                    }
                })
                .delete({
                    let f = f;
                    move |Path(params): Path<Params>| async move {
                        let id: MongoId = params["id"].clone().into();
                        f(db(), &params).delete(id.clone()).await?;
                        ApiResult::Ok(StatusCode::NO_CONTENT)
                    }
                }),
            )
    }
}
use axum::Router;
use category::{Category, CategoryTable};
use vantage_axum::crud;
use vantage_mongodb::prelude::*;

#[tokio::main]
async fn main() -> VantageResult<()> {
    db::init("mongodb://localhost:27017", "learn3").await?;

    let app = Router::new()
        .nest("/categories", crud(|db, _| Category::table(db).clone()))
        .nest(
            "/categories/{cat_id}/products",
            crud(|db, p| {
                let mut c = Category::table(db).clone();
                c.add_condition(c.id().eq(p["cat_id"].as_str()));
                c.ref_products()
            }),
        );

    let listener = tokio::net::TcpListener::bind("0.0.0.0:3001").await.unwrap();
    axum::serve(listener, app).await.unwrap();

    Ok(())
}

Adding a new entity to this server is now two things:

1. Write a Table::new(...) constructor for it — declarative, one function, same shape we learned in chapter 2.
2. Mount it.

use std::collections::HashMap;
use std::sync::OnceLock;
use axum::Router;
use vantage_mongodb::prelude::*;
type Params = HashMap<String, String>;
fn crud<E, F>(make_table: F) -> Router
where
    F: Fn(MongoDB, &Params) -> Table<MongoDB, E> + Send + Sync + 'static,
    E: Entity<AnyMongoType> + Serialize + DeserializeOwned + Send + Sync + 'static,
    <E as vantage_types::TryIntoRecord<AnyMongoType>>::Error: std::fmt::Debug,
{
    let _ = make_table;
    Router::new()
}
#[entity(MongoType)]
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct Widget {
    pub name: String,
}
impl Widget {
    pub fn table(db: MongoDB) -> &'static Table<MongoDB, Widget> {
        static CACHE: OnceLock<Table<MongoDB, Widget>> = OnceLock::new();
        CACHE.get_or_init(|| {
            Table::new("widget", db)
                .with_id_column("_id")
                .with_column_of::<String>("name")
        })
    }
}
fn main() {
let _app: Router = Router::new()
.nest("/widgets", crud(|db, _| Widget::table(db).clone()))
;
}

The full surface: GET list, POST create, GET /{id}, PATCH /{id}, DELETE /{id}, pagination via ?page=&per_page=, full-text search via ?q=, 404s for missing ids, 400s for malformed bodies, structured JSON errors. No new handler code. No per-entity error mapping. No per-entity query-param struct. One line per entity.

Every route plays by the same rules because every route is served by the same crud — the single description of “what this route does” lives in the entity’s Table definition, and the HTTP boundary just routes to it. That is the “one description, many operations” principle from chapter 2’s Table carried all the way to the wire, unbroken.

The vantage_axum module is generic enough to lift directly into a larger codebase — it has no knowledge of Category, Product, or your particular routes. Drop it into your own binary, give it a db() accessor, write entity files, mount routes.

One thing it deliberately doesn’t do is authentication — every endpoint here is anonymous. Auth belongs to the HTTP layer: wrap the Router in your usual tower middleware; Vantage stays out of it.

Vista — the Universal Data Handle

Chapters 1–3 built a typed data layer: Table<SqliteDB, Product>, conditions, relationships, CRUD. That layer is great when you know the entity at compile time. But step 3’s generic crud() helper already showed the limitation — it could only exist because we erased the entity type with Serialize + DeserializeOwned and went through JSON.

A CLI that lists “any table from any backend”, a web admin that draws forms from a YAML schema, a UI data grid that works with whatever you point it at — none of these know your Product struct. They need a handle that carries its own schema, speaks a single value type, and works regardless of which database sits underneath.

That handle is Vista.

What Vista actually is

A Vista wraps a typed Table<DB, E> and erases both the backend and the entity. All data flows through Record<CborValue> — an ordered map of string keys to CBOR values. All schema lives on the Vista itself: columns (with types and flags), references, id column, and a set of capabilities.

Think of the progression:

Table<SqliteDB, Product>   — typed entity, typed backend, compile-time safe
Vista                      — fully erased: schema-bearing, CborValue, no generics

Vista trades away compile-time knowledge for runtime flexibility: everything is a string key and a CBOR value, and the schema travels with the handle rather than with your types. Wrapping a typed Table — this chapter’s path — is also not the only way to get one: every driver ships a VistaFactory that can materialize a Vista from a declarative YAML spec, with Rhai scripts for the expressions YAML can’t state. That path has its own guide — Config-Driven Vistas — and this chapter stays on the typed one. Scripts read and write Vistas through their own short vocabulary (table("product").where(…)), taught in Scripting with Rhai; the methods below are the Rust side of the same operations.

Wrapping a typed Table

Each backend ships a factory that turns a typed table into a Vista. For SQLite, the database hands you one:

use vantage_sql::prelude::*;
use vantage_types::prelude::*;
use vantage_vista::prelude::*;
#[entity(SqliteType)]
#[derive(Debug, Clone, Default)]
struct Product { name: String, price: i64 }
impl Product {
    fn table(db: SqliteDB) -> Table<SqliteDB, Product> {
        Table::new("product", db)
            .with_id_column("id")
            .with_column_of::<String>("name")
            .with_column_of::<i64>("price")
            .with_column_of::<i64>("category_id")
    }
}
#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
let db = SqliteDB::connect("sqlite:products.db").await.context("connect")?;
let table = Product::table(db.clone());
let vista: Vista = db.vista_factory().from_table(table)?;
Ok(())
}

That’s it. The factory harvests columns, id field, title fields, and references from the typed table definition you already built in chapter 2. No extra mapping code.

For MongoDB the factory is a MongoVistaFactory, for AWS an AwsVistaFactory — same shape, different import. The resulting Vista is the same type regardless of which factory produced it.

This chapter’s snippets read products.db, the file chapters 1–3 built, so they compile but are not run by the book’s tests.

Reading schema

Once you have a Vista, everything is runtime introspection:

use vantage_sql::prelude::*;
use vantage_types::prelude::*;
use vantage_vista::prelude::*;
#[entity(SqliteType)]
#[derive(Debug, Clone, Default)]
struct Product { name: String, price: i64 }
#[entity(SqliteType)]
#[derive(Debug, Clone, Default)]
struct Category { name: String }
impl Product {
    fn table(db: SqliteDB) -> Table<SqliteDB, Product> {
        Table::new("product", db)
            .with_id_column("id")
            .with_column_of::<String>("name")
            .with_column_of::<i64>("price")
            .with_column_of::<i64>("category_id")
            .with_one("category", "category_id", Category::table)
    }
}
impl Category {
    fn table(db: SqliteDB) -> Table<SqliteDB, Category> {
        Table::new("category", db)
            .with_id_column("id")
            .with_column_of::<String>("name")
            .with_many("products", "category_id", Product::table)
    }
}
#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
let db = SqliteDB::connect("sqlite:products.db").await.context("connect")?;
let vista = db.vista_factory().from_table(Product::table(db.clone()))?;
// Columns — name, original type, flags
for (name, column) in vista.columns() {
    let ty = column.original_type.as_deref().unwrap_or("?");
    println!("{name}: {ty}");
}
// price: i64
// …

// ID column
if let Some(id_column) = vista.id_column() {
    println!("id column: {id_column}");
}

// Title columns — the human-readable ones
for title in vista.title_columns() {
    println!("title: {title}");
}

// References — relationships declared on the typed table
for (name, reference) in vista.references() {
    println!("ref: {name} ({})", reference.kind.as_str());
}
// ref: category (has_one)
Ok(())
}

Column carries an original_type (the Rust type name the typed column was declared with, or None when nothing declares one) and a set of ColumnFlags. The flags the data layer acts on:

Any other name is kept as ColumnFlag::Other for the layer that reads it — a UI’s presentation flag, for instance. Vista itself never interprets those.

Narrowing

Vista doesn’t carry its own condition type. Instead, it delegates to a TableShell — the per-driver executor every Vista wraps, and the piece each factory actually builds — which translates the value into whatever the backend speaks:

use vantage_sql::prelude::*;
use vantage_types::prelude::*;
use vantage_vista::prelude::*;
#[entity(SqliteType)]
#[derive(Debug, Clone, Default)]
struct Product { name: String, price: i64 }
impl Product {
    fn table(db: SqliteDB) -> Table<SqliteDB, Product> {
        Table::new("product", db)
            .with_id_column("id")
            .with_column_of::<String>("name")
            .with_column_of::<i64>("price")
            .with_column_of::<i64>("category_id")
    }
}
#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
let db = SqliteDB::connect("sqlite:products.db").await.context("connect")?;
let vista = db.vista_factory().from_table(Product::table(db.clone()))?;
let cakes = vista.clone().with_condition("category_id", FilterOp::Eq, 1)?;

let rows = cakes.list_values().await?;
// Only products with category_id == 1; `vista` still holds every product
Ok(())
}

The value converts into a CborValue. The driver’s shell translates the condition into a native one — an Expression for SQL, a bson::Document for MongoDB, an AwsCondition for AWS — and pushes it onto the wrapped table. FilterOp has the other comparisons too (Gt, In, …); every shell takes Eq, and the rest need the FilterOps capability.

Why with_* consumes

A Vista is a value, the same as the Table it wraps. It owns its query state, so vista.clone().with_condition(..) hands back a new, smaller set and leaves vista exactly as it was. That’s the property generic code leans on: a function that receives a Vista can narrow it for its own purposes without changing what the caller sees, and a cached “all products” Vista can be the starting point for every request that needs a subset.

The with_* methods take self and return Result<Self>, so they chain:

use vantage_sql::prelude::*;
use vantage_types::prelude::*;
use vantage_vista::prelude::*;
#[entity(SqliteType)]
#[derive(Debug, Clone, Default)]
struct Product { name: String, price: i64 }
impl Product {
    fn table(db: SqliteDB) -> Table<SqliteDB, Product> {
        Table::new("product", db)
            .with_id_column("id")
            .with_column_of::<String>("name")
            .with_column_of::<i64>("price")
            .with_column_of::<i64>("category_id")
    }
}
#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
let db = SqliteDB::connect("sqlite:products.db").await.context("connect")?;
let vista = db.vista_factory().from_table(Product::table(db.clone()))?;
let v = vista
    .clone()
    .with_condition("category_id", FilterOp::Eq, 1)?
    .with_order("price", SortDirection::Descending)?;
Ok(())
}

When you hold a Vista of your own and want to change it in place — a UI handle whose search box the user keeps editing — the &mut self methods do the same work: add_condition, set_search, set_order. They exist for the owner of the value, not for narrowing someone else’s.

A narrowed Vista is a different, smaller set — not a view you toggle filters on. Each clause the shell accepted is recorded on query(), so the layers above (the cache in the next chapter, for one) can tell two narrowed Vistas apart.

Narrowing by id

A common pattern — “I have an id and want the row”:

use vantage_sql::prelude::*;
use vantage_types::prelude::*;
use vantage_vista::prelude::*;
#[entity(SqliteType)]
#[derive(Debug, Clone, Default)]
struct Product { name: String, price: i64 }
impl Product {
    fn table(db: SqliteDB) -> Table<SqliteDB, Product> {
        Table::new("product", db)
            .with_id_column("id")
            .with_column_of::<String>("name")
            .with_column_of::<i64>("price")
            .with_column_of::<i64>("category_id")
    }
}
#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
let db = SqliteDB::connect("sqlite:products.db").await.context("connect")?;
let vista = db.vista_factory().from_table(Product::table(db.clone()))?;
let row = vista.clone().with_id(7)?.get_some_value().await?;
// Some(("7", record)) when the row exists
Ok(())
}

with_id reads the id column name from the schema and adds the equality condition for you.

Search and ordering

use vantage_sql::prelude::*;
use vantage_types::prelude::*;
use vantage_vista::prelude::*;
#[entity(SqliteType)]
#[derive(Debug, Clone, Default)]
struct Product { name: String, price: i64 }
impl Product {
    fn table(db: SqliteDB) -> Table<SqliteDB, Product> {
        Table::new("product", db)
            .with_id_column("id")
            .with_column_of::<String>("name")
            .with_column_of::<i64>("price")
            .with_column_of::<i64>("category_id")
    }
}
#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
let db = SqliteDB::connect("sqlite:products.db").await.context("connect")?;
let vista = db.vista_factory().from_table(Product::table(db.clone()))?;
// Quicksearch — fans across columns flagged Searchable — then sort by column
let v = vista
    .clone()
    .with_search("tart")?
    .with_order("price", SortDirection::Descending)?;

let rows = v.list_values().await?;
Ok(())
}

Both have replace semantics — a second search drops the first. Both return an error if the driver doesn’t support them, and ordering also refuses a column that isn’t flagged Orderable. Check capabilities first, or try and handle the error.

Generic code branches on the capabilities instead, here on a Vista it owns:

use vantage_sql::prelude::*;
use vantage_types::prelude::*;
use vantage_vista::prelude::*;
#[entity(SqliteType)]
#[derive(Debug, Clone, Default)]
struct Product { name: String, price: i64 }
impl Product {
    fn table(db: SqliteDB) -> Table<SqliteDB, Product> {
        Table::new("product", db)
            .with_id_column("id")
            .with_column_of::<String>("name")
            .with_column_of::<i64>("price")
    }
}
#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
let db = SqliteDB::connect("sqlite:products.db").await.context("connect")?;
let vista = db.vista_factory().from_table(Product::table(db.clone()))?;
let mut v = vista.clone();
if v.capabilities().has(Capability::Search) {
    v.set_search(Some("query"))?;
}
if v.capabilities().has(Capability::Order) {
    v.set_order(Some(("name", SortDirection::Ascending)))?;
}
Ok(())
}

Reading windows

Vista has two ways to read part of a set. Not every driver supports both — capabilities tell you which to use.

A window by offset: fetch_window

Random access — any [offset, offset + limit) slice of the set, in its order:

use vantage_sql::prelude::*;
use vantage_types::prelude::*;
use vantage_vista::prelude::*;
#[entity(SqliteType)]
#[derive(Debug, Clone, Default)]
struct Product { name: String, price: i64 }
impl Product {
    fn table(db: SqliteDB) -> Table<SqliteDB, Product> {
        Table::new("product", db)
            .with_id_column("id")
            .with_column_of::<String>("name")
            .with_column_of::<i64>("price")
    }
}
#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
let db = SqliteDB::connect("sqlite:products.db").await.context("connect")?;
let vista = db.vista_factory().from_table(Product::table(db.clone()))?;
let first = vista.fetch_window(0, 25).await?;    // rows 0..25
let second = vista.fetch_window(25, 25).await?;  // rows 25..50

println!("{} rows, total {:?}", first.rows.len(), first.total);
Ok(())
}

A Page holds the rows and, when the fetch learned it, the set’s total — None means “didn’t say”, never zero. The leading window (offset == 0) works on every driver that lists; any other offset needs the Window capability. SQL databases and MongoDB have it, anything with LIMIT … OFFSET.

A cursor: fetch_cursor

Forward only — each call returns up to limit rows and a token for the next call:

use vantage_sql::prelude::*;
use vantage_types::prelude::*;
use vantage_vista::prelude::*;
#[entity(SqliteType)]
#[derive(Debug, Clone, Default)]
struct Product { name: String, price: i64 }
impl Product {
    fn table(db: SqliteDB) -> Table<SqliteDB, Product> {
        Table::new("product", db)
            .with_id_column("id")
            .with_column_of::<String>("name")
            .with_column_of::<i64>("price")
    }
}
#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
let db = SqliteDB::connect("sqlite:products.db").await.context("connect")?;
let vista = db.vista_factory().from_table(Product::table(db.clone()))?;
if vista.capabilities().has(Capability::Cursor) {
    let (page, token) = vista.fetch_cursor(None, 100).await?;   // first page
    let (page, token) = vista.fetch_cursor(token, 100).await?;  // second page
    // token is None when the set is exhausted
}
Ok(())
}

The token is opaque — its shape is driver-private (a DynamoDB LastEvaluatedKey, a REST nextToken, the last key S3 returned). Just round-trip it. Requires the Cursor capability, which SQLite doesn’t advertise and the S3 listing in the next chapter does.

Traversing references

Same-persistence references (the with_many / with_one from chapter 2) come through automatically. Given a parent row, get_ref resolves the relationship:

use vantage_sql::prelude::*;
use vantage_types::prelude::*;
use vantage_vista::prelude::*;
#[entity(SqliteType)]
#[derive(Debug, Clone, Default)]
struct Product { name: String, price: i64 }
#[entity(SqliteType)]
#[derive(Debug, Clone, Default)]
struct Category { name: String }
impl Product {
    fn table(db: SqliteDB) -> Table<SqliteDB, Product> {
        Table::new("product", db)
            .with_id_column("id")
            .with_column_of::<String>("name")
            .with_column_of::<i64>("price")
            .with_column_of::<i64>("category_id")
            .with_one("category", "category_id", Category::table)
    }
}
impl Category {
    fn table(db: SqliteDB) -> Table<SqliteDB, Category> {
        Table::new("category", db)
            .with_id_column("id")
            .with_column_of::<String>("name")
            .with_many("products", "category_id", Product::table)
    }
}
#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
let db = SqliteDB::connect("sqlite:products.db").await.context("connect")?;
let categories = db.vista_factory().from_table(Category::table(db.clone()))?;

// Given a category row, traverse to its products
if let Some((_, category)) = categories.get_some_value().await? {
    let products = categories.get_ref("products", Some(&category))?;
    for (id, row) in products.list_values().await? {
        println!("  {id} — {:?}", row.get("name"));
    }
}
Ok(())
}

get_ref takes the parent row as a Record<CborValue>. It reads the join field (e.g. category_id) out of that row to build the condition on the target, so a traversal starts from a row: fetch the parent with get_some_value() or list_values(), then traverse from each row. With None in place of the row, you get the bare target — the set a new related row goes into.

get_ref resolves two kinds of relation:

1. Contained relations — records embedded in a column of the parent row (an order’s lines array) come back as an editable sub-Vista.
2. Foreign-key references — the shell forwards to the with_one / with_many relationships declared on the typed table, and re-wraps the narrowed target as a fresh Vista.

Either way the result is another Vista, so traversal chains without ever leaving the universal surface. Traversal needs the Traverse capability.

Cross-backend references

What if your categories live in PostgreSQL and your products in MongoDB? That is deliberately not a Vista’s job — a Vista honestly describes one backend, and there is no engine that could execute a join across two. Cross-persistence traversal lives one layer up, in a Catalog: register a loader per model name, and reference.narrow(catalog.vista(&reference.target)?, &row) builds the target in its backend and narrows it by values read from the parent row. The consumer still just receives a Vista — see Config-Driven Vistas for the catalog in action.

Capabilities — the explicit contract

VistaCapabilities is a set of Capability values. The driver puts in exactly what it implements:

use vantage_sql::prelude::*;
use vantage_types::prelude::*;
use vantage_vista::prelude::*;
#[entity(SqliteType)]
#[derive(Debug, Clone, Default)]
struct Product { name: String, price: i64 }
impl Product {
    fn table(db: SqliteDB) -> Table<SqliteDB, Product> {
        Table::new("product", db)
            .with_id_column("id")
            .with_column_of::<String>("name")
            .with_column_of::<i64>("price")
    }
}
#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
let db = SqliteDB::connect("sqlite:products.db").await.context("connect")?;
let vista = db.vista_factory().from_table(Product::table(db.clone()))?;
let caps = vista.capabilities();
for capability in caps.iter() {
    println!("{capability}");
}
// order, search, filter_ops, window, traverse, insert, update, delete, …

if caps.has(Capability::Insert) {
    println!("writable");
}
Ok(())
}

Each capability names one method family: Insert, Update, Delete, Import, Subscribe, Order, Search, FilterOps, NativeConditions, Window, Cursor, Traverse and ConfineWrites (writes through a narrowed Vista stay inside its set). Listing, getting a row, counting and equality conditions have no capability: every driver answers them.

A CSV file advertises Window and Traverse — it’s read-only. SQLite advertises reads, writes, Window and Traverse, but not Cursor. An S3 listing advertises Cursor and Traverse: no random access, no search, no writes.

Putting it together

Here’s a small function that takes any Vista and prints a summary — works with any backend:

#![allow(unused)]
fn main() {
use vantage_core::Result as VantageResult;
use vantage_vista::prelude::*;
async fn print_vista(vista: &Vista) -> VantageResult<()> {
    let columns: Vec<&String> = vista.columns().keys().collect();

    // Header
    print!("  {:>12} ", vista.id_column().unwrap_or("id"));
    for col in &columns {
        print!("{col:>16} ");
    }
    println!();

    // Count — every driver answers it
    println!("  ({} rows)", vista.count().await?);

    // Rows
    let rows = vista.list_values().await?;
    for (id, record) in &rows {
        print!("  {id:>12} ");
        for col in &columns {
            let val = record
                .get(col.as_str())
                .map(|v| format!("{v:?}"))
                .unwrap_or_default();
            print!("{val:>16} ");
        }
        println!();
    }
    Ok(())
}
}

No generics. No entity type. No backend knowledge. print_vista works with a SQLite Vista, a MongoDB Vista, an AWS Vista — anything the framework can produce.

What we covered

Choose Your Path

You now have a Vista — a runtime handle over your data that works the same no matter what’s behind it. Everything up to here was the foundation. Everything past here is the reactive stack: a local cache, an event bus, and watchable views that update as the data changes.

That stack is backend-agnostic, and the rest of the guide proves it by forking here into two paths. Both build the same layers — Dio, Lens, Augmentation, Scenery, and a watch-streaming HTTP server — and differ only in the backend underneath. Pick the one shaped like your problem; the code you write is nearly identical either way.

Which one is yours?

Why the guide leads with S3

If you own a SQL database, an S3 bucket may look like a strange place to start teaching caching and reactivity. That is deliberate. A slow, read-only, capability-poor backend makes the value of the layer above visible: every millisecond the cache saves, and every sort the backend cannot do but Diorama can, is obvious. A capable SQL database would hide the very problem the reactive stack solves — so Path A teaches it where it shows, and Path B proves it was never about S3.

Both paths converge on the same idea: once a Dio wraps a Vista, nothing above it knows or cares what the backend is. Read whichever path is yours — or both, and watch the code stay the same while the backend changes underneath it.

Dio & Lens — Caching and Events

Chapter 4 gave you Vista — a universal handle that works with any backend. But every call still hits the backend. That was easy to forgive with a local SQLite file; this chapter picks a data source where it hurts: a real cloud API, hundreds of milliseconds away, read-only, and unable to sort or search. We’ll build a small CLI around it — an inventory of a public S3 bucket, with a cache.

Diorama (vantage-diorama) is the layer that makes this pleasant. It sits between your Vista and whatever consumes it, and does three things:

1. Transparent caching. Keep a local copy of your data. Reads come from cache, not from the master — a listing that costs seconds over the network costs microseconds after the first fetch.
2. Capability injection. A Vista backed by S3 can’t sort, search, or read a window at an offset. Diorama caches the dataset locally and answers those queries from cache — the consumer sees a richer handle than the backend actually offers.
3. Custom write routing. Writes don’t have to go to the master — useful when the master can’t take them at all. You decide what a write means: a queue, a different store, an API call.

Table, Vista, Dio

This is the third handle to the same records — and like the query-vs-table comparison back in chapter 2, each one trades something away for something new:

Caching

Diorama caches at two levels. Page segments hold windows of an ordered query result — they’re what lets a viewport scroll a huge listing without re-asking the master, and they power the loading strategies you’ll meet in chapter 7. Beneath them sits the key/value record store — one entry per record id, plain and dumb. We keep it simple for now: everything in this chapter runs on the record store alone.

A Dio owns exactly the two blocks above: a master Vista — the source of truth, however far away — and a cache, the local copy. What it deliberately does not decide is policy: when the cache fills, when it goes stale, what a write means. There are too many valid answers for one default — seed once and keep forever, re-fetch on a timer, reconcile from a push stream, write through to the master, write somewhere else entirely. That’s what the Lens is for: it describes how the cache is used. The simplest useful Lens pumps the master into the cache once, when the Dio is built:

use serde::{Deserialize, Serialize};
use vantage_aws::prelude::*;
use vantage_diorama::prelude::*;
use vantage_diorama::{Cache, Loader, LoaderCx};
use vantage_vista::prelude::*;
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
struct File {
    #[serde(rename = "Key")]
    filename: String,
    #[serde(rename = "Size")]
    size: String,
}
impl File {
    fn table(aws: AwsAccount, bucket: &str, prefix: &str) -> Table<AwsAccount, File> {
        Table::new(
            "restxml/Contents@continuation-token=NextContinuationToken:s3/GET /{Bucket}?list-type=2",
            aws,
        )
        .with_id_column("Key")
        .with_column_of::<String>("Size")
        .with_condition(eq("Bucket", bucket))
        .with_condition(eq("prefix", prefix))
        .with_condition(eq("max-keys", 100))
    }
}
#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
let aws = AwsAccount::public("us-east-1");
let master = aws
    .vista_factory()
    .from_table(File::table(aws, "noaa-ghcn-pds", "csv/by_station/GM"))?;
let lens = Lens::builder()
    .cache(Cache::redb("cache.redb"))
    .loader(Loader::eager(|cx: LoaderCx| async move {
        // Pump: read everything from the master, make the cache hold exactly that.
        let rows = cx.master().list_values().await?;
        cx.reconcile(rows).await
    }))
    .build()?;

let dio = lens.dio(master).build().await?;   // runs the eager start, returns a warm Dio
Ok(())
}

With the Dio in place, there are two ways to consume it. The first is the facade Vista — dio.cached() — for proactive querying: you ask, it answers, exactly like the backend’s own Vista from chapter 4. Same interface, same records — except the answers come from the cache, and the capability set can be wider than the master’s: the facade searches, orders and reads windows whatever the master can do, and a read-only master gains writes when the Lens routes them somewhere. The second way in is the Scenery — a standing, reactive view that keeps itself current and tells you when it changed. UIs bind to sceneries; chapter 7 lives there.

The facade Vista

Hand dio.cached() to chapter 4’s print_vista and it just works — it never learns a cache is underneath. Each call returns a fresh, independent Vista, so you narrow it with with_* like any other and no other consumer sees the change.

What makes the facade interesting is its capability set. Chapter 4’s contract was honest but rigid: whatever the backend can’t do, your application can’t have. The facade advertises what the pipeline can do: Search, Order, FilterOps and Window always (a clause the master takes is pushed down, and the rest is evaluated over the cached rows), Subscribe because the event bus backs it, and the writes the master can take — or all of them once a Lens writer gives writes somewhere to go. The honesty contract still holds; it just describes a different engine.

The project: a weather-station inventory

NOAA publishes its daily climate archive — GHCN, one CSV per weather station — as a public S3 bucket in the AWS Open Data registry. It’s a perfect Diorama subject: listing it is slow, every listing request is paid again on every run, and the API can’t sort or search. We’ll grow one small tool across three chapters:

• This chapter — a CLI that lists the station files from a persistent local cache.
• Chapter 6 — augmentation: each file gains columns computed from its contents (how many readings, how recent).
• Chapter 7 — a live terminal UI that scrolls the whole archive and fetches per-file data for exactly the rows on screen.

Set up the project:

cargo new learn-4 && cd learn-4
cargo add serde --features derive
cargo add tokio --features full
cargo add vantage-aws vantage-diorama vantage-vista

vantage-aws is the S3/DynamoDB/IAM driver — it signs (or deliberately doesn’t sign) requests itself, so there’s no AWS SDK in the tree. vantage-diorama is what this chapter is about. The code splits into two files: files.rs holds the table definition, main.rs uses it.

Every snippet that talks to S3 or opens cache.redb compiles in the book’s tests but doesn’t run there; the definitions without either do run.

files.rs — the listing as a table

One row per file in the bucket. This is chapter 2’s pattern — an entity, and a table() constructor that describes the source — pointed at a cloud API instead of a database:

#![allow(unused)]
fn main() {
use serde::{Deserialize, Serialize};
use vantage_aws::prelude::*;

/// One file in the bucket. Field names match S3's wire XML
/// (`<Contents><Key/><Size/></Contents>`).
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct File {
    #[serde(rename = "Key")]
    pub filename: String,
    #[serde(rename = "Size")]
    pub size: String,
}
}

The serde renames map S3’s wire names onto the fields we actually want to write in Rust. Size is a String because that’s what the XML carries — no silent coercion.

#![allow(unused)]
fn main() {
use serde::{Deserialize, Serialize};
use vantage_aws::prelude::*;
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct File {
    #[serde(rename = "Key")]
    pub filename: String,
    #[serde(rename = "Size")]
    pub size: String,
}
impl File {
    /// `ListObjectsV2` narrowed to one bucket and prefix. S3 sends at
    /// most `max-keys` keys per response; the `@continuation-token`
    /// cursor tells the framework to keep requesting pages until the
    /// listing is complete.
    pub fn table(aws: AwsAccount, bucket: &str, prefix: &str) -> Table<AwsAccount, File> {
        Table::new(
            "restxml/Contents@continuation-token=NextContinuationToken:s3/GET /{Bucket}?list-type=2",
            aws,
        )
        .with_id_column("Key")
        .with_column_of::<String>("Size")
        .with_condition(eq("Bucket", bucket))
        .with_condition(eq("prefix", prefix))
        .with_condition(eq("max-keys", 100))
    }
}
}

The table name is doing a lot of work here — for vantage-aws, it’s the wire protocol spelled out: restxml is the protocol S3 speaks; Contents is the response element holding the rows; s3/GET /{Bucket}?list-type=2 is the service and request. Conditions complete the request the same way chapter 2’s conditions completed a WHERE clause: Bucket fills the path placeholder, and anything else (prefix, max-keys) becomes a query parameter.

The @continuation-token=NextContinuationToken part is auto-pagination. S3 answers at most max-keys files per response, plus a continuation token when more exist; the cursor declaration tells the driver to keep re-issuing the request — token folded back in — until the listing is complete. One list() call, as many HTTP requests as it takes.

{protocol}/{array_key}[@cursor|@request=response]:{service}/{METHOD} {path}?{query}

A first listing

main.rs, shortest possible version. It starts with mod files;, then builds the table, lists it and prints:

mod files {
    use serde::{Deserialize, Serialize};
    use vantage_aws::prelude::*;
    #[derive(Debug, Clone, Default, Serialize, Deserialize)]
    pub struct File {
        #[serde(rename = "Key")]
        pub filename: String,
        #[serde(rename = "Size")]
        pub size: String,
    }
    impl File {
        pub fn table(aws: AwsAccount, bucket: &str, prefix: &str) -> Table<AwsAccount, File> {
            Table::new(
                "restxml/Contents@continuation-token=NextContinuationToken:s3/GET /{Bucket}?list-type=2",
                aws,
            )
            .with_id_column("Key")
            .with_column_of::<String>("Size")
            .with_condition(eq("Bucket", bucket))
            .with_condition(eq("prefix", prefix))
            .with_condition(eq("max-keys", 100))
        }
    }
}
use files::File;
use vantage_aws::prelude::*;

const BUCKET: &str = "noaa-ghcn-pds";
const PREFIX: &str = "csv/by_station/GM";

#[tokio::main]
async fn main() {
    if let Err(e) = run().await {
        e.report();
    }
}

async fn run() -> VantageResult<()> {
    let aws = AwsAccount::public("us-east-1");
    let files = File::table(aws, BUCKET, PREFIX);

    for (filename, file) in files.list().await? {
        println!("{:>10}  {filename}", file.size);
    }
    Ok(())
}

The GM prefix narrows the archive to Germany’s stations — 1122 files, small enough to look at, big enough to feel:

   4796352  csv/by_station/GM000001153.csv
   7057976  csv/by_station/GM000001474.csv
   2978238  csv/by_station/GM000002277.csv
   ...
1122 files in 2.3s

2.3 seconds: twelve HTTPS round-trips (1122 files at 100 per page), paid on every run, forever, because nothing remembers the answer. This is the itch the rest of the chapter scratches.

The master Vista, and what it can’t do

The Dio wants a Vista for its master, so we erase the typed table the same way chapter 4 did:

use serde::{Deserialize, Serialize};
use vantage_aws::prelude::*;
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
struct File {
    #[serde(rename = "Key")]
    filename: String,
    #[serde(rename = "Size")]
    size: String,
}
impl File {
    fn table(aws: AwsAccount, bucket: &str, prefix: &str) -> Table<AwsAccount, File> {
        Table::new(
            "restxml/Contents@continuation-token=NextContinuationToken:s3/GET /{Bucket}?list-type=2",
            aws,
        )
        .with_id_column("Key")
        .with_column_of::<String>("Size")
        .with_condition(eq("Bucket", bucket))
        .with_condition(eq("prefix", prefix))
        .with_condition(eq("max-keys", 100))
    }
}
const BUCKET: &str = "noaa-ghcn-pds";
const PREFIX: &str = "csv/by_station/GM";
#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
let aws = AwsAccount::public("us-east-1");
let master = aws
    .vista_factory()
    .from_table(File::table(aws, BUCKET, PREFIX))?;
Ok(())
}

Ask this Vista what it can do, and you’ll see why this backend needs a Diorama:

use serde::{Deserialize, Serialize};
use vantage_aws::prelude::*;
use vantage_vista::prelude::*;
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
struct File {
    #[serde(rename = "Key")]
    filename: String,
    #[serde(rename = "Size")]
    size: String,
}
impl File {
    fn table(aws: AwsAccount, bucket: &str, prefix: &str) -> Table<AwsAccount, File> {
        Table::new(
            "restxml/Contents@continuation-token=NextContinuationToken:s3/GET /{Bucket}?list-type=2",
            aws,
        )
        .with_id_column("Key")
        .with_column_of::<String>("Size")
        .with_condition(eq("Bucket", bucket))
        .with_condition(eq("prefix", prefix))
        .with_condition(eq("max-keys", 100))
    }
}
#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
let aws = AwsAccount::public("us-east-1");
let master = aws
    .vista_factory()
    .from_table(File::table(aws, "noaa-ghcn-pds", "csv/by_station/GM"))?;
let caps = master.capabilities();
// Cursor and Traverse — and that's it.
assert!(caps.has(Capability::Cursor));
assert!(!caps.has(Capability::Search) && !caps.has(Capability::Order));
assert!(!caps.has(Capability::Window) && !caps.has(Capability::Insert));
Ok(())
}

Cursor is worth pausing on — it’s the one capability this chapter leans on. fetch_cursor(token, limit) is Vista’s cursor pagination from chapter 4: pass None for the first page, pass back the returned token for the next one. For S3 the driver defines the token as the last key of the previous page, because S3 lists keys in order and accepts any key as a starting point (start-after). That has a consequence the generic contract doesn’t promise: this cursor survives process restarts. Any key you already hold — say, the last key in a cache — resumes the listing right after it.

The Lens: pump pages, resume where you left off

Now the Lens. One setting does the work — an eager loader that, when the Dio is built, synchronizes the cache with the bucket:

use std::time::Instant;
use serde::{Deserialize, Serialize};
use vantage_aws::prelude::*;
use vantage_diorama::prelude::*;
use vantage_diorama::{Cache, Loader, LoaderCx};
use vantage_vista::prelude::*;
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
struct File {
    #[serde(rename = "Key")]
    filename: String,
    #[serde(rename = "Size")]
    size: String,
}
impl File {
    fn table(aws: AwsAccount, bucket: &str, prefix: &str) -> Table<AwsAccount, File> {
        Table::new(
            "restxml/Contents@continuation-token=NextContinuationToken:s3/GET /{Bucket}?list-type=2",
            aws,
        )
        .with_id_column("Key")
        .with_column_of::<String>("Size")
        .with_condition(eq("Bucket", bucket))
        .with_condition(eq("prefix", prefix))
        .with_condition(eq("max-keys", 100))
    }
}
const CACHE_FILE: &str = "cache.redb";
const PAGE_SIZE: usize = 1000;
async fn sync(cx: &LoaderCx) -> VantageResult<()> {
    let mut token: Option<CborValue> =
        cx.cache().list_values().await?.keys().last().cloned().map(Into::into);
    loop {
        let start = Instant::now();
        let (page, next) = cx.master().fetch_cursor(token, PAGE_SIZE).await?;
        let count = page.rows.len();
        cx.cache().insert_values(page.rows.into_iter().collect()).await?;
        println!("fetched {count} files in {:?}", start.elapsed());
        if next.is_none() {
            return Ok(());
        }
        token = next;
    }
}
#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
let aws = AwsAccount::public("us-east-1");
let master = aws
    .vista_factory()
    .from_table(File::table(aws, "noaa-ghcn-pds", "csv/by_station/GM"))?;
let lens = Lens::builder()
    .cache(Cache::redb(CACHE_FILE))
    .loader(Loader::eager(|cx: LoaderCx| async move { sync(&cx).await }))
    .build()
    .context("Failed to build lens")?;
let dio = lens.dio(master).build().await?;
Ok(())
}

• Lens::builder() collects the settings; .build() checks them and returns the Lens, already in an Arc because every Dio it builds shares it.
• .cache(Cache::redb(CACHE_FILE)) opens a redb file on disk. This is why the cache survives process restarts. Cache::memory() is the alternative for when persistence isn’t wanted.
• Loader::eager(...) runs once per Dio, inside build(), which waits for it. The callback gets a LoaderCx: an owned, cheap-to-clone context with the Dio’s master() and cache(), so the future can hold it across .await without borrowing anything.
• lens.dio(master).build().await does the rest in one call: opens a cache table named after the master, spawns the write worker, runs the eager start, and hands back the Dio.

sync is where the cursor pays off. Instead of one big list_values(), it pulls the listing one page at a time and seeds the cursor from the cache:

#![allow(unused)]
fn main() {
use std::time::Instant;
use vantage_aws::prelude::*;
use vantage_diorama::LoaderCx;
use vantage_vista::prelude::*;
const PAGE_SIZE: usize = 1000;
/// Pump the master listing into the cache, one page per request. S3's
/// paging cursor is simply "the last key seen" — so the last key already
/// in the cache resumes the listing, and pages loaded by an earlier run
/// (even one that was interrupted) are never fetched again.
async fn sync(cx: &LoaderCx) -> VantageResult<()> {
    let mut token: Option<CborValue> = cx
        .cache()
        .list_values()
        .await?
        .keys()
        .last()
        .cloned()
        .map(Into::into);
    loop {
        let start = Instant::now();
        let (page, next) = cx.master().fetch_cursor(token, PAGE_SIZE).await?;
        let count = page.rows.len();
        cx.cache()
            .insert_values(page.rows.into_iter().collect())
            .await?;
        println!("fetched {count} files in {:?}", start.elapsed());
        if next.is_none() {
            return Ok(());
        }
        token = next;
    }
}
}

Read the first statement again: the initial token is the last filename already cached. On a cold cache that’s None — start from the top. On a warm cache it’s the end of what we have — S3 continues from there. Kill the process halfway through a sync and run it again: the pages you already paid for are never fetched twice. The cache isn’t just an answer store; it’s the resume point. PAGE_SIZE is the most keys one request asks for; S3 returns at most 1000.

Reads come from the cache

The listing loop stops asking S3 and starts asking the Dio — through the facade Vista from the first half of the chapter. Notice who talks to what: sync, being Lens plumbing, addresses the Dio’s two sides through its LoaderCx (cx.master() to fetch, cx.cache() to store and resume); a consumer asks dio.cached() and never learns what’s underneath. The records come back as Record<CborValue> — the erased form from chapter 4 — so field access goes through the CborValueExt helpers the prelude brings in:

use std::time::Instant;
use serde::{Deserialize, Serialize};
use vantage_aws::prelude::*;
use vantage_diorama::prelude::*;
use vantage_diorama::Cache;
use vantage_vista::prelude::*;
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
struct File {
    #[serde(rename = "Key")]
    filename: String,
    #[serde(rename = "Size")]
    size: String,
}
impl File {
    fn table(aws: AwsAccount, bucket: &str, prefix: &str) -> Table<AwsAccount, File> {
        Table::new(
            "restxml/Contents@continuation-token=NextContinuationToken:s3/GET /{Bucket}?list-type=2",
            aws,
        )
        .with_id_column("Key")
        .with_column_of::<String>("Size")
        .with_condition(eq("Bucket", bucket))
        .with_condition(eq("prefix", prefix))
        .with_condition(eq("max-keys", 100))
    }
}
#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
let aws = AwsAccount::public("us-east-1");
let master = aws
    .vista_factory()
    .from_table(File::table(aws, "noaa-ghcn-pds", "csv/by_station/GM"))?;
let dio = Lens::builder().cache(Cache::redb("cache.redb")).build()?.dio(master).build().await?;
let start = Instant::now();
let listing = dio.cached().list_values().await?;
for (filename, file) in &listing {
    let size = file.get("Size").and_then(|v| v.as_str()).unwrap_or("");
    println!("{size:>10}  {filename}");
}
println!("{} files from cache in {:?}", listing.len(), start.elapsed());
Ok(())
}

First run — cold cache, the sync pump narrating each page:

fetched 100 files in 215ms
fetched 100 files in 217ms
...
fetched 22 files in 114ms
   4796352  csv/by_station/GM000001153.csv
   7057976  csv/by_station/GM000001474.csv
   ...
1122 files from cache in 18ms

Second run — same command, new process:

fetched 0 files in 370ms
1122 files from cache in 17ms

One round-trip — the resume request from the last cached key, confirming nothing new exists — and then the full listing in 17 milliseconds. New files that do appear under the prefix arrive on exactly that request, without re-fetching the thousand we already hold.

Invalidating

A resuming cache has one blind spot: it only ever looks past what it holds, so files deleted from the bucket linger locally. The honest fix is to start over. The Lens opens cache.redb when it is built, so the place to start over is before that: delete the file, and the eager start finds an empty cache and lists the bucket from the top.

use vantage_aws::prelude::*;
const CACHE_FILE: &str = "cache.redb";
fn main() -> VantageResult<()> {
// In `run`, before the Lens is built:
if std::env::args().any(|a| a == "--invalidate") {
    remove_cache()?;
}
Ok(())
}

fn remove_cache() -> VantageResult<()> {
    match std::fs::remove_file(CACHE_FILE) {
        Err(e) if e.kind() != std::io::ErrorKind::NotFound => {
            Err(e).context("Failed to remove the cache")
        }
        _ => Ok(()),
    }
}

$ cargo run -- --invalidate
fetched 100 files in 215ms
fetched 100 files in 217ms
...
1122 files from cache in 18ms

The CLI is complete: two seconds once, milliseconds forever after, resumable mid-sync, and an escape hatch back to the truth.

The event bus

One more thing build() set up, invisibly: every Dio carries a broadcast event bus, which dio.subscribe_events() listens to. Anything that changes data announces it there as a DioEvent: Seeded when an eager start has put the whole set in the cache, DatasetChanged when the set was rewritten wholesale, RecordChanged / RecordInserted / RecordRemoved for row-level changes, Refreshing and RefreshFailed around a re-read, and WritePending / WriteReverted / WriteFailed as writes move through the queue. Nothing in this chapter listens yet; chapter 7’s sceneries subscribe to this bus to know when to recompute, and chapter 6’s hydration uses it to report progress.

One rule is worth forming as a habit now: direct cache writes are silent. Our sync calls cx.cache().insert_values(...), which stores rows and announces nothing. Inside an eager start that’s fine — the Dio publishes Seeded once the start returns, and every scenery reloads from the cache. Anywhere else, use the calls that write and announce in one motion: cx.reconcile(rows) makes the cache hold exactly the set you hand it (writing what differs, deleting what’s gone) and publishes one DatasetChanged; dio.apply_change(...) takes one upstream ChangeEvent — Inserted, Updated, Deleted or Invalidated — and publishes the matching row event.

Writes, on a read-only master

S3’s listing Vista doesn’t advertise Insert, and the facade won’t pretend otherwise — by default. A write enters a Dio as a ChangeFlash handed to dio.flash(...) (the facade’s own insert_value and friends become flashes too). The flash is staged in the cache at once, joins the Dio’s write queue, and the queue writes it through the master. A master that can’t take it fails the write: the staged row is rolled back, WriteReverted goes out on the bus, and flash returns the error — never a silent drop.

use serde::{Deserialize, Serialize};
use vantage_aws::prelude::*;
use vantage_diorama::prelude::*;
use vantage_diorama::Cache;
use vantage_types::Record;
use vantage_vista::prelude::*;
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
struct File {
    #[serde(rename = "Key")]
    filename: String,
    #[serde(rename = "Size")]
    size: String,
}
impl File {
    fn table(aws: AwsAccount, bucket: &str, prefix: &str) -> Table<AwsAccount, File> {
        Table::new(
            "restxml/Contents@continuation-token=NextContinuationToken:s3/GET /{Bucket}?list-type=2",
            aws,
        )
        .with_id_column("Key")
        .with_column_of::<String>("Size")
        .with_condition(eq("Bucket", bucket))
        .with_condition(eq("prefix", prefix))
        .with_condition(eq("max-keys", 100))
    }
}
#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
let aws = AwsAccount::public("us-east-1");
let master = aws
    .vista_factory()
    .from_table(File::table(aws, "noaa-ghcn-pds", "csv/by_station/GM"))?;
let dio = Lens::builder().cache(Cache::redb("cache.redb")).build()?.dio(master).build().await?;
let mut row = Record::new();
row.insert("Size".to_string(), CborValue::from("0"));

let written = dio.flash(ChangeFlash::insert("csv/by_station/NOTE.csv", row)).await;
// Err: the S3 listing takes no inserts, and the staged row is rolled back
Ok(())
}

To make the write mean something else, give the Lens a writer. Every flash the queue runs is then handed to your closure instead of the master — append to a journal, call a different API, publish to a queue. This is exactly the introduction’s “a read-only CSV file accepts writes by routing them into a queue”. Ok confirms the write and keeps the staged row; Err rolls it back. A writer also makes every Dio of the Lens advertise the writes on its facade:

use vantage_aws::prelude::*;
use vantage_diorama::prelude::*;
use vantage_diorama::{Cache, FlashOutcome, LoaderCx};
#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
let lens = Lens::builder()
    .cache(Cache::redb("cache.redb"))
    .writer(|_cx: LoaderCx, flash: ChangeFlash| async move {
        println!("journal: {:?} {:?}", flash.kind(), flash.id());
        Ok(FlashOutcome::applied())
    })
    .build()?;
Ok(())
}

The four settings

A Lens has four settings, each with its own moment:

Everything a Dio adds beyond these — augmentation in chapter 6, a live follow with .live() — is chosen on lens.dio(master) before build(), because it belongs to that one Dio, not to every Dio the Lens builds.

Why a builder that can refuse?

Lens::builder().build() returns a Result, and it does refuse things. Some settings only make sense together, and each wrong combination would otherwise fail much later, in a way that’s hard to trace back: Loader::blocking on a loader with no eager start (there’s nothing to block on), Refresher::every(Duration::ZERO) (a timer with no period), a write queue of size zero (a queue that can never take a write), a Lens built outside a tokio runtime (nowhere to spawn its tasks). build() checks the combination once and names the setting at fault, so a misconfiguration is an error at startup, not a stall at the first write:

use std::time::Duration;
use vantage_diorama::prelude::*;
use vantage_diorama::{Cache, LoaderCx, Refresher};
use vantage_vista::prelude::*;
#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
let refused = Lens::builder()
    .cache(Cache::memory())
    .refresher(
        Refresher::new(|cx: LoaderCx| async move {
            let rows = cx.master().list_values().await?;
            cx.reconcile(rows).await
        })
        .every(Duration::ZERO),
    )
    .build();

assert!(refused.is_err()); // "Refresher::every needs a non-zero interval"
Ok(())
}

The cost is one ? at the call site. A builder that took anything and guessed would save that, and turn each of these mistakes into a hang.

What we covered

Augmentation — Enriching Rows

Chapter 5’s inventory knows every station file’s name and size — the two things an S3 listing gives away for free. Everything interesting is inside the files: how many readings a station has recorded, and how recently. Getting that costs one download per file, which is exactly the kind of expense you want to pay once and remember.

Augmentation is Diorama’s answer: enrich the master’s rows, one row at a time, from a detail source — and let the Dio’s cache hold the result. The listing stays the cheap, fast spine it was in chapter 5; each row gains columns it never had, hydrated on demand and persisted alongside the row. This chapter adds two such columns to the inventory:

      SIZE     ROWS     LATEST  FILENAME
   4796352   140629   19911231  csv/by_station/GM000001153.csv

ROWS — how many readings the file holds; LATEST — the date of the most recent one. Neither exists anywhere in S3’s listing; both are computed from the file’s contents.

We build on chapter 5’s crate unchanged — learn-5 starts as a copy of learn-4 (files.rs and main.rs exactly as they were) and this chapter only adds. One new file, readings.rs, describes the detail side; a handful of lines in main.rs wire it up.

The detail side: readings.rs

Every station file is a CSV of date-ordered readings:

ID,DATE,ELEMENT,DATA_VALUE,M_FLAG,Q_FLAG,S_FLAG,OBS_TIME
GM000001153,18910101,TMAX,-20,,,I,
GM000001153,18910102,TMAX,-40,,,I,
...

So the two columns we’re after are cheap derivations: rows is the line count minus the header, latest is the DATE field of the last line. The expensive part is getting the contents at all.

The detail source is a table — the same ListObjectsV2 listing as the master, in fact, with two differences: it declares no prefix (the augmentation will narrow it to a single file per fetch), and it carries the derived columns. Start readings.rs with its entity:

#![allow(unused)]
fn main() {
use serde::{Deserialize, Serialize};
use vantage_aws::prelude::*;

/// One file, seen through the augmenter: the listing row plus columns
/// computed from the file's contents.
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct Readings {
    #[serde(rename = "Key")]
    pub filename: String,
    pub rows: i64,
    pub latest: String,
}
}

Lazy expressions

Where do rows and latest come from? Chapter 2’s with_expression won’t do it — those expressions lower into the backend’s query (a SQL subselect), and S3 can’t compute anything server-side. What we need runs on our side, after the row comes back.

That’s a lazy expression: a column computed in Rust, on the returned record. Lazy expressions apply in declaration order, and each callback borrows the record as built so far — including the columns earlier lazy expressions added. That ordering rule is the whole trick:

#![allow(unused)]
fn main() {
use serde::{Deserialize, Serialize};
use vantage_aws::prelude::*;
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct Readings {
    #[serde(rename = "Key")]
    pub filename: String,
    pub rows: i64,
    pub latest: String,
}
impl Readings {
    pub fn table(aws: AwsAccount, bucket: &str) -> Table<AwsAccount, Readings> {
        let bucket = bucket.to_string();
        Table::new("restxml/Contents:s3/GET /{Bucket}?list-type=2", aws.clone())
            .with_id_column("Key")
            .with_condition(eq("Bucket", bucket.clone()))
            .with_lazy_expression("contents", move |row| {
                let aws = aws.clone();
                let bucket = bucket.clone();
                let key = row.get("Key").and_then(|v| v.as_str()).unwrap_or_default().to_string();
                async move { Ok(s3::get_object(&aws, &bucket, &key).await?.into()) }
            })
            .with_lazy_expression("rows", |row| {
                // Every line after the CSV header is one reading.
                let contents = row.get("contents").and_then(|v| v.as_str()).unwrap_or_default();
                let rows = contents.lines().count().saturating_sub(1) as i64;
                async move { Ok(rows.into()) }
            })
            .with_lazy_expression("latest", |row| {
                // Readings are date-ordered; take the last line's DATE column.
                let contents = row.get("contents").and_then(|v| v.as_str()).unwrap_or_default();
                let latest = contents
                    .lines()
                    .last()
                    .and_then(|line| line.split(',').nth(1))
                    .unwrap_or_default()
                    .to_string();
                async move { Ok(latest.into()) }
            })
    }
}
}

Read it as a pipeline over one record. The listing returns {Key, Size, …}; then:

1. contents reads Key off the record and downloads the file — s3::get_object is the driver’s raw fetch, unsigned here just like the listing. Its return value is inserted into the record under contents. This is the expensive step, and it happens once.
2. rows never touches the network: it reads row.get("contents") — the column the previous expression just added — and counts lines.
3. latest reads the same contents and takes the last line’s date.

One download feeds every derived column declared after it. Each callback clones what it needs out of the borrowed record before going async, and each expression’s name is also registered as a column on the table, so the derived fields are part of its schema like any other.

Why not just list this table?

It’s tempting to stop here — the augmenter table already produces every column we want, so why not make it the Dio’s master? Because lazy expressions run when data is fetched. A list() on this table triggers the whole pipeline for every row it returns: a thousand files means a thousand downloads inside one blocking call, and nothing comes back until the last one lands. That’s not a listing — it’s a batch job wearing a listing’s interface.

What we’re after is different. The list of files should reach the user instantly — it’s chapter 5’s cached listing, it costs milliseconds. The expensive columns should then be fetched one row at a time, each result written into the Dio and announced on the event bus, so anything watching updates progressively. Loading details for every station takes a long time no matter what; the win is in never blocking the listing on it, and in choosing the order — the next chapter puts a viewport in charge, so the rows the user is currently looking at are hydrated first, and rows nobody scrolls to are never fetched at all.

That split — a cheap master everyone lists, an expensive detail source consulted per row — is what augmentation declares.

Wiring the augmentation

Back in main.rs. The augmenter becomes a Vista like any table, and the Dio is told how to use it. Chapter 5 built its Dio with lens.dio(master).build(); the augmentation slots in between the two calls, and that’s the only structural change to run():

#![allow(unused)]
fn main() {
use std::sync::Arc;
use vantage_aws::prelude::*;
use vantage_diorama::{Augmentation, Detail, Fetch, Lens, MergeRule, Source};
use vantage_types::EmptyEntity;
use vantage_vista::Vista;
const BUCKET: &str = "noaa-ghcn-pds";
struct Readings;
impl Readings {
    fn table(aws: AwsAccount, bucket: &str) -> Table<AwsAccount, EmptyEntity> {
        Table::new("restxml/Contents:s3/GET /{Bucket}?list-type=2", aws)
            .with_id_column("Key")
            .with_condition(eq("Bucket", bucket))
    }
}
async fn run(aws: AwsAccount, lens: Arc<Lens>, master: Vista) -> vantage_core::Result<()> {
let augmenter = aws
    .vista_factory()
    .from_table(Readings::table(aws.clone(), BUCKET))?;

let dio = lens
    .dio(master)
    .augment(vec![Augmentation::new(
        Detail::Fixed(Arc::new(augmenter)),
        Source::Column {
            from: "Key".into(),
            to: Some("prefix".into()),
        },
        Fetch::PerRow,
        MergeRule::lift(["rows", "latest"]),
    )])
    .build()
    .await?;
let _ = dio;
Ok(())
}
}

An Augmentation answers four questions:

• detail: where do detail records come from? Detail::Fixed holds our augmenter Vista directly. The config-driven form is Detail::Resolved { resolver, name }: a config carries only a name, so the detail Vista is looked up by that name on each fetch, through a VistaResolver such as a Catalog. The resolver travels with the detail, so the Dio needs no catalog of its own and augment takes nothing but the list.
• source — how does a master row select its detail record? Source::Column maps the master’s Key onto the detail table’s prefix condition. A full filename used as an S3 prefix matches exactly one object — so each fetch narrows the augmenter to a one-row listing, and the lazy expressions run for precisely that file.
• fetch — Fetch::PerRow: one detail fetch per master row.
• merge — which detail columns land on the master row? Just rows and latest. contents is deliberately absent: it exists only inside the detail fetch, feeds the derived columns, and is never cached. The megabytes stay out of the Dio; the two numbers stay in.

Reads hydrate

Nothing has fetched anything yet — declaring an augmentation is free. The work happens on read: rows a facade read returns come back hydrated — any of them still missing its augment columns runs the detail fetch first, and the result is written back to the cache as complete.

Not every read, though. The listing must stay what chapter 5 made it — instant — so list_values through the facade returns the cheap rows untouched. Hydration belongs to bounded reads: get_value for one record, fetch_window for a range. The rows you ask for are the rows that pay:

#![allow(unused)]
fn main() {
use std::time::Instant;
use vantage_aws::prelude::*;
use vantage_diorama::Dio;
async fn run(dio: Dio) -> vantage_core::Result<()> {
// The listing stays instant — cheap rows, no downloads.
let start = Instant::now();
let listing = dio.cached().list_values().await?;
println!("{} files (listed in {:?})", listing.len(), start.elapsed());

// Details are paid for by the rows you ask for: a window of ten.
let start = Instant::now();
let window = dio.cached().fetch_window(0, 10).await?;
for (filename, file) in &window.rows {
    let size = file.get("Size").and_then(|v| v.as_str()).unwrap_or("");
    let rows = file.get("rows").and_then(|v| v.as_i64()).unwrap_or(0);
    let latest = file.get("latest").and_then(|v| v.as_str()).unwrap_or("");
    println!("{size:>10} {rows:>8} {latest:>10}  {filename}");
}
println!("{} files detailed in {:?}", window.rows.len(), start.elapsed());
Ok(())
}
}

dio.cached() is the Dio’s facade: a Vista with the master’s schema that reads through the cache. fetch_window returns a page, whose rows keep the window’s order.

Ten downloads is still a wait worth narrating, and the Dio announces it: one DioEvent::Hydrating with the pending count before the first fetch, then a RecordChanged per row as each lands. A dozen lines of event plumbing become the progress display:

#![allow(unused)]
fn main() {
use vantage_diorama::{Dio, DioEvent};
fn run(dio: Dio) {
let mut events = dio.subscribe_events();
tokio::spawn(async move {
    while let Ok(event) = events.recv().await {
        match event {
            DioEvent::Hydrating { pending } => println!("hydrating {pending} files…"),
            DioEvent::RecordChanged { id } => println!("  {id}"),
            _ => {}
        }
    }
});
}
}

First run — chapter 5’s sync, the instant listing, then the window doing its downloads:

fetched 100 files in 215ms
...
1122 files (listed in 16ms)
hydrating 10 files…
  csv/by_station/GM000001153.csv
  csv/by_station/GM000001474.csv
  ...
   4796352   140629   19911231  csv/by_station/GM000001153.csv
   7057976   206874   20260531  csv/by_station/GM000001474.csv
   2978238    87207   19990131  csv/by_station/GM000002277.csv
   ...
10 files detailed in 20.2s

Real data, and readable at a glance: station GM000001474 has 206,874 readings and is still reporting (May 2026); its neighbour GM000001153 went silent at the end of 1991.

Second run — new process, warm cache:

fetched 0 files in 340ms
1122 files (listed in 16ms)
   4796352   140629   19911231  csv/by_station/GM000001153.csv
   ...
10 files detailed in 16ms

No hydrating line at all: those ten rows already carry their augment columns, so the window finds no gaps and never touches the network. Twenty seconds became sixteen milliseconds; the derived numbers live in cache.redb with the rest of the row. Ask for a different window — fetch_window(500, 10) — and only its gaps download. (--invalidate still clears everything, derived columns included — derived data is data.)

What we covered

Scenery — Reactive Views

Chapter 6’s CLI asks for one window of details and exits — the asking is still on the programmer. A real interface is a standing view: the user scrolls, and the window should follow; details land one by one, and the rows should repaint; the bucket changes, and the listing should notice. And it has to hold up at scale — the full GHCN archive is about 122,000 station files, of which a screen shows forty. What a UI actually needs is an ordered row set it can read by index, hydration that follows the user’s attention, and a signal whenever anything on screen changes.

Scenery is that layer. Each Scenery is a reactive view onto a Dio, exposing one access pattern:

• TableScenery — ordered rows by index, with a viewport
• RecordScenery — a single record by id
• ValueScenery — a computed scalar (count, sum, custom)

All three share one reactivity mechanism: a Generation counter that bumps whenever the view’s state changes. Consumers subscribe() to a watch channel and redraw on each bump — the channel only ever holds the latest generation, so a burst of changes costs one repaint, not one per change.

A Scenery is not another handle to your data — it hands you no records to keep. It maintains one particular view of the Dio’s copy, precomputed and ready for a widget to read:

That last cell is where this chapter ends up. The consumer — a terminal table here, a desktop grid or a remote client such as a React page elsewhere — drives its scenery in one repeating loop:

1. The consumer declares what it shows: set_viewport(40..80, 40..80) — nothing more than “these rows are on screen.”
2. The scenery turns that into work for the Dio: list pages for the spine, detail fetches for the rows in view.
3. Results land in the Dio’s cache, and the event bus announces each one — RecordChanged, row by row.
4. The scenery reacts to the bus: it updates the affected slot and bumps its Generation.
5. The consumer’s watch channel ticks; it re-reads row(idx) for what’s visible and repaints.

The code that runs this loop for one particular consumer is the adapter — the same handful of lines whether it paints a terminal, a desktop toolkit, or a wire to the browser.

Back to the inventory

Chapter 6 left the CLI listing one prefix and detailing ten fixed rows. What we actually want from the inventory is the tool you’d reach for daily — and that sets the requirements:

• the whole archive, not a prefix — all 122,000 station files, scrollable end to end;
• instant open, even on the first run — an empty table that fills as the listing arrives, never a frozen prompt;
• details where the user is looking — the rows on screen sprout ROWS and LATEST as their files are read, and the cursor drags that attention with it;
• standing freshness — the bucket changes, the table follows, plus a grand total of readings across the archive.

Measure chapter 6’s ending against that. It read from the Dio with dio.cached().fetch_window(0, 10) — and it works fine, for a CLI: the read is bounded to ten records, and it resolves once all ten have their details. Nothing blocks — it’s an async wait, the runtime stays free — but the user waits all the same: twenty seconds of silence on a cold cache, and every widening of the window widens the wait. For a better experience we need a more responsive UI:

1. display results as soon as we have them — don’t hold rows hostage to their details;
2. show data immediately and fix it reactively as better values arrive;
3. register what the user is currently looking at, reducing what is fetched and transmitted.

The Scenery implements exactly this:

• rows are served as soon as the list pass has them, with … standing in for details still pending;
• every landed detail updates its own row and bumps the Generation — one tick, one repaint;
• set_viewport(loaded, shown) registers the user’s attention, and only those rows hydrate.

For the consumer, this chapter reaches for ratatui — a terminal-UI library — which Vantage updates in real time through a ready-made adapter. The dataset-ui-adapters crate ships such adapters for several UI frameworks (egui, Slint, GPUI, Cursive, Tauri among them), and they all speak the same scenery loop. Here is ours a few seconds into a cold start: the first two visible rows already hydrated, and the total still waiting for the rest of the archive:

FILENAME                              SIZE        ROWS      LATEST
csv/by_station/ACW00011604.csv        41673       1231      19490814
csv/by_station/ACW00011647.csv        481021      14355     20260710
csv/by_station/AE000041196.csv        1998959     …         …
…
 8000 rows · 2 augmented · total rows 0 · ↑/↓ PgUp/PgDn scroll · r refresh · q quit

learn-6 starts as a copy of learn-5’s data side — files.rs, readings.rs, and the whole Dio setup carry over. Three things change: the prefix widens to the entire archive, the Lens learns to serve a UI, and the printing loop is replaced by a scenery-bound terminal table.

cargo add dataset-ui-adapters --features ratatui

Widening the scope is two touches. The prefix loses its country filter:

#![allow(unused)]
fn main() {
const PREFIX: &str = "csv/by_station/";
}

And the listing pages grow. Every page is one HTTP round-trip: chapter 5 fetched 100 keys at a time, a comfortable twelve requests for its 1,122 files — but at 122,000 files the same setting means 1,220 round-trips of mostly latency. S3 caps a single response at 1,000 keys, so files.rs asks for the cap and the whole archive syncs in about 122 requests:

#![allow(unused)]
fn main() {
use vantage_aws::prelude::*;
use vantage_types::EmptyEntity;
fn table(aws: AwsAccount) -> Table<AwsAccount, EmptyEntity> {
    Table::new("restxml/Contents:s3/GET /{Bucket}?list-type=2", aws)
        .with_id_column("Key")
        // S3's per-response maximum — we're listing the whole station set.
        .with_condition(eq("max-keys", 1000))
}
}

A Lens that serves a UI

Chapter 5’s Lens had one job: warm the cache before anyone reads. A UI inverts the priorities — the screen must appear immediately, on whatever is known so far, and data streams in behind it. Two changes to the builder:

#![allow(unused)]
fn main() {
use std::time::Duration;
use vantage_aws::prelude::*;
use vantage_diorama::{Cache, Lens, Loader, LoaderCx, Refresher};
async fn sync(_cx: &LoaderCx) -> vantage_core::Result<()> {
    Ok(())
}
fn run() -> vantage_core::Result<()> {
let lens = Lens::builder()
    .cache(Cache::redb("cache.redb"))
    // Don't wait for the sync: it warms the cache in the background,
    // while the view lists the bucket itself.
    .loader(Loader::eager(|cx: LoaderCx| async move { sync(&cx).await }).blocking(false))
    // Reconcile against the bucket once a minute: new files appear
    // as un-hydrated rows, vanished ones drop out, changed ones are
    // demoted for re-hydration. The augmentation's refresh pass does
    // the work; the refresher only sets the interval.
    .refresher(Refresher::new(|_cx: LoaderCx| async { Ok(()) }).every(Duration::from_secs(60)))
    .build()
    .context("Failed to build lens")?;
let _ = lens;
Ok(())
}
}

.blocking(false) detaches the sync. An eager start blocks the Dio’s build() by default, which is what chapter 5 wanted: nobody read before the cache was warm. Here build() returns at once and the sync runs as a background task, so the UI’s first frame never waits on 122 requests. The sync itself is chapter 5’s, minus its println! narration (a terminal UI owns the screen now). It writes the cache directly, and writes to the raw cache raise no events, so the Dio does the announcing: when the detached start finishes it sends DioEvent::Seeded, and every open view re-reads.

That covers the cache. What a TableScenery does with it needs a little more context. An augmented Dio drives two-pass loading: a list pass fetches cheap rows and establishes the view’s order (its spine), and a detail pass hydrates individual rows’ augment columns as the viewport reaches them. The detail pass is chapter 6’s augmentation. The list pass picks its source when a view first lists:

• while the cache isn’t known to hold the whole set (a first run, or a sync still running), it asks the master, page by page;
• once the Dio holds a whole copy, which the eager start records when it finishes, it reads the cache instead: one local read, zero network.

So on a cold start the table lists the bucket itself while the sync runs, and on Seeded it re-lists from the warm cache. Nothing in the application chooses between the two; the Dio knows which answer it can trust.

The refresher completes the freshness story. Once a minute the Dio reconciles its cache against the master (this does walk the listing: 122 requests, in the background). New files appear as un-hydrated rows, deleted ones drop out, and a file whose listing entry changed keeps showing its old numbers but is marked stale; the next time it’s on screen, the detail pass re-fetches it. Chapter 5 needed --invalidate to notice deletions; a standing view just notices. The refresher’s own callback does nothing here: on an augmented Dio the reconcile is the augmentation’s refresh pass, and Refresher::new(..).every(..) only sets how often it runs.

Why not always list from the cache?

Because an empty cache and an empty bucket look the same. A view that read the cache during a first run would show zero files and call the listing complete, then jump to 122,000 rows minutes later. A half-written cache is worse: it reads as a complete listing of whichever files the sync reached first. The whole-copy record is how the Dio tells “the cache is the listing” from “the cache is what we have so far”, and until it can say the first, the master answers. The cost is the master walk on a cold start, and that walk is the same one the sync is making anyway.

Creating the reactive UI

Our UI needs two things from the Dio: the rows, and a grand total. Each one is a scenery. Let’s open them one at a time.

The table

#![allow(unused)]
fn main() {
use vantage_diorama::Dio;
async fn run(dio: Dio) -> vantage_core::Result<()> {
// One list page covers the whole cached listing (~122k station files);
// the viewport's detail pass hydrates whatever is on screen first.
let scenery = dio.table_scenery().page_size(200_000).open().await?;
let _ = scenery;
Ok(())
}
}

.open() builds the scenery’s rows (or joins the rows another open of the same query already built), spawns its reactor (the task that watches the Dio’s event bus), and hands back Arc<dyn TableScenery>. What you get is deliberately small and synchronous:

• row_count() and row(idx), an EnrichedRecord: the record plus a per-row status;
• set_viewport(loaded, shown), more on this in a moment;
• subscribe(), the generation channel from the start of the chapter;
• refresh_source(), which asks the Dio to re-read its master; the view reloads as the fresh rows land.

The builder can also chain .order(column, SortDirection::Ascending), .search(text) and .filter(condition), every one of them served locally on a backend that can do none of it. We don’t use them here; the table shows the archive as-is.

#![allow(unused)]
fn main() {
use vantage_diorama::Dio;
use vantage_vista::SortDirection;
async fn run(dio: Dio) -> vantage_core::Result<()> {
// The same table, from the end of the alphabet.
let reversed = dio
    .table_scenery()
    .order("Key", SortDirection::Descending)
    .open()
    .await?;
let _ = reversed;
Ok(())
}
}

A UI rarely commits to one order at open time, though, and it doesn’t have to reopen. The handle mutates in place: set_order(Some((column, direction))) and set_search(Some(text)), bound to a key or a header click, re-point the view at the ordered index for the new query, swap the visible rows in one atomic step (the grid never blanks mid-reorder), and restart hydration for whatever is on screen. Sorting back reuses the already-built index: zero list calls. Runtime filters change the same way, through set_filters(conditions), which takes a Vec<Condition> and replaces the previous set (an empty vec clears it).

One thing probably caught your eye: page_size(200_000)?! That’s the list pass: how many spine rows one list page asks for. It’s a separate axis from the viewport, which drives only the detail pass over rows the spine already holds. Left at the default, the spine would stop at the first hundred files and nothing in this UI asks for page two. Set past the archive size, one list page builds the entire spine. Once the Dio holds its whole copy, that page is a single read of the local cache.

The viewport

The viewport is the scenery’s answer to “which rows is the user actually looking at?” — a plain range of row indexes, like 40..80. It’s declared through set_viewport(loaded, shown), the method on the handle we just opened. loaded is the range to load and hydrate; shown is the range actually on screen, which feeds the view’s “showing a–b” statistics and nothing else. A grid usually passes the same range twice. You rarely call it yourself: whoever renders the table calls it as the user scrolls. Later in this chapter, that’s the ratatui adapter’s job.

It is the load-bearing call, because the viewport drives hydration. The rows a consumer declares as its viewport are the rows the detail pass works on. Framework code calls this the demand gate: no row is detailed unless some live view demands it. Everything else stays a cheap list row — and rows never observed never cost a download.

Recognize it? This is chapter 6’s bounded read with the asking automated: there the programmer chose the window; here the viewport is the window, and it moves with the user.

The grand total

#![allow(unused)]
fn main() {
use vantage_diorama::Dio;
use vantage_diorama::aggregate::Sum;
async fn run(dio: Dio) -> vantage_core::Result<()> {
// Grand total of the ROWS column. `rows` is augmented, so each
// computation first hydrates every cached file: the total appears once
// they have all landed, then follows the bucket's changes.
let totals = dio.value_scenery(Sum::of("rows")).open().await?;
let _ = totals;
Ok(())
}
}

A value scenery runs one aggregation (here Sum) over the rows the Dio admits from its cache. Nothing is asked of the master. It recomputes when the Dio announces a change, and bumps its generation only when the value actually moved. open() computes the first value before it returns.

Hold on: how can it sum rows when most files aren’t augmented yet? It doesn’t sum the gaps. The aggregation declares which columns it reads, and when one of them is an augmented column, the Dio hydrates the rows missing it before the aggregation sees them. A sum that skipped un-hydrated rows would show a number that looks like a total and isn’t one. So the value is always a total over every cached file, and on a cold cache that means it waits for every file to download.

That’s an expensive number on 122,000 files, and it’s meant to be: a total of readings genuinely needs every file read once. The cost is paid once, because the hydrated columns stay in cache.redb; a warm restart totals from the cache. When you want a cheap number instead, aggregate something the listing already has. Count::rows() reads no column at all, so it counts the cached rows without hydrating anything.

(There is a third kind, RecordScenery: dio.record_scenery(id).open(), one record under the same contract: read it, subscribe, redraw on bump. A detail pane beside the table would use one; our table doesn’t need it.)

Open freely, drop when done

Sceneries are inexpensive. A page opens as many as it needs (a grid, a total, a detail pane), and every open() returns a view of its own: its own viewport, its own order, search and filters. Views of the same query still read one set of rows underneath, so two grids over one query make one request and hydrate each file once. The other half of that contract: release them when the page closes. Dropping the handle stops the view’s tasks and withdraws its demand, so a closed page stops pulling data.

Why does each open get its own view?

Because sharing the view meant sharing its setters. When identical opens returned one shared instance, a quick-search typed into one grid reshaped every other grid of the same query: set_search on one widget filtered its neighbour’s rows too, and two remote clients asking the same question couldn’t scroll to different pages. Sharing was right for the rows and wrong for everything the user controls.

So the split runs along that line. The rows (the per-query index, the cached records, the hydration already paid for) are shared between every view of one query. The view (viewport, shown range, order, search, filters) belongs to whoever opened it. A setter moves only its own view to the rows of its new query, joining another view’s rows when one already answers it. Chapter 8 leans on this: every HTTP connection opens its own view, and the downloads are still paid once.

Binding to a terminal

What remains is rendering — and none of it is specific to this app. Scrolling a virtualized table, forwarding the visible range as the viewport, redrawing on generation bumps: that’s a reusable binding, and dataset-ui-adapters ships it for ratatui in its ratatui_dio module. The entire UI:

#![allow(unused)]
fn main() {
use std::sync::Arc;
use dataset_ui_adapters::ratatui_dio;
use vantage_aws::prelude::*;
use vantage_diorama::{TableScenery, ValueScenery};
async fn ui(
    scenery: Arc<dyn TableScenery>,
    totals: Arc<dyn ValueScenery>,
) -> vantage_core::Result<()> {
    ratatui_dio::SceneryTable::new(scenery)
        .with_column("FILENAME", "Key", 0)
        .with_column("SIZE", "Size", 10)
        .with_column("ROWS", "rows", 8)
        .with_column("LATEST", "latest", 10)
        .with_status_value("total rows", totals)
        .run()
        .await
        .context("terminal UI failed")
}
}

Line by line:

• ratatui_dio::SceneryTable::new(scenery) — hands the scenery to the adapter. The full module path marks the boundary: everything before this line was framework, everything after is the ratatui binding.
• .with_column(header, field, width) — one table column: the header text, the record field it reads, and a width in characters (0 = flexible fill). A field the row doesn’t have yet renders as … — which is exactly how un-hydrated rows look.
• .with_status_value(label, scenery) — pins a ValueScenery into the status bar; it repaints whenever that scenery’s generation bumps, so total rows follows the Dio.
• .run() — takes over the terminal until q. It draws only the visible rows (at 122,000 you don’t build widgets for the rest), keeps the scenery’s viewport on a ten-row band around the cursor — details load for the record the user is on and its neighbours, not the whole screen. Band-not-screen is the adapter’s policy, not the scenery’s: each detail fetch here is a multi-second download, and hydrating all forty visible rows would waste most of that work every time the user scrolls on. It listens to every subscribed generation for repaints, and renders the status bar: row count, an augmented counter (how many rows on hand are fully hydrated), your pinned values, and the key legend. r calls the scenery’s refresh_source(): the Dio’s reconcile on demand, ahead of the timer.

Running it

cargo run on a cold cache is the whole system visible at once. The table appears instantly, empty, while the view lists the bucket and the background sync copies it; when the sync lands, the view re-lists from the cache and the row counter passes 122,000. Meanwhile the rows around the cursor sprout numbers, one file at a time, as the detail pass works through the band (… becoming 14355 20260710), and the augmented counter ticks upward with each one. Move the cursor, and the band follows; jump to End, and the last stations of the alphabet get their turn. total rows stays at its first value, zero over an empty cache, until the total has every file it needs. Quit, run again: everything already observed is back instantly, warm from cache.redb, and hydration resumes wherever you look next.

Notice what the application never wrote: a render loop, a fetch, an event match. It reads scenery.row(idx) and totals.value() through a binding that repaints when a generation channel says so. Nothing more passes between data and display — and it’s precisely how a real UI binds to Vantage.

What we covered

Serving Scenery — Axum & Watch Streams

Chapter 7’s client of the Dio was a terminal driving two sceneries — one showing the rows, the other aggregating a running sum. Sceneries scale well beyond that: the framework can drive many active viewers at once, and nothing requires them to live in the same process. This chapter sends them across the network — every connected browser tab becomes its own viewer, looking at its own page of the archive, with details streaming in as they land.

The traditional plumbing for live updates over HTTP is a WebSocket, server-sent events, or the style the Kubernetes API made standard — the watch: a plain HTTP response that never ends, delivering one JSON line per change over chunked transfer encoding. We take the Kubernetes shape: every endpoint answers a plain GET with a snapshot, and the same URL with ?watch=true keeps the connection open and streams changes for as long as the client stays. A watch is not a polling loop — it is a Scenery on the far end of an HTTP connection.

This chapter re-uses chapter 3’s Axum server, wired to the Dio through a new adapter crate — and finishes with a small React app browsing the bucket the way the AWS S3 Explorer does. Worth pausing on what the frontend gets for free: the page renders instantly from the cache, cells fill themselves in as augmentation lands, and every open tab stays current — the responsiveness chapter 7 built for the terminal, now delivered over a wire.

One flight per row

Before any of that can be safe, the framework needs an answer to a question chapter 7 never had to ask: what happens when several views drive augmentation at once?

With one terminal and one viewport, it never mattered who ran the detail fetches. If each view ran its own, inline, for its own viewport, two browser tabs watching overlapping pages would each download the same CSVs; two tabs on disjoint pages would race each other with no ordering at all; and a tab opened mid-download would have no way to say “me next”.

So the fetches are the Dio’s job, not the view’s. Every consumer that wants rows hydrated — a scenery’s viewport, a facade read blocking on its window — registers a queue with the Dio’s augment scheduler and enqueues row ids into it. A worker pool drains the queues:

• Round-robin across consumers. The worker takes one id from each queue in turn, so two views with disjoint pages interleave — neither starves behind the other’s backlog.
• One flight per row. An id already being fetched is never fetched again; every queue waiting on it is notified by the same completion. Overlapping views cost one download, not one per view.
• Closing a view withdraws its work. The queue registration is owned by the view; when its handle drops, its queued ids vanish. A fetch already in the air completes and lands in the cache — paid-for work is kept.
• Workers are configurable. The default single worker keeps fetch order deterministic; Lens::builder().tuning(Tuning::default().with_augment_workers(4)) hydrates four rows at a time when the detail source can take it.

Nothing in the example code asks for this: the scheduler sits under the same set_viewport and fetch_window calls the previous chapters used. What it buys is what you can safely do here: open as many sceneries as you have HTTP connections.

Every open is its own view (chapter 7, “Why does each open get its own view?”). Two identical opens share one set of rows, which is right for two widgets showing the same grid, and each still keeps its own viewport, which is right for two clients watching different pages of the same query. Every view counts in the demand union — the Dio hydrates the union of the columns every open scenery demands — and is released on drop.

The adapter: DioRouter

Everything HTTP lives in a new adapter crate, vantage-api-adapters — the server-side sibling of the dataset-ui-adapters crate that provided chapter 7’s ratatui binding. Where a UI adapter binds a scenery to a widget, an API adapter binds it to a route:

#![allow(unused)]
fn main() {
use vantage_api_adapters::axum_dio::DioRouter;
use vantage_diorama::Dio;
fn api(dio: Dio) -> axum::Router {

let api = DioRouter::new(dio.clone())
    .with_column("filename", "Key")
    .with_column("size", "Size")
    .with_column("rows", "rows")
    .with_column("latest", "latest")
    .with_page_size(50)
    .into_router();
api
}
}

Each .with_column(name, field) maps a record field to a JSON key — and doubles as the watch sceneries’ demand: naming rows and latest here is exactly what makes a watch connection drive their hydration (chapter 7’s demand gate, now per connection). into_router() yields a plain axum::Router with two routes, each in two modes:

The split embodies the demand philosophy: a plain GET is not a standing view, so it serves the Dio’s current knowledge instantly — augmented columns appear once some view has paid for them. A watch is the standing view: it declares its page as the viewport, hydration follows, and every change streams back as a Kubernetes-style NDJSON line:

{"type":"ADDED","object":{"index":3,"filename":"…","size":"4880965","rows":null,"latest":null}}
{"type":"MODIFIED","object":{"index":3,"filename":"…","size":"4880965","rows":143676,"latest":"20260531"}}

The stream diffs against what it already sent — a generation bump that changed nothing on this page costs nothing on the wire. And the scenery is owned by the response stream: when the client disconnects, the stream drops, the scenery’s guard aborts its tasks, its queued fetches are withdrawn, and its demand drains. A closed tab stops pulling — the same lifecycle rule as chapter 7’s closing grid, now enforced by HTTP.

One honesty note against the Kubernetes original: there is no resume token (no resourceVersion). A client that reconnects gets a fresh snapshot of ADDED lines and a new watch — not a replay of what it missed while away.

The server

learn-7 is learn-6 with the terminal swapped for the router — and the prefix narrowed back to chapter 5’s GM (1,122 files): a server should boot in seconds, and everything below works the same way on the full archive. The data plumbing is otherwise identical — files.rs, the augmenter, the Lens — with two differences. First, the cache is opened by hand, because we want a second table in the same redb file:

use std::sync::Arc;
use vantage_aws::prelude::*;
use vantage_diorama::{CacheBackend, CacheTable, RedbCache};
struct ContentsCache;
impl ContentsCache {
    fn new(_table: Arc<dyn CacheTable>) -> Arc<Self> {
        Arc::new(Self)
    }
}
#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
let cache = Arc::new(RedbCache::open("cache.redb").context("Failed to open cache")?);
let contents = ContentsCache::new(cache.open_table("contents").await?);
let _ = contents;
Ok(())
}

A Lens normally opens cache.redb itself (Cache::redb(path)); Cache::backend(cache) hands it ours instead. The Dio claims one named table for the listing, and open_table("contents") claims another for the cache we’re about to meet.

Second, the eager start stays blocking. That’s Loader::eager’s default; learn-6 added .blocking(false) to open its UI on an empty table. A server should answer its first request from a warm cache, so the Dio’s build() runs the chapter-5 sync to completion before axum::serve ever binds the port:

#![allow(unused)]
fn main() {
use std::sync::Arc;
use std::time::Duration;
use vantage_aws::prelude::*;
use vantage_diorama::{Cache, Lens, Loader, LoaderCx, RedbCache, Refresher};
async fn sync(_cx: &LoaderCx) -> vantage_core::Result<()> {
    Ok(())
}
fn run(cache: Arc<RedbCache>) -> vantage_core::Result<()> {
let lens = Lens::builder()
    .cache(Cache::backend(cache))
    // Pre-fetch: the eager start blocks the Dio's build (the default),
    // so the server starts answering only once the listing is cached.
    .loader(Loader::eager(|cx: LoaderCx| async move { sync(&cx).await }))
    .refresher(Refresher::new(|_cx: LoaderCx| async { Ok(()) }).every(Duration::from_secs(60)))
    .build()
    .context("Failed to build lens")?;
let _ = lens;
Ok(())
}
}

A cold start lists the 1,122 files in two requests before the port opens; a restart resumes from redb and confirms with a single request:

fetched 1000 files in 831.454208ms
fetched 122 files in 194.010875ms
serving on http://localhost:3007

The rest of main.rs is mounting:

#![allow(unused)]
fn main() {
use tower_http::services::ServeDir;
use vantage_aws::prelude::*;
async fn serve(api: axum::Router) -> vantage_core::Result<()> {
let app = axum::Router::new()
    .nest("/api/files", api)
    .fallback_service(ServeDir::new("frontend/dist"));

let listener = tokio::net::TcpListener::bind("0.0.0.0:3007")
    .await
    .context("Failed to bind :3007")?;
println!("serving on http://localhost:3007");
axum::serve(listener, app).await.context("server failed")
}
}

Concurrency needs no further code. Dio is a cheap clone over shared state, redb reads run concurrently, and no lock is held across a network await anywhere in the read path — every simultaneous request proceeds on its own.

Neither does authentication — because there is none. The endpoints are anonymous, as in chapter 3: wrap the router in your tower auth middleware before exposing it; Vantage deliberately stays out of authn.

A cache that earns its keep

The detail endpoint returns everything about one file, augment columns included — which means the first GET /{id} downloads the CSV. Chapter 6 deliberately kept contents out of the cache: at 122,000 stations, storing every casually-viewed file would grow cache.redb by gigabytes. But a server sees repeat traffic — the same popular stations, again and again — and re-downloading those is just as wasteful.

ContentsCache splits the difference with lazy admission: a file must be requested twice before its contents earn a slot. It is application code, not framework — the framework contributes open_table (a named key-value table in the Dio’s own redb file); the admission policy is entirely ours to write.

#![allow(unused)]
fn main() {
use std::collections::HashSet;
use std::sync::{Arc, Mutex};
use vantage_diorama::CacheTable;
pub struct ContentsCache {
    table: Arc<dyn CacheTable>,
    /// Keys downloaded at least once — the admission ledger.
    seen: Mutex<HashSet<String>>,
}
}

Its one operation is cache-first with the admission rule on the miss path:

#![allow(unused)]
fn main() {
use std::collections::HashSet;
use std::future::Future;
use std::sync::{Arc, Mutex};
use vantage_aws::prelude::{CborValueExt, VantageResult};
use vantage_diorama::CacheTable;
use vantage_vista::prelude::*;
pub struct ContentsCache {
    table: Arc<dyn CacheTable>,
    seen: Mutex<HashSet<String>>,
}
impl ContentsCache {
pub async fn get_or_fetch<F, Fut>(&self, key: &str, fetch: F) -> VantageResult<String>
where
    F: FnOnce() -> Fut,
    Fut: Future<Output = VantageResult<String>>,
{
    if let Some(row) = self.table.get_value(key).await?
        && let Some(body) = row.get("contents").and_then(|v| v.as_str())
    {
        return Ok(body.to_string());
    }
    let body = fetch().await?;
    let repeat = !self.seen.lock().unwrap().insert(key.to_string());
    if repeat {
        let record = [("contents".to_string(), CborValue::from(body.clone()))]
            .into_iter()
            .collect();
        self.table.insert_value(key, &record).await?;
    }
    Ok(body)
}
}
}

A hit is served from redb. A miss runs fetch either way, but only a key already in seen (a repeat request) gets its body written to the contents table. One-off requests leave nothing behind but their key. (The ledger itself is never trimmed — bare keys, fine at 122,000 files; a bigger keyspace would want an eviction rule, and the policy being application code makes that your call.) The augmenter’s contents lazy expression routes its download through it, and everything downstream — rows, latest, the detail endpoint — is unchanged:

#![allow(unused)]
fn main() {
use std::future::Future;
use std::sync::Arc;
use vantage_aws::prelude::*;
use vantage_types::EmptyEntity;
pub struct ContentsCache;
impl ContentsCache {
    pub async fn get_or_fetch<F, Fut>(&self, _key: &str, fetch: F) -> VantageResult<String>
    where
        F: FnOnce() -> Fut,
        Fut: Future<Output = VantageResult<String>>,
    {
        fetch().await
    }
}
fn table(aws: AwsAccount, bucket: String, contents: Arc<ContentsCache>) -> Table<AwsAccount, EmptyEntity> {
Table::new("restxml/Contents:s3/GET /{Bucket}?list-type=2", aws.clone())
    .with_id_column("Key")
.with_lazy_expression("contents", move |row| {
    let aws = aws.clone();
    let bucket = bucket.clone();
    let contents = contents.clone();
    let key = row
        .get("Key")
        .and_then(|v| v.as_str())
        .unwrap_or_default()
        .to_string();
    async move {
        let fetch_key = key.clone();
        let body = contents
            .get_or_fetch(&key, || async move {
                s3::get_object(&aws, &bucket, &fetch_key).await
            })
            .await?;
        Ok(body.into())
    }
})
}
}

Watching it work

A plain GET is the cache, instantly — rows and latest are null because nothing has demanded them yet:

$ curl 'localhost:3007/api/files?offset=0&limit=3'
{"total":1122,"offset":0,"limit":3,"items":[
  {"index":0,"filename":"csv/by_station/GM000001153.csv","size":"4796352","rows":null,"latest":null},
  …

A detail GET blocks on the bounded facade read — 1.8 s to download and digest 4.8 MB — and the same request again is a 21 ms cache hit:

$ curl 'localhost:3007/api/files/csv%2Fby_station%2FGM000001153.csv'
{"Key":"csv/by_station/GM000001153.csv", …"Size":"4796352", …"rows":140629,"latest":"19911231"}

Now the standing view. curl -N holds the connection; the page arrives as ADDED rows, then each CSV lands as a MODIFIED line the moment its download completes:

$ curl -N 'localhost:3007/api/files?offset=3&limit=3&watch=true'
{"type":"ADDED","object":{"index":3,"filename":"csv/by_station/GM000002288.csv","size":"4880965","rows":null,"latest":null}}
{"type":"ADDED","object":{"index":4,"filename":"csv/by_station/GM000002698.csv","size":"6351302","rows":null,"latest":null}}
{"type":"ADDED","object":{"index":5,"filename":"csv/by_station/GM000002716.csv","size":"5042384","rows":null,"latest":null}}
{"type":"MODIFIED","object":{"index":3,…"rows":143676,"latest":"20260531"}}
{"type":"MODIFIED","object":{"index":4,…"rows":186088,"latest":"20081031"}}
{"type":"MODIFIED","object":{"index":5,…"rows":147011,"latest":"20250824"}}

And the scheduler is visible from outside. Two watches on disjoint pages, side by side: their MODIFIED lines alternate — B, A, B, A — the single worker taking one row from each view’s queue in turn:

19:35:35 [B] {"type":"MODIFIED","object":{"index":42,…"rows":201382,…}}
19:35:36 [A] {"type":"MODIFIED","object":{"index":40,…"rows":77595,…}}
19:35:38 [B] {"type":"MODIFIED","object":{"index":43,…"rows":132579,…}}
19:35:40 [A] {"type":"MODIFIED","object":{"index":41,…"rows":128335,…}}

Two watches on the same page: each row’s MODIFIED reaches both connections in the same second — one download, fanned out by the event bus to every scenery holding the row:

19:36:18 [B] {"type":"MODIFIED","object":{"index":50,…"rows":135802,…}}
19:36:18 [A] {"type":"MODIFIED","object":{"index":50,…"rows":135802,…}}

Run the first GET again afterwards: the page the watches paid for now answers with its numbers filled in. Current knowledge grew.

A tiny React client

The frontend is one component. It fetches its page with watch=true and reads the NDJSON stream directly off fetch — no client library, twenty lines:

const res = await fetch(`/api/files?offset=${offset}&limit=${LIMIT}&watch=true`, {
  signal: ctl.signal,
})
const reader = res.body.getReader()
// …accumulate chunks, split on '\n'…
const event = JSON.parse(line)
setRows(rs => ({ ...rs, [event.object.index]: event.object }))

ADDED and MODIFIED merge the same way — the object lands at its index — so the table renders the listing immediately and the Rows / Latest cells flip from … to numbers as the watch delivers them, exactly like chapter 7’s terminal cells did. Paging aborts the fetch via the AbortController, which closes the connection, which drops the scenery server-side. Clicking a row hits the detail endpoint into a side panel. npm run build in learn-7/frontend emits dist/, the ServeDir fallback serves it, and http://localhost:3007 is a bucket explorer: title, file count, Prev/Next, and a table filling itself in.

What we covered

Dio & Lens over SQL

This path builds a live, cached, watchable view in front of a SQL database you own. We start from a Vista over a SQLite product table: a runtime handle that can already sort, search, paginate and write. That is a capable backend, which raises a fair question. If the database already does all that, what is a caching-and-events layer for?

For the two things a capable backend doesn’t hand you on its own. A Dio keeps a live local copy of a data segment, so reads are answered from memory instead of round-trips, and it announces every change on an event bus, so a view can watch the data instead of re-polling it. A slow, read-only API needs a third thing from Diorama: the injection of capabilities the backend lacks, like sorting a listing it cannot sort. A SQL database needs none of that. Here the live cache and the change stream are the whole story.

We’ll build a bar’s shelf: a handful of products, each with a stock count, behind a cached, watchable API. A background “till” sells items so the data moves. This chapter builds the cache; the next serves it as a live watch; the one after moves the whole thing to PostgreSQL without touching the model.

The full crate is learn-8.

The model

The Introduction’s Product, with one column added: stock, the units on the shelf.

#![allow(unused)]
fn main() {
// src/product.rs
use vantage_sql::prelude::*;
use vantage_types::prelude::*;

#[entity(SqliteType)]
#[derive(Debug, Clone, Default)]
pub struct Product {
    pub name: String,
    pub price: i64,
    pub stock: i64,
}

impl Product {
    pub fn table(db: SqliteDB) -> Table<SqliteDB, Product> {
        Table::new("product", db)
            .with_id_column("id")
            .with_column_of::<String>("name")
            .with_column_of::<i64>("price")
            .with_column_of::<i64>("stock")
    }
}
}

Nothing here knows it is about to be cached, watched, or served from Postgres. The entity is written once, and the layers above it decide what to do with it.

The master

The master is the product table as a Vista, built the way the Introduction built one: hand the table to its datasource’s Vista factory.

use vantage_sql::prelude::*;
use vantage_types::prelude::*;

#[entity(SqliteType)]
#[derive(Debug, Clone, Default)]
pub struct Product {
    pub name: String,
    pub price: i64,
    pub stock: i64,
}

impl Product {
    pub fn table(db: SqliteDB) -> Table<SqliteDB, Product> {
        Table::new("product", db)
            .with_id_column("id")
            .with_column_of::<String>("name")
            .with_column_of::<i64>("price")
            .with_column_of::<i64>("stock")
    }
}

#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
const DB_PATH: &str = concat!(env!("CARGO_MANIFEST_DIR"), "/products.db");

let url = format!("sqlite:{DB_PATH}?mode=rwc");
let db = SqliteDB::connect(&url).await.context("connect sqlite")?;
let master = db.vista_factory().from_table(Product::table(db.clone()))?;
Ok(())
}

The database is a file beside the crate, and mode=rwc creates it on first run. A file rather than sqlite::memory: is deliberate: the next chapter edits it from another terminal with the sqlite3 CLI and watches the change arrive.

Ask this Vista what it can do and its capabilities cover everything a listing needs: Order, Search, FilterOps and Window, plus Insert, Update and Delete. Contrast that with the S3 path, where the Vista admits it cannot sort or search. There is no gap to fill here, so this path injects no capabilities. What we still want is the cache.

The lens: eager and reactive

The S3 path’s loader walks the bucket a page at a time, resuming from a cursor, and writes each page into a redb file. A SQL table needs none of that: it hands back the whole set in one call. So this path uses an eager loader that lists the table once, and a refresher that lists it again on a timer.

use std::time::Duration;

use vantage_diorama::prelude::*;
use vantage_diorama::{Cache, Loader, LoaderCx, Refresher};
use vantage_sql::prelude::*;
use vantage_types::prelude::*;

#[entity(SqliteType)]
#[derive(Debug, Clone, Default)]
pub struct Product {
    pub name: String,
    pub price: i64,
    pub stock: i64,
}

impl Product {
    pub fn table(db: SqliteDB) -> Table<SqliteDB, Product> {
        Table::new("product", db)
            .with_id_column("id")
            .with_column_of::<String>("name")
            .with_column_of::<i64>("price")
            .with_column_of::<i64>("stock")
    }
}

#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
let db = SqliteDB::connect("sqlite:products.db?mode=rwc").await.context("connect sqlite")?;
let master = db.vista_factory().from_table(Product::table(db.clone()))?;
let lens = Lens::builder()
    .cache(Cache::memory())
    .loader(Loader::eager(reload))
    .refresher(Refresher::new(reload).every(Duration::from_secs(1)))
    .build()
    .context("build lens")?;
let dio = lens.dio(master).build().await?;
Ok(())
}

/// Make the cache hold exactly the master's rows.
async fn reload(cx: LoaderCx) -> VantageResult<()> {
    let rows = cx.master().list_values().await?;
    cx.reconcile(rows).await
}

Three settings carry the behaviour:

• Cache::memory(): the cache is ephemeral, and a restart re-reads the table. The S3 path persists its cache with Cache::redb("cache.redb") because re-listing a remote bucket is expensive. Re-reading a local table is not.
• Loader::eager(reload): before the server answers a request, list_values() pulls the whole table into the cache. lens.dio(master).build() waits for it by default, so the first read is already warm.
• Refresher::new(reload).every(..): once a second, reload runs again. This is what makes the copy live: a sale the till commits, or an edit typed into sqlite3, lands on the next tick.

The loader and the refresher share one function because both answer the same question: what should the cache hold? LoaderCx::reconcile takes the master’s answer and makes the cache hold exactly that set. It writes the rows that differ, deletes the cached rows the master no longer returns (a product that sold out), leaves unchanged rows alone, and publishes one DatasetChanged for watchers.

Why reconcile instead of clearing and reloading?

The obvious refresh clears the cache and inserts the fresh rows. It has two problems.

The first is the gap. Between the clear and the insert the cache is empty, and a reader that lands in that gap sees an empty shelf. Reconcile never empties the cache: each row is kept, rewritten, or removed.

The second is writes in flight. A Dio takes writes optimistically: the cache shows the new value at once, and the master catches up. A refresh that listed the master just before that write landed would put the old value back. Reconcile skips any row with a write in flight, and any row whose write settled after this refresh began listing, so a refresh can’t undo a write it raced.

The cost is a comparison per row on every tick. For a shelf of products that is noise. For a table of a million rows, a one-second full reconcile is the wrong tool, and the PostgreSQL chapters replace the timer with push.

Reads come from the cache

From here on, reads don’t touch SQLite. Everything a consumer asks of the Dio, a listing, a page or a single record, is answered from the in-memory cache the eager loader warmed. The database is consulted by the once-a-second reconcile and by nothing else. The next chapter puts a DioRouter over this Dio, and a plain GET returns the shelf without a query reaching the table.

The backend stays the source of truth, but the handle you read is a local, live mirror of it.

Checklist

• The model is a plain Table with an #[entity] struct; nothing in it mentions caching.
• The master is db.vista_factory().from_table(..), and a SQL Vista needs no capability injected.
• One reload function lists the master and hands the rows to cx.reconcile; the loader and the refresher both call it.
• Cache::memory() because re-reading a local table is cheap; Cache::redb(..) when it isn’t.
• Refresher::new(..).every(..) is what brings outside writes in, until the PostgreSQL chapters switch to push.

Next: serve this Dio as a Kubernetes-style watch, and turn on the till so the shelf moves while you look at it.

Serving & Watching Live Stock

The last chapter left a Dio holding a live copy of the product table. Now we put it behind HTTP, using the same adapter the S3 path uses. DioRouter doesn’t know or care that its Dio is backed by SQL instead of S3: it binds a Dio to a pair of routes, GET and watch, and that is all it needs.

use vantage_api_adapters::axum_dio::DioRouter;
use std::time::Duration;

use vantage_diorama::prelude::*;
use vantage_diorama::{Cache, Loader, LoaderCx, Refresher};
use vantage_sql::prelude::*;
use vantage_types::prelude::*;

#[entity(SqliteType)]
#[derive(Debug, Clone, Default)]
pub struct Product {
    pub name: String,
    pub price: i64,
    pub stock: i64,
}

impl Product {
    pub fn table(db: SqliteDB) -> Table<SqliteDB, Product> {
        Table::new("product", db)
            .with_id_column("id")
            .with_column_of::<String>("name")
            .with_column_of::<i64>("price")
            .with_column_of::<i64>("stock")
    }
}

async fn reload(cx: LoaderCx) -> VantageResult<()> {
    let rows = cx.master().list_values().await?;
    cx.reconcile(rows).await
}

#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
let db = SqliteDB::connect("sqlite:products.db?mode=rwc").await.context("connect sqlite")?;
let master = db.vista_factory().from_table(Product::table(db.clone()))?;
let lens = Lens::builder()
    .cache(Cache::memory())
    .loader(Loader::eager(reload))
    .refresher(Refresher::new(reload).every(Duration::from_secs(1)))
    .build()
    .context("build lens")?;
let dio = lens.dio(master).build().await?;
let listener = tokio::net::TcpListener::bind("0.0.0.0:3008").await.context("bind :3008")?;

let api = DioRouter::new(dio.clone())
    .with_column("name", "name")
    .with_column("price", "price")
    .with_column("stock", "stock")
    .with_page_size(50)
    .into_router();

let app = axum::Router::new().nest("/api/products", api);
axum::serve(listener, app).await.context("server failed")?;
Ok(())
}

Each with_column maps a JSON key to a record field. The columns double as the demand of every watch the router opens, which matters on an augmented Dio and is harmless here.

A plain GET is the cache, answered at once, never a query against SQLite:

$ curl 'localhost:3008/api/products?offset=0&limit=10'
{"total":5,"offset":0,"limit":10,"items":[
  {"index":0,"name":"Espresso","price":280,"stock":12},
  {"index":1,"name":"Cappuccino","price":340,"stock":8},
  {"index":2,"name":"Cold Brew","price":420,"stock":5},
  {"index":3,"name":"Croissant","price":260,"stock":6},
  {"index":4,"name":"Cheesecake","price":520,"stock":2}]}

A till to make it move

A cache over a table nobody writes is a slow constant. To see the reactive stack work, the data has to change, so the app runs a till: a background task that sells one unit of a random product every 800ms, removes anything that sells out, and every tenth tick takes a delivery. These are ordinary SQL writes, sent with sqlx::query(..).execute(db.pool()) on the same pool the master reads from (src/sim.rs):

-- sell one unit of a random product that still has stock
UPDATE product SET stock = stock - 1
 WHERE id = (SELECT id FROM product WHERE stock > 0 ORDER BY RANDOM() LIMIT 1);

-- anything that hit zero leaves the shelf
DELETE FROM product WHERE stock <= 0;

The server never calls the till and the till never calls the server. They meet only through the database and the once-a-second reconcile, so any writer, in any process, moves the shelf. The sqlite3 CLI will do as well, as we’ll see.

Why not have the till write through the Dio?

A Dio is writable, and a write through it shows up in every watch at once instead of on the next tick. For an app that owns all its writes, that is the better path.

This app doesn’t own all its writes. A delivery script, a back-office tool, or a person with sqlite3 changes the table behind the server’s back, and a Dio that only learned of its own writes would drift from the database. The till writes straight to SQL on purpose: it stands in for every writer the server can’t see. The price is latency, up to one refresh interval, and the PostgreSQL chapters remove it.

The watch

curl -N holds the connection open. The page arrives as ADDED lines, the current shelf, and then every sale streams back as a MODIFIED line once the reconcile picks it up:

$ curl -N 'localhost:3008/api/products?watch=true'
{"type":"ADDED","object":{"index":0,"name":"Espresso","price":280,"stock":12}}
{"type":"ADDED","object":{"index":1,"name":"Cappuccino","price":340,"stock":8}}
{"type":"ADDED","object":{"index":2,"name":"Cold Brew","price":420,"stock":5}}
{"type":"ADDED","object":{"index":3,"name":"Croissant","price":260,"stock":6}}
{"type":"ADDED","object":{"index":4,"name":"Cheesecake","price":520,"stock":2}}
{"type":"MODIFIED","object":{"index":4,"name":"Cheesecake","price":520,"stock":1}}
{"type":"MODIFIED","object":{"index":2,"name":"Cold Brew","price":420,"stock":4}}
{"type":"MODIFIED","object":{"index":0,"name":"Espresso","price":280,"stock":9}}

That is the reactive stack running over SQL, end to end: a SQL write, reconciled into the cache, one DatasetChanged event, a scenery re-derive, a diff against what this connection last saw, one NDJSON line on the wire. Nothing in that chain is specific to SQL. It is the same machinery the S3 path serves the NOAA bucket with.

It really is the database

The strongest proof that this isn’t an in-process illusion: change the table from outside the app. With a watch open, in another terminal:

$ sqlite3 learn-8/products.db "UPDATE product SET price=999 WHERE id='p1'"

and the open watch stream reflects it within the second:

{"type":"MODIFIED","object":{"index":0,"name":"Espresso","price":999,"stock":11}}

No API call and no shared memory: a separate process wrote a row, the reconcile read it, and the change fanned out to every watcher. The Dio is a mirror of the database, not a copy that happens to agree with it.

Checklist

• DioRouter::new(dio).with_column(..).into_router() serves GET and watch over any Dio; nest it under the resource’s path.
• A plain GET reads the cache; a ?watch=true opens a scenery and streams NDJSON.
• Writers outside the server reach watchers through the refresher’s reconcile, within one interval.
• Without .key_by(..), a watch diffs by position and never sends DELETED.

Next: the same app, the same shelf, the same watch, served from PostgreSQL instead of a file and changed by flipping one compile flag.

Moving to PostgreSQL

A file on disk is a fine place for a bar to start. When it outgrows one, with concurrent writers, a second instance or a backup story, you reach for a database server. In most stacks that is a migration project. Here it is a refactor that adds one file and changes two attributes, and then a compile flag decides which backend the binary speaks.

The finished crate is learn-9: the same app as learn-8, made backend-parametric.

Name the backend once

learn-8 wrote SqliteDB in a handful of places. The refactor pulls every one of them into a single alias, so the concrete backend is named in one file:

#![allow(unused)]
fn main() {
// src/db.rs: the only file that names a database
use vantage_sql::prelude::*;

#[cfg(not(feature = "pg"))]
pub type Db = vantage_sql::sqlite::SqliteDB;
#[cfg(feature = "pg")]
pub type Db = vantage_sql::postgres::PostgresDB;

pub async fn connect() -> VantageResult<Db> {
    #[cfg(not(feature = "pg"))]
    let url = format!("sqlite:{}?mode=rwc", concat!(env!("CARGO_MANIFEST_DIR"), "/products.db"));
    #[cfg(feature = "pg")]
    let url = std::env::var("DATABASE_URL")
        .map_err(|_| vantage_core::error!("DATABASE_URL must be set for the `pg` build"))?;

    Db::connect(&url).await.context("connect db")
}
}

PostgresDB::connect, db.pool() and db.vista_factory().from_table(..) are the same calls as their SQLite twins, so connect() is the whole of the connection difference: a file path on one side, a DATABASE_URL on the other. db.rs also holds the CREATE TABLE; it is plain SQL both backends accept.

Two attributes on the model

The entity marker names whichever backend is compiled in, and the table builder takes the Db alias instead of SqliteDB. That is the entire change to the model:

#![allow(unused)]
fn main() {
use vantage_sql::prelude::*;
use vantage_types::prelude::*;
#[cfg(feature = "pg")]
use vantage_sql::postgres::AnyPostgresType;
#[cfg(not(feature = "pg"))]
type Db = vantage_sql::sqlite::SqliteDB;
#[cfg(feature = "pg")]
type Db = vantage_sql::postgres::PostgresDB;
// src/product.rs
#[cfg_attr(not(feature = "pg"), entity(SqliteType))]
#[cfg_attr(feature = "pg", entity(PostgresType))]
#[derive(Debug, Clone, Default)]
pub struct Product {
    pub name: String,
    pub price: i64,
    pub stock: i64,
}

impl Product {
    pub fn table(db: Db) -> Table<Db, Product> {
        Table::new("product", db)
            .with_id_column("id")
            .with_column_of::<String>("name")
            .with_column_of::<i64>("price")
            .with_column_of::<i64>("stock")
    }
}
}

The builder body, the columns and the id, doesn’t move. #[entity(..)] takes a list of the backends a struct serves, not a choice of one: #[entity(SqliteType, PostgresType)] generates both sets of conversions. learn-9 picks one with cfg_attr only so that the build it isn’t running doesn’t need the other driver compiled in.

Everything else is unchanged: main.rs, sim.rs, the lens, the Dio and the DioRouter. They were written against Db and the portable data-set methods, so they never named a backend. The reconcile still calls cx.master().list_values(); the till still runs the same UPDATE … ORDER BY RANDOM(); the watch still streams ADDED/MODIFIED.

Why a compile flag and not a runtime switch?

You could switch at runtime. Everything from the master up is a type-erased Vista, so a match on the URL scheme that builds the master from either factory would hand the same Dio code either backend.

The parts below the master are typed by backend, though. Table<Db, Product> and the till’s db.pool() are a SQLite type or a Postgres type, so a runtime switch would compile both drivers and write those parts twice, once per arm. learn-9 serves one backend per deployment, so it takes the flag: one alias, one driver in the binary. An app that must speak both at once keeps the match, and pays for both drivers.

Run it on Postgres

Start a local Postgres (the repo ships the script), point DATABASE_URL at it, and build with the flag:

$ vantage-sql/scripts/postgres/start.sh          # docker: postgres:17-alpine on :5433
$ export DATABASE_URL='postgres://vantage:vantage@localhost:5433/vantage'
$ cargo run -p learn-9 --no-default-features --features pg
serving on http://localhost:3009 — try: curl -N 'localhost:3009/api/products?watch=true'

The shelf serves and streams as it did on SQLite, with the same routes, the same NDJSON and the same till:

$ curl -N 'localhost:3009/api/products?watch=true'
{"type":"ADDED","object":{"index":0,"name":"Espresso","price":280,"stock":12}}
{"type":"ADDED","object":{"index":1,"name":"Cappuccino","price":340,"stock":8}}
...
{"type":"MODIFIED","object":{"index":4,"name":"Cheesecake","price":520,"stock":1}}

The it-really-is-the-database proof works over Postgres too. Edit a row with psql and watch it land:

$ psql "$DATABASE_URL" -c "UPDATE product SET price=777 WHERE id='p2'"
{"type":"MODIFIED","object":{"index":1,"name":"Cappuccino","price":777,"stock":6}}

What this proves

The reactive stack, cache, events, sceneries and watch server, ran unchanged over a file-backed SQLite database and a networked PostgreSQL server, and the code that moved between them was one alias and two attributes. The repo’s vantage-sql/tests/postgres/6_vista.rs exercises the same Vista over a real Postgres.

Checklist

• One Db alias in db.rs names the backend; nothing else does.
• #[cfg_attr(.., entity(..))] picks the entity marker for the compiled backend.
• main.rs, the lens, the refresher and the router don’t change.
• The till’s statements and the schema stay within the SQL both dialects accept.
• --no-default-features --features pg and a DATABASE_URL run the same app on Postgres.

There is one thing left on the table. The app reconciles on a one-second timer: it re-reads the table whether or not anything changed, and it is always up to a beat behind. A file on disk can’t do better; a server can. Next, we commit to Postgres and trade that poll for push: the database tells us the moment a row changes.

Real-Time Push with LISTEN/NOTIFY

The app so far reconciles on a timer: Refresher::new(reload).every(Duration::from_secs(1)). That is polling. Once a second the Dio re-reads the master and reconciles. It works, but it is a beat behind every change, and it re-queries even when nothing moved. A file-backed SQLite database has no better option, because it can’t call you back. A PostgreSQL server can: it can tell you the instant a row changes, and then the poll goes away.

Having shown the app is portable, this chapter commits to Postgres: it drops the feature flag, deletes sim.rs, moves the writer into its own process, and replaces the refresher with one call on the Dio builder, .live(). The finished crate is learn-10.

Teach the database to announce changes

Postgres LISTEN/NOTIFY is a publish/subscribe channel built into the server. A trigger calls pg_notify on every write, and anyone LISTENing on that channel wakes up. So the only schema added beyond the table is a trigger. db::setup sends these statements, each through sqlx::query(..).execute(db.pool()), after CREATE TABLE product (..):

CREATE OR REPLACE FUNCTION product_notify() RETURNS trigger AS $$
BEGIN PERFORM pg_notify('product_changed', ''); RETURN NULL; END;
$$ LANGUAGE plpgsql;

DROP TRIGGER IF EXISTS product_notify_trg ON product;

CREATE TRIGGER product_notify_trg
AFTER INSERT OR UPDATE OR DELETE ON product
FOR EACH STATEMENT EXECUTE FUNCTION product_notify();

FOR EACH STATEMENT fires the notify once per statement, not per row: a single “something changed” ping is all the Dio needs to reconcile. The payload is empty on purpose (more on that below). The channel is product_changed because the Postgres Vista listens on {table}_changed.

Tell the Vista the trigger exists

Postgres offers no way to ask “is anything going to notify me on this channel?”. A LISTEN on a channel nobody feeds succeeds and then waits forever. A Vista that assumed it could push because the table is writable would advertise a feed that may never arrive, and no consumer could tell that apart from a quiet table. So the application, the only party that knows, says so:

use vantage_sql::postgres::{AnyPostgresType, PostgresDB};
use vantage_sql::prelude::*;
use vantage_types::prelude::*;
use vantage_vista::SortDirection;

#[entity(PostgresType)]
#[derive(Debug, Clone, Default)]
pub struct Product {
    pub name: String,
    pub price: i64,
    pub stock: i64,
    pub created: i64,
}

impl Product {
    pub fn table(db: PostgresDB) -> Table<PostgresDB, Product> {
        Table::new("product", db)
            .with_id_column("id")
            .with_column_of::<String>("name")
            .with_column_of::<i64>("price")
            .with_column_of::<i64>("stock")
            .with_column_of::<i64>("created")
    }
}

#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
let url = std::env::var("DATABASE_URL").context("DATABASE_URL must be set")?;
let db = PostgresDB::connect(&url).await.context("connect postgres")?;
let mut master = db
    .vista_factory()
    .with_notify(true) // db::setup installed product_notify_trg
    .from_table(Product::table(db.clone()))?;
master.set_order(Some(("created", SortDirection::Ascending)))?;
Ok(())
}

with_notify(true) is what lets the Vista advertise Subscribe. Leave it off and the live follow below does nothing. That matters here, because this chapter deletes the refresher: push and a refresher’s timer are the two ways the cache learns about a write from outside, and an app that declines both never updates.

The model gained a created column, an epoch-millis stamp set on delivery. The shelf is ordered by it, so a drink keeps its place as it sells and new deliveries append at the end.

One call replaces the poll

The server builds the Lens as before, with an eager in-memory cache and a loader that reconciles, but with no refresher. The Dio builder gets .live() instead:

use vantage_diorama::prelude::*;
use vantage_diorama::{Cache, Loader, LoaderCx};
use vantage_sql::postgres::{AnyPostgresType, PostgresDB};
use vantage_sql::prelude::*;
use vantage_types::prelude::*;

#[entity(PostgresType)]
#[derive(Debug, Clone, Default)]
pub struct Product {
    pub name: String,
    pub price: i64,
    pub stock: i64,
    pub created: i64,
}

impl Product {
    pub fn table(db: PostgresDB) -> Table<PostgresDB, Product> {
        Table::new("product", db)
            .with_id_column("id")
            .with_column_of::<String>("name")
            .with_column_of::<i64>("price")
            .with_column_of::<i64>("stock")
            .with_column_of::<i64>("created")
    }
}

#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
let url = std::env::var("DATABASE_URL").context("DATABASE_URL must be set")?;
let db = PostgresDB::connect(&url).await.context("connect postgres")?;
let master = db
    .vista_factory()
    .with_notify(true)
    .from_table(Product::table(db.clone()))?;
let lens = Lens::builder()
    .cache(Cache::memory())
    .loader(Loader::eager(|cx: LoaderCx| async move {
        let rows = cx.master().list_values().await?;
        cx.reconcile(rows).await
    }))
    .build()
    .context("build lens")?;

let dio = lens.dio(master).live().build().await?;
Ok(())
}

That is the whole change. live() checks whether the master advertises Subscribe. Because we opted in above, the Postgres Vista does, and the Dio opens a LISTEN on product_changed. Each notification arrives as a coarse change, “the set changed”, and the Dio answers it with a refresh. With no refresher on the Lens, a refresh re-runs the eager loader, so the same cx.reconcile(rows) that warmed the cache brings it up to date. Over a master that can’t push, live() does nothing and a refresher carries the load instead.

Why not listen for NOTIFY yourself?

You can. A PgListener on product_changed and a loop is a dozen lines, and an event source the Vista doesn’t model (a webhook, a Kafka topic, a SaaS callback) needs exactly that. The loop hands each event to the Dio with apply_change:

use vantage_diorama::prelude::*;
use vantage_diorama::{Cache, Loader, LoaderCx};
use vantage_sql::postgres::{AnyPostgresType, PostgresDB};
use vantage_sql::prelude::*;
use vantage_types::prelude::*;

#[entity(PostgresType)]
#[derive(Debug, Clone, Default)]
pub struct Product {
    pub name: String,
    pub price: i64,
    pub stock: i64,
    pub created: i64,
}

impl Product {
    pub fn table(db: PostgresDB) -> Table<PostgresDB, Product> {
        Table::new("product", db)
            .with_id_column("id")
            .with_column_of::<String>("name")
            .with_column_of::<i64>("price")
            .with_column_of::<i64>("stock")
            .with_column_of::<i64>("created")
    }
}

#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
let url = std::env::var("DATABASE_URL").context("DATABASE_URL must be set")?;
let db = PostgresDB::connect(&url).await.context("connect postgres")?;
let master = db.vista_factory().from_table(Product::table(db.clone()))?;
let lens = Lens::builder()
    .cache(Cache::memory())
    .loader(Loader::eager(|cx: LoaderCx| async move {
        let rows = cx.master().list_values().await?;
        cx.reconcile(rows).await
    }))
    .build()
    .context("build lens")?;
let dio = lens.dio(master).build().await?;
// A payload-less ping: re-read the set.
dio.apply_change(ChangeEvent::Invalidated).await?;

// A source that names the row: drop it from the cache, no re-read.
dio.apply_change(ChangeEvent::Deleted { id: "d42".into() }).await?;
Ok(())
}

live() applies changes the same way, so for a Vista that advertises Subscribe the hand-written loop buys nothing and costs two things live() already gets right. It subscribes before the eager loader copies the set and applies what arrived once the copy lands, so a write during the copy is neither lost nor overwritten by it. And a follow that falls behind the stream re-reads the set instead of dropping changes. Write the loop for sources the Vista can’t reach; use live() for the ones it can.

The till becomes its own process

To drive it, learn-10 ships a second binary, the mutator, run alongside the server:

$ cargo run -p learn-10                 # the reactive server, port 3010
$ cargo run -p learn-10 --bin mutator   # a separate writer process

The mutator writes through Vantage’s active-entity API, with no hand-written SQL. list_entities() hands back drinks that each carry their own id and datasource, so selling one is:

use rand::seq::SliceRandom;
use vantage_dataset::prelude::ActiveEntitySet;
use vantage_sql::postgres::{AnyPostgresType, PostgresDB};
use vantage_sql::prelude::*;
use vantage_types::prelude::*;

#[entity(PostgresType)]
#[derive(Debug, Clone, Default)]
pub struct Product {
    pub name: String,
    pub price: i64,
    pub stock: i64,
    pub created: i64,
}

impl Product {
    pub fn table(db: PostgresDB) -> Table<PostgresDB, Product> {
        Table::new("product", db)
            .with_id_column("id")
            .with_column_of::<String>("name")
            .with_column_of::<i64>("price")
            .with_column_of::<i64>("stock")
            .with_column_of::<i64>("created")
    }
}

#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
let url = std::env::var("DATABASE_URL").context("DATABASE_URL must be set")?;
let db = PostgresDB::connect(&url).await.context("connect postgres")?;
let table = Product::table(db.clone());
let mut shelf = table.list_entities().await?;
let drink = shelf.choose_mut(&mut rand::thread_rng()).unwrap();
if drink.stock <= 1 {
    drink.delete().await?; // last unit: off the shelf
} else {
    drink.stock -= 1;
    drink.save().await?; // one sale
}
Ok(())
}

Two independent processes that share nothing but the database and the notify channel: the strongest proof that the reactive stack tracks the data, not an in-process shortcut.

Removals, and a UI to see them

This app also keys its watch by identity, so a sold-out drink is reported as a removal rather than a list that shrinks:

use vantage_api_adapters::axum_dio::DioRouter;
use vantage_diorama::prelude::*;
use vantage_diorama::{Cache, Loader, LoaderCx};
use vantage_sql::postgres::{AnyPostgresType, PostgresDB};
use vantage_sql::prelude::*;
use vantage_types::prelude::*;

#[entity(PostgresType)]
#[derive(Debug, Clone, Default)]
pub struct Product {
    pub name: String,
    pub price: i64,
    pub stock: i64,
    pub created: i64,
}

impl Product {
    pub fn table(db: PostgresDB) -> Table<PostgresDB, Product> {
        Table::new("product", db)
            .with_id_column("id")
            .with_column_of::<String>("name")
            .with_column_of::<i64>("price")
            .with_column_of::<i64>("stock")
            .with_column_of::<i64>("created")
    }
}

#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
let url = std::env::var("DATABASE_URL").context("DATABASE_URL must be set")?;
let db = PostgresDB::connect(&url).await.context("connect postgres")?;
let master = db
    .vista_factory()
    .with_notify(true)
    .from_table(Product::table(db.clone()))?;
let lens = Lens::builder()
    .cache(Cache::memory())
    .loader(Loader::eager(|cx: LoaderCx| async move {
        let rows = cx.master().list_values().await?;
        cx.reconcile(rows).await
    }))
    .build()
    .context("build lens")?;
let dio = lens.dio(master).live().build().await?;
let api = DioRouter::new(dio.clone())
    .with_column("id", "id")
    .with_column("name", "name")
    .with_column("price", "price")
    .with_column("stock", "stock")
    .key_by("id") // identity-keyed: emits DELETED when a row leaves
    .with_page_size(50)
    .into_router();
Ok(())
}

With key_by("id"), the watch stream gains a third event, DELETED, alongside ADDED and MODIFIED. The key field has to be one of the with_column fields, which is why id is exposed now. A small React page (frontend/index.html, served by the same axum app through ServeDir) consumes the stream and animates each drink: arriving with a flash, ticking down as it sells, and leaving with a “sold out” stamp when a DELETED lands. It is plain React over the watch endpoint, not part of Vantage, and the next chapter reuses the same file unchanged.

Watch it on the wire:

$ curl -N 'localhost:3010/api/products?watch=true'
{"type":"ADDED","object":{"index":0,"id":"d51...","name":"Mojito","price":1000,"stock":11}}
{"type":"ADDED","object":{"index":1,"id":"d59...","name":"Boulevardier","price":1400,"stock":12}}
{"type":"MODIFIED","object":{"index":0,"id":"d51...","name":"Mojito","price":1000,"stock":10}}
{"type":"DELETED","object":{"index":0,"id":"d59...","name":"Boulevardier","price":1400,"stock":1}}

Every line is a database write, announced by Postgres, reconciled into the cache, diffed by the scenery and fanned out, with no timer anywhere in the loop.

Checklist

• A FOR EACH STATEMENT trigger calls pg_notify('{table}_changed', '') on every write.
• vista_factory().with_notify(true) declares the trigger, so the Vista advertises Subscribe.
• lens.dio(master).live().build() follows the channel; each ping refreshes the Dio, and with no refresher that re-runs the eager loader’s reconcile.
• Without with_notify(true) and without a refresher, the cache never sees an outside write.
• dio.apply_change(ChangeEvent::..) is the entry for sources the Vista can’t subscribe to.
• .key_by("id") on the router turns sell-outs into DELETED events.

Next: the same app, the same frontend, a different database. SurrealDB’s live queries push the change itself rather than a ping.

Switching to SurrealDB

Postgres was a migration within SQL: a different server, the same dialect family. SurrealDB is a different database: multi-model, schemaless by default, and reached over a WebSocket rather than a pooled SQL connection. Adopting it tests the claim this path has been making. The goal is to change as little as possible, and then to cash in the one thing SurrealDB does better than a SQL server: it pushes the changed row, not a ping.

The finished crate is learn-11. Set it next to learn-10 and the diff is small, which is the result we want.

What changes

Three places name the backend, and nothing else does.

The entity marker: swap the backend in the attribute.

#![allow(unused)]
fn main() {
use vantage_surrealdb::types::AnySurrealType;
use vantage_types::entity;
#[entity(SurrealType)] // was: #[entity(PostgresType)]
#[derive(Debug, Clone, Default)]
pub struct Product {
    pub name: String,
    pub price: i64,
    pub stock: i64,
    pub created: i64,
}
}

The table builder: the same columns, built against SurrealDB. SurrealDB’s native id is a Thing (product:⟨key⟩).

#![allow(unused)]
fn main() {
use vantage_surrealdb::surrealdb::SurrealDB;
use vantage_surrealdb::types::AnySurrealType;
use vantage_table::table::Table;
use vantage_types::entity;

#[entity(SurrealType)]
#[derive(Debug, Clone, Default)]
pub struct Product {
    pub name: String,
    pub price: i64,
    pub stock: i64,
    pub created: i64,
}

impl Product {
pub fn surreal_table(db: SurrealDB) -> Table<SurrealDB, Product> {
    Table::new("product", db)
        .with_id_column("id")
        .with_column_of::<String>("name")
        .with_column_of::<i64>("price")
        .with_column_of::<i64>("stock")
        .with_column_of::<i64>("created")
}
}
}

Connect and setup: a DSN instead of a DATABASE_URL, and no trigger. Where Postgres needed a pg_notify function and an AFTER … FOR EACH STATEMENT trigger, SurrealDB emits change frames on its own. Setup makes sure the table exists:

use surreal_client::SurrealConnection;
use vantage_core::{Result, error};
use vantage_surrealdb::surrealdb::SurrealDB;

#[tokio::main(flavor = "current_thread")]
async fn main() -> Result<()> {
let dsn = std::env::var("SURREALDB_URL")
    .unwrap_or_else(|_| "cbor://root:root@localhost:8000/bar/v1".to_string());

let client = SurrealConnection::dsn(&dsn)
    .map_err(|e| error!("invalid SURREALDB_URL DSN", details = e.to_string()))?
    .connect()
    .await
    .map_err(|e| error!("connect surrealdb", dsn = &dsn, details = e.to_string()))?;

// The one bit of schema: a table for LIVE SELECT to attach to.
client
    .query("DEFINE TABLE IF NOT EXISTS product SCHEMALESS", None)
    .await
    .map_err(|e| error!("define product table", details = e.to_string()))?;

let db = SurrealDB::new(client);
Ok(())
}

The till (the mutator) is the same active-entity code as before. Only the id it hands new_entity is a Thing:

use vantage_core::{Context, Result};
use vantage_dataset::prelude::ActiveEntitySet;
use vantage_surrealdb::surrealdb::SurrealDB;
use vantage_surrealdb::thing::Thing;
use vantage_surrealdb::types::AnySurrealType;
use vantage_table::table::Table;
use vantage_types::entity;

#[entity(SurrealType)]
#[derive(Debug, Clone, Default)]
pub struct Product {
    pub name: String,
    pub price: i64,
    pub stock: i64,
    pub created: i64,
}

impl Product {
    pub fn surreal_table(db: SurrealDB) -> Table<SurrealDB, Product> {
        Table::new("product", db)
            .with_id_column("id")
            .with_column_of::<String>("name")
            .with_column_of::<i64>("price")
            .with_column_of::<i64>("stock")
            .with_column_of::<i64>("created")
    }
}

async fn deliver(
    table: &Table<SurrealDB, Product>,
    key: String,
    name: String,
    price: i64,
    stock: i64,
    created: i64,
) -> Result<()> {
table
    .new_entity(
        Thing::new("product", key), // was: a String id
        Product { name, price, stock, created },
    )
    .save()
    .await?;
Ok(())
}

fn main() {}

What doesn’t change

The server’s reactive core is untouched. The Vista comes from SurrealVistaFactory::new(db.clone()) (the factory db.vista_factory() returns on a SurrealDB) instead of the Postgres factory, and the Lens and the Dio builder are line for line the Postgres chapter’s, including the call that matters:

use surreal_client::SurrealConnection;
use vantage_core::{Context, Result, error};
use vantage_dataset::prelude::*;
use vantage_diorama::prelude::*;
use vantage_diorama::{Cache, Loader, LoaderCx};
use vantage_surrealdb::surrealdb::SurrealDB;
use vantage_surrealdb::types::AnySurrealType;
use vantage_surrealdb::vista::SurrealVistaFactory;
use vantage_table::table::Table;
use vantage_types::entity;
use vantage_vista::SortDirection;

#[entity(SurrealType)]
#[derive(Debug, Clone, Default)]
pub struct Product {
    pub name: String,
    pub price: i64,
    pub stock: i64,
    pub created: i64,
}

impl Product {
    pub fn surreal_table(db: SurrealDB) -> Table<SurrealDB, Product> {
        Table::new("product", db)
            .with_id_column("id")
            .with_column_of::<String>("name")
            .with_column_of::<i64>("price")
            .with_column_of::<i64>("stock")
            .with_column_of::<i64>("created")
    }
}

#[tokio::main(flavor = "current_thread")]
async fn main() -> Result<()> {
let dsn = std::env::var("SURREALDB_URL")
    .unwrap_or_else(|_| "cbor://root:root@localhost:8000/bar/v1".to_string());
let client = SurrealConnection::dsn(&dsn)
    .map_err(|e| error!("invalid SURREALDB_URL DSN", details = e.to_string()))?
    .connect()
    .await
    .map_err(|e| error!("connect surrealdb", dsn = &dsn, details = e.to_string()))?;
let db = SurrealDB::new(client);
let mut master =
    SurrealVistaFactory::new(db.clone()).from_table(Product::surreal_table(db.clone()))?;
master.set_order(Some(("created", SortDirection::Ascending)))?;

let lens = Lens::builder()
    .cache(Cache::memory())
    .loader(Loader::eager(|cx: LoaderCx| async move {
        let rows = cx.master().list_values().await?;
        cx.reconcile(rows).await
    }))
    .build()
    .context("build lens")?;

let dio = lens.dio(master).live().build().await?; // the same call, now backed by LIVE SELECT
Ok(())
}

And the frontend is the same file as learn-10. The React page consumes ?watch=true and animates the shelf; it never learns which database is behind the stream. Diff the two frontend/index.html files and you get nothing: the reactive layer, up to and including the UI, does not depend on the backend.

The payoff: precise push, no re-read

live() looks the same, but underneath it does something better than it could on Postgres. SurrealDB’s LIVE SELECT delivers a typed, per-row notification for every change: the action (CREATE, UPDATE, DELETE) and the affected record. So the Dio applies each change directly to its cache, as ChangeEvent::Inserted, Updated or Deleted would through dio.apply_change. A create inserts one row, an update repaints one row, a delete removes one row. There is no “something changed, re-read everything” step, because the change arrived in full.

$ curl -N 'localhost:3011/api/products?watch=true'
{"type":"ADDED","object":{"index":0,"id":{"@@TAGGED@@":[8,["product","d51..."]]},"name":"Mojito","price":1000,"stock":11}}
{"type":"MODIFIED","object":{"index":0,"id":{"@@TAGGED@@":[8,["product","d51..."]]},"name":"Mojito","price":1000,"stock":10}}
{"type":"DELETED","object":{"index":0,"id":{"@@TAGGED@@":[8,["product","d59..."]]},"name":"Boulevardier","price":1400,"stock":1}}

The id arrives as a SurrealDB Thing rather than a bare string. The frontend’s idOf helper normalises either shape to a stable key, which is why the same UI serves both backends.

Why does SurrealDB need no with_notify(true)?

Because there is nothing to forget. Postgres pushes only through a trigger the application installs, and the driver can’t see whether it exists, so the application has to declare it. SurrealDB’s LIVE SELECT is part of the server: any table can be followed, with no schema of yours involved. The Surreal Vista advertises Subscribe from the moment you build it, and live() uses it.

The asymmetry is the backend’s, not Vantage’s. A capability the driver can verify is advertised by default; one that depends on application setup is opt-in, so a Vista never promises a feed that may be silent.

Checklist

• The entity marker becomes #[entity(SurrealType)]; the columns don’t move.
• SurrealConnection::dsn(..).connect() replaces DATABASE_URL, and DEFINE TABLE is the only schema.
• New records take a Thing id: Thing::new("product", key).
• SurrealVistaFactory::new(db).from_table(..) builds the master; the Lens and .live() are unchanged.
• Each LIVE SELECT frame is applied as one row change; nothing is re-read.

From here the reference half of the book takes over: SurrealDB for the driver details, SQL for the dialects, Config-Driven Vistas for declaring all of this from YAML, Live Data for what every backend does about freshness, and Adding a New Persistence when your backend isn’t one Vantage already speaks.

The Vantage Journey

Vantage didn’t start as a multi-backend entity framework. It started as a weekend experiment to see if Rust could build SQL queries without feeling like Rust.

This page walks you through each release — what changed, why it mattered, and how the API evolved from raw Postgres queries to a universal persistence layer.

• 0.0 — “DORM” (April–November 2024)
  • Milestones
• 0.1 — Vantage is born (December 2024)
• 0.2 — Entity Framework & MDA (February 2025)
• 0.3 — The Great Separation (July–October 2025)
• 0.4 — The Type System Rewrite (November 2025–April 2026)
• 0.5 — The Vista Era (May–June 2026)
  • Capabilities — the explicit contract
  • Config-driven: YAML specs and Rhai scripting
  • Contained relations and nested writes
  • Diorama — caching, events, and reactive views
  • More backends, more reach
• 0.6 — Hardening (June 2026)
• The bigger picture

graph LR
    D[0.0 DORM] -->|rename| V1[0.1 Vantage]
    V1 -->|entity framework| V2[0.2 MDA]
    V2 -->|crate split| V3[0.3 UI Adapters]
    V3 -->|type rewrite| V4[0.4 Type System]
    V4 -->|universal handle| V5[0.5 Vista]
    V5 -->|hardening| V6[0.6 Robustness]
    style D fill:#555,color:#fff
    style V1 fill:#4a7c59,color:#fff
    style V2 fill:#2d6a8f,color:#fff
    style V3 fill:#8f5a2d,color:#fff
    style V4 fill:#7c2d8f,color:#fff
    style V5 fill:#2d8f6a,color:#fff
    style V6 fill:#8f2d4a,color:#fff

0.0 — “DORM” (April–November 2024)

The project was originally called DORM — the Dry ORM. It was Postgres-only, monolithic, and proudly opinionated. Everything lived in one crate.

The core idea was already there: Data Sets. Instead of loading records eagerly, you describe what you want and let the framework figure out the query.

let clients = Client::table();           // Table<Postgres, Client>
let paying = clients.with_condition(
    clients.is_paying_client().eq(&true)
);
let orders = paying.ref_orders();         // Table<Postgres, Order>

for order in orders.get().await? {
    println!("#{} total: ${:.2}", order.id, order.total as f64 / 100.0);
}

Behind this innocent-looking code, DORM generated a single SQL query with subqueries, joins, and soft-delete filters — all derived from model definitions:

SELECT id,
    (SELECT name FROM client WHERE client.id = ord.client_id) AS client_name,
    (SELECT SUM((SELECT price FROM product WHERE id = product_id) * quantity)
     FROM order_line WHERE order_line.order_id = ord.id) AS total
FROM ord
WHERE client_id IN (SELECT id FROM client WHERE is_paying_client = true)
  AND is_deleted = false

No hand-written SQL. No query strings. The framework composed everything from relationship definitions and table extensions like SoftDelete.

Milestones

0.1 — Vantage is born (December 2024)

On December 12, the framework was renamed from DORM to Vantage and published to crates.io for the first time. The API stayed the same — this was a branding milestone, not an architectural one.

The same month brought the first Axum integration (bakery_api), proving that data sets could drive REST endpoints naturally:

async fn list_orders(
    client: axum::extract::Query<OrderRequest>,
    pager: axum::extract::Query<Pagination>,
) -> impl IntoResponse {
    let orders = Client::table()
        .with_id(client.client_id.into())
        .ref_orders();

    let mut query = orders.query();
    query.add_limit(Some(pager.per_page));
    Json(query.get().await.unwrap())
}

Tags v0.1.0 and v0.1.1 were published the same day. SQLx data source landed 11 days later.

0.2 — Entity Framework & MDA (February 2025)

Version 0.2 repositioned Vantage as a full Entity Framework with Model-Driven Architecture. The README doubled in size. The vision expanded from “clever query builder” to “how enterprises should structure business logic.”

The key insight: entities aren’t just database rows. They’re business objects that might live in SQL, NoSQL, a REST API, or a message queue — and your code shouldn’t care which.

impl Client {
    fn table() -> Table<Client, Oracle> { /* ... */ }
    fn registration_queue() -> impl Insertable<Client> { /* Kafka */ }
    fn admin_api() -> impl DataSet<Client> { /* REST */ }
    fn read_csv(file: String) -> impl ReadableDataSet<Client> { /* CSV */ }
}

This release also introduced the idea of struct projection — using different Rust types against the same data set to control which fields get queried:

struct MiniClient { name: String }
struct FullClient { name: String, email: String, balance: Decimal }

// Only fetches `name` from the database
let name = clients.get_id_as::<MiniClient>(42).await?.name;

The monolith was getting heavy, though. Everything still lived in one crate, and adding a new database meant touching core code.

0.3 — The Great Separation (July–October 2025)

Version 0.3 broke the monolith into dedicated crates and bet heavily on SurrealDB as the primary backend. The trait-based architecture that defines Vantage today was born here.

Table definitions moved from static initialization to a builder pattern:

// 0.2 — static, Postgres-only
Table::new_with_entity("bakery", postgres())
    .with_id_column("id")
    .with_column("name")
    .with_many("clients", "bakery_id", || Box::new(Client::table()))

// 0.3 — builder, any datasource
Table::<SurrealDB, Client>::new("client", ds.clone())
    .with_id_column("id")
    .with_column("name")
    .with_column("email")
    .with_many("orders", "client_id", || Client::order_table())

Field accessors now return Expression instead of column objects — making them composable across query builders:

// Build conditions from expressions
let active = clients.is_paying_client().eq(true);
let big_spenders = clients.balance().gt(1000);
let query = clients.with_condition(active).with_condition(big_spenders);

The AnyTable type-erasure system arrived, enabling generic code that works with any datasource:

let tables: Vec<AnyTable> = vec![
    AnyTable::new(Client::table()),   // SurrealDB
    AnyTable::new(Product::table()),  // SQLite
];

for table in &tables {
    println!("{}: {} records", table.name(), table.count().await?);
}

This release culminated with UI adapters for six frameworks — egui, GPUI, Slint, Tauri, Ratatui, and Cursive — all driven by the same AnyTable interface.

0.4 — The Type System Rewrite (November 2025–April 2026)

0.5 — The Vista Era (May–June 2026)

Version 0.4 made the type system precise. Version 0.5 asks the next question: how does code that doesn’t know your entity type talk to your data? A CLI that lists any table, a web admin that draws forms from a YAML schema, a UI data grid pointed at whatever you give it — none of them know your Product struct.

0.4’s answer was AnyTable: type-erasure that funnelled everything through JSON. 0.5 replaces it with Vista — a universal, schema-bearing data handle. A Vista wraps any typed Table<DB, E>, erases both backend and entity, and carries its own schema (columns, references, capabilities) while delegating execution to a per-driver TableShell.

// Wrap any typed table — SQLite, MongoDB, AWS, all the same shape
let vista = SqliteVistaFactory::new(db).from_table(Product::table(db.clone()))?;

// Everything is now runtime introspection — no generics, no entity type
for name in vista.get_column_names() {
    let col = vista.get_column(name).unwrap();
    println!("{}: {}", col.name, col.original_type);
}

let mut v = vista.clone();
v.add_condition_eq("category_id", 1.into())?;     // driver translates to native condition
v.add_search("tart")?;                            // fans across SEARCHABLE columns
v.add_order("price", SortDirection::Descending)?;
let rows = v.fetch_page(1).await?;

The AnyTable carrier was deleted outright in vantage-table 0.5.2 — type erasure now lives one layer up, in Vista. That decommission is what the 0.5 version bump marks.

Capabilities — the explicit contract

Not every backend can do everything. A CSV file can’t sort server-side; DynamoDB can only order by its sort key; a REST API may paginate by cursor but not by page number. Vista makes this explicit with a struct of capability flags, and each driver declares exactly what it supports:

let caps = vista.capabilities();
if caps.can_search { v.add_search("query")?; }
if caps.can_fetch_page { /* random-access pager */ } else { /* load-more button */ }

Calling a method the driver doesn’t advertise returns an Unsupported error — never a silent match-all, never a panic. UI adapters branch on these flags to decide which controls to render.

Config-driven: YAML specs and Rhai scripting

A Vista no longer needs a hand-written Rust definition. Tables, columns, and relations can be declared in a YAML spec (VistaSpec) and loaded by the driver’s factory. For anything YAML can’t express — vendor-specific expressions, derived sources, scripted reference traversal — there’s an optional Rhai DSL that compiles to native queries:

let users = table("users").alias("u");
select()
    .from(users)
    .expression(users["name"])
    .where(users["age"] >= 18)
    .order_by(users["name"], "asc")

The same script renders dialect-correct SQL for SQLite, Postgres, and MySQL — automatic identifier quoting, date_format() mapping to strftime/TO_CHAR/DATE_FORMAT, group_concat() becoming GROUP_CONCAT/STRING_AGG. SurrealDB gets its own vocabulary (graph traversal, RELATE, record ids). YAML stays the primary format; Rhai is the targeted, serializable escape hatch.

Contained relations and nested writes

Embedded objects and arrays — an order’s lines, a row’s JSON column — now surface as a fully editable sub-Vista. SurrealDB backs them with native nested objects; SQL backends store them as JSON columns and patch the host column on writeback. And insert_value learned to walk relations: hand it a record whose keys name a relation and it sequences the writes, stamping foreign keys automatically.

Diorama — caching, events, and reactive views

The biggest new subsystem is vantage-diorama — a layer that sits between a Vista and whatever consumes it. Three concepts:

• Lens — long-lived shared infrastructure: a cache backend, lifecycle callbacks, refresh policy. Built once per application.
• Dio — a Vista bound to a Lens. Owns the cache table, a write queue, an event bus, and a refresh task.
• Scenery — a reactive view onto a Dio (ordered tables, individual records, aggregates) that a UI binds to.

let lens = Arc::new(
    Lens::new()
        .cache_at("./cache.redb")
        .on_start(|dio| { let dio = dio.clone(); async move {
            let rows = dio.master().list_values().await?;
            dio.cache().insert_values(rows).await?;
            Ok(())
        }})
        .refresh_every(Duration::from_secs(300))
        .build()?,
);
let products = lens.make_dio(products_vista).await?;
let mut v = products.vista();          // facade: reads from cache, writes through the queue

More backends, more reach

0.5 also widened the backend roster around the Vista abstraction: redb (embedded key-value store), AWS (DynamoDB and friends, all returning Vistas directly), a Cmd source that turns a shell script into a queryable table, and an API pool. Cross-persistence reference traversal — categories in Postgres, products in MongoDB — moved out of Vista and into vantage-vista-factory’s VistaCatalog, keeping each Vista strictly single-backend.

0.6 — Hardening (June 2026)

If 0.5 was about reach, 0.6 is about trust. The release is a coordinated, workspace-wide sweep with a single theme: no panics, no silent failures. Every path that could abort the process deep inside a write, or quietly return a wrong answer, was made fallible and explicit.

• Serialization is fallible. A new TryIntoRecord trait replaces the infallible blanket: a failing Serialize (non-string map keys, an out-of-range number) now becomes a recoverable error instead of a process abort mid-write. Entity write paths (insert / replace / patch) propagate it.
• No silent zeros. get_count unwraps the common [{"count": N}] shape and surfaces unrecognized result shapes as errors instead of returning 0.
• No silent match-all. Search on a backend that can’t search server-side (CSV, redb) returns an Unsupported error — never a quietly unfiltered result set.
• Expression safety. f64::IntoValue degrades NaN/Infinity to Null instead of panicking; Expression::preview() uses single-pass interleaving so a {} inside a rendered value can’t corrupt the output. select_column became fallible.
• Clearer error classification. vantage-core’s error-kind annotators were renamed to mark_unsupported / mark_unimplemented (the is_* names are now real predicates), and trace emission was decoupled — mark_* no longer logs implicitly; chain .traced() when you want the tracing::error! event.
• Security: SurrealDB identifier injection. Identifier escaping was unified into surreal-client’s single escape_identifier authority and fixed for SurrealDB 3.x — a crafted ⟩ inside a column name could previously break out of ⟨…⟩ quoting and inject arbitrary SurrealQL. similarity() and time_group() literals are now bound as parameters rather than interpolated, closing the same hole on the Rhai-exposed search path.

Alongside the hardening, 0.6 folded the standalone vantage-data-script crate into vantage-vista’s rhai feature, added random-access windowing (fetch_window) with REST lazy-loading, and introduced ColumnFlag::Label for status-tag display hints.

The bigger picture

Looking at the trajectory:

What started as 16 commits in April 2024 is now 800+ commits, 300+ pull requests, and a framework that can drive the same business logic across a dozen backends and six UI frameworks — typed when you know the entity, universal (via Vista) when you don’t.

The destination hasn’t changed since day one: describe your data once, use it everywhere. Each version just made “everywhere” a little bigger.

What’s New in Vantage 0.4

Version 0.4 is a ground-up rewrite of the type system and a massive expansion of persistence support. If 0.3 proved the architecture, 0.4 makes it production-ready.

• Progressive model — implement only what you need
• Persistence-specific Type Systems
• Table — the interface you get for free
• ActiveEntity and ActiveRecord
• Record<V> — persistence-native value bags
• CBOR everywhere
• New persistences
• Unified error handling
• AnyTable goes universal
• What’s still coming

Progressive model — implement only what you need

In earlier versions, adding a new backend meant implementing everything. Version 0.4 flips this: capabilities unlock progressively based on which traits your persistence (backend) implements. There are two parallel trait hierarchies — one for typed entities, one for schema-less records.

DataSet works with your Rust structs:

// A CSV file is read-only — and that's fine
impl ReadableDataSet<Product> for CsvFile { /* ... */ }

// A message queue can only append
impl InsertableDataSet<Event> for KafkaTopic { /* ... */ }

// A full database gets the complete toolkit
impl ReadableDataSet<Client> for SurrealDB { /* ... */ }
impl WritableDataSet<Client> for SurrealDB { /* ... */ }
// ReadableDataSet + WritableDataSet automatically unlocks ActiveEntitySet!

ValueSet is the schema-less counterpart — same hierarchy, but operates on Record<Value> instead of entities. Useful for dynamic tables, config-driven schemas, or when you don’t have (or want) a struct:

// Read raw records — no struct needed
impl ReadableValueSet for CsvFile { /* returns Record<CsvType> */ }

// Full CRUD on raw values
impl WritableValueSet for SurrealDB { /* accepts Record<AnySurrealType> */ }
// ReadableValueSet + WritableValueSet automatically unlocks ActiveRecordSet!

Each layer is opt-in. A CSV persistence never pretends it can build select queries.

The DataSource layer is progressive too. These traits aren’t a strict hierarchy — they combine independently, and each persistence implements only what its engine supports:

DataSource                        ← marker, all backends
 ├── TableSource                  ← CRUD, columns, conditions
 ├── ExprDataSource               ← execute expressions, defer()
 └── SelectableDataSource         ← query builder (Selectable)

TableExprSource                   ← count/sum/max/min as composable expressions
 requires: TableSource + ExprDataSource

TableQuerySource                  ← table definition → full query
 requires: TableSource + SelectableDataSource

Here’s what each persistence implements:

CSV
 └── TableSource (read-only, in-memory conditions)

REST API  (vantage-api-client)
 ├── TableSource (read-only, HTTP GET)
 └── ExprDataSource (basic resolve)

MongoDB
 ├── TableSource (full CRUD, native bson::Document conditions)
 └── SelectableDataSource (field projection, $match pipeline)
     └── aggregates via native $sum/$max/$min pipeline

Postgres / MySQL / SQLite  (via sqlx)
 ├── TableSource (full CRUD)
 ├── ExprDataSource (parametric SQL execution)
 ├── SelectableDataSource (JOINs, CTEs, window functions)
 └── TableQuerySource

SurrealDB
 ├── TableSource (full CRUD)
 ├── ExprDataSource (CBOR execution)
 ├── SelectableDataSource (SurrealQL builder)
 ├── TableExprSource (composable aggregates)
 ├── TableQuerySource
 └── vendor extensions (graph queries, RELATE, live queries)

There’s also MockTableSource in vantage-table and ImDataSource in vantage-dataset — a fully in-memory persistence backed by HashMap<String, IndexMap<String, Record<Value>>>. Great for tests and prototyping without any database.

Persistence-specific Type Systems

The vantage-types crate is the foundation of 0.4. Each persistence defines its own type universe via the vantage_type_system! macro — no more funnelling everything through JSON:

vantage_type_system!(AnySurrealType, SurrealDB);   // CBOR-native types
vantage_type_system!(AnyMongoType, MongoDB);        // BSON-native types
vantage_type_system!(AnySqliteType, SqliteDB);      // CBOR with integer/real/text markers
vantage_type_system!(AnyPostgresType, PostgresDB);  // CBOR with full PG type coverage
vantage_type_system!(AnyMysqlType, MysqlDB);        // CBOR with MySQL type coverage

All SQL backends (SQLite, Postgres, MySQL) now use CBOR as their internal value representation — not JSON. This preserves type fidelity that JSON loses (integer vs float, binary blobs, precise decimals) while keeping the same vantage_type_system! ergonomics.

Strict conversions replace silent casting — try_into() is explicit and fallible:

let val: i64 = surreal_value.try_into()?;   // fails if it's actually a string
let val: String = mongo_value.try_into()?;   // fails if it's actually a number

These types flow through the entire framework, not just storage. Columns carry their type as a generic parameter (Column<i64>, Column<String>) — surviving type-erasure via original_type so UI adapters can inspect what a column actually holds. AssociatedExpression<'a, DS, T, R> is parameterised by the value type T, so a count query on SurrealDB returns AnySurrealType while the same query on Postgres returns AnyPostgresType. AssociatedQueryable<R> then converts that into your expected Rust type — all type-checked at compile time.

// Column preserves type through erasure
let price = Column::<i64>::new("price").with_flag(ColumnFlag::Indexed);
let erased = Column::<AnyType>::from_column(price);
assert_eq!(erased.get_type(), "i64");  // original type survives

// AssociatedExpression carries the persistence's value type
let count: AssociatedExpression<'_, SurrealDB, AnySurrealType, usize> =
    table.get_expr_count();
let n: usize = count.get().await?;  // executes, converts AnySurrealType → usize

This means the full range of your persistence’s native types is preserved end-to-end — from column definition through query building to result extraction — without narrowing down to JSON variants.

Table — the interface you get for free

Once a persistence implements TableSource, the framework hands you Table<DB, Entity> — a fully-featured abstraction over your data with columns, conditions, ordering, pagination, relationships, and aggregates. You don’t build any of this yourself; it comes from vantage-table.

Table<DB, E> auto-implements a wide range of traits from the DataSet and ValueSet hierarchies: ReadableDataSet<E>, WritableDataSet<E>, InsertableDataSet<E>, ActiveEntitySet<E>, ReadableValueSet, WritableValueSet, InsertableValueSet, and ActiveRecordSet. All of these come for free once your persistence implements TableSource.

let products = Product::table(db)
    .with_condition(products["is_deleted"].eq(false))
    .with_order(products["price"].desc())
    .with_pagination(Pagination::ipp(25));

// ReadableDataSet, WritableDataSet, ActiveEntitySet — all auto-implemented
for (id, product) in products.list().await? { /* ... */ }
let count = products.get_count().await?;

Additionally, table.select() yields a vendor-specific query builder — SurrealSelect, SqliteSelect, PostgresSelect — giving you full access to the persistence’s native query capabilities when you need to go beyond what the generic Table API offers.

This dramatically simplifies implementing new persistences. You implement TableSource methods (read, write, aggregate) and get the entire Table API for free — conditions compose automatically, columns carry types, references traverse between tables, and AnyTable type-erasure just works.

graph LR
    P[Select Persistence]
    M[Create Model Layer]
    B[Build Business Logic]
    E[Export via FFI or API]
    P ==> M ==> B ==> E
    style P fill:#4a7c59,color:#fff
    style M fill:#2d6a8f,color:#fff
    style B fill:#8f5a2d,color:#fff
    style E fill:#7c2d8f,color:#fff

Pick your persistence (SurrealDB, Postgres, CSV — or several). Define entities and tables in a shared model crate. Build business logic against Table and DataSet traits — persistence-agnostic and testable with mocks. Finally, expose your model to other languages via FFI (C ABI, PyO3, UniFFI, WASM) or through API layers (Axum, gRPC). The model stays in Rust; consumers don’t need to know. Read more in Model-Driven Architecture and Three Paths for Developers.

ActiveEntity and ActiveRecord

The vantage-dataset crate introduces two flavours of the active record pattern:

ActiveEntity<D, E> — wraps a typed entity. Derefs straight to your struct, tracks the ID, and saves back to any WritableDataSet:

let mut user = users.get_entity(&id).await?.unwrap();
user.email = "new@example.com".to_string();  // modify via DerefMut
user.save().await?;                           // persists the change

ActiveRecord<D> — same idea, but schema-less. Works with Record<Value> instead of a concrete struct, perfect for dynamic tables or config-driven entities:

let mut rec = table.get_value_record(&id).await?;
rec["status"] = json!("active");
rec.save().await?;

Both auto-unlock via blanket impls — if your datasource implements ReadableDataSet + WritableDataSet, you get ActiveEntitySet for free. No extra code.

let mut user = users.get_entity(&id).await?
    .unwrap_or_else(|| users.new_entity(id, User::default()));
user.name = "Alice".into();
user.save().await?;

Record<V> — persistence-native value bags

Record<V> is an IndexMap<String, V> wrapper — but V is not serde_json::Value. It’s your persistence’s own type. A SurrealDB table returns Record<AnySurrealType>, Postgres returns Record<AnyPostgresType>, MongoDB returns Record<AnyMongoType>. This ensures the full range of your persistence’s native types is preserved — not narrowed down to what JSON can represent.

// Each persistence speaks its native types
let record: Record<AnySurrealType> = surreal_table.get_value(&id).await?;
let record: Record<AnyPostgresType> = pg_table.get_value(&id).await?;
let record: Record<AnyMongoType> = mongo_table.get_value(&id).await?;

// Type-safe extraction — respects the persistence's type boundaries
let name: String = record["name"].try_get::<String>().unwrap();

Record is the common currency across the framework — ReadableValueSet, WritableValueSet, ActiveRecord, and entity conversion all work through it. JSON conversion only happens at the boundary when you need AnyTable type-erasure (via from_table()).

CBOR everywhere

The SurrealDB client switched from JSON to binary CBOR, improving serialization performance and type fidelity. All backends followed — Postgres, MySQL, SQLite.

New persistences

MongoDB, CSV, and REST API persistences joined SurrealDB and SQLite. CSV evaluates conditions in-memory. MongoDB uses native BSON filters — no expression translation needed:

// Same handle_commands function — different persistences
handle_commands(SurrealDB::table("product")).await?;
handle_commands(CsvFile::table("products.csv")).await?;
handle_commands(MongoDB::table("product")).await?;

Unified error handling

vantage-core introduced VantageError with structured context — replacing the patchwork of Box<dyn Error> and .unwrap() calls:

use vantage_core::{error, util::error::Context, Result};

connection.connect()
    .with_context(|| error!("Failed to connect", dsn = &dsn, timeout = 30))?;

AnyTable goes universal

AnyTable::from_table() now wraps any datasource whose types convert to/from JSON, using an internal JsonAdapter for on-the-fly conversion. This means your generic code works with persistences that don’t even share a value type:

// Wrap a MongoDB table for use in generic code
let any: AnyTable = AnyTable::from_table(mongo_products);
let any: AnyTable = AnyTable::from_table(csv_products);
// Both work through the same interface

What’s still coming

What’s New in Vantage 0.5

Version 0.4 made the type system precise — every backend carries its own types, and a persistence implements only what its engine supports. Version 0.5 answers the question that follows: how does code that doesn’t know your entity type talk to your data?

A CLI that lists any table, a web admin that draws forms from a YAML schema, a UI data grid pointed at whatever you hand it — none of them know your Product struct. 0.4’s answer was AnyTable: type-erasure funnelled through JSON. 0.5 replaces it with Vista, a universal, schema-bearing data handle, and builds an entire reactive stack on top of it.

• Vista — the universal data handle
• CBOR, not JSON
• Capabilities — the explicit contract
• Config-driven: YAML specs and Rhai scripting
• Contained relations and nested writes
• Diorama — caching, events, and reactive views
• Cross-persistence traversal: VistaCatalog
• More backends
• Migrating off AnyTable
• What’s still coming

Vista — the universal data handle

A Vista wraps any typed Table<DB, E> and erases both the backend and the entity. All data flows through Record<CborValue> — an ordered map of string keys to CBOR values. All schema lives on the Vista itself: columns (with types and flags), references, the id column, and a set of capability flags. Execution is delegated to a per-driver TableShell.

Table<SqliteDB, Product>   — typed entity, typed backend, compile-time safe
Vista                      — fully erased: schema-bearing, CborValue, no generics

Each backend ships a factory that turns a typed table into a Vista. The factory harvests columns, id field, title fields, and references from the table definition you already built — no extra mapping code:

use vantage_sql::prelude::*;
use vantage_vista::Vista;

let table = Product::table(db.clone());
let vista = SqliteVistaFactory::new(db).from_table(table)?;

For MongoDB it’s MongoVistaFactory, for AWS AwsVistaFactory — same shape, different import. The resulting Vista is identical regardless of which factory produced it. From there, everything is runtime introspection:

// Schema — no entity type required
for name in vista.get_column_names() {
    let col = vista.get_column(name).unwrap();
    println!("{}: {}", col.name, col.original_type);  // name: String, price: i64, ...
}
for (name, kind) in vista.list_references() {
    println!("ref: {} ({:?})", name, kind);           // ref: products (HasMany)
}

// Query — Vista delegates the value to the driver's native condition type
let mut v = vista.clone();
v.add_condition_eq("category_id", 1.into())?;
v.add_search("tart")?;                                 // fans across SEARCHABLE columns
v.add_order("price", SortDirection::Descending)?;
let rows = v.fetch_page(1).await?;

CBOR, not JSON

Where AnyTable narrowed every value to serde_json::Value, Vista carries ciborium::Value end to end. CBOR preserves the type fidelity JSON loses — integer vs float, binary blobs, precise decimals — so a Record<CborValue> round-trips through the Vista layer without lossy hops. JSON conversion happens only at the boundary, when you actually need it (an HTTP response, for example), and it’s a one-liner.

This is the same carrier the backends already use internally (Surreal and SQL store CBOR; Mongo stores BSON), so wrapping a typed table into a Vista no longer crosses a JSON funnel.

Capabilities — the explicit contract

Not every backend can do everything. A CSV file can’t sort or search server-side; DynamoDB orders only by its declared sort key; a token-paginated REST API offers a forward cursor but no page numbers. Vista makes this explicit with VistaCapabilities — a struct of booleans where each driver declares exactly what it supports:

let caps = vista.capabilities();
if caps.can_search { v.add_search("query")?; }
if caps.can_fetch_page {
    let page = v.fetch_page(2).await?;          // random access
} else if caps.can_fetch_next {
    let (rows, token) = v.fetch_next(None).await?;  // forward cursor
}

UI adapters branch on these flags directly: a data grid checks can_fetch_page to decide between a scrollbar (random access) and a “load more” button (cursor-based).

Config-driven: YAML specs and Rhai scripting

A Vista no longer needs a hand-written Rust definition. Tables, columns, and relations can be declared in a YAML spec (VistaSpec) and materialized by the driver’s factory:

table: product
columns:
  - { name: name, flags: [title, searchable] }
  - { name: price }
references:
  - { name: orders, kind: has_many, foreign_key: product_id }

For everything YAML can’t express — vendor-specific expressions, derived sources, scripted reference traversal — there’s an optional Rhai DSL that compiles to native queries. The same script renders dialect-correct SQL across all three SQL backends:

let users = table("users").alias("u");
select()
    .from(users)
    .expression(users["name"])
    .where(users["age"] >= 18)
    .order_by(users["name"], "asc")

• Automatic identifier quoting (backticks for MySQL, double quotes for Postgres/SQLite)
• Dialect-aware primitives: date_format() → strftime() / TO_CHAR() / DATE_FORMAT()
• group_concat() → GROUP_CONCAT (SQLite/MySQL) / STRING_AGG (Postgres)

SurrealDB gets its own Rhai vocabulary — graph traversal (graph()/recurse()), record ids (thing), $parent references, and SurrealDB-namespaced aggregates. A per-reference Rhai script can even override the default foreign-key traversal, evaluated lazily with the parent row in scope.

Contained relations and nested writes

Embedded objects and arrays — an order’s lines array, a row’s JSON column — now surface as a fully editable sub-Vista:

let order = Order::table(db)
    .with_contained_many("lines", |line| {
        line.with_column("product_id").with_column("quantity")
    });

SurrealDB backs contained relations with native nested objects and arrays; the SQL backends store them as JSON columns and patch the host column on writeback. Reads project the column into records; writes re-serialize the whole collection and patch the parent row. Contained records can even traverse out to real tables (line.product).

Insert learned to walk relations, too. Hand insert_value a record whose keys name a relation instead of a column, and Vista sequences the writes so foreign keys populate automatically: a has-one child is inserted first and its id stamped into the parent’s FK; has-many children are inserted after the parent with the parent’s id stamped into each. Arbitrary depth, same-persistence relations only.

Diorama — caching, events, and reactive views

The largest new subsystem is vantage-diorama, a layer that sits between a Vista and whatever consumes it. It does three things: caches transparently, injects capabilities the backend lacks, and routes writes wherever you want.

graph LR
    M[Master Vista] --> D[Dio]
    L[Lens<br/>cache + callbacks] --> D
    D --> F[Facade Vista]
    D --> S[Scenery]
    D -. cache + events .-> D
    style M fill:#4a7c59,color:#fff
    style L fill:#2d6a8f,color:#fff
    style D fill:#8f5a2d,color:#fff
    style S fill:#7c2d8f,color:#fff
    style F fill:#7c2d8f,color:#fff

Four words you’ll see throughout:

• Vista — a single-backend data source (the master).
• Lens — long-lived shared infrastructure: cache backend, lifecycle callbacks, refresh policy. Built once per application.
• Dio — a Vista bound to a Lens. Owns the cache table, a write queue, an event bus, and a refresh task. Produced by lens.make_dio(vista).
• Scenery — a reactive view onto a Dio (ordered tables, individual records, aggregates). The UI binds here.