Augmentation

A Vista reads one table from one backend. But a row is often only half the story: an S3 bucket listing tells you a file’s key and size, not how many resources the terraform state inside it declares. A REST list of repositories doesn’t carry the CI status that lives behind a shell command. The data you want to show lives in a second source, keyed off the first.

Augmentation wires two Vistas into one Dio: a master that is listed, and a detail source that is loaded one row at a time and merged on top. The two can be the same Vista (a backend whose list and detail are separate operations) or entirely different backends — a REST master enriched by a cmd detail, or the reverse. The detail source is resolved by name through a VistaResolver, usually a Catalog, so it is persistence-agnostic.

Why this lives in the Dio, not the Vista

This is a join, and joins are governed by the capability contract. A SQL Vista can join its own tables; a REST Vista cannot; two different backends can never push a join down to either engine. Where the backend can’t, the layer above fills the gap — that is the whole job of the Dio.

So augmentation is deliberately a Dio-layer feature. A Vista honestly describes one backend and has no cache, no viewport, and no catalog; declaring “enrich from another Vista, loaded lazily and cached” on it would be a promise it can’t keep. The Dio has all three, so it owns the augmentation: it lists the master cheaply, and for each row the user actually scrolls to, it fetches the detail and stitches the columns together. It is a per-row join, executed client-side, exactly where a push-down is impossible.

How it runs

Augmentation engages the two-pass machinery:

1. List pass — the master is listed cheaply (a page at a time). Each row is written to the cache as Incomplete, carrying only the columns the list returns. When the master can serve windows (Capability::Window) the page is pushed down; otherwise the set is listed and windowed locally.
2. Detail pass — viewport-driven. For each visible Incomplete row, every augmentation resolves its detail Vista, fetches the matching record, and merges the chosen columns onto the cached row, which flips to Fresh. Rows off-screen are never fetched; rows already hydrated are never re-fetched.

A failed detail fetch marks only that row, leaving its cheap columns intact — the rest of the page hydrates normally.

Declaring it

Augmentation is a property of the Dio, not the Lens: it is fixed when the Dio is built. Hand the DioBuilder one or more augmentations between lens.dio(master) and build():

use std::sync::Arc;

use vantage_diorama::{Augmentation, Cache, Detail, Fetch, Lens, MergeRule, Source};
use vantage_vista::mocks::MockShell;
use vantage_vista::{Column, Vista, VistaMetadata};

fn meta(columns: &[&str]) -> VistaMetadata {
    columns
        .iter()
        .fold(VistaMetadata::new(), |m, c| m.with_column(Column::new(*c, "String")))
        .with_id_column("id")
}

fn detail() -> Vista {
    Vista::new("tfstate_detail", MockShell::new().with_metadata(meta(&["id", "resources", "serial"])))
}

#[tokio::main(flavor = "current_thread")]
async fn main() -> vantage_core::Result<()> {
let master = Vista::new("buckets", MockShell::new().with_metadata(meta(&["id", "key", "size"])));
let lens = Lens::builder().cache(Cache::memory()).build()?;
let augmentation = Augmentation::new(
    Detail::Fixed(Arc::new(detail())),
    Source::Column { from: "key".into(), to: None },
    Fetch::PerRow,
    MergeRule::lift(["resources", "serial"]),
);
let dio = lens
    .dio(master)
    .augment(vec![augmentation]) // ≥1 engages two-pass
    .build()
    .await?;
let _ = dio;
Ok(())
}

Each Augmentation has four parts. Here the detail is resolved by name through a Catalog:

#![allow(unused)]
fn main() {
use std::sync::Arc;

use vantage_diorama::{Augmentation, Detail, Fetch, MergeRule, Source};
use vantage_vista::mocks::MockShell;
use vantage_vista::{Catalog, Vista, VistaResolver};

fn detail() -> Vista {
    Vista::new("tfstate_detail", MockShell::new())
}

let resolver: Arc<dyn VistaResolver> =
    Arc::new(Catalog::new().with("tfstate_detail", |_: &str| Ok(detail())));

let augmentation = Augmentation::new(
    Detail::Resolved { resolver, name: "tfstate_detail".into() }, // resolved by name
    Source::Column { from: "key".into(), to: None },
    Fetch::PerRow,
    MergeRule::lift(["resources", "serial"]),
);
let _ = augmentation;
}

Detail::Fixed(Arc<Vista>) takes a detail Vista directly instead, for a side table no resolver knows.

source and fetch are two orthogonal axes:

Id and id-keyed Column sources read by key through get_value — the uniform “one record by key” primitive (a cmd detail script, a SQL WHERE id =, a REST GET /{id}). Build returns an arbitrary narrowed Vista, so it is per-row only.

Declaring one

An Augmentation is built in Rust. There is no serde mirror of these types: a consumer already has its own config vocabulary — vantage-ui reads an augment: block naming a source table, a key and a column list — and maps that onto Augmentation directly. A second, parallel YAML shape in diorama would be one more spelling of the same idea for each consumer to ignore.

Rhai as a closure factory

The runtime types carry closures, not script strings — Source::Build is a Fn(&row, base) -> Vista. Rhai is one factory that produces such a closure; hand-written Rust is another. A consumer that wants a scripted narrowing builds the closure itself and hands it to Source::Build. The diorama core never sees a script string. The vocabulary such a script would use is the table handle’s, covered in Scripting with Rhai.

Merging

merge.columns lists the detail columns to lift; an empty list lifts all of them. The detail record is the authoritative hydration of the row, so on a name clash its value wins — it overwrites the cheap list-pass value and adds its new columns. (An empty list plus overwrite is exactly the old cmd two-pass: the detail script returns the full record, which replaces the stub.) The augmented Dio therefore advertises the union of the master’s columns and the lifted detail columns — the “Dio advertises a superset” principle: below it, sources are partial; above it, the view is whole.

Anticipated objections

Why not model the detail as a foreign-key relation?

A relation resolves within one persistence and can be pushed down as a join. The detail here may be a different backend entirely; there is nothing to join on and no engine that could honour it. Augmentation is the cross-Vista form, stitched by the Dio.

Why one row at a time?

Because the expensive work should follow the user’s attention. The list pass is cheap and immediate; only rows that reach the viewport pay for their detail, and they pay once.

What’s next

Three things are planned but not yet implemented; each fails honestly today rather than degrading silently:

• Batched fetch — collect the window’s distinct keys into one set query and scatter the results back, for Id/Column sources.
• Detail-key cache — dedupe fetches across master rows that share a key (non-unique Column sources).
• Scripted fetch — a script choosing how to pull.

Checklist

To augment a master with a second source:

1. Register the detail model in a Catalog by name (Catalog::new().with(..)), or hold it directly for Detail::Fixed.
2. Build the Augmentation — Detail::Resolved { resolver, name }, a Source (id, column, or a closure), a Fetch (PerRow, or Custom), and the MergeRule columns.
3. Wire the Dio — lens.dio(master).augment(vec![..]).build().await?; the two-pass machinery engages for every table scenery opened on it.
4. Show the columns — the lifted detail columns appear on hydrated rows alongside the master’s.

A runnable end-to-end example (two in-memory Vistas, list pass then detail pass) lives in vantage-diorama/examples/augmentation.rs:

cargo run -p vantage-diorama --example augmentation