16. A vault of documents¶

The shelf of chapter 6 was a list written by hand. Here the same books are documents in a folder, and HQL reads the folder. This part of the book is about knowledge: documents, the links between them, their meaning, and the agents that read them.

A vault is a folder¶

A vault is a directory of Markdown and HyperMarkDown documents. The tutorial vault looks like this:

examples/tutorial/
├── authors/
│   ├── frank-herbert.hmd
│   ├── jrr-tolkien.hmd
│   ├── mary-shelley.hmd
│   ├── virginia-woolf.hmd
│   └── walt-whitman.hmd
├── books/
│   ├── a-room-of-ones-own.hmd
│   ├── dune.hmd
│   ├── frankenstein.hmd
│   ├── leaves-of-grass.hmd
│   ├── mrs-dalloway.hmd
│   ├── the-common-reader.hmd
│   └── the-hobbit.hmd
├── club.hmd
├── frankenstein-inspires-dune.hmd
├── librarian.hmd
└── hql.toml

Each document is a card. A card has a header of metadata and a body someone wrote:

---
title: Dune
metadata:
  category: book
  author: Frank Herbert
  year: 1965
  genre: novel
  pages: 412
  status: reading
---
# Dune

A novel by [[frank-herbert]]. A desert planet, a spice, and a family that inherits both.

A program run inside a vault runs against it: hql takes the nearest directory at or above the current one that holds .hmd/ or hql.toml. --vault <directory> names one instead:

hql --vault examples/tutorial eval 'cards | size'

cards | size

`cards`¶

cards is every card of the vault, as a sequence. Its order is the order a sorted walk of the folder reaches each document — the entries of each directory by name, depth first — so a program answers the same on every machine:

cards | take(4) | map(c => c.name)

["frank-herbert", "jrr-tolkien", "mary-shelley", "virginia-woolf"]

A card has the fields of a document: its name (the file stem), title, path, format, body, its kind, and metadata, the header's tree.

A link is an optional card¶

@[[name]] refers to a card by name: @ followed by the link a document writes, [[name]]. A link may point at a document nobody has written yet, so its type is Option<Card>, and reading a field through it reads through the option:

@[[dune]].title

"Dune"

@[[nowhere]].title

none

Metadata is Data¶

A card's metadata is the Data of chapter 2, so every question from chapters 2 and 6 applies. The books are the cards whose category says so:

cards
| filter(c => c.metadata.category == "book")
| map(c => c.title)

["A Room of One's Own", "Dune", "Frankenstein", "Leaves of Grass", "Mrs Dalloway", "The Common Reader", "The Hobbit"]

The reading list¶

The club's reading list is the books still to read, by title:

cards
| filter(c => c.metadata.status == "todo")
| sort(by = c => c.title)
| map(c => c.title)

["Leaves of Grass", "The Common Reader", "The Hobbit"]

Read it as three steps, each consuming the one before: filter decides each card on its own, sort needs every card before it can place the first, and map turns each card into its title. Nothing is stored in between, and nothing is a loop.

No one maintains this list. Mark a book status: done in its header and the next run of the query no longer lists it.

Writing cards¶

The formats of chapter 8 apply to a card as to any value. A card written as Markdown is its body; a list of cards, a list of links:

cards | filter(c => c.metadata.category == "author") | markdown

- [[frank-herbert]]
- [[jrr-tolkien]]
- [[mary-shelley]]
- [[virginia-woolf]]
- [[walt-whitman]]

Grouping cards¶

group partitions the books by a header value, as it partitioned the shelf:

cards
| filter(c => c.metadata.category == "book")
| group(by = c => c.metadata.status)
| map(g => {status: g.key, books: g.items | map(c => c.title)})

{{books: ["A Room of One's Own", "Frankenstein"], status: "done"}, {books: ["Dune", "Mrs Dalloway"], status: "reading"}, {books: ["Leaves of Grass", "The Common Reader", "The Hobbit"], status: "todo"}}

A typed view of a card works as the views of chapter 6 did, and the pages per author are a map over each group and a sum — map, then reduce:

fn pages(c: Card) : Int = match c.metadata.pages { some(Int n) => n, _ => 0 }

cards
| filter(c => c.metadata.category == "book")
| group(by = c => c.metadata.author)
| map(g => {author: g.key, pages: g.items | map(pages) | sum})

{{author: "Virginia Woolf", pages: 610}, {author: "Frank Herbert", pages: 412}, {author: "Mary Shelley", pages: 280}, {author: "Walt Whitman", pages: 145}, {author: "J. R. R. Tolkien", pages: 310}}

Kinds of card¶

A card whose header declares a relation between two other cards is a RelationCard; every other card is a ConceptCard. of_type keeps the cards of one kind:

cards | of_type(RelationCard) | map(c => c.title)

["Frankenstein inspires Dune"]

Chapter 17 is about what that relation means: cards and the links between them form a graph.

The vault itself¶

cards is a binding the standard library makes, cards = vault.cards, over vault, the value the host seeds when it is given a vault:

[vault.name, vault.cards | size | text]

["tutorial", 15]

Outside every vault and without --vault there is no vault, and naming one says so:

cards | size

check error[unknown-name]: `cards` needs a vault: run inside one or pass --vault <directory>
 --> <expression>:1:1
  |
1 | cards | size
  | ^^^^^

A vault is a namespace¶

A card is a file with a place in a vault, and the place is what makes it addressable by name: [[dune]] names a card because there is one tree to resolve the name in. That tree is a namespace, rooted at the vault's directory, and a vault is exactly one namespace. cards holds what a wikilink can reach and nothing else, and a card's path is its place in the vault, relative to the root, so the cards of one folder are a filter on the path:

cards | filter(c => c.path | starts_with("books/")) | size

Other vaults¶

A second vault is a second namespace. It is reached as a configured source of kind vault (sources and targets by id), with its own cards and graph, and its cards are never merged into cards: a wikilink in one vault never resolves into another, and a relation between cards of two vaults is a join's result, not a link. The field station of the data sources examples configures two:

journal = source("journal")
archive = source("archive")
[journal.cards | map(c => c.name), archive.cards | map(c => c.name)]

[["kite", "owl", "wren"], ["finch", "heron"]]

What is next¶

A document that writes [[frank-herbert]] links to another. Chapter 17 follows the links, and the cards and links together are a graph.