21. A vault of documents¶
The shelf of chapter 6 was a list written by hand. Here the same books are documents in a folder, and HQL reads the folder. This part of the book is about knowledge: documents, the links between them, their meaning, and the agents that read them.
A vault is a folder¶
A vault is a directory of Markdown and HyperMarkDown documents, marked as
one by an .hmd/ directory at its root. The reading-club vault looks like this:
examples/reading-club/
├── authors/
│ ├── frank-herbert.hmd
│ ├── jrr-tolkien.hmd
│ ├── mary-shelley.hmd
│ ├── virginia-woolf.hmd
│ └── walt-whitman.hmd
├── books/
│ ├── a-room-of-ones-own.hmd
│ ├── dune.hmd
│ ├── frankenstein.hmd
│ ├── leaves-of-grass.hmd
│ ├── mrs-dalloway.hmd
│ ├── the-common-reader.hmd
│ └── the-hobbit.hmd
├── club.hmd
├── frankenstein-inspires-dune.hmd
├── librarian.hmd
├── .hmd/config.toml
└── .hql/config.toml
.hmd/ makes the directory a vault. .hql/config.toml makes it an HQL
project too, which configures the programs run in it; the two are separate,
and a vault need not be a project
(chapter 13).
The vault's .hmd/config.toml says where its cards start, with the key
wiki: a path against the vault root, doc/wiki when the file does not set
it. The reading club keeps its cards at the vault root, so it says so:
wiki = "."
That directory is the namespace root. A document is a card exactly when
it lies below it, and a card's path is written from it, so books/dune.hmd
above. A file of the vault outside it, a README.md beside .hmd/ in a
vault whose wiki is doc/wiki, is not a card. A card is a .hmd file, or a
.md or .markdown file whose body is plain Markdown.
Each document is a card. A card has a header of metadata and a body someone wrote:
---
title: Dune
metadata:
category: book
author: Frank Herbert
year: 1965
genre: novel
pages: 412
status: reading
---
# Dune
A novel by [[frank-herbert]]. A desert planet, a spice, and a family that inherits both.
A program run inside a vault reads it: hql takes the nearest directory at
or above the program's file that holds .hmd/, or at or above the working
directory for an expression or standard input. --vault <directory> names
one instead, and --bare runs with none. The cards are read through the
builtin package hmd, which the reading-club's project imports for every
program ("The vault itself" below):
hql --vault examples/reading-club eval 'cards | size'
cards | size
15
cards¶
cards is every card of the vault, as a sequence. Its order is the order a
sorted walk of the folder reaches each document — the entries of each
directory by name, depth first — so a program answers the same on every
machine:
cards | take(4) | map(c => c.name)
["frank-herbert", "jrr-tolkien", "mary-shelley", "virginia-woolf"]
A card has the fields of a document: its name (the file stem), title,
path, format, body, its kind, and metadata, the header's tree.
A link is an optional card¶
@[[name]] refers to a card by name: @ followed by the link a document
writes, [[name]]. A link
may point at a document nobody has written yet, so its type is Option<Card>,
and reading a field through it reads through the option:
@[[dune]].title
"Dune"
@[[nowhere]].title
none
Metadata is Data¶
A card's metadata is the Data of chapter 2, so every question
from chapters 2 and 6 applies. The books are the cards whose category says so:
cards
| filter(c => c.metadata.category == "book")
| map(c => c.title)
["A Room of One's Own", "Dune", "Frankenstein", "Leaves of Grass", "Mrs Dalloway", "The Common Reader", "The Hobbit"]
The reading list¶
The club's reading list is the books still to read, by title:
cards
| filter(c => c.metadata.status == "todo")
| sort(by = c => c.title)
| map(c => c.title)
["Leaves of Grass", "The Common Reader", "The Hobbit"]
Read it as three steps, each consuming the one before: filter decides each
card on its own, sort needs every card before it can place the first, and
map turns each card into its title. Nothing is stored in between, and
nothing is a loop.
No one maintains this list. Mark a book status: done in its header and the
next run of the query no longer lists it.
Writing cards¶
The formats of chapter 8 apply to a card as to any value. A card written as Markdown is its body; a list of cards, a list of links:
cards | filter(c => c.metadata.category == "author") | markdown
- [[frank-herbert]]
- [[jrr-tolkien]]
- [[mary-shelley]]
- [[virginia-woolf]]
- [[walt-whitman]]
Grouping cards¶
group partitions the books by a header value, as it partitioned the shelf:
cards
| filter(c => c.metadata.category == "book")
| group(by = c => c.metadata.status)
| map(g => {status: g.key, books: g.items | map(c => c.title)})
{{books: ["A Room of One's Own", "Frankenstein"], status: "done"}, {books: ["Dune", "Mrs Dalloway"], status: "reading"}, {books: ["Leaves of Grass", "The Common Reader", "The Hobbit"], status: "todo"}}
A typed view of a card works as the views of chapter 6 did, and the pages per author are a map over each group and a sum — map, then reduce:
fn pages(c: Card) : Int = match c.metadata.pages { some(Int n) => n, _ => 0 }
cards
| filter(c => c.metadata.category == "book")
| group(by = c => c.metadata.author)
| map(g => {author: g.key, pages: g.items | map(pages) | sum})
{{author: "Virginia Woolf", pages: 610}, {author: "Frank Herbert", pages: 412}, {author: "Mary Shelley", pages: 280}, {author: "Walt Whitman", pages: 145}, {author: "J. R. R. Tolkien", pages: 310}}
Kinds of card¶
A card whose header declares a relation between two other cards is a
RelationCard; every other card is a ConceptCard. of_type keeps the cards
of one kind:
cards | of_type(RelationCard) | map(c => c.title)
["Frankenstein inspires Dune"]
Chapter 22 is about what that relation means: cards and the links between them form a graph.
The vault itself¶
cards is a binding of the builtin package hmd, cards = vault.cards, over
vault, the value the host seeds when it is given a vault:
[vault.name, vault.cards | size | text]
["reading-club", 15]
A vault is HyperMarkdown's, and so are vault and cards. They are bound
while the package hmd is active, and a run that has a vault activates
it: the vault found at or above the program, or the one --vault names. A
program in a vault writes cards with no import.
A run with no vault activates hmd by importing it: import hmd in the
program, --import hmd on the command line, or [modules] import in the
project's .hql/config.toml:
[modules]
import = ["hmd"]
With no vault and no import, cards names what is missing:
cards | size
check error[unknown-name]: `cards` is HyperMarkdown's: write `import hmd`, or run with `--import hmd`, to bind it
--> <expression>:1:1
|
1 | cards | size
| ^^^^^
With hmd imported and no vault, cards has nothing to read, and the
check says so:
import hmd
cards | size
check error[unknown-name]: `cards` needs a vault: run inside one or pass --vault <directory>
--> <expression>:2:1
|
2 | cards | size
| ^^^^^
A vault is a namespace¶
A card is a file with a place in a vault, and the place is what makes it
addressable by name: [[dune]] names a card because there is one tree to
resolve the name in. That tree is a namespace, rooted at the vault's
namespace root, and a vault is exactly one namespace. cards holds what a
wikilink can reach and nothing else, and a card's path is its place in the
vault, relative to the namespace root, so the cards of one folder are a filter
on the path:
cards | filter(c => c.path | starts_with("books/")) | size
7
HQL resolves a reference exactly as HyperMarkdown does, through
HyperMarkdown's own resolver, so hql and hmd lint always name the same
card. A [[name]] in a card is looked up from that card: a path that opens
with /, ./ or ../ is followed; otherwise a name the card imports, then
the card's own folder and each folder above it, nearest first, then the
folders it imports with *, and last a search of the whole vault. A search
that finds two cards chooses neither, and the reference has to be
qualified, [[books/dune]]. @[[dune]] in a program is resolved as though
written at the namespace root. Where dune.hmd and dune.md sit side by
side, [[dune]] is the .hmd card and [[dune.md]] the other, whose name
keeps its extension.
Other vaults¶
A second vault is a second namespace. It is reached as a configured endpoint of
kind vault (endpoints by id), with its own
cards and graph, and its cards are never merged into cards: a wikilink
in one vault never resolves into another, and a relation between cards of two
vaults is a join's result, not a link. The field station of the data sources
examples configures two:
journal = source("journal")
archive = source("archive")
[journal.cards | map(c => c.name), archive.cards | map(c => c.name)]
[["kite", "owl", "wren"], ["finch", "heron"]]
What is next¶
A document that writes [[frank-herbert]] links to another.
Chapter 22 follows the links, and the cards and links together
are a graph.